......................... Purpose, two execution modes, "What mlld IS/ISN'T," mental model shift. ..... Only lines beginning with `/` are mlld; everything else is text. ......... Create with /var @x = … ; reference with @x in templates/commands. .... All commands use braces; language specifiers; /exe shortcuts. .......... Only /show, /run, /output emit output. .. Backticks/double-colon/triple-colon, quotes, and command contexts. ............ Dot + index access for objects/arrays; array slicing [start:end]; .data/.json parse JSON strings. ... Reusable templates and functions via /exe. ............ Angle brackets load file contents; globs; AST selectors with wildcards/type-filters/name-lists; metadata. ................. Modules, paths, registries; "as namespace." .......... When forms (bare, first, simple) – full detail in CONTROL_FLOW. ..... foreach and /for – full detail in CONTROL_FLOW. .............. Logical, comparison, ternary operators for expressions. ....... State writes via state:// protocol; hydration via dynamic modules. ...................... Types, primitives, command results, code execution. ...................... Backticks, ::…::, :::…:::; interpolation rules. ................... Angle brackets, AST selectors (wildcards/types/names), metadata, glob behavior. ............. Glob patterns, array access, "as" templates, common patterns. ................ Array and string methods: includes, indexOf, join, split, etc. ...................... Operator |, transformers, @ctx/@p context, retry/hints, inline effects, parallel groups with ||. ....................... ">>" and "<<" comments (start/end-of-line). ............. @now, @input, @base, @debug. .................. Decision tree, security & syntax differences, param passing. ................ Defining shell/js/node/template/section executables; shadow env intro. ............... Write to files/stdout/stderr with optional formats. .................. Syntactic sugar for /output to stdout; works in action contexts. ............... Append JSONL/plain records via /append or `| append` builtin. ...................... Enable live chunks with stream keyword or /stream; suppress via flags. ................. Bare/first/simple forms, /exe...when patterns, wildcard (*), none keyword. ...................... foreach vs /for; nested loops; objects with _key; collection form. .................. Design flows instead of exits; patterns. .......................... Module-first philosophy; imports; helpers; shadow environments; isolation. ......................... Tool orchestration; data pipelines; conditional workflows; guarded/step-by-step. .......... Allow-list, @input import. .................... Document metadata block. ................. Quoted/unquoted paths, dynamic paths, URL loading. ...................... Prefixes, resolution priority, import type modifiers, custom config. ....................... Publishing workflow, version management, required frontmatter, commands. ..... Auth, repo layout, commands. ............ document/structured/stream/debug modes for SDK consumption. ....... Runtime context injection without filesystem I/O. .................. File-based execution with state management and caching. ................. Static analysis for tool discovery and validation. ............. Create variables with @; directives use /. ........... @run is invalid; use /run on LHS and run / implicit RHS. ..... Use @var not ${var}; mlld ≠ JS/shell templates. .......... Interpolation only in directive/template contexts. ...... mlld ≠ template engine; use /show or /exe. ......... Angle brackets load content; quotes hold literal filenames. ....... is plain text; is a file ref (has ./*/@). ...... Don't shell-call @func inside commands; assign then use. .......... Prefer templates over procedural concatenation. .......... Move algorithms into helpers/modules. ........ Know when to use angle brackets for path resolvers. .......... Model flows via /when rather than exits. ........................ Execution context table, syntax summary, quick transforms list. ......................... Guards, data labels, expression tracking, @ctx.guard context. ........................ Live chunks with stream keyword or /stream; suppress via flags. ......................... Documentation, examples repository, source code. mlld is a modular prompt scripting language for dynamically assembling context and orchestrating LLMs—think Make + npm for the LLM era, or a Unix pipe for chaining discrete AI/tool steps. It's designed to be written directly in Markdown: readable as docs, executable as workflows. Two modes: 1. Markdown mode (default): any line not starting with `/` 2. mlld mode: lines starting with `/` (directives) What mlld IS: * 📋 A workflow orchestrator (like Make + npm for the AI era) * 🎭 A conductor's baton (you conduct, tools play) * 📝 Executable documentation (reads like a guide, runs like a script) * 🚦 A logical router (route data and actions based on conditions) What mlld ISN'T: * ❌ A template engine (not Jinja/Handlebars) * ❌ A shell script replacement (it orchestrates shells; doesn't replace them) Mental model shift: * ❌ "How do I implement this?" → ✅ "What tool/module handles this?" * ❌ "I need an if-statement" → ✅ "I need a decision point" * ❌ "Let me concatenate strings" → ✅ "Let me create a template" Think Docker Compose or GitHub Actions: declare what happens, don't program how. Only directives on lines beginning with `/` are executed. ```mlld ❌ Hello @name! Let me /show something. ✅ /var @greeting = `Hello @name!` /show @greeting ``` Create with `/var @name = "Alice"`. Reference with `@name` in templates/commands. ```mlld /var @name = "Alice" /show `Hello @name!` /run cmd {echo "User: @name"} /run js (@name) { console.log("Hi", name) } ``` Every command needs braces. Use language specifiers: `cmd {}` for simple commands, `sh {}` for shell scripts, `js {}` for code. ```mlld ❌ /run cmd echo "hello" ✅ /run cmd {echo "hello"} # Language specifiers /run cmd {echo "hello"} # simple commands (pipes only) /run sh {npm test && build} # shell scripts (&&, ||, multi-line) /run js {console.log("hi")} # JavaScript code # Executables /exe @hi() = cmd {echo hi} /exe @script() = sh { echo "multi"; ls -la } /exe @calc(x) = js { return @x * 2 } ``` Only these produce output: `/show`, `/run`, `/output`. Everything else mutates state. ```mlld /var @secret = "hidden" # no output /show `Visible` /run cmd {echo "Also visible"} ``` Default to `::...::` for inline templates, `.att` files for external templates (5+ lines). Switch to `:::...:::` or `.mtt` ONLY for Discord `<@userid>` mentions or heavy social media `@handle` usage. **Quick Reference:** | Syntax | Interpolation | Pipes | Loops | Use For | |--------|---------------|-------|-------|---------| | `::...::` | `@var` `` `@exe()` | ✓ | ✓ | **Default inline** | | `.att` | `@var` `` `@exe()` | ✓ | ✓ | **Default external (5+ lines)** | | `` `...` `` | `@var` `` `@exe()` | ✓ | ✓ | Same as `::...::` (preference) | | `"..."` | `@var` `` `@exe()` | ✓ | ✗ | Single-line only | | `:::...:::` | `{{var}}` only | ✗ | ✗ | Discord/social escape hatch | | `.mtt` | `{{var}}` only | ✗ | ✗ | Discord/social external | | `'...'` | **None (literal)** | ✗ | ✗ | **Literal text** | | `{...}` | `@var` `` | ✗ | ✗ | Commands/code | Key: Backticks, `::...::`, and `"..."` all use `@var` interpolation. Only `'...'` (single quotes) keeps text literal without interpolation. **Inline templates:** ```mlld >> Double-colon (default) /var @msg = ::Hello @name!:: /var @doc = ::Use `npm test` before @env:: /var @report = :: Status: @status Config: <@base/config.json> Data: @data|@json :: >> Backticks (alternative) /var @msg = `Hello @name!` /var @multi = ` Line 1: @var Line 2: @other ` >> Double quotes (single-line only) /var @path = "@base/files/@filename" /run cmd {echo "Processing @file"} >> Triple-colon (Discord/social only) /var @alert = :::Alert <@{{adminId}}>! Issue from <@{{userId}}>::: /var @tweet = :::Hey @{{user}}, check this! cc: @{{team1}} @{{team2}}::: >> Single quotes (literal) /var @literal = '@name stays literal' ``` **External templates:** ```mlld >> .att files (default for 5+ lines) >> file: templates/deploy.att # Deployment: @env Status: @status Config: <@base/config/@env.json> >> usage /exe @deploy(env, status) = template "templates/deploy.att" /show @deploy("prod", "success") >> .mtt files (Discord/social only) >> file: templates/discord.mtt 🚨 Alert <@{{adminId}}>! Reporter: <@{{reporterId}}> Severity: {{severity}} >> usage /exe @alert(adminId, reporterId, severity) = template "templates/discord.mtt" ``` **Loops (::, backticks, .att only):** ```mlld /var @list = :: /for @item in @items - @item.name: @item.value /end :: >> Requirements: /for and /end at line start >> NOT supported in :::...:::, .mtt, or "..." ``` **Trade-offs when using Discord/social escape hatch:** | Feature | `::...::` / `.att` | `:::...:::` / `.mtt` | |---------|-------------------|----------------------| | `@var` interpolation | ✓ | ✗ Use `{{var}}` | | `` loading | ✓ | ✗ | | `@exe()` calls | ✓ | ✗ | | Pipes `\|` | ✓ | ✗ | | Loops | ✓ | ✗ | | Discord `<@id>` | Escape `\<@id\>` | ✓ Natural | | Many `@handles` | Works | ✓ Cleaner | **Common mistakes:** ```mlld >> ✗ Using {{}} in ::...:: /var @msg = ::Hello {{name}}:: >> {{name}} is literal /var @msg = ::Hello @name:: >> ✓ >> ✗ Using @var in :::...::: /var @msg = :::Hello @name::: >> @name is literal /var @msg = :::Hello {{name}}::: >> ✓ >> ✗ Using ::: without Discord/social need /var @msg = :::Status: {{status}}::: >> Loses all features /var @msg = ::Status: @status:: >> ✓ Full features >> ✗ Importing template files /import { @tpl } from "./file.att" >> Error /exe @tpl(x) = template "./file.att" >> ✓ ``` Objects/arrays use dot+index in directives and `{{ }}` in ::: templates only. Arrays support slicing with `[start:end]` and builtin methods. ```mlld /var @user = {"name":"Alice","scores":[10,20,30]} # Directives: /show @user.name # Alice /show @user.scores.1 # 20 # Array slicing: /var @arr = [1,2,3,4,5] /show @arr[0:3] # [1,2,3] /show @arr[-2:] # [4,5] - last 2 /show @arr[:-1] # [1,2,3,4] - all except last /show @arr[2:] # [3,4,5] - from index 2 >> JSON string accessors: /var @jsonStr = '[{"name":"Alice"},{"name":"Bob"}]' /exe @process(items) = js { return items.filter(u => u.name === "Alice") } /var @result = @process(@jsonStr.data) << .data/.json parse JSON /var @length = @jsonStr.text.length << .text/.content preserve strings >> Builtin methods on arrays/strings: /var @list = ["apple", "banana", "cherry"] /var @text = "Hello World" /show @list.includes("banana") << true /show @list.indexOf("cherry") << 2 /show @list.join(", ") << "apple, banana, cherry" /show @text.toLowerCase() << "hello world" /show @text.split(" ") << ["Hello", "World"] /show @text.includes("World") << true >> Double-colon templates: /show ::@user.name has @user.scores.0:: ``` Use `/exe` for parameterized templates or commands. ```mlld /exe @greet(name) = cmd {echo "Hello @name"} /run @greet("Bob") /exe @welcome(name, role) = ::Welcome @name, our new @role!:: /show @welcome("Rae","developer") ``` Angle brackets load file contents (not filenames). Works in variables, templates, commands, and with globs. Only `<…>` containing `.`, `/`, `*`, or `@` are treated as file references—so pseudo-XML tags like `` are safe, plain text. ```mlld /var @content = # file contents /show # prints file contents /var @filesTxt = ["a.md","b.md"] # literal names /var @contents = [, ] # loaded contents # Field access on loaded JSON /var @authorEmail = .author.email # Globs + "as" templates /var @toc = as "- [<>.ctx.fm.title](<>.ctx.relative)" ``` Data flows as native types: ```mlld /var @cfg = # object, not JSON string /var @users = @cfg.users # array access works directly /var @text = @cfg.text # stringified JSON when needed /var @parsed = @text.data # parse JSON string to object ``` Inline transforms (pipes): ```mlld /var @pretty = | @json /var @summary = `User data: @data|@json` ``` Security: By default, restricts to project root. Use `--allow-absolute` flag to permit absolute paths outside project. Module imports and path imports. Import type keywords control resolution behavior. ```mlld # Modules (no quotes) /import { @parallel, @retry } from @mlld/core /import @corp/utils as @corp # Paths (quote & resolver via <> when needed) /import { @readme } from "@base/README.md" /import { @helper } from "./utils.mld" # Import types (optional, auto-inferred when omitted) /import module { @api } from @corp/tools # pre-installed registry module /import static { @config } from "./config.json" # embedded at parse time /import live as @status # fresh every execution /import cached(30m) as @feed # cached with TTL (5m, 1h, 7d) /import local { @helper } from @me/dev-module # llm/modules/ during dev ``` - Prefix identifiers in the braces with `@` so imported names match runtime variable syntax (e.g., `{ @helper }`). - Aliases after `as` must include `@` (e.g., `as @utils`), and wildcard imports require `* as @alias`. - Import types: `module` (offline after install), `static` (zero runtime cost), `live` (always fresh), `cached(TTL)` (smart caching), `local` (dev modules). - When omitted, mlld infers the safest option. /when drives decisions (bare, first, simple). Full details in . ```mlld /when @isProd => show "Prod" /when first [ @role=="admin" => show "Admin" * => show "Guest" ] ``` Use `foreach` to transform collections; use `/for` to execute/collect. Batch pipelines (`=> |`) run after iteration completes. Full details in . ```mlld /var @names = ["Alice","Bob"] /exe @greet(name) = ::Hi @name!:: /var @gs = foreach @greet(@names) /for @n in @names => show `Name: @n` # Batch pipelines - process collected results /var @numbers = for @n in [1,2,3,4] => @n => | @sum /var @pairs = foreach @duplicate(@data) => | @flatten # Optional parallel forms /for parallel @n in @names => show @n # uses MLLD_PARALLEL_LIMIT (default 4) /var @caps = for parallel(2) @n in @names => @greet(@n) # per-loop cap override /for parallel(3, 1s) @n in @names => show @n # cap 3 with 1s pacing between starts ``` Logical, comparison, and ternary operators for expressions and conditions. Comparison: `<`, `>`, `<=`, `>=`, `==`, `!=` Logical: `&&`, `||`, `!` Ternary: `condition ? trueVal : falseVal` ```mlld # Expressions /var @isValid = @score > 80 && @submitted /var @status = @isPro ? "premium" : "basic" /var @canEdit = @isOwner || (@role == "editor" && !@isLocked) /var @opposite = !@condition # Parentheses for precedence /var @complex = (@a || @b) && (@c != @d) # In /when conditions /when @tokens > 1000 && @mode == "production" => show "High usage" /when (@role == "admin" || @role == "mod") && @active => show "Privileged" ``` Type coercion: `"true" == true`, `"false" == false`, `null == undefined` Precedence: `!` → comparison → `&&` → `||` → `?:` State updates via state:// protocol instead of filesystem writes. ```mlld /output @value to "state://path" # Captured as StateWrite, not file ``` State hydration via dynamic modules: ```mlld /var @count = @state.count + 1 /output @count to "state://count" ``` Runtime provides state: ```typescript executeRoute(file, payload, { state: { count: 0 } }); // Returns: { stateWrites: [{ path: 'count', value: 1, ... }] } ``` Application handles persistence. SDK captures writes without filesystem I/O. Create primitives, arrays, objects, or assign from command/code results. Primitives preserve JS types in `/exe js`. ```mlld /var @n = 42 /var @price = 19.99 /var @ok = true /var @arr = [1,2,3] /var @obj = {"key":"value"} /var @merged = { ...@obj, "extra": 1 } # object spread; later entries override earlier /exe @add(a,b) = js { return a + b } /var @sum = @add(@n, 8) # 50 (number) /var @date = cmd {date} # result of command /var @readme = # file contents ``` Object spread merges objects left-to-right; each spread target must be an object or it throws. Prefer backticks; use :: for backticks-in-text; ::: when many @mentions. ```mlld /var @message = `Hello @name, welcome!` /var @doc = ::Use `mlld` to orchestrate:: /var @tweet = :::Hey @{{handle}}—{{msg}}::: When-expressions are first-class: ```mlld # Direct assignment /var @status = when [ @score > 90 => "A" * => "F" ] # In array literals /var @arr = [ 1, when [ @flag => 2 ], 3 ] # As function arguments /exe @fmt(x) = `val:@x` /show @fmt(when [ @cond => "yes" * => "no" ]) ``` ``` Inline template control: - `/for … /end` is available in backticks and `::…::` templates. - `:::…:::` templates do not support loops; they support `{{var}}` interpolation and inline `/show` only. Examples: ```mlld # Backtick wrapper /var @tpl = ` /for @x in ["A","B"] - @x /end ` /show @tpl # Double-colon wrapper /var @items = ["A","B"] /var @msg = :: /for @x in @items - @x /end :: /show @msg ``` Notes: - Line-start only: `/for` and `/end` must begin at a line start inside the template body. - No loops in `[[...]]` or `:::...:::` templates. Angle brackets load, quotes store literal paths. Field access and globs are supported. ```mlld /var @author = .author /var @firstEmail = .users[0].email /var @md = as "# <>.ctx.filename" ``` AST selection: add `{ ... }` inside the angle brackets to pull specific definitions or usages. ```mlld # Exact names /var @user = /var @callers = # Wildcards /var @handlers = # prefix: handleRequest, handleError /var @validators = # suffix: emailValidator, phoneValidator /var @requests = # contains: getUserRequest, createUserRequest # Type filters /var @funcs = # all functions/methods /var @vars = # all variables/constants /var @classes = # all classes /var @everything = # all top-level definitions # Name listing (returns string arrays, not code) /var @names = # all definition names /var @funcNames = # function names only /var @classNames = # class names only # Section listing (markdown) /var @headings = # all heading titles /var @h2s = # H2 headings only # Variable interpolation /var @type = "fn" /var @defs = # dynamic type filter /var @nameList = # dynamic name listing # Usage patterns (find functions that USE matched symbols) /var @callers = # functions using validateEmail /var @serviceUsers = # functions using any *Service ``` Glob + name-list behavior: - Single file: `` → plain array `["name1", "name2"]` - Glob pattern: `<**/*.ts { ?? }>` → per-file objects `[{ names: [...], file, relative, absolute }]` - Example: `/for @f in <**/*.py { class?? }> => show "@f.names.length classes in @f.relative"` Supported: `.js`, `.ts`, `.jsx`, `.tsx`, `.mjs`, `.py`, `.pyi`, `.rb`, `.go`, `.rs`, `.sol`, `.java`, `.cs`, `.c`, `.cpp`, `.h`, `.hpp`. Type keywords: `fn`, `var`, `class`, `interface`, `type`, `enum`, `struct`, `trait`, `module`. Missing patterns return `null` preserving request order. Cannot mix content selectors with name-lists. Detection rule: only `<…>` with `.`, `/`, `*`, or `@` are treated as file refs. XML-like `` is plain text. Metadata lives under `.ctx` (e.g., `@file.ctx.filename`); shorthand like `@file.filename` still maps to the same fields for backward compatibility. File metadata fields: `.ctx.content` (default), `.ctx.filename`, `.ctx.relative`, `.ctx.absolute`, `.ctx.tokens`, `.ctx.tokest`, `.ctx.fm` (frontmatter), `.ctx.json` (for .json files) URL metadata fields: All file fields plus `.ctx.url`, `.ctx.domain`, `.ctx.title`, `.ctx.description`, `.ctx.html`, `.ctx.text`, `.ctx.md`, `.ctx.headers`, `.ctx.status`, `.ctx.contentType` Glob patterns load multiple files at once. Use `` to match recursively and access results as an array. ```mlld # Load all markdown files /var @docs = /show @docs.length # number of files loaded /show @docs[0].ctx.filename # first file's name /show @docs[0].content # first file's content # Generate table of contents /var @toc = as "- [<>.ctx.fm.title](<>.ctx.relative)" # Find specific test directories /var @tests = # all example.md files /var @dirs = @tests.map(t => t.ctx.relative.replace("/example.md", "")) # Field access on each file /for @doc in @docs => show @doc.ctx.fm.title ``` Glob metadata (each matched file has): - All standard file fields: `content`, `filename`, `relative`, `absolute` - `file` property traces source path - Frontmatter via `fm` property - Use `as "template"` to transform each match with `<>` placeholder Common patterns: ```mlld # All TypeScript files /var @sources = # All JSON configs /var @configs = <**/*.json> # Specific pattern /var @handlers = ``` Variables support builtin methods for common operations. Methods are called with parentheses and can accept arguments. **Array Methods:** - `@array.includes(value)` - Returns true if array contains value - `@array.indexOf(value)` - Returns index of value, or -1 if not found - `@array.length()` - Returns array length - `@array.join(separator)` - Joins array elements into string **String Methods:** - `@string.includes(substring)` - Returns true if string contains substring - `@string.indexOf(substring)` - Returns index of substring, or -1 - `@string.length()` - Returns string length - `@string.toLowerCase()` - Converts to lowercase - `@string.toUpperCase()` - Converts to uppercase - `@string.trim()` - Removes leading/trailing whitespace - `@string.startsWith(prefix)` - Returns true if starts with prefix - `@string.endsWith(suffix)` - Returns true if ends with suffix - `@string.split(separator)` - Splits into array ```mlld /var @fruits = ["apple", "banana", "cherry"] /var @message = "Hello World" # Array operations /show @fruits.includes("banana") # true /show @fruits.indexOf("cherry") # 2 /show @fruits.join(" and ") # "apple and banana and cherry" # String operations /show @message.toLowerCase() # "hello world" /show @message.split(" ") # ["Hello", "World"] /show @message.includes("World") # true # With variable arguments /var @search = "banana" /show @fruits.includes(@search) # true ``` Note: Methods are parsed as field access exec patterns (`@obj.method(args)`) and executed as ExecInvocations internally. Chain stages with `|`. Built-ins: @json, @xml, @csv, @md (upper/lower accepted). All `/exe` functions work in pipelines (native mlld, JS, shell). Stages access context via `@ctx` and pipeline history via `@p`. JSON parsing modes: `@json` (loose by default), `@json.strict` (enforce strict JSON), `@json.loose` (explicit relaxed parsing with single quotes, trailing commas, comments), `@json.llm` (extract JSON from LLM responses with code fences/prose, returns false if none found). ```mlld /var @users = cmd {cat users.json} | @json | @csv /exe @double(n) = js {(@n*2)} /var @x = cmd {echo "5"} | @double # JSON parsing variants /var @relaxed = @input | @json.loose # JSON5: single quotes, trailing commas, comments /var @strict = @input | @json.strict # strict JSON only /var @auto = @input | @json # loose mode with helpful errors /var @extracted = @llmResponse | @json.llm # extract from code fences/prose, false if none # Native mlld functions in pipelines /exe @filterHigh(items) = for @item in @items => when [ @item.priority == "high" => @item none => skip ] /var @data = '[{"priority":"high"},{"priority":"low"}]' /var @filtered = @data.data | @filterHigh | @json # Retry with hints /exe @validator(input) = when [ @input.valid => @input @ctx.try < 3 => retry "need more detail" * => "fallback" ] /var @res = @raw | @validator # Best-of-N pattern /exe @selectBest(response) = when [ @response.score > 8 => @response @ctx.try < 5 => retry { temperature: 0.9 } * => @selectHighestScore(@p.retries.all) ] ``` Context object (@ctx): `try` (attempt #), `tries` (array), `stage`, `input`, `hint` (from retry), `lastOutput`, `isPipeline` Pipeline array (@p): `@p[0]` (input), `@p[-1]` (previous output), `@p.retries.all` (full history) Inline effects: `@data | @transform | log` (stderr), `@data | output to "file.txt"` (inline write) Inline effects run through guard hooks; use `op:output`/`op:show`/`op:append`/`op:log` filters to cover directives and inline forms. Formats (`with { format: "json|csv|xml|text" }`): guide parsing for pipeline stages. Parallel groups (`||`): run multiple commands as one stage in parallel. ```mlld # Two transforms run concurrently; outputs are collected in order /exe @left(input) = `L:@input` /exe @right(input) = `R:@input` /exe @combine(input) = js { const arr = JSON.parse(input); // parallel stage returns a JSON array string return arr.join(' + '); } /var @out = "x" with { pipeline: [ @left || @right, @combine ] } /show @out # → "L:x + R:x" # Leading || runs parallel stages immediately /var @results = || @fetchA() || @fetchB() || @fetchC() /run || @fetchA() || @fetchB() || @fetchC() # also works in /run /exe @parallel() = || @helper1() || @helper2() | @combine # and /exe # With concurrency caps: (n, delay) /var @capped = || @a() || @b() || @c() || @d() (2, 100ms) # cap=2, 100ms pacing ``` Data flows as native types: - Loaders return parsed data: `` yields object, not JSON string - Pipeline stages (including batch `=> |`) receive native types: `@data | @process` passes arrays/objects - JavaScript functions get parsed values (no `JSON.parse()` needed) - Use `.text` for stringified form, `.data` for structured access Semantics: * Grouping: `A || B || C` forms a single stage executed concurrently with a global cap. * Leading `||`: Start pipeline with parallel execution. Syntax: `|| @a() || @b() || @c()` runs all three immediately. * Equivalence: `|| @a() || @b()` produces same AST as `"" with { pipeline: [[@a, @b]] }` * Ordering: Results preserve command order; next stage receives native array/object. * Concurrency: Limited by `MLLD_PARALLEL_LIMIT` (default `4`). * Retries: Returning `retry` inside a parallel group is not supported and aborts the pipeline. If you need retries, run validation after the group and request a retry of the previous (non-parallel) stage. * Rate limits: 429/"rate limit" errors trigger per-command exponential backoff. * Inline effects: Effects attached to commands in the group run after each command succeeds. Use `>>` or `<<` for comments at start or end of lines. ```mlld >> start-of-line /show "Hello" >> end-of-line /var @x = 5 << end-of-line ``` * `@now` – current timestamp * `@input` – stdin/env (must be allowed) * `@base` – project root * `@debug` – environment info Decision tree: * Single line + pipes only (`|`) → `cmd { … }` (safe, recommended) * Needs `&&`, `||`, control flow, or multi-line → `sh { … }` (permissive) * JavaScript/Python code → `js { … }` / `python { … }` Language specifiers work consistently across `/run`, `/var`, and `/exe`: ```mlld # cmd (pipes only, safe) /run cmd {echo Hello | tr '[:lower:]' '[:upper:]'} /var @date = cmd {date} /exe @list(dir) = cmd {ls -la @dir | head -5} # sh (shell scripts with &&, ||, multi-line) /run sh { npm test && npm run build || echo "Build failed" } /var @output = sh {echo "test" && ls} /exe @deploy(env) = sh { npm test && npm run deploy:$env } # js/python (code execution) /run js {console.log("hello")} /var @result = js {return 42} /exe @calc(x) = js { return @x * 2 } # stdin support (all forms) /var @data = '[{"name":"Alice"},{"name":"Bob"}]' /run cmd { cat | jq '.[] | select(.name == "Alice")' } with { stdin: @data } >> OR pipe sugar: /run @data | { cat | jq '.[] | select(.name == "Alice")' } /exe @processJson(data) = run @data | { cat | jq '.[]' } ``` Parameter syntax: * `cmd`: interpolate with `@param` * `sh`: use shell variables as `$param` * `js/python`: parameters passed as variables Security: `cmd` forbids `&&`/`||` to reduce injection risk; use `sh` when necessary. Backward compatibility: `run {…}` still works (equivalent to `cmd {…}`). Large variables: For bash/sh executables, variables >128KB are automatically handled via heredocs (when `MLLD_BASH_HEREDOC=1`). Define reusable commands/code/templates/sections. ```mlld # Commands (simple vs shell scripts) /exe @list(dir) = cmd {ls -la @dir | head -5} /exe @script() = sh { echo "Start"; ls -la } # JavaScript /exe @add(a,b) = js { return a + b } # Templates /exe @greet(name) = `Hello @name!` # Section extractor /exe @intro(file) = <@file # Introduction> # With stdin (explicit with clause) /exe @processJson(data) = cmd { cat | jq '.[]' } with { stdin: @data } # With stdin (pipe sugar) /exe @filterActive(data) = run @data | { cat } | @transformFilter ``` Shadow environments (JS/Node): Expose JS helpers across `js {}` blocks in the same module: ```mlld /exe @double(n) = js { return n*2 } /exe @cap(s) = js { return s[0].toUpperCase()+s.slice(1) } /exe js = { double, cap } # expose /var @out = js { cap("hello") + ": " + double(5) } # "Hello: 10" ``` Variable scoping (security): JS/Node blocks only access parameters passed explicitly—no auto-injection of outer scope variables: ```mlld /var @data = [1, 2, 3] # ❌ Cannot access outer variables /var @result = node { return data.map(x => x * 2); # ReferenceError: data not defined } # ✅ Pass variables as parameters /exe @double(arr) = js { return arr.map(x => x * 2); } /var @result = @double(@data) # [2, 4, 6] # ✅ Multiple parameters /exe @process(data, multiplier) = js { return data.map(x => x * multiplier); } /var @result = @process(@data, 2) # [2, 4, 6] ``` This isolation prevents accidental variable leakage and makes dependencies explicit. Write data to files or streams, with optional format. ```mlld /output @content to "out.txt" /output @data to "config.json" /output @html to "page.html" as html /output @message to stdout /output @error to stderr /output @config to "settings.yaml" as yaml ``` Syntactic sugar for `/output to stdout`. Works in action contexts. ```mlld /log @message # same as /output @message to stdout /log `Processing: @item` # In action contexts /for @item in @items => log @item /when @debug => log "Debug info" /when [ @valid => log "Success" none => log "Failed validation" ] ``` Append newline-delimited records via `/append` or the pipeline builtin `| append`. ```mlld /append @record to "events.jsonl" # JSON object per line (validated) /append "raw line" to "events.log" /var @runs = ["alpha","beta"] /var @_ = for @name in @runs => append @name to "pipeline.log" /var @_ = "done" | append "completion.log" /var @_ = "ignored" | append @runs to "runs.jsonl" ``` `.jsonl` enforces JSON serialization. Any other extension writes text. `.json` extensions are blocked. Forms: * Simple: `/when @cond => action` * Bare (evaluate all matches): ```mlld /when [ @score > 90 => show "Excellent!" @isBonus => show "Bonus!" none => show "No matches" ] ``` * Switch (`first` stops at first match): ```mlld /when first [ @role=="admin" => show "Admin" @role=="user" => show "User" * => show "Guest" ] ``` `/exe...when` patterns (value-returning): ```mlld # Basic handler with fallback /exe @handler(input) = when [ @input.valid => @input.value @ctx.try < 3 => retry "validation failed" none => "default" ] # Switch-style with first /exe @parser(type, data) = when first [ @type == "json" => @parseJSON(@data) @type == "xml" => @parseXML(@data) @type == "csv" => @parseCSV(@data) * => @data ] # Conditional transformer /exe @process(item) = when [ @item.priority > 5 => @urgent(@item) @item.archived => null * => @standard(@item) ] ``` Local variables in when blocks: ```mlld /when @mode: [ let @prefix = "Status:" "active" => show "@prefix Active" * => show "@prefix Unknown" ] /exe @format(name) = when [ let @greeting = "Hello" * => "@greeting @name!" ] ``` `let` declares a variable scoped to the when block. It's evaluated before conditions are checked. Augmented assignment (`+=`) modifies local variables: ```mlld /exe @collect() = when [ let @items = [] @items += "a" @items += "b" * => @items # ["a", "b"] ] ``` `+=` semantics: * Arrays: concat (`[1, 2] += 3` → `[1, 2, 3]`, `[1] += [2, 3]` → `[1, 2, 3]`) * Strings: append (`"hello" += " world"` → `"hello world"`) * Objects: shallow merge (`{a: 1} += {b: 2}` → `{a: 1, b: 2}`) * Only works with local `let` bindings (not global variables) Notes: * RHS uses action words (no `/show` slash): `show`, `@func()` * Use `let @var = value` for local variables in when blocks (not `@var = value`) * Use `@var += value` to accumulate into local variables * `*` wildcard matches anything (catch-all) * `none` executes only if no conditions matched (must appear after all regular conditions, incompatible with `*`) * One action per line (repeat condition for multiple actions) * Parentheses & `&&`/`||` supported in conditions `foreach` returns a transformed array; `/for` executes an action per item or collects results. ```mlld # foreach (transform) /var @names = ["alice","bob","charlie"] /exe @greet(name) = ::Hi @name!:: /var @greetings = foreach @greet(@names) # /for (execute) /for @n in @names => show `Name: @n` # collection form /var @nums = [1,2,3] /var @doubled = for @x in @nums => js { return @x * 2 } # [2,4,6] # objects and keys /var @cfg = {"host":"localhost","port":3000} /for @v in @cfg => show `@v_key: @v` # nested for loops /for @x in ["A","B"] => for @y in [1,2] => show `@x-@y` # Output: A-1, A-2, B-1, B-2 # triple nesting /for @x in @outer => for @y in @middle => for @z in @inner => show `@x/@y/@z` ``` When-expressions in for RHS: ```mlld /var @xs = [1, null, 2, null, 3] # Keep only non-null values /var @filtered = for @x in @xs => when [ @x != null => @x none => skip ] # @filtered => ["1","2","3"] ``` Define foreach in /exe RHS and invoke: ```mlld /exe @wrap(x) = `[@x]` /exe @wrapAll(items) = foreach @wrap(@items) /show @wrapAll(["a","b"]) | @join(',') # => [a],[b] ``` Use /show foreach with options: ```mlld /var @names = ["Ann","Ben"] /exe @greet(n) = `Hello @n` /show foreach @greet(@names) with { separator: " | ", template: "{{index}}={{result}}" } # => 0=Hello Ann | 1=Hello Ben ``` Empty arrays are safe: `/for` emits nothing; `for … =>` returns `[]`. Batch pipelines - process collected results after iteration: ```mlld /var @numbers = for @n in [1,2,3,4] => @n => | @sum /var @pairs = foreach @duplicate(@data) => | @flatten /var @sorted = for @item in @items => @process(@item) => | @sortBy("priority") ``` Batch stage receives the collected array (or object) directly, so helpers work with native arrays/objects without manual parsing. Parallel /for (optional): ```mlld # Default cap from env (MLLD_PARALLEL_LIMIT, default 4) /for parallel @x in @items => show @x # Cap override and pacing between starts (units: s, m, h, d, w) /for parallel(2, 1s) @x in @items => show @x # Collection form preserves input order /var @out = for parallel(3) @x in ["a","b","c"] => @upper(@x) ``` Notes: - Directive form streams results as iterations finish (order not guaranteed); expression form returns results in input order. - Concurrency capped by `MLLD_PARALLEL_LIMIT` unless overridden per loop. - Pacing: optional minimum delay between iteration starts via `/for parallel(n, interval) ...` (units: ms, s, m, h). - Rate limits: per‑iteration 429/"rate limit" errors use exponential backoff. mlld has no `return/exit`. Model outcomes with `/when` and flags. ```mlld /var @check = @validate(@input) /when [ @check.valid => @process(@input) !@check.valid => show `Error: @check.message` ] ``` Module-first philosophy: keep `.mld` files readable; move complexity into focused modules that expose capabilities (like microservices). Avoid "kitchen sink" modules or side effects on import. **Core patterns:** * `/export { @func }` required (auto-export is legacy) * Import types: `module` (registry), `static` (embedded), `live` (fresh), `cached(TTL)`, `local` (dev) * Namespace imports avoid collisions: `as @alias` * Executables preserve module scope (sibling variables always accessible) **Creating modules:** ```mlld --- name: text-utils author: alice version: 1.0.0 about: String helpers needs: [js] license: CC0 --- /exe @upper(s) = js { return s.toUpperCase() } /exe @trim(s) = js { return s.trim() } /export { @upper, @trim } ``` **Import patterns:** ```mlld # Registry (offline after install) /import { @helper } from @alice/utils /import { @helper } from @alice/utils@1.0.0 # version /import { @helper } from @alice/utils@beta # tag # Local files /import { @config } from "./config.mld" /import { @config } from <@base/config.mld> # Namespace (avoid collisions) /import @alice/utils as @alice /import @bob/utils as @bob # Import types (control caching) /import module { @api } from @corp/tools # cached /import static { @prompt } from "./prompt.md" # embedded /import live as @status # always fresh /import cached(1h) as @feed # TTL cache /import local { @dev } from @alice/experimental # llm/modules/ ``` **Local development:** ```bash mlld install @alice/utils # install from registry mlld update @alice/utils # update to latest mlld ls # list installed ``` Dev modules in `llm/modules/` (flat structure, matched by frontmatter): ``` llm/modules/ ├── my-utils.mld.md # author: alice, name: experimental └── helpers.mld # author: bob, name: tools ``` ```mlld /import local { @helper } from @alice/experimental # finds my-utils.mld.md ``` **Lock files (mlld-lock.json):** * Auto-generated, tracks versions/hashes * Commit to version control * Only registry modules validated Tool orchestration: ```mlld /var @areas = [ {"name":"auth","files":["auth/*.ts"],"tests":["test/auth/*"]}, {"name":"api","files":["api/*.ts"],"tests":["test/api/*"]} ] /exe @runQA(area) = cmd {echo "Testing @area.name" | cat} /var @results = foreach @runQA(@areas) ``` Data pipeline: ```mlld /import { @fetchData, @validate, @transform } from @data/pipeline /var @raw = @fetchData("https://api.example.com/users") /var @valid = @validate(@raw, { schema: "user" }) /var @report = @transform(@valid, { format: "report" }) /show `Processed @report.count users` ``` Conditional workflows: ```mlld /import { @getPR, @commentOnPR } from @company/github /var @pr = @getPR(@MLLD_PR_NUMBER) /when first [ @pr.mergeable => @status = "ready" * => @status = "blocked" ] /when [ @status=="ready" => @commentOnPR(@MLLD_PR_NUMBER,"Ready to merge") @status=="blocked" => show "Needs attention" ] ``` Step-by-step & guarded execution: ```mlld /var @processed = @data | @validate | @normalize | @analyze /when [ @processed.ok => @emitReport(@processed) !@processed.ok => show "❌ Validation failed" ] ``` Guards work on expressions and transformations—labels are preserved through helper chains, templates, iterators, and pipelines. mlld uses dual configuration: - `mlld-config.json`: Your project settings (dependencies, preferences, resolver config) - `mlld-lock.json`: Auto-generated locks (versions, hashes, sources) Edit `mlld-config.json` manually; `mlld-lock.json` updates automatically. Allow environment variables in `mlld-lock.json` (prefix `MLLD_`), then import via `@input`. ```json { "security": { "allowedEnv": ["MLLD_NODE_ENV","MLLD_API_KEY","MLLD_GITHUB_TOKEN"] } } ``` ```mlld /import { @MLLD_NODE_ENV, @MLLD_API_KEY } from @input /show `Running in @MLLD_NODE_ENV` ``` Use frontmatter to describe the pipeline: ```yaml --- description: Pipeline description version: 1.0.0 author: Your Name --- ``` Paths may be literal (single quotes), interpolated (double), or bare resolver names; URLs are supported as sources. ```mlld /var @dir = "./docs" /var @userFile = "data/@username/profile.json" /var @template = 'templates/@var.html' # literal '@' /show /var @remote = ``` Prefixes map @ references to content sources. Built-in types: LOCAL, HTTP, GITHUB, REGISTRY. **Built-in resolvers (no config needed):** * `@author/module` → RegistryResolver (mlld-lang/registry) * `@base/file` → ProjectPathResolver (project root) * Local paths `./file.mld` → LocalResolver (fuzzy extension matching: .mld, .mld.md, .md) **Quick setup:** ```bash mlld alias --name notes --path ~/notes # local prefix mlld alias --name shared --path ../shared --global # global prefix mlld setup --github # private repo wizard ``` **Custom prefixes (mlld-config.json):** ```json { "resolvers": { "prefixes": [ { "prefix": "@lib/", "resolver": "LOCAL", "config": { "basePath": "./src/lib" } }, { "prefix": "@company/", "resolver": "GITHUB", "config": { "repository": "company/private-modules", "branch": "main", "basePath": "modules" } }, { "prefix": "@cdn/", "resolver": "HTTP", "config": { "baseUrl": "https://cdn.example.com", "cacheTTL": "1h" } } ] } } ``` **Resolution priority:** 1. Longest prefix match (`@company/auth/` beats `@company/`) 2. Resolver priority values: ProjectPath (10) → Registry (15) → Local/HTTP/GitHub (20) 3. Import types modify caching behavior, NOT which resolver is selected **Import type modifiers:** * Control caching/timing after resolver selects the source * Resolver chosen by prefix/pattern; import type adjusts caching semantics * `module` → content-addressed cache * `static` → embedded in AST * `live` → always fresh * `cached(TTL)` → time-based cache * `local` → dev mode (llm/modules/) Public registry at github.com/mlld-lang/registry. First module requires PR review; updates publish directly. **Publishing workflow:** ```bash # First time (creates PR) mlld publish my-tool.mld.md # Updates (direct publish after first merge) mlld publish my-tool.mld.md # prompts for version bump # Force PR workflow mlld publish --pr my-tool.mld.md ``` **Required frontmatter:** ```yaml --- name: my-tool author: alice # must match GitHub username version: 1.0.0 about: Brief description needs: [js, sh] # runtime dependencies needs-js: lodash # package deps (legacy format) needs-sh: git, curl # command deps (legacy format) license: CC0 # required --- ``` **Version management:** ```mlld /import { @helper } from @alice/utils # latest /import { @helper } from @alice/utils@1.0.0 # exact version /import { @helper } from @alice/utils@^1.0.0 # semver range /import { @helper } from @alice/utils@beta # tag ``` **Commands:** ```bash mlld install @alice/utils # install module mlld install @alice/utils@1.0.0 # specific version mlld update @alice/utils # update to latest mlld outdated # check for updates mlld ls # list installed # Publishing mlld publish my-tool.mld.md # publish/update mlld publish --tag beta my-tool.mld.md # publish with tag mlld publish --dry-run my-tool.mld.md # validate without publishing # Auto-detect dependencies mlld add-needs my-tool.mld.md ``` **Module source:** * GitHub repo → references commit SHA (immutable) * Gist → automatic creation if not in repo * Version locked in mlld-lock.json Authenticate and publish to your private repo. ```bash mlld auth login mlld auth status mlld publish my-module.mld.md --private --path lib/modules --pr ``` Four modes control how SDK consumers receive output: **document** (default): Returns string ```typescript const output = await processMlld(script); ``` **structured**: Returns `{ output, effects, exports, environment, stateWrites, metrics }` ```typescript const result = await interpret(script, { mode: 'structured' }); console.log(result.effects); // All effects with security metadata console.log(result.stateWrites); // State updates ``` **stream**: Returns `StreamExecution` handle with real-time events ```typescript const handle = interpret(script, { mode: 'stream' }); handle.on('stream:chunk', event => process.stdout.write(event.text)); handle.on('effect', event => console.log(event.effect)); await handle.done(); ``` **debug**: Returns `{ ast, variables, trace, durationMs, ...structured }` ```typescript const result = await interpret(script, { mode: 'debug' }); console.log(result.trace); // All events with timestamps ``` Events: `stream:chunk`, `stream:progress`, `command:start`, `command:complete`, `effect`, `execution:complete` Runtime context injection without filesystem I/O: ```typescript processMlld(template, { dynamicModules: { '@state': { count: 0, messages: [...] }, '@payload': { text: 'user input', userId: '123' } } }); ``` In script: ```mlld /var @count = @state.count + 1 /var @input = @payload.text ``` Security: All dynamic imports labeled `src:dynamic`, marked untrusted. Guards can check: ```mlld /guard before secret = when [ @input.ctx.taint.includes('src:dynamic') => deny "Cannot use dynamic data as secrets" * => allow ] ``` DynamicModuleResolver has priority 1 (checked before filesystem/registry). File-based execution with state management: ```typescript const result = await executeRoute('./agent.mld', payload, { state: { conversationId: '123', messages: [...] }, timeout: 30000 }); // Apply state updates for (const write of result.stateWrites) { await updateState(write.path, write.value); } ``` Features: - In-memory AST caching (mtime-based invalidation) - State hydration via `@state` module - Payload injection via `@payload` - State writes via `state://` protocol - Full effects and metrics - Timeout and cancellation Static analysis without execution: ```typescript const analysis = await analyzeModule('./tools.mld'); if (!analysis.valid) { console.error('Errors:', analysis.errors); } // Discover exports const tools = analysis.executables .filter(e => analysis.exports.includes(e.name)); // Check security const networkFns = analysis.executables .filter(e => e.labels.some(l => l.startsWith('net:'))); // Get capabilities console.log(analysis.needs); // Required console.log(analysis.wants); // Tiered preferences ``` Use cases: MCP proxy tool discovery, module registry validation, IDE/LSP features, security auditing. ```mlld ❌ /var greeting = "Hello" ✅ /var @greeting = "Hello" ``` ```mlld ❌ /var @result = @run {echo "hello"} ✅ /var @result = cmd {echo "hello"} ``` ```mlld ❌ /var @msg = "Hello ${name}" # JS/shell syntax doesn't work ❌ /show `Result: ${count}` ✅ /var @msg = "Hello @name" # Use @ not ${} ✅ /show `Result: @count` ``` mlld uses `@var` not `${var}`. Don't mix JavaScript/shell template syntax. ```mlld ❌ Hello @name! # plain Markdown, no interp ✅ /show "Hello @name!" ``` ```mlld ❌ This is {{name}}'s doc ✅ /show ::This is @name's doc:: ``` ```mlld /var @content = # loads contents /var @path = "README.md" # literal string ``` `` is plain text; `` (has `.`) is a file ref. ```mlld ❌ # inside shell trying to call @func /run cmd { RESULT=$(@helper("x")) # won't work echo $RESULT } ✅ /var @r = @helper("x") /run @r | { cat } ``` Prefer templates over manual concatenation. Move heavy logic to `/exe` helpers or modules; keep `.mld` orchestration simple. Know when to use angle brackets for resolver paths; modules don’t use them. Model alternative outcomes with `/when` and flags rather than aborting. Execution Context – where interpolation applies: | Context | Syntax | Example | Notes | | ---------------------- | --------- | ------------------------ | ------------------ | | Backticks | `@var` | `` `Hello @name` `` | Primary template | | Double-colon `::…::` | `@var` | `::Use \`cmd\` @name::\` | Escape backticks | | Triple-colon `:::…:::` | `{{var}}` | `:::Hi @{{handle}}:::` | Escape lots of `@` | | Commands `{…}` | `@var` | `{echo "@msg"}` | Interpolates | | Double quotes | `@var` | `"Hi @name"` | Interpolates | | Single quotes | '@var' | `'Hi @name'` | Literal | | Directive level | `@var` | `/show @greeting` | Direct reference | Transforms (built-in): `@json`, `@xml`, `@csv`, `@md` Aliases: `@p` ≡ `@pipeline` inside pipeline stages Globs: use `` and `as "template"` with `<>` placeholder for each matched file Guards protect data and operations. Label sensitive data, define policies: /var secret @apiKey = "sk-12345" # Label as 'secret' /var pii @email = "user@example.com" # Label as 'pii' /guard @noShellSecrets before secret = when [ @ctx.op.type == "run" => deny "Secrets blocked from shell" * => allow ] /run cmd { echo @apiKey } # Blocked by guard Syntax: /guard [@name] TIMING LABEL = when [...] - TIMING: before, after, or always (required) - Syntactic sugar: 'for' is equivalent to 'before' (e.g., /guard for secret = /guard before secret) Before guards (input validation): /guard @sanitize before untrusted = when [ * => allow @input.trim().slice(0, 100) ] After guards (output validation): /guard @validateJson after op:exe = when [ @isValidJson(@output) => allow * => deny "Invalid JSON" ] Transform with allow @value: /guard @redact before secret = when [ @ctx.op.type == "show" => allow @redact(@input) * => allow ] Denied handlers: /exe @handler(value) = when [ denied => `Blocked: @ctx.guard.reason` * => @value ] Labels track through: methods (@secret.trim()), templates, field access, iterators, pipelines. **Automatic labels**: - `src:exec` - Added to /run and /exe results - `src:file` - Added to file loads - `src:dynamic` - Added to dynamic module imports - `dir:/path` - Added for file directories (all parents) Example directory guards: ```mlld /guard before op:run = when [ @input.any.ctx.taint.includes('dir:/tmp/uploads') => deny "Cannot execute uploaded files" * => allow ] /guard before secret = when [ @input.ctx.taint.some(t => t.startsWith('dir:/app/config')) => deny "Config directory may contain secrets" * => allow ] ``` Guards are: non-reentrant, ordered (file top-to-bottom), composable (all run, deny > retry > allow @value > allow). Details: docs/user/security.md Show progress during execution instead of buffering: stream @claude("prompt") # Keyword (sugar for with { stream: true }) /stream @generateReport() # Directive form stream @a() || stream @b() # Parallel streams (concurrent, buffered results) NDJSON auto-parsing for stream executors; live output with thinking (💭) and tool-use (🔧) formatting. Suppress: --no-stream flag or MLLD_NO_STREAM=true Debug: --show-json mirrors NDJSON to stderr; --append-json saves to JSONL Details: docs/user/streaming.md Documentation: https://mlld.ai/docs Examples: https://github.com/mlld-lang/mlld/tree/main/tests/cases/valid/feat (comprehensive test cases demonstrating features) Source: https://github.com/mlld-lang/mlld