Why It Works

The design has three properties that compound:

• Self-similarity — Every depth runs the same prompt, same tools, same agent. No specialized "scout" or "planner" roles. The intelligence is in decomposition, not specialization. The system prompt teaches one pattern — size-first → search → chunk → delegate → combine — and it works at every scale.

• Self-hosting — The extension is the canonical recursion machinery. When the shell helper is enabled (the ypi wrapper, or any load with YPISHELLHELPER=1), the prompt also includes its source for inspection and modification. A bare pi -e / npm extension install uses the native tool only and does not require that shell file.

• Bounded recursion — Five concentric guardrails (depth limit, PATH scrubbing, call count, budget, timeout) guarantee termination. The system prompt also installs cognitive pressure: deeper agents are told to be more conservative, preferring direct action over spawning more children.

• Symbolic access — Anything the agent needs to manipulate precisely is a file, not just tokens in context. $CONTEXT holds the data, $RLMPROMPTFILE holds the original prompt, and hashline provides line-addressed edits. Agents grep/sed/cat instead of copying tokens from memory.

Guardrails

| Feature | Env var | What it does | |---------|---------|-------------| | Budget | RLM_BUDGET=0.50 | Max dollar spend for entire recursive tree; native extension mode requires JSON output so child cost can be measured | | Timeout | RLM_TIMEOUT=60 | Wall-clock limit for entire recursive tree | | Call limit | RLMMAXCALLS=20 | Max total rlm_query invocations | | Model routing | RLMCHILDMODEL=haiku | Use cheaper model for sub-calls | | Depth limit | RLMMAXDEPTH=3 | How deep recursion can go | | jj disable | RLMJJ=0 | Skip workspace isolation; child agents are read-only unless RLMUNSAFENOJJ_WRITE=1 | | Plain text | RLM_JSON=0 | Disable JSON mode (no cost tracking) | | Tracing | PITRACEFILE=$HOME/scratch/trace.log | Log all calls with timing + cost |

The agent can check spend at any time:

rlm_cost          # "$0.042381"
rlm_cost --json   # {"cost": 0.042381, "tokens": 12450, "calls": 3}

Pi Compatibility

ypi is a thin layer on top of Pi. We strive not to break or duplicate what Pi already does:

| Pi feature | ypi behavior | Tests | |---|---|---| | Session history | Uses Pi's native session manager when a parent session exists. Child sessions go in the same dir with trace-encoded filenames. RLMSHAREDSESSIONS=0 uses --no-session and clears child session env. No separate session store. | G24–G30 | | Extensions | Child processes disable ambient extension discovery and explicitly reload the ypi extension. RLMEXTENSIONS=0 disables recursion extension loading; RLMCHILD_EXTENSIONS=0 disables it for child depths. | G34–G38, N8 | | Native recursion | The canonical extensions/recursive.ts extension registers a native Pi rlm_query tool. Minimal mode works with only Pi plus extension files: no ypi launcher, no shell helper, no jj. | extension smoke, pure-extension E2E | | System prompt | The extension injects SYSTEMPROMPT.md when present and falls back to a minimal built-in prompt when it is not. If the shell rlmquery file exists, its source is appended as optional compatibility context. Standalone shell rlm_query falls back to Pi's --system-prompt. | T8–T9, parity E2E | | -p mode | All child Pi calls run non-interactive (-p). ypi never fakes a terminal. | T3–T4 | | --session flag | Used when session sharing is enabled and Pi has a session dir; --no-session otherwise. Never both. | G24, G28 | | Provider/model | Never hardcoded. ypi and rlmquery use Pi's defaults unless the user sets RLMPROVIDER/RLM_MODEL. | T14, T14c |

If Pi changes how sessions or extensions work, our guardrail tests should catch it.

Troubleshooting

If ypi or recursion seems broken, run make doctor first. The most common cause is the wrong host pi: either the old @mariozechner/pi-coding-agent shadowing the current @earendil-works/pi-coding-agent, or a version older than .pi-version. make doctor reports the exact mismatch and the one-line fix (bun add -g @earendil-works/pi-coding-agent@<pinned>). It honors YPIPIBIN, so it checks the same binary recursion actually spawns.

Package Boundary

There are two published packages, built from one canonical source:

| Package | Audience | Entry point | Includes | |---|---|---|---| | pi-recursive | Pi users who want recursion inside plain pi | pi install npm:pi-recursive or pi -e npm:pi-recursive | The native rlm_query tool, prompt injection, depth/status/env handling. No bin; host pi is a peer dependency. | | ypi | Users who want a preconfigured recursive CLI | npm install -g ypi / bun install -g ypi | The same extension plus launcher defaults, the shell-compatible rlm_query (pipes/async), cost/session helpers. Bundles pi so the CLI runs without a separate global install. |

Both ship the same extensions/ source. pi-recursive is the extension-only publish view, staged from the repo root by scripts/build-pi-recursive; ypi ships the extension plus its launcher and shell helpers. The shell helper is opt-in (YPISHELLHELPER=1, set by the ypi wrapper), so installing pi-recursive gives you the native tool only.

Contributing

Project Structure

ypi/
├── ypi                    # Thin launcher: sets env, loads extensions/recursive.ts
├── rlm_query              # Optional shell-compatible recursive sub-call command
├── extensions/
│   ├── recursive.ts       # Canonical ypi Pi extension
│   ├── ypi/               # Native tool, env, prompt, status modules
│   └── ypi.ts             # Compatibility alias for recursive.ts
├── SYSTEM_PROMPT.md       # Teaches the LLM to be recursive + edit code
├── AGENTS.md              # Meta-instructions for the agent (read by ypi itself)
├── Makefile               # test targets
├── tests/
│   ├── test_unit.sh       # Mock pi, test bash logic (no LLM, fast)
│   ├── test_guardrails.sh # Test guardrails (no LLM, fast)
│   └── test_e2e.sh        # Real LLM calls (slow, costs ~$0.05)
├── pi-mono/               # Git submodule: upstream Pi coding agent
└── README.md

Version Control

This repo strongly prefers jj for version control. Git remains the remote-facing substrate.

jj status                    # What's changed
jj describe -m "message"     # Describe current change
jj new                       # Start a new change
jj bookmark set master       # Point master at current change
jj git push                  # Push to GitHub

Prefer jj for local changes, especially recursive agent work. Use the repo's safe push/land helpers for remote-facing operations.

Testing

make test-fast         # unit + guardrails
make test-extensions   # latest Pi + extension compatibility, including minimal mode
make pre-push-checks   # shared local/CI gate (recommended before push)
make test-e2e          # real LLM calls, costs money
make test-recursion-e2e # focused live proof that ypi invokes rlm_query
make test-extension-recursion-e2e # direct pi -e native tool recursion proof
make test-parity-e2e   # wrapper vs direct-extension parity proof
make test              # all of the above

Install hooks once per clone to run checks automatically on git push:

make install-hooks

Release/update helper:

make release-preflight   # same checks + upstream dry-run in one command make land                # deterministic-ish landing helper

Before any change to rlmquery: run make test-fast. After: run it again. rlmquery is a live dependency of the agent's own execution — breaking it breaks the agent.

CI helper commands:

make ci-status N=15      # recent workflow runs make ci-last-failure     # dump latest failing workflow log

History

ypi went through five approaches before landing on the current design:

• Tool-use REPL (exp 010/012) — Pi's completeWithTools(), ReAct loop. 77.6% on LongMemEval.
• Python bridge — HTTP server between Pi and Python RLM. Too complex.
• Pi extension — Custom provider with search tools. Not true recursion.
• Bash RLM (rlmquery + SYSTEMPROMPT.md) — True recursion via bash.
• Pi-native extension RLM — extensions/recursive.ts registers native recursion; ypi and shell rlm_query are compatibility/ergonomics layers. Current approach.

See Also

• Pi coding agent — the underlying agent
• Recursive Language Models — the library that inspired this
• rlm-cli — Python RLM CLI (budget, timeout, model routing)