Synthadoc: An open-source LLM knowledge compilation engine that turns raw documents into structured, local-first wikis. A transparent, human-readable alternative to traditional RAG, which can be self-managed and self-improved without the use of any tools.
Synthadoc
.-+###############+-.
.## ##.
## .----. .----. ##
## /######\ /######\ ##
## |######| |######| ##
## | [SD] | | wiki | ##
## |######| |######| ##
## \######/ \######/ ##
## '----' '----' ##
'## ##'
'-+###############+-'
S Y N T H A D O C Community Edition v1.2.0 ──────────────────────────────── Domain-agnostic LLM wiki engine
Document version: v1.2.0
Engineered for solo users and enterprises alike, providing a domain-specific knowledge base that scales seamlessly while maintaining accuracy through autonomous self-optimization.
Built for individuals, small teams, and large organizations who need a knowledge base that stays accurate as documents accumulate.
Synthadoc reads your raw source documents — PDFs, spreadsheets, PPTs, web pages, images, videos, Word files, TXTs, and AI session transcripts (.jsonl) — and uses an LLM to synthesize them into a persistent, structured wiki. Cross-references are built automatically, contradictions are detected and surfaced, orphan pages are flagged, and every answer cites its sources. Outputs are stored as local Markdown files, ensuring seamless integration and autonomous management within Obsidian or any wiki-compliant ecosystem.
▶ From Documents to Wiki |
▶ Four Interfaces: CLI, Obsidian, Web UI & MCP |
▶ Agentic Maintenance Workflow |
Table of Contents
- Who Is It For?
- Inspiration and Vision
- Problems Addressed
- Why Synthadoc?
- Architecture
- What's Included
- Installation
- Quick-Start Guide
- Creating Your Own Wiki
- Configuration
- Command Reference by Use Case
- Administrative Reference
- Understanding Logs and the Audit Trail
- Customization
- Links
Who Is It For?
Synthadoc scales from a single researcher to a company-wide knowledge platform:
| Team size | Typical use case | | ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Solo / 1–2 people | Personal research wiki, freelance knowledge base, indie hacker documentation - run it free on Gemini Flash or a local Ollama model with zero ongoing cost | | Small team (3–20) | Centralized internal knowledge base for startups and departments that aggregates diverse individual data sources into a unified, high-integrity wiki. The system automatically resolves contradictions and scales autonomously, ensuring organizational intelligence grows in tandem with your team | | Medium / enterprise | Compliance-sensitive knowledge bases that must stay local; per-department wikis on separate ports; audit trail for every ingest and cost event; hook system for CI/CD integration; OpenTelemetry for ops dashboards |
No cloud account. No vendor lock-in. The wiki is plain Markdown — open it in any editor, back it up with git, sync it with any cloud drive.
Inspiration and Vision
"The LLM should be able to maintain a wiki for you."
— Andrej Karpathy, LLM Wiki gist
Most knowledge-management tools retrieve and summarize at query time. Synthadoc inverts this: it compiles knowledge at ingest time. Every new source enriches and cross-links the entire corpus, not just appends a new chunk. The wiki is the artifact — readable, editable, and browsable without any tool running.
Long-term alignment:
| Direction | How Synthadoc moves there | | ------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Agent orchestration | Orchestrator dispatches parallel ingest, query, and lint sub-agents with cost guards and retry backoff | | Sub-agent skills/plugins | Featuring a 3-tier lazy-load capability system, the platform allows for the injection of custom skills and hooks via a plug-and-play interface, ensuring core stability is never compromised during extension | | LLM wiki vs. RAG | Pre-compiled structured knowledge beats query-time synthesis for contradiction detection, graph traversal, and offline access | | CLI / HTTP | A unified interface via CLI and RESTful endpoints, the system streamlines full-spectrum integration: from data ingestion and querying to automated linting, security auditing, and job orchestration | | Local-first | All data stays on your machine; localhost-only network binding; no cloud dependency except the LLM API itself | | Provider choice | LLM backends including free-tier Gemini and Groq, paid Anthropic/OpenAI/DeepSeek/MiniMax/Qwen (DashScope), local Ollama and Qwen, and coding-tool CLI providers (Claude Code, Opencode) — no API key required if you already have a subscription |
Problems Addressed
RAG retrieves document chunks at query time. Synthadoc compiles knowledge at ingest — synthesising sources into a linked, audited wiki graph so contradictions are caught, claims are traced to sources, and the artifact survives outside the tool.
| Problem | Synthadoc approach | | --- | --- | | Contradictions blended silently | Ingest-time conflict detection; page flagged status: contradicted; auto-resolve or queue for human review | | No links between related content | [[wikilinks]] auto-built on every ingest pass; weighted graph (wikilink + co-source signals) with Louvain clustering in web UI | | Orphan pages never surfaced | Lint reports unreferenced pages with ready-to-paste index entries | | LLM output can be overconfident | Adversarial second-LLM pass flags overstated claims, unsupported superlatives, and contestable facts per page | | Claims lack source traceability | ^[file:L-L] citation on every claim; Source Viewer in Obsidian; PDF page resolution; broken-citation lint | | Knowledge lifecycle invisible | 5-state machine (draft → active → contradicted / stale → archived); auto-transitions via lint; immutable event log | | Repeat ingest is expensive | 3-layer cache (embedding, LLM, provider prompt) — repeat lint on unchanged pages costs near-zero tokens | | Knowledge locked in proprietary tools | Plain Markdown + YAML frontmatter; OKF v0.1 compatible; fully offline-readable in any editor | | Wiki structure drifts with growth | scaffold regenerates index, AGENTS.md, and purpose.md from current wiki state without touching linked pages | | Migration requires full re-ingestion | Single-zip backup + restore with port/domain rewriting; no re-ingestion needed | | Cost and compliance exposure | Localhost-only; per-job token+cost log; configurable soft-warn and hard-gate thresholds |
Citation quality: Generated pages include inline citations linking every claim to its source lines. Pages without citations trigger a model-compatibility warning — use Gemini 2.5 Flash or higher for reliable citation annotation.>
Active page protection: Pages promoted to active status are protected — sources that contradict them are flagged for review rather than overwriting the human-reviewed content.
Why Synthadoc?
Every Yes below is a built-in feature — no add-ons or upgrades required.
Knowledge Quality
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Ingest-time synthesis — sources compiled into the wiki at ingest; not re-summarised at query time | Yes | No | Partial | No | | Contradiction detection & resolution — conflicting claims flagged status: contradicted; auto-resolve available; full conflict history | Yes | No | No | No | | Adversarial claim review — concurrent second-LLM pass flags overstated claims, unsupported superlatives, and contestable facts per page | Yes | No | No | No | | Claim-level provenance — ^[file:L-L] citation on every claim; Source Viewer in Obsidian; PDF page resolution; broken-citation lint | Yes | No | Partial | No | | 5-state lifecycle machine — draft → active → contradicted / stale → archived; auto-transitions via lint; immutable event log; cascade link cleanup on archive (immediate, no lint run required) | Yes | No | No | No | | Pre-LLM source sanitizer — strips zero-width chars, bidi overrides, hidden HTML, and instruction-override phrases before any LLM call | Yes | No | No | No |
Knowledge Structure
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Weighted knowledge graph + D3 visualisation — [[wikilinks]] auto-built at ingest; co-source edges connect pages compiled from the same document; edge thickness reflects combined weight; dashed edges for co-source-only relationships; Louvain cluster colouring; click node to query; Obsidian graph panel: same graph inside Obsidian via Canvas, type filter, click node to open page | Yes | No | Partial | No | | Orphan page detection — unreferenced pages surfaced by lint with ready-to-paste index entries | Yes | No | No | No | | Query-scoped routing — ROUTING.md maps wiki branches to page slugs; queries auto-select relevant branches; new pages auto-slotted | Yes | No | No | No | | Candidates staging — ingest pages to a staging area first; review, promote, or discard before they enter the live wiki | Yes | No | No | No | | Scaffold automation — regenerates index categories, AGENTS.md/CLAUDE.md/GEMINI.md, and purpose.md from current wiki state; protected pages never overwritten | Yes | No | No | No |
Search & Query
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Query decomposition + gap detection — compound questions split into parallel BM25 sub-queries; thin results trigger a knowledge-gap callout with suggested web searches | Yes | Partial | No | No | | BM25 TF fallback + compound identifier search — reliable results on small corpora (IDF collapse → TF fallback); underscore identifiers expanded at index and query time so capex growth matches capexgrowth | Yes | No | No | No | | Web search → wiki pages — Tavily search fans out into parallel URL ingest jobs; gap callout in web UI suggests searches inline | Yes | No | No | No | | Semantic re-ranking — optional vector re-ranking (BAAI/bge-small-en-v1.5) improves recall on conceptually related queries; BM25 stays as fallback | Yes (optional) | Varies | No | No | | Streaming output + query cache — token-by-token streaming; cache key = question + wiki version; auto-invalidates on ingest or lifecycle change | Yes | Partial | Partial | Partial | | Proportional context budget — sources allocated proportionally to model context window (60 % wiki / 20 % history / 15 % system / 5 % index); replaces fixed top-N cap | Yes | No | No | No |
Interfaces & Integration
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Obsidian integration — native plugin: ingest modal, streaming query, lint report, lifecycle controls, context pack builder, provenance viewer, export modal, knowledge graph panel (Canvas force graph, type filter, hover tooltip, click-to-open page); background vault monitoring (auto-snapshot on every file save, 2 s debounce, dedup so unchanged saves are free); Reading View set as default on install so citation chips are visible immediately | Yes | No | No | No | | Web chat UI — synthadoc web: streaming answers, session sidebar, multi-turn history, knowledge-gap callouts, knowledge graph tab | Yes | No | Yes | Yes | | MCP server — 12 tools; Claude Desktop (stdio), Claude Code (SSE), n8n/LangGraph (HTTP/SSE); brain+memory architecture; no double-LLM cost for reads | Yes | No | No | No | | Context packs — goal → sub-questions → token-budget evidence pack; REST + MCP callable; paste into any LLM chat as grounded context | Yes | No | No | No | | Export formats — llms.txt, llms-full.txt, GraphML, JSON (provenance + lifecycle), OKF v0.1 bundle; lifecycle-filtered; zero extra LLM calls | Yes | No | Partial | No | | Multi-platform agent skill files — AGENTS.md (Codex/OpenCode), CLAUDE.md (Claude Code), GEMINI.md (Gemini CLI); all include full CLI quick-reference, domain guidelines, MCP tool table; regenerated by scaffold | Yes | No | No | No |
Content Sources
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Multi-format ingest — PDF, DOCX, PPTX, XLSX/CSV, Markdown, TXT, images (vision), web URLs, YouTube transcripts, AI session transcripts (.jsonl) | Yes | Varies | Partial | Partial | | Web search decomposition — broad topics decomposed into focused Tavily keyword searches; results merged and deduplicated | Yes | No | No | Partial | | YouTube transcript ingest — timestamped transcript + executive summary; no API key; auto-generated captions supported | Yes | No | Yes | No | | Multilingual / CJK queries — Chinese, Japanese, Korean — no false knowledge gaps | Yes | Limited | No | No | | Multiple LLM providers + coding tools — Gemini, Groq, Qwen, MiniMax, DeepSeek, Anthropic, OpenAI, Ollama; Claude Code and Opencode (no API key needed) | Yes | No | No | No |
Operations & Trust
| Capability | Synthadoc | Typical RAG | NotebookLM | Notion AI | | --- | --- | --- | --- | --- | | Local-first + offline artifact — source documents never leave your machine; compiled wiki is plain Markdown, fully readable offline in any editor without the server | Yes | Varies | No | No | | Portable backup / restore — single zip: wiki pages + audit/lifecycle DB + config; port and domain rewriting on restore; migrate machines without re-ingesting | Yes | No — re-ingest required | No — AI metadata lost | No | | Lifecycle content snapshots + rollback — page body captured at every lifecycle transition (manual CLI/Obsidian, lint-driven auto-transition, or Obsidian vault save); deduplicated so unchanged saves cost nothing; browse per-page version history with lifecycle history (newest-first, with reason); restore any prior version with lifecycle rollback; rollback saves the current body first so it is always undoable | Yes | No | No | Partial — Notion has time-based edit history (plan-gated, no lifecycle tie, no auditable rollback) | | Cost guard + full audit trail — per-job token + cost log; soft-warn and hard-gate thresholds; audit citations validates every claim citation; immutable event log | Yes | No | No | No | | Resumable job queue + retry — every ingest/lint job persisted with status and error; batch a hundred documents and resume after a crash | Yes | No | No | No | | Custom skills + CI hooks — subclass BaseSkill for new file formats; 2 hook events (oningestcomplete + onlint_complete); example git auto-commit hook included; blocking hooks can gate operations | Yes | Limited | No | No | | Per-source truncation flag — --max-source-chars caps any source (PDF, DOCX, web page, plain text) before the LLM call; truncated sources flagged with truncated: true in frontmatter and warned in lint output | Yes | No | No | No | | Multi-wiki isolation — each wiki on its own port with independent config, audit trail, and job queue; switch with synthadoc use | Yes | No | Partial | No | | Agentic maintenance workflows — conversational wiki maintenance via agentic tool-call loop; "re-ingest stale pages" bulk-reingests every stale page; "re-ingest the [slug] page" re-ingests any single page by slug regardless of lifecycle state (active, draft, or stale); agent confirms before touching anything, streams inline progress, and runs lint on completion; graph sidebar maintenance chips in the web UI trigger the same workflows with one click — no terminal required | Yes | No | No | No |
Business value
| Value | How | | --- | --- | | Faster onboarding | New team members query the wiki instead of digging through documents | | Audit trail | Every ingest recorded in audit.db with source hash, token count, and timestamp | | Cost control | Configurable thresholds; 3-layer cache reduces repeat spend | | Compliance | Local-first — source documents and compiled knowledge never leave your machine | | Extensibility | Hooks fire on every event; custom skills load without a server restart |
Architecture
For full architecture details, data models, API reference, and plugin development guide see docs/design.md.
What's Included
See docs/design.md — Appendix A: Release Feature Index for a full feature list by version.
Installation
Production
Prerequisites: Python 3.11+. No Node.js, no Git, no build steps.
pip install synthadoc
synthadoc --version # confirm it worksThe Obsidian plugin is bundled inside the package. New wikis created with synthadoc install have it installed automatically. If you are upgrading an existing Synthadoc installation, run synthadoc plugin upgrade to push the updated plugin binary to all registered wikis.
Development
For developers modifying the Python engine, running the test suite, or developing the Obsidian plugin TypeScript.
Additional prerequisites: Git (any). Node.js 18+ only if modifying the plugin TypeScript source.
Step 1 — Clone and install
git clone https://github.com/axoviq-ai/synthadoc.git
cd synthadoc
pip install -e ".[dev]"[dev] adds pytest, respx, and the other test dependencies. Tests require a source checkout — they are not included in the pip wheel.
Step 2 — Run the test suite
pytest --ignore=tests/performance/ -qExpected: all tests pass, 0 failures. Performance benchmarks (optional):
pytest tests/performance/ -v --benchmark-disableStep 3 — Obsidian plugin development (optional)
The compiled plugin is bundled with the package and kept up to date by CI — no build step is needed to work on the Python side or run tests.
If you modify the TypeScript source under obsidian-plugin/src/, recompile and sync it into the Python package manually:
cd obsidian-plugin
npm install # first time only, or after package.json changes
edit src/main.ts or other source files
npm run build # compile TypeScript → main.js (also syncs to synthadoc/data/obsidian-plugin/)
npm test # run Vitest unit tests
cd ..Set your API keys
At least one LLM API key is required — unless you use Claude Code or Opencode as your provider (no separate API key needed — see Coding tool CLI providers).
Synthadoc defaults to Gemini Flash — free tier, no credit card, 1 million tokens per day. Get a key at aistudio.google.com/app/apikey (click "Create API key").
| Provider | Free tier | Vision | Get key | | ---------------- | -------------------------------------------------------- | --------------- | ----------------------------------------------------------------- | | Gemini Flash | Yes — 15 RPM / 1M tokens/day, no credit card | Yes | aistudio.google.com | | Groq | Yes — rate-limited | No | console.groq.com | | Ollama | Yes — runs locally, no key (GPU required) | Model-dependent | ollama.com | | Qwen | Yes — 1M free tokens (90-day trial), then paid DashScope | Model-dependent | bailian.console.aliyun.com | | MiniMax | No — pay-per-token | Yes | platform.minimax.io | | DeepSeek | No — pay-per-token (very cheap text rates) | No | platform.deepseek.com | | Anthropic | No | Yes | console.anthropic.com | | OpenAI | No | Yes | platform.openai.com | | Claude Code | Included with subscription — no API key | No | Set provider = "claude-code" in config.toml | | Opencode | Free via Opencode Zen — no API key | No | Set provider = "opencode", model = "opencode/big-pickle" in config.toml; connect first: run opencode → /connect → select Zen |
# macOS / Linux — add to ~/.bashrc or ~/.zshrc to persist
export GEMINIAPIKEY=AIza… # default — free tier, 1M tokens/day
export GROQAPIKEY=gsk_… # alternative free tier — 100K tokens/day
export ANTHROPICAPIKEY=sk-ant-… # paid — highest quality
export OPENAIAPIKEY=sk-… # paid
export MINIMAXAPIKEY=… # paid — text rates (image support)
export DEEPSEEKAPIKEY=… # paid — text rates (no image support)
export QWENAPIKEY=… # DashScope cloud — 1M free tokens trial
export TAVILYAPIKEY=tvly-… # web search (optional)
Windows cmd — current session only
set GEMINIAPIKEY=AIza…
set GROQAPIKEY=gsk_…
set ANTHROPICAPIKEY=sk-ant-…
set OPENAIAPIKEY=sk-…
set MINIMAXAPIKEY=…
set DEEPSEEKAPIKEY=…
set QWENAPIKEY=…
set TAVILYAPIKEY=tvly-…Web search uses Tavily (TAVILYAPIKEY) — optional, only needed for synthadoc ingest "search for: …" jobs. Get a free key at tavily.com.
Install a wiki and start the engine
A wiki is a self-contained knowledge base — a folder of Markdown pages maintained and cross-referenced automatically by Synthadoc. The fastest way to get started is the History of Computing demo (13 pre-built pages, no LLM API key required to browse).
# Linux / macOS
synthadoc install history-of-computing --target ~/wikis --demo
Windows (cmd.exe)
synthadoc install history-of-computing --target %USERPROFILE%\wikis --demoThen start the engine:
# Foreground — logs stream to the console
synthadoc serve -w history-of-computing
Background — releases the terminal
synthadoc serve -w history-of-computing --backgroundThe server binds to http://127.0.0.1:7070 (localhost-only). Leave it running while you work — the Obsidian plugin, CLI ingest commands, and query commands all talk to it.
To switch LLM provider, edit [agents] in <wiki-root>/.synthadoc/config.toml and restart synthadoc serve. See Appendix — Switching LLM providers for step-by-step instructions.
To stop a background server:
# Linux / macOS
kill <PID>
Windows (cmd)
taskkill /PID <PID> /FThe PID is printed on start and saved to <wiki-root>/.synthadoc/server.pid.
Upgrading: after updating synthadoc (via pip install --upgrade synthadoc or git pull), restart the server to pick up the new code, then run these to keep registered wikis in sync:
synthadoc plugin upgrade # push updated Obsidian plugin binary to all registered wikis
synthadoc demo sync --force # demo-installed wikis — update pages and pick up citation markersNeither command requires the server to be running.
Quick-Start Guide
The History of Computing demo includes 13 pre-built pages, raw source files covering clean-merge, contradiction, and orphan scenarios, and a full walkthrough of key Synthadoc feature.
Full step-by-step walkthrough: docs/user-quick-start-guide.md
The guide covers:
- Verify the demo server started (banner, health check)
- Install the Synthadoc plugin (auto-installs Dataview) and open the vault
- Review wiki structure and key files (index, purpose, AGENTS.md, dashboard)
- Query the pre-built wiki — including knowledge gap detection
- Batch ingest all demo source files
- Run lint — auto-promote clean pages to active
- Manage page lifecycle — 5-state machine (draft → active → stale/contradicted/archived), manual transitions, immutable audit trail
- Resolve a contradiction
- Fix an orphan page
- Run the adversarial lint pass — flag overstated claims across all pages
- Web search ingestion with automatic decomposition
- Ingest a YouTube video
- Enrich the wiki with scaffold (regenerate/update index, purpose, AGENTS.md)
- Audit features (token cost, history, events)
- Schedule recurring operations
- Set up query-scoped routing with ROUTING.md
- Stage and review candidate pages before promoting them
- Build a context pack for grounded LLM prompts
- Verify claim provenance — source-line citations, broken citation audit, global provenance table
- Export your wiki — llms.txt, llms-full.txt, GraphML wikilink graph, agent-ready JSON with provenance and lifecycle history, OKF v0.1 bundle for zero-code agent consumption
- Use the web chat UI — streaming answers, session-aware hint chips, citations in-browser
- Query caching — understand how answers are cached and how to bypass with
--no-cache - Backup and restore — create a portable wiki zip, restore on a different machine
- Knowledge graph — weighted edges (wikilink + co-source signals), explore clusters in the web UI Graph tab, click a node to query it
- Ingest an AI session transcript — turn Claude Code, Codex, or chat conversations into structured wiki pages
- Agentic maintenance workflows — re-ingest stale pages or any named page conversationally from the web UI or Obsidian, with confirmation and real-time progress
Creating Your Own Wiki
New to building your own wiki? Work through the AquaFlow Capital Workshop Walkthrough first — a complete end-to-end workshop using a real M&A due-diligence wiki with pre-built pages, evaluation queries, and benchmark results. Once you are comfortable with the flow, come back here to build your own wiki from scratch.
Unlike the demo (which ships with pre-built pages), your own wiki starts from a domain description and grows as you feed it sources:
synthadoc install market-condition-canada --target ~/wikis --domain "Market conditions and trends in Canada"
synthadoc use market-condition-canada # set as the default wiki — no -w needed from here onBefore starting the server, open ~/wikis/market-condition-canada/.synthadoc/config.toml and set your LLM provider — see Appendix C in the Quick-Start Guide for the full provider list and API key setup. Then start:
synthadoc serve
synthadoc status # confirm the wiki registered correctly (should show 0 pages)--domain is a free-text description of the subject area — the LLM uses it to generate five domain-aware starter files via scaffold:
| File | Purpose | | ------------------- | --------------------------------------------------------------------------- | | wiki/index.md | Table of contents — domain-relevant categories with [[wikilinks]] | | wiki/purpose.md | Scope declaration — used by the ingest agent to filter out-of-scope sources at ingest time, and injected as a pinned preamble into every query synthesis prompt so the LLM understands the wiki's domain boundaries when answering | | AGENTS.md | LLM behaviour guidelines (tone, terminology, synthesis style) — read by Codex CLI, OpenCode, and generic OpenAI Agents tooling | | CLAUDE.md | Same content as AGENTS.md — loaded automatically by Claude Code when the wiki folder is open | | GEMINI.md | Same content as AGENTS.md — loaded automatically by Gemini CLI |
wiki/dashboard.md is also created during install (a static template — not LLM-generated). ROUTING.md is optional and generated separately via synthadoc routing init after pages accumulate.
Before ingesting any content, run scaffold once to build a clean starting index and purpose files based on your domain:
synthadoc scaffoldsynthadoc install also copies both the Synthadoc plugin and the Dataview plugin directly into the vault's plugins folder, pre-enables them, and sets the correct server URL — no separate plugin step is required. Open the wiki folder in Obsidian — both plugins are active immediately, no manual toggling needed.
The Quick-Start Guide covers the full Obsidian setup in detail — see docs/user-quick-start-guide.md.
Local Web UI — once the server is running, you can also query the wiki from your browser without Obsidian:
synthadoc webThis opens a local chat interface at http://localhost:{port}/app. The Web UI is local-only and is not accessible from the network — authentication and authorisation are not configured by default in the Community Edition.
Recommended growth loop:
1. Seed with web searches — pull in real content for the topics you care about:
synthadoc ingest "search for: Economy, employment and labour market analysis in Toronto GTA"
synthadoc ingest "search for: Bank of Canada interest rate outlook 2025"
synthadoc jobs list # watch progressEach search fans out into up to 20 parallel URL ingest jobs. Both query and web search automatically decompose broad inputs into focused parallel sub-tasks — see Quick-Start Guide for examples.
2. Review candidates (optional quality gate) — enable staging before large ingest batches so pages below your confidence threshold wait for review rather than entering BM25 immediately:
synthadoc staging policy threshold # pages below high confidence → wiki/candidates/
synthadoc candidates list # see what's waiting
synthadoc candidates promote early-internet-history # approve individually
synthadoc candidates promote --all # or approve everything at once
synthadoc candidates discard punch-card-era # discard pages that don't belongSkip this step if you trust all your sources — staging policy off is the default.
3. Re-run scaffold — after pages accumulate, scaffold regenerates a richer index that reflects actual content. Pages already linked in index.md are never overwritten:
synthadoc scaffold4. Lint and query — check for contradictions, flag overstated claims, verify citations, and confirm the wiki answers your questions:
synthadoc lint run # full lint: structural checks + adversarial pass (default)
synthadoc lint run --no-adversarial # structural only — skip the adversarial LLM review
synthadoc lint report # view all issues including citation violations (Check 5)
synthadoc audit citations --broken # list claim citations that failed validation
synthadoc query "What are the current employment trends in the Toronto GTA?"5. Set up routing — once the wiki spans distinct topic areas, routing narrows each query to the relevant branch, cutting latency and reducing noise in synthesis:
synthadoc routing init # generate ROUTING.md from current index.md (one-time)From this point, queries automatically scope to the 1–2 most relevant topic branches. New pages created by ingest are auto-slotted into ROUTING.md — no manual maintenance needed. See Appendix H in the Quick-Start Guide for latency benchmarks across corpus sizes.
6. Build a context pack — assemble cited wiki excerpts within a token budget for use in an external agent prompt:
synthadoc context build "Toronto GTA real estate market" --tokens 4000Returns ranked page excerpts with relevance scores, confidence levels, and source paths — no synthesis. The POST /context/build REST endpoint and synthadoc_context MCP tool make this callable from any agent pipeline. To connect Claude Code to your wiki's MCP server:
# Replace 7070 with the port shown when you ran synthadoc serve
claude mcp add --transport sse synthadoc-market-condition-canada http://127.0.0.1:7070/mcp/sseThen ask Claude Code: "Build a context pack on Toronto GTA real estate market" and it will call synthadoccontext automatically. See docs/design.md — Context packs for the knowledge backend pattern.
7. Schedule recurring updates — keep the wiki fresh and the routing table clean automatically:
synthadoc schedule add --op "ingest --batch raw_sources/" --cron "0 2 *"
synthadoc schedule add --op "lint run" --cron "0 3 0"
synthadoc schedule add --op "scaffold" --cron "0 4 0"
synthadoc schedule add --op "routing clean" --cron "0 5 0"Run order matters: lint first (removes dead wikilinks), scaffold next (regenerates index), routing clean last (prunes ROUTING.md entries for deleted pages).
Semantic re-ranking (vector search)
BM25 keyword search is the default. Optional vector re-ranking (BAAI/bge-small-en-v1.5 cosine similarity) improves recall on conceptually related queries — enable it by installing fastembed and setting [search] vector = true in config. The ~130 MB model is downloaded once; BM25 stays active as fallback.
See docs/design.md — Semantic re-ranking for configuration options and performance notes.
Knowledge gap workflow
When a query returns thin or empty results, the wiki doesn't yet cover the topic. Fill the gap with a targeted web search ingest, wait for jobs, then re-query. Each ingest cycle makes the wiki denser — future queries need the web less.
See docs/design.md — Knowledge gap workflow for the full pattern.
See docs/design.md for a full description of how ingest, contradiction detection, and orphan tracking work under the hood.
Configuration
You do not need to configure anything to run the demo. The demo wiki ships with its own settings and sensible built-in defaults cover everything else. Set your API key env var, run synthadoc serve, and go.
For the full configuration reference — layer precedence, global vs. per-project config, all keys and defaults — see Appendix E — Configuration in the Quick-Start Guide, or docs/design.md — Configuration for the complete technical reference.
Command Reference by Use Case
Setting up a wiki
# Create a new wiki (LLM scaffold runs automatically; port is auto-assigned to avoid conflicts)
synthadoc install my-wiki --target ~/wikis --domain "Machine Learning"
Pin a specific port manually
synthadoc install my-wiki --target ~/wikis --domain "Machine Learning" --port 7071
Install the demo (includes pre-built pages and raw sources — no LLM call needed)
synthadoc install history-of-computing --target ~/wikis --demo
List available demo templates
synthadoc demo list
Sync new source files into an existing demo install (additive only, no overwrites)
synthadoc demo sync history-of-computing
Update existing demo pages from the latest template (overwrites demo pages)
synthadoc demo sync history-of-computing --force
Reinstall the Obsidian plugin into a wiki's vault — normally done automatically by synthadoc install
synthadoc plugin install history-of-computing
Push the updated plugin binary to every registered wiki after a Synthadoc upgrade
synthadoc plugin upgradeSwitching the active wiki
# Set a wiki as the default so -w is not required for any subsequent command
synthadoc use my-wiki
Check which wiki is currently active
synthadoc use
Clear the saved default (revert to requiring -w on every command)
synthadoc use --clearRefreshing wiki scaffold
After install, you can re-run the LLM scaffold at any time to regenerate domain-specific content (index categories, AGENTS.md guidelines, purpose.md scope). Pages already linked in index.md are protected and preserved.
# Regenerate scaffold for an existing wiki
synthadoc scaffold -w my-wiki
Schedule weekly refresh (runs every Sunday at 4 AM)
synthadoc schedule add --op "scaffold" --cron "0 4 0" -w my-wikiconfig.toml and dashboard.md are never modified by scaffold.
Running the server
# Start HTTP API + job worker (foreground — terminal stays attached)
synthadoc serve -w my-wiki
Detach to background — banner shown, then shell is released
All logs go to <wiki>/.synthadoc/logs/synthadoc.log
synthadoc serve -w my-wiki --background
Custom port
synthadoc serve -w my-wiki --port 7071
Verbose debug logging to console
synthadoc serve -w my-wiki --verboseIngesting sources
# Single file or URL
synthadoc ingest report.pdf -w my-wiki
synthadoc ingest https://example.com/article -w my-wiki
Entire folder (parallel, up to maxparallelingest at a time)
synthadoc ingest --batch raw_sources/ -w my-wiki
Manifest file — ingest a curated list of sources in one shot.
sources.txt: one entry per line; each line is either an absolute file path
(PDF, DOCX, PPTX, MD, …) or a URL. Blank lines and # comments are ignored.
Each entry becomes a separate job in the queue, processed sequentially.
#
Example sources.txt:
/home/user/docs/research-paper.pdf
/home/user/slides/keynote.pptx
https://en.wikipedia.org/wiki/Alan_Turing
# this line is ignored
synthadoc ingest --file sources.txt -w my-wiki
Force re-ingest (bypass deduplication and cache)
synthadoc ingest --force report.pdf -w my-wiki
Web search — triggers a Tavily search, then ingests each result URL as a child job.
Prefix the query with any recognised intent: "search for:", "find on the web:",
"look up:", or "web search:" (prefix is stripped before the search is sent)
Requires TAVILYAPIKEY to be set.
#
Note: web search content is NOT saved to raw_sources/. The flow is direct:
query → Tavily → URLs → each URL fetched → wiki pages written
raw_sources/ is for user-provided local files (PDF, DOCX, PPTX, etc.) only.
The wiki pages themselves are the persistent output of a web search.
synthadoc ingest "search for: Bank of Canada interest rate decisions 2024" -w my-wiki
synthadoc ingest "find on the web: unemployment trends Ontario Q1 2025" -w my-wiki
Limit how many URLs are enqueued (default 20, overrides [websearch] maxresults)
synthadoc ingest "search for: quantum computing basics" --max-results 5 -w my-wiki
Multiple web searches at once via a manifest file
web-searches.txt:
search for: Bank of Canada interest rate decisions 2024
find on the web: unemployment trends Ontario Q1 2025
look up: Toronto housing market affordability index
synthadoc ingest --file web-searches.txt -w my-wiki
YouTube video — transcript extracted automatically, no API key needed.
The video must have captions (auto-generated or manual).
Check: open the video on YouTube → ... → Show transcript.
synthadoc ingest "https://www.youtube.com/watch?v=O5nskjZ_GoI" -w my-wiki
synthadoc ingest "https://youtu.be/O5nskjZ_GoI" -w my-wiki
YouTube URLs returned by web search are also routed automatically:
if Tavily returns a YouTube URL, the transcript is ingested instead of the page HTML.
synthadoc ingest "search for: history of computing lecture" -w my-wikiEach YouTube wiki page opens with an executive summary — what the video is about, the main topics covered, and the key takeaway — followed by the full timestamped transcript for precise citation.
Querying
# Ask a question — answer streams token-by-token as the LLM generates it
synthadoc query "What is Moore's Law?" -w my-wiki
Blocking mode (no streaming) — useful in scripts or pipes
synthadoc query "What is Moore's Law?" --no-stream -w my-wiki
Skip the cache — always call the LLM even if the answer is cached
synthadoc query "What is Moore's Law?" --no-cache -w my-wiki
Save the answer as a new wiki page
synthadoc query "What is Moore's Law?" --save -w my-wikiQuery answers are cached automatically by question content and wiki version. Repeated identical questions return instantly from cache. The cache invalidates automatically when you ingest new content or change a page's lifecycle state.
Web Chat UI
# Open the browser-based chat interface for a wiki
synthadoc web -w my-wiki→ Full walkthrough (session modes, multi-turn conversation, session history sidebar): Quick-Start Guide — Step 22
Linting
# Run a full lint pass (enqueues job)
synthadoc lint run -w my-wiki
Only contradictions
synthadoc lint run --scope contradictions -w my-wiki
Auto-apply high-confidence resolutions
synthadoc lint run --auto-resolve -w my-wiki
Skip adversarial review (structural checks only; also clears existing warnings)
synthadoc lint run --no-adversarial -w my-wiki
Skip lifecycle checks (structural and adversarial checks only)
synthadoc lint run --no-lifecycle -w my-wiki
Instant report (reads wiki files directly, no server needed)
synthadoc lint report -w my-wikiManaging page lifecycle
# Show page counts by lifecycle state
synthadoc status -w my-wiki
Promote a draft page to active after manual review
synthadoc lifecycle activate <slug> -w my-wiki --reason "reviewed and verified"
Archive a page whose source has been superseded
synthadoc lifecycle archive <slug> -w my-wiki --reason "replaced by updated source"
After archiving, any [[slug]] references in other pages are cleaned up automatically.
The output reports which pages were updated:
Cascade: [[old-page]] removed from 2 page(s): cpu-design, transistor-history
Restore an archived page back to draft for re-review
synthadoc lifecycle restore <slug> -w my-wiki --reason "source re-added"
View full state history for a page (or all pages)
synthadoc lifecycle log <slug> -w my-wiki
synthadoc lifecycle log -w my-wiki
Browse content snapshots captured at each lifecycle transition
synthadoc lifecycle history <slug> -w my-wiki
synthadoc lifecycle history <slug> --index 1 -w my-wiki # inspect newest snapshot
synthadoc lifecycle history <slug> --index 2 --show-content -w my-wiki # print full body
Restore the page body to a prior snapshot (rollback is itself snapshotted, so it is undoable)
synthadoc lifecycle rollback <slug> --index 2 --reason "restoring original body" -w my-wiki
Purge old lifecycle events to reclaim audit.db space
synthadoc audit lifecycle purge -w my-wiki --before 2026-01-01
synthadoc audit lifecycle purge -w my-wiki --keep-latest 100Synthadoc captures the page body in three situations: a manual lifecycle transition (activate, archive, restore), a lint-driven auto-transition (stale detection, contradiction auto-resolve, draft promotion), or a file save detected by the Obsidian plugin (2 s debounce, deduplicated — unchanged saves cost nothing). Use synthadoc lifecycle history <slug> to browse snapshots and synthadoc lifecycle rollback <slug> --index N --reason "..." to restore any prior version — the rollback itself is recorded as an auditable event so it can be undone.
Monitoring jobs
# List all jobs (oldest first by default)
synthadoc jobs list -w my-wiki
Sort column — created_at (default) | status | operation
synthadoc jobs list --sort created_at -w my-wiki # oldest first (default)
synthadoc jobs list --sort status -w my-wiki # alphabetical by status
synthadoc jobs list --sort operation -w my-wiki # alphabetical by operation type
Sort direction — asc (default) | desc
synthadoc jobs list --order desc -w my-wiki # newest first
synthadoc jobs list --sort status --order desc -w my-wiki # status Z→A
Filter by status — pending | in_progress | completed | failed | skipped | dead | cancelled
synthadoc jobs list --status pending -w my-wiki
synthadoc jobs list --status failed -w my-wiki
synthadoc jobs list --status dead -w my-wiki
Combine sort, order, and status freely
synthadoc jobs list --status failed --sort created_at --order desc -w my-wiki
Single job detail
synthadoc jobs status <job-id> -w my-wiki
Retry a dead job
synthadoc jobs retry <job-id> -w my-wiki
Cancel all pending jobs at once (e.g. after a bad batch ingest)
synthadoc jobs cancel -w my-wiki # prompts for confirmation
synthadoc jobs cancel --yes -w my-wiki # skip confirmation
Remove old records
synthadoc jobs purge --older-than 30 -w my-wikiInspecting ingest results
# Preview how a source will be analysed without writing pages
synthadoc ingest report.pdf --analyse-only -w my-wiki
→ {"entities": [...], "tags": [...], "summary": "..."}
Audit trail
# Ingest history: timestamp, source file, wiki page, tokens, cost
synthadoc audit history -w my-wiki # last 50 records (shows total)
synthadoc audit history -n 100 -w my-wiki # last 100 records
synthadoc audit history -n 50 --offset 50 -w my-wiki # page 2 (records 51-100)
synthadoc audit history --json -w my-wiki # raw JSON for scripting
Query history: question, sub-questions, tokens, cost
synthadoc audit queries -w my-wiki # last 50 records (shows total)
synthadoc audit queries -n 100 -w my-wiki # last 100 records
synthadoc audit queries -n 50 --offset 50 -w my-wiki # page 2 (records 51-100)
synthadoc audit queries --json -w my-wiki # raw JSON for scripting
Token usage: totals + daily breakdown
synthadoc audit cost -w my-wiki # last 30 days
synthadoc audit cost --days 7 -w my-wiki # last 7 days
Audit events: contradictions found, auto-resolutions, cost gate triggers
synthadoc audit events -w my-wiki # last 100 events (shows total)
synthadoc audit events -n 100 --offset 100 -w my-wiki # page 2 (events 101-200)
synthadoc audit events --json -w my-wiki # raw JSON for scripting
Claim citations: source-line provenance for every annotated claim
synthadoc audit citations -w my-wiki # all citations (last 50)
synthadoc audit citations --page alan-turing -w my-wiki # citations for one page
synthadoc audit citations --source turing.pdf -w my-wiki # citations from one source
synthadoc audit citations --broken -w my-wiki # validation failures only
synthadoc audit citations --json -w my-wiki # raw JSON for scriptingScheduling recurring jobs
Relative paths in --op (e.g. raw_sources/) are resolved against the wiki root directory, not the working directory of the shell that runs the schedule. This means they work correctly even when the OS scheduler fires the task with a different working directory (e.g. C:\Windows\System32 on Windows).
# Register a nightly ingest
synthadoc schedule add --op "ingest --batch raw_sources/" --cron "0 2 *" -w my-wiki
Weekly lint
synthadoc schedule add --op "lint run" --cron "0 3 0" -w my-wiki
Bulk-register all jobs declared in [[schedule.jobs]] in config.toml (alternative to schedule add)
See docs/design.md § "schedule sub-commands" for the config.toml format
synthadoc schedule apply -w my-wiki
List scheduled jobs (shows schedule, next run, last run, last result)
synthadoc schedule list -w my-wiki
Remove a scheduled job
synthadoc schedule remove <id> -w my-wiki
Run a scheduled operation immediately and record the result in the audit trail
synthadoc schedule run --op "lint run" -w my-wiki
Show recent scheduled run history
synthadoc schedule history -w my-wiki
synthadoc schedule history --limit 50 -w my-wikiCron expression format: minute hour day-of-month month day-of-week
| Field | Range | Examples | |-------|-------|---------| | minute | 0–59 | 0 = on the hour | | hour | 0–23 | 2 = 2 AM, 22 = 10 PM | | day of month | 1–31 | * = every day | | month | 1–12 | * = every month | | day of week | 0–6 | 0 = Sunday, 1 = Monday |
Common schedules:
| Expression | Meaning | |------------|---------| | 0 2 * | Every day at 2 AM | | 0 22 * | Every night at 10 PM | | 0 3 0 | Every Sunday at 3 AM | | 0 /6 | Every 6 hours | | 30 8 1-5 | Weekdays at 8:30 AM |
Routing
ROUTING.md maps wiki branches to page slugs so queries and ingest jobs are scoped to the relevant section of the wiki. Create it once from your existing index.md, then let Synthadoc maintain it automatically as new pages are added.
# Bootstrap ROUTING.md from current index.md branch structure (run once)
synthadoc routing init -w my-wiki
Report dangling slugs (pages listed in ROUTING.md that no longer exist)
synthadoc routing validate -w my-wiki
Auto-remove dangling slugs from ROUTING.md
synthadoc routing clean -w my-wikiCandidates staging
When staging is enabled, ingest writes new pages to wiki/candidates/ for human review instead of the main wiki. Useful when you want to approve AI-generated pages before they become canonical.
# Show current staging policy
synthadoc staging policy -w my-wiki
Route all new pages to staging (review everything)
synthadoc staging policy all -w my-wiki
Only stage pages below a confidence threshold (auto-promote high-confidence)
synthadoc staging policy threshold --min-confidence high -w my-wiki
Turn staging off (pages go directly to wiki/)
synthadoc staging policy off -w my-wiki
List candidate pages awaiting review
synthadoc candidates list -w my-wiki
Promote a specific page (moves it from candidates/ to wiki/)
synthadoc candidates promote my-page-slug -w my-wiki
Promote all candidates at once
synthadoc candidates promote --all -w my-wiki
Discard a specific candidate
synthadoc candidates discard my-page-slug -w my-wiki
Discard all candidates
synthadoc candidates discard --all -w my-wikiContext packs
A context pack decomposes a goal into sub-questions, runs parallel BM25 searches, and packs the highest-scoring excerpts into a single cited Markdown document within a token budget.
Typical use cases:
- Paste into an external LLM chat (Claude.ai, ChatGPT) as grounded context before asking a question
- Save next to a document you are writing as a cited research brief
- Pipe into another CLI tool that reads Markdown
# Print to terminal — inspect, copy, or pipe
synthadoc context build "How did transistors change computing?" -w my-wiki
Copy to clipboard and paste into an LLM chat (macOS)
synthadoc context build "early computing pioneers" -w my-wiki | pbcopy
Custom token budget (default 4000)
synthadoc context build "Early programming languages" --tokens 8000 -w my-wiki
Save next to a document you are writing
synthadoc context build "Rise of microprocessors" --output ~/drafts/computing-brief.md -w my-wikiExporting
Export your wiki in machine-readable formats for RAG pipelines, LLM context windows, graph analysis tools, and OKF-compliant agents. All formats are assembled server-side with zero additional LLM calls. Requires synthadoc serve to be running.
```bash
Active pages as LLM context (llms.txt spec)
synthadoc export --format llms.txt --status active -w my-wikiFull content dump with provenance footnotes preserved
synthadoc export --format llms-full.txt --output exports/wiki-full.txt -w my-wikiExport wikilink graph as GraphML — open in yEd, Gephi, or Cytoscape
synthadoc export --format graphml --output exports/wiki.graphmREADME truncated. View on GitHub
