paperpipe

The problem: You're implementing a paper. You need the exact equations, want to verify your code matches the math, and your coding agent keeps hallucinating details. Reading PDFs is slow; copy-pasting LaTeX is tedious.

The solution: paperpipe maintains a local paper database with PDFs, LaTeX source (when available), extracted equations, and coding-oriented summaries. It integrates with coding agents (Claude Code, Codex, Gemini CLI) so they can ground their responses in actual paper content.

Typical workflow

# 1. Add papers you're implementing (multiple at once, mixed sources OK)
papi add 2303.08813 1706.03762 "Attention Is All You Need"
Or one at a time
papi add 2303.08813                    # LoRA paper
papi add https://arxiv.org/abs/1706.03762  # URL
papi add "Attention Is All You Need"   # Search by title
2. Check what equations you need to implement
papi show lora --level eq             # prints equations to stdout
3. Verify your code matches the paper
   (or let your coding agent do this via the /papi skill)
papi show lora --level tex            # exact LaTeX definitions
4. Ask cross-paper questions (requires RAG backend)
papi ask "How does LoRA differ from full fine-tuning in terms of parameter count?"
5. Keep implementation notes
papi notes lora                       # opens notes.md in $EDITOR

Installation

# Basic (uv recommended)
uv tool install paperpipe
With features
uv tool install paperpipe --with "paperpipe[llm]"      # better summaries via LLMs
uv tool install paperpipe --with "paperpipe[paperqa]"  # RAG via PaperQA2
uv tool install paperpipe --with "paperpipe[leann]"    # local RAG via LEANN
uv tool install paperpipe --with "paperpipe[figures]"  # figure extraction from LaTeX/PDF
uv tool install paperpipe --with "paperpipe[mcp]"      # MCP server integrations (Python 3.11+)
uv tool install paperpipe --with "paperpipe[all]"      # everything

pip install paperpipe
pip install 'paperpipe[llm]'
pip install 'paperpipe[paperqa]'  # PaperQA2 + multimodal PDF parsing
pip install 'paperpipe[leann]'
pip install 'paperpipe[figures]'        # figure extraction from LaTeX/PDF
pip install 'paperpipe[mcp]'
pip install 'paperpipe[all]'

git clone https://github.com/hummat/paperpipe && cd paperpipe
pip install -e ".[all]"

What paperpipe stores

~/.paperpipe/                         # override with PAPERDBPATH
├── index.json
├── .pqa_papers/                      # staged PDFs for RAG (created on first papi ask)
├── .pqa_index/                       # PaperQA2 index cache
├── .leann/                           # LEANN index cache
├── papers/
│   └── lora/
│       ├── paper.pdf                 # for RAG backends
│       ├── source.tex                # full LaTeX (if available from arXiv)
│       ├── equations.md              # extracted equations with context
│       ├── summary.md                # coding-oriented summary
│       ├── tldr.md                   # one-paragraph TL;DR
│       ├── meta.json                 # metadata + tags
│       ├── notes.md                  # your implementation notes
│       └── figures/                  # extracted figures (if available)
│           ├── figure1.png
│           └── figure2.pdf

Why this structure matters:

• equations.md — Key equations with variable definitions. Use for code verification.
• source.tex — Original LaTeX. Use when you need exact notation or the equation extraction missed something.
• summary.md — High-level overview focused on implementation (not literature review). Use for understanding the approach.
• tldr.md — Quick 2-3 sentence overview of the paper's contribution.
• figures/ — Architecture diagrams, network structures, and result plots extracted from LaTeX source or PDF.
• .pqa_papers/ — Staged PDFs only (no markdown) so RAG backends don't index generated content.

Core commands

| Command | Purpose | |---------|---------| | papi add <id-or-url-or-title>... | Add one or more papers (downloads PDF + LaTeX, generates summary/equations/TL;DR) | | papi add --pdf file.pdf | Add a local PDF or URL | | papi add --from-file list.json | Import papers from a JSON list or text file | | papi list | List papers (filter with --tag) | | papi search "query" | Search across titles, tags, summaries, equations (--rg for grep-style, -p paper1,paper2 to limit scope) | | papi index --backend search | Build/update ranked search index (search.db) | | papi show <paper> --level eq | Print equations (best for agent sessions) | | papi show <paper> --level tex | Print LaTeX source | | papi show <paper> --level summary | Print summary | | papi show <paper> --level tldr | Print TL;DR | | papi export <papers...> --to ./dir | Export context files into a repo (--level summary\|equations\|full) | | papi notes <paper> | Open/print implementation notes | | papi regenerate <papers...> | Regenerate summary/equations/tags/TL;DR | | papi remove <papers...> | Remove papers | | papi ask "question" | Cross-paper RAG query (requires PaperQA2 or LEANN) | | papi index | Build/update the retrieval index | | papi tags | List all tags (--audit to find duplicates, --merge OLD NEW, --delete TAG) | | papi path | Print database location | | papi docs | Print agent integration snippet (for CLAUDE.md/AGENTS.md) | | papi rebuild-index | Rebuild index.json from on-disk paper directories (recovery) |

Run papi --help or papi <command> --help for full options.

Import/Export

Share your paper collection with others or back it up.

Export:

# Export full list to JSON papi list --json > my_papers.json

Export specific tag
papi list --tag "computer-vision" --json > cv_papers.json

Import:

# Import from JSON (preserves custom names and tags) papi add --from-file my_papers.json

Import from text file (one arXiv ID per line)
papi add --from-file paper_ids.txt --tags "imported"
Import from BibTeX file (requires bibtexparser)
papi add --from-file papers.bib
or install with BibTeX support:
uv tool install paperpipe --with "paperpipe[bibtex]"

Title Search:

# Add papers by title (auto-selects if high confidence match) papi add "Attention Is All You Need" papi add "NeRF: Representing Scenes as Neural Radiance Fields"

Semantic Scholar Support:

# Add papers from Semantic Scholar papi add https://www.semanticscholar.org/paper/... papi add 0123456789abcdef0123456789abcdef01234567  # S2 paper ID

Multiple papers at once (mixed sources OK):

papi add 2303.08813 1706.03762 "Attention Is All You Need" papi add 2303.08813 https://www.semanticscholar.org/paper/... "NeRF"

Exact text search (fast, no LLM required):

papi search --rg "AdamW"                # case-insensitive, literal string (default)
papi search --rg --case-sensitive "NeRF"  # match exact case
papi search --rg --regex "AdamW|Lion"   # regex or OR mode (opt-in)

Ranked search (BM25 via SQLite FTS5, no LLM required):

papi index --backend search --search-rebuild    # builds <paper_db>/search.db
papi search "surface reconstruction"             # uses FTS if available (default)
papi search --no-fts "surface reconstruction"    # force in-memory scan (disables FTS, uses fuzzy matching)
papi search --no-fts --exact "exact phrase"      # force scan with exact matching only

Hybrid ranked+exact search:

papi search --hybrid "surface reconstruction"
papi search --hybrid --show-grep-hits "surface reconstruction"

Limit search to specific papers:

papi search "attention" -p attention-is-all-you-need
papi search "loss" -p paper1,paper2,paper3

What are FTS and BM25?

• FTS = Full-Text Search. Here it means SQLite’s FTS5 extension, which builds an inverted index so searches don’t

• BM25 = Okapi BM25, a standard relevance-ranking function used by many search engines. It ranks results based on

References (external):

https://sqlite.org/fts5.html https://en.wikipedia.org/wiki/Okapi_BM25

export PAPERPIPESEARCHMODE=auto   # auto|fts|scan|hybrid

[search]
mode = "auto" # auto|fts|scan|hybrid

Agent integration

paperpipe is designed to work with coding agents. Install the skill and MCP servers:

papi install                          # installs skill + MCP for detected CLIs
or be specific:
papi install skill --claude --codex --gemini
papi install mcp --claude --codex --gemini

After installation, your agent can:

• Use /papi to get paper context (skill)
• Call MCP tools like retrieve_chunks for RAG retrieval
• Verify code against paper equations

Custom skills

| Skill | Description | |--------|-------------| | /papi | Route questions to the cheapest papi command | | /papi-init | Add/update PaperPipe integration in your project's AGENTS.md/CLAUDE.md | | /verify-with-paper | Verify code against paper equations | | /ground-with-paper | Ground responses in paper excerpts | | /compare-papers | Compare multiple papers for a decision | | /curate-paper-note | Create a project note from paper excerpts |

For a ready-to-paste snippet for your repo's agent instructions, run papi docs or see AGENT_INTEGRATION.md.

What the agent sees

When you (or your agent) run papi show <paper> --level eq, you get structured output like:

## Equation 1: LoRA Update
$$h = W0 x + \Delta W x = W0 x + BA x$$
where:
$W_0 \in \mathbb{R}^{d \times k}$: pretrained weight matrix (frozen)
$B \in \mathbb{R}^{d \times r}$, $A \in \mathbb{R}^{r \times k}$: low-rank matrices
$r \ll \min(d, k)$: the rank (typically 1-64)

# Install MCP servers for all supported agents (user scope) papi install mcp

Install for specific agents
papi install mcp --claude
papi install mcp --codex
papi install mcp --gemini
Install repo-local MCP configs (Claude + Gemini) and Codex globally
papi install mcp --repo
Customize embedding model
papi install mcp --embedding text-embedding-3-small

RAG backends (papi ask)

paperpipe supports two RAG backends for cross-paper questions:

| Backend | Install | Best for | |---------|---------|----------| | PaperQA2 | paperpipe[paperqa] | Agentic synthesis with citations (cloud LLMs) | | LEANN | paperpipe[leann] | Local retrieval (Ollama) |

# PaperQA2 (default if installed)
papi ask "What regularization techniques do these papers use?"
LEANN (local)
papi ask "..." --backend leann

The first query builds an index (cached under .pqa_index/ or .leann/). Use papi index to pre-build.

# API keys should be in env
export OPENAIAPIKEY=...
export ANTHROPICAPIKEY=...
export GEMINIAPIKEY=...
export VOYAGEAPIKEY=...
export OPENROUTERAPIKEY=...
Ollama (local) + Ollama embeddings
papi index --backend pqa --pqa-llm ollama/olmo-3:7b --pqa-embedding ollama/nomic-embed-text
GPT + OpenAI Embeddings
papi index --backend pqa --pqa-llm gpt-4.1 --pqa-summary-llm gpt-4.1-mini --pqa-embedding text-embedding-3-small
Gemini + Google Embeddings
papi index --backend pqa --pqa-llm gemini/gemini-3-flash-preview --pqa-embedding gemini/gemini-embedding-001
Claude + Voyage Embeddings
papi index --backend pqa --pqa-llm claude-sonnet-4-5 --pqa-summary-llm claude-haiku-4-5 --pqa-embedding voyage/voyage-4
OpenRouter + Voyage Embeddings
papi index --backend pqa --pqa-llm openrouter/google/gemini-3.5-flash --pqa-embedding voyage/voyage-4

# Ollama (local)
papi ask "how is neus different from nerf?" --backend pqa --pqa-llm ollama/olmo-3:7b --pqa-embedding ollama/nomic-embed-text
GPT
papi ask "how is neus different from nerf?" --backend pqa --pqa-llm gpt-4.1 --pqa-summary-llm gpt-4.1-mini --pqa-embedding text-embedding-3-small
Gemini
papi ask "how is neus different from nerf?" --backend pqa --pqa-llm gemini/gemini-3-flash-preview --pqa-embedding gemini/gemini-embedding-001
Claude
papi ask "how is neus different from nerf?" --backend pqa --pqa-llm claude-sonnet-4-5 --pqa-summary-llm claude-haiku-4-5 --pqa-embedding voyage/voyage-4
OpenRouter
papi ask "how is neus different from nerf?" --backend pqa --pqa-llm openrouter/google/gemini-3.5-flash --pqa-embedding voyage/voyage-4

export OPENAIAPIKEY=...
papi index --backend pqa --pqa-embedding text-embedding-3-small

export GEMINIAPIKEY=...
papi index --backend pqa --pqa-embedding gemini/gemini-embedding-001

export VOYAGEAPIKEY=...
papi index --backend pqa --pqa-embedding voyage/voyage-4

export OPENAIAPIBASE=https://api.voyageai.com/v1
export OPENAIAPIKEY="$VOYAGEAPIKEY"
papi index --backend pqa --pqa-embedding openai/voyage-4

papi ask "..." --backend leann --leann-provider ollama --leann-model qwen3:8b
papi ask "..." --backend leann --leann-host http://localhost:11434
papi ask "..." --backend leann --leann-top-k 12 --leann-complexity 64

# API keys should be in env
export OPENAIAPIKEY=...
export ANTHROPICAPIKEY=...
export GEMINIAPIKEY=...
export VOYAGEAPIKEY=...
export OPENROUTERAPIKEY=...
Ollama (local) + Ollama embeddings
papi index --backend leann --leann-embedding-mode ollama --leann-embedding-model nomic-embed-text
OpenAI + OpenAI embeddings
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model text-embedding-3-small --leann-embedding-api-key $OPENAIAPIKEY
Gemini + Gemini embeddings (OpenAI-compatible)
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model gemini-embedding-001 --leann-embedding-api-base https://generativelanguage.googleapis.com/v1beta/openai/ --leann-embedding-api-key $GEMINIAPIKEY
Voyage embeddings (OpenAI-compatible) — endpoint + key auto-routed from VOYAGEAPIKEY
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model voyage-4

# Ollama (local)
papi ask "how is neus different from nerf?" --backend leann --leann-provider ollama --leann-model olmo-3:7b --leann-index papersollamanomic-embed-text
OpenAI
papi ask "how is neus different from nerf?" --backend leann --leann-provider openai --leann-model gpt-4.1 --leann-api-key $OPENAIAPIKEY --leann-index papersopenaitext-embedding-3-small
Anthropic + Voyage embeddings
papi ask "how is neus different from nerf?" --backend leann --leann-provider anthropic --leann-model claude-sonnet-4-5 --leann-api-key $ANTHROPICAPIKEY --leann-index papersopenaivoyage-4
OpenRouter + Voyage embeddings
papi ask "how is neus different from nerf?" --backend leann --leann-provider openai --leann-model google/gemini-3.5-flash --leann-api-base https://openrouter.ai/api/v1 --leann-api-key $OPENROUTERAPIKEY --leann-index papersopenaivoyage-4
Gemini (OpenAI-compatible)
papi ask "how is neus different from nerf?" --backend leann --leann-provider openai --leann-model gemini-3-flash-preview --leann-api-base https://generativelanguage.googleapis.com/v1beta/openai/ --leann-api-key $GEMINIAPIKEY --leann-index papersopenaigemini-embedding-001

# Ollama (local)
papi index --backend leann --leann-embedding-mode ollama --leann-embedding-model nomic-embed-text
OpenAI
export OPENAIAPIKEY=...
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model text-embedding-3-small --leann-embedding-api-key $OPENAIAPIKEY
Gemini (OpenAI-compatible)
export GEMINIAPIKEY=...
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model gemini-embedding-001 --leann-embedding-api-base https://generativelanguage.googleapis.com/v1beta/openai/ --leann-embedding-api-key $GEMINIAPIKEY
Voyage (OpenAI-compatible) — endpoint + key auto-routed from VOYAGEAPIKEY
export VOYAGEAPIKEY=...
papi index --backend leann --leann-embedding-mode openai --leann-embedding-model voyage-4

[leann]
indexbyembedding = false

[leann] llm_provider = "ollama" llm_model = "qwen3:8b" embedding_model = "nomic-embed-text" embedding_mode = "ollama"

[leann] llm_model = "openrouter/deepseek/deepseek-v4-pro"

papi index --backend leann
Override common LEANN build knobs (maps to leann build ...):
papi index --backend leann --leann-embedding-mode ollama --leann-embedding-model nomic-embed-text
papi index --backend leann --leann-embedding-mode ollama --leann-embedding-host http://localhost:11434
papi index --backend leann --leann-doc-chunk-size 350 --leann-doc-chunk-overlap 128

LLM configuration

paperpipe uses LLMs for generating summaries, extracting equations, and tagging. Without an LLM, it falls back to regex extraction and metadata-based summaries.

# Set your API key (pick one)
export GEMINIAPIKEY=...       # default provider
export OPENAIAPIKEY=...
export ANTHROPICAPIKEY=...
export VOYAGEAPIKEY=...       # for Voyage embeddings (recommended with Claude)
export OPENROUTERAPIKEY=...   # 200+ models
Override the default model
export PAPERPIPELLMMODEL=gpt-4o
export PAPERPIPELLMTEMPERATURE=0.3  # default: 0.3

Local-only via Ollama

export PAPERPIPELLMMODEL=ollama/qwen3:8b
export PAPERPIPEEMBEDDINGMODEL=ollama/nomic-embed-text
Either env var name works (paperpipe normalizes both):
export OLLAMA_HOST=http://localhost:11434
export OLLAMAAPIBASE=http://localhost:11434

Ollama defaults to a small context window (~4k tokens) and silently truncates longer prompts, so a full paper's LaTeX would be cut off. paperpipe avoids this by sizing the context window to each prompt, capped at 32768 tokens. Lower the cap to save memory, or raise it for very long papers:

export PAPERPIPEOLLAMANUMCTX=16384   # or set [llm] ollamanum_ctx in config.toml

This applies to local and Ollama Cloud (ollama/<model>:cloud, after ollama signin) models.

Many reasoning-capable models "think" by default and can spend their whole output budget on hidden reasoning, returning empty content for extraction prompts. paperpipe disables thinking for Ollama models by default since it does structured extraction, not reasoning. Re-enable it if you want:

export PAPERPIPEOLLAMATHINK=true   # or set [llm] ollama_think in config.toml

Via the Claude Code CLI (no API key)

If you have the Claude Code CLI installed and signed in (including subscription/OAuth logins), paperpipe can route summary/equation/tag/title generation through it instead of an API key:

export PAPERPIPELLMMODEL=claude-cli/sonnet   # or claude-cli/opus, claude-cli/<full-model-name>

paperpipe shells out to claude -p for each generation step using the CLI's own authentication. Notes:

• Summarization only. papi ask (PaperQA2/RAG) and embeddings still need an API key or local Ollama — the CLI backend cannot drive them.
• Calls count against your Claude Code usage/rate limits, not API billing.
• Each invocation boots the CLI runtime (~3s), so generation is slower than an API backend.
• temperature is not configurable on this backend.

papi models                    # probe default models for your configured keys
papi models latest             # probe latest model candidates (gpt-5, Gemini via OpenRouter/Gemini, Claude, Voyage 4)
papi models last-gen           # probe previous generation
papi models all                # probe broader superset
papi models --verbose          # show underlying provider errors

Tagging

Papers are auto-tagged from:

• arXiv categories (cs.CV → computer-vision)
• LLM-generated semantic tags (biased toward existing tags for consistency)
• Your --tags flag

papi add 1706.03762 --tags my-project,priority papi list --tag attention

Edit tags on one or more papers (no LLM)
papi regenerate <paper>... --tags nerf,3d        # add
papi regenerate <paper>... --remove-tags 3d      # remove
papi regenerate <paper>... --set-tags nerf,slam  # replace all
papi regenerate <paper>... --clear-tags          # remove all
Edit tags across the whole database
papi tags --audit                  # find duplicate/similar tags
papi tags --merge old-tag new-tag  # rename a tag across all papers
papi tags --delete junk-tag        # remove a tag from all papers

Non-arXiv papers

papi add ./paper.pdf                                       # local PDF (auto-detected)
papi add "https://example.com/paper.pdf"                   # PDF URL (auto-detected)
papi add --pdf ./paper.pdf --title "My Paper" --no-llm     # --pdf for explicit metadata options
papi add --pdf "https://example.com/paper.pdf" --tags siggraph

Configuration file

For persistent settings, create ~/.paperpipe/config.toml (override location with PAPERPIPECONFIGPATH):

[llm]
model = "gemini/gemini-2.5-flash"
temperature = 0.3
ollamanumctx = 32768   # max context window for ollama/* models (default 32768)
ollama_think = false     # enable Ollama "thinking" for reasoning models (default false)
timeout = 120            # per-request seconds; raise for slow local/reasoning models (default 120)
[embedding] model = "gemini/gemini-embedding-001"
[paperqa] settings = "default" indexdir = "~/.paperpipe/.pqaindex" summary_llm = "gpt-4o-mini" enrichment_llm = "gpt-4o-mini" 
agent_llm drives PaperQA2's search agent. PaperQA2's own default is gpt-4o; paperpipe
 
instead inherits the answer llm (above) unless you set this. Set agent_type = "fake" for
 
deterministic, low-token retrieval that skips the agent LLM's tool-calling loop.
 
agent_llm = "gpt-4o-mini"
 
agent_type = "fake"

Optional: override LEANN separately (otherwise it follows [llm]/[embedding] for openai/ollama model ids)
[leann]
llm_provider = "ollama"
llm_model = "qwen3:8b"
embedding_model = "nomic-embed-text"
embedding_mode = "ollama"
[tags.aliases] cv = "computer-vision" nlp = "natural-language-processing"

Precedence: CLI flags > env vars > config.toml > built-in defaults.

Development

git clone https://github.com/hummat/paperpipe && cd paperpipe
pip install -e ".[dev]"
make check                            # format + lint + typecheck + test

# Bump version in pyproject.toml, then:
make release

Credits

• PaperQA2 by Future House — RAG backend.

• LEANN — (local) RAG backend.

License

MIT