NexusRAG supports two document parsers, switchable via NEXUSRAGDOCUMENTPARSER env config:

| Feature | Docling (default) | Marker | |---|---|---| | Math/Formula | Basic (known LaTeX issues) | Superior LaTeX via Surya | | GPU footprint | ~18-20GB VRAM (formula enrichment) | ~2-4GB VRAM | | Formats | PDF, DOCX, PPTX, HTML | PDF, DOCX, PPTX, XLSX, HTML, EPUB | | Chunking | HybridChunker (semantic + structural) | Heading-aware + page-based | | Image extraction | Via Docling pipeline | Via Marker pipeline | | Table extraction | Structured export | Markdown tables |

Both parsers share the same output contract (ParsedDocument) — downstream pipeline (dedup, embedding, KG, retrieval) works identically regardless of parser choice.

Common features across both parsers:

• Structural preservation — Heading hierarchy, page boundaries, paragraph grouping
• Multi-format — PDF, DOCX, PPTX, TXT with consistent output
• Page-aware metadata — Every chunk carries its page number, heading path, and references to images/tables on the same page
• LLM captioning — Images and tables captioned by vision/text LLM for semantic search

# Switch parser in .env NEXUSRAGDOCUMENTPARSER=marker   # or "docling" (default)

| Stage | Technology | Details | |---|---|---| | Vector Embedding | BAAI/bge-m3 | 1024-dim multilingual bi-encoder (100+ languages) | | KG Embedding | Gemini / Ollama / sentence-transformers | Configurable: Gemini (3072d), Ollama, or local sentence-transformers (e.g. bge-m3 1024d) | | Vector Search | ChromaDB | Cosine similarity, over-fetch top-20 candidates | | Knowledge Graph | LightRAG | Entity/relationship extraction, keyword-to-entity matching | | Reranking | BAAI/bge-reranker-v2-m3 | Cross-encoder joint scoring — encodes (query, chunk) pairs together | | Generation | Gemini / Ollama | Agentic streaming chat with function calling |

Why two embedding models? Vector search needs speed (local bge-m3, 1024-dim). Knowledge graph extraction needs semantic richness for entity recognition — choose Gemini Embedding (3072-dim, cloud), Ollama, or sentence-transformers (fully local, no API needed). Each model is optimized for its role.

Retrieval flow:

• Parallel retrieval — Vector over-fetch (top-20) + KG entity lookup run simultaneously
• Cross-encoder reranking — All 20 candidates scored jointly with the query through a transformer (far more precise than cosine similarity alone)
• Filtering — Keep top-8 above relevance threshold (0.15), with fallback to top-3 if all below
• Media discovery — Find images and tables on the same pages as retrieved chunks

Images and tables are embedded into chunk vectors — not stored separately. When the parser extracts an image on page 5, its LLM-generated caption is appended to the text chunks on that page before embedding. This means searching for "revenue chart" finds chunks that contain the chart description, without needing a separate image search index.

Image Pipeline

• Parser (Docling or Marker) extracts images from PDF/DOCX/PPTX (up to 50 per document)
• Vision LLM (Gemini Vision or Ollama multimodal) generates captions: specific numbers, labels, trends
• Captions appended to page chunks: [Image on page 5]: Graph showing 12% revenue growth YoY
• Chunk is embedded → image becomes vector-searchable through its description
• During retrieval, images on matched pages are surfaced as [IMG-p4f2] references

• Parser exports tables as structured Markdown (preserving rows, columns, dimensions)
• Text LLM summarizes each table: purpose, key columns, notable values (max 500 chars)
• Summaries appended to page chunks: [Table on page 5 (3x4)]: Annual sales by region
• Table summaries injected back into document Markdown as blockquotes for the document viewer

Enhance RAG accuracy and organization by attaching custom key-value metadata during document upload:

• Metadata Filtering — Perform hybrid search (semantic + metadata filtering) to narrow down the search space and prevent hallucinations.
• Flexible Organization — Tag documents with attributes like year, category, or author without needing separate workspaces.
• Optimized Retrieval — Pre-filtering in ChromaDB reduces processing time and latency during vector search.
• Supported Endpoints — Pass custommetadata (list of key-values) in the upload API, and metadatafilter in query/chat APIs.

Every answer is grounded in source documents with 4-character citation IDs (e.g., [a3z1]):

• Inline citations — Clickable badges embedded directly in the answer text
• Source cards — Each citation shows filename, page number, heading path, and relevance score
• Cross-navigation — Click a citation to jump to the exact section in the document viewer
• Image references — Visual content cited separately as [IMG-p4f2] with page tracking
• Strict grounding — The LLM is instructed to only cite sources that directly support claims, max 3 per sentence

Interactive force-directed graph built from extracted entities and relationships:

• Entity types — Person, Organization, Product, Location, Event, Technology, Financial Metric, Date, Regulation (configurable)
• Force simulation — Repulsion + spring forces + center gravity with real-time physics
• Pan & zoom — Mouse drag, scroll wheel (0.3x-3x), keyboard reset
• Node interaction — Click to select, hover to highlight connected edges, drag to reposition
• Entity scaling — Node radius proportional to connectivity (degree)
• Query modes — Naive, Local (multi-hop), Global (summary), Hybrid (default)
• No extra services — LightRAG uses file-based storage (NetworkX + NanoVectorDB), zero Docker overhead

Switch between cloud and local models with a single environment variable:

Gemini (Cloud)

| Model | Best For | Thinking | |---|---|---| | gemini-2.5-flash | General chat, fast responses | Budget-based (auto) | | gemini-3.1-flash-lite | High throughput, cost-effective Recommended default| Level-based: minimal / low / medium / high |

Extended thinking is automatically configured — Gemini 2.5 uses thinkingbudgettokens, Gemini 3.x uses thinking_level.

Ollama (Local / Self-hosted)

| Model | Parameters | Tool Calling | Recommendation | |---|---|---|---| | gemma4:e4b | 4.5B effective (8B total) | Native | Recommended default — best quality/size ratio, 128K context, vision + thinking + native tool calling | | gemma4:e2b | 2.3B effective (5.1B total) | Native | Ultra-lightweight, fast responses. Needs thinking enabled for reliable tool calls | | qwen3.5:9b | 9B | Native | Good multilingual support, solid tool calling | | qwen3.5:4b | 4B | Native | Lightweight, works on 8GB RAM. May miss some tool calls | | gemma3:12b | 12B | Prompt-based | Best balance of quality and speed for older Ollama versions |

Tip: Gemma 4 models require Ollama v0.20.0+. NexusRAG auto-detects native tool calling support — models that support it use Ollama's native tools API (more reliable), others fall back to prompt-based tool calling automatically.

Tip: For Knowledge Graph extraction, larger models (12B+) produce significantly better entity/relationship quality. Smaller models (4B) may extract zero entities on complex documents.

Provider switching — Comment/uncomment blocks in .env:

# Cloud (Gemini)
LLM_PROVIDER=gemini
GOOGLEAIAPI_KEY=your-key
Local (Ollama) — uncomment to switch
LLM_PROVIDER=ollama
OLLAMA_MODEL=gemma4:e4b

KG Embedding Providers

The Knowledge Graph embedding model is configured separately from the chat LLM:

| Provider | Config | API Required | Dimension | |---|---|---|---| | Gemini (default) | KGEMBEDDINGPROVIDER=gemini | Google AI API key | 3072 | | Ollama | KGEMBEDDINGPROVIDER=ollama | Ollama server | Varies | | sentence-transformers | KGEMBEDDINGPROVIDER=sentence_transformers | None (fully local) | Model-dependent (e.g. 1024 for bge-m3) |

# Fully local KG embeddings — no API or external service needed
KGEMBEDDINGPROVIDER=sentence_transformers
KGEMBEDDINGMODEL=BAAI/bge-m3
KGEMBEDDINGDIMENSION=1024

Tip: sentence_transformers reuses the same BAAI/bge-m3 model already downloaded for vector search — zero extra disk space, zero API costs, fully offline.

The chat system uses a semi-agentic architecture with real-time SSE streaming:

• Agent steps — Visual timeline: Analyzing → Retrieving → Generating → Done (with live timers)
• Extended thinking — Gemini/Ollama reasoning displayed in a collapsible panel
• Function calling — 3-tier: Native (Gemini), Native (Ollama — Gemma 4, Qwen 3.5), or prompt-based fallback (older models). Auto-detected via probe
• Force-search mode — Pre-retrieval before LLM generation for guaranteed grounded answers
• Heartbeat — 15s SSE keepalive prevents TCP timeout on slow responses
• Fallback — If Ollama produces empty output, auto-triggers search + retry
• Chat history — Persistent per workspace with message ratings (thumbs up/down)

Theme & Layout

• Dark / Light mode with smooth transition, persisted preference
• Collapsible sidebar with workspace navigation (icon-only mode at narrow width)
• Responsive grid layouts — mobile to desktop

• Streaming token rendering with memoized paragraph blocks (only active block re-renders)
• Inline citation badges with hover tooltips (source file, page, heading path, relevance %)
• Agent step timeline with spinner animations and elapsed timers
• Thinking panel — scrollable, auto-follow, collapsible after completion
• Code blocks with syntax highlighting (Python, JS, SQL, etc.) and one-click copy

• Drag-and-drop upload (PDF, DOCX, PPTX, TXT, MD — up to 50MB)
• Status badges with shimmer animation during processing
• Per-document chips: pages, chunks, images, tables, file size, processing time

• 4 query modes: Hybrid, Vector, Local KG, Global KG
• Adjustable result count (1-20) with slider + direct input
• Document scope filtering (multi-select)
• Relevance score bars with color coding (green / amber / red)

• Stat cards: documents, indexed, chunks, images, entities, relationships
• Entity type distribution bar with animated widths
• Top entities ranked by connectivity
• Per-document chunk breakdown chart

• Framer Motion animations throughout (staggered entrances, layout transitions)
• Loading skeletons, toast notifications, empty state illustrations
• Keyboard shortcuts: / to focus search, Enter to send, Escape to cancel

• Multiple isolated knowledge bases, each with its own documents, ChromaDB collection, and KG
• Custom system prompt per workspace (override default Q&A behavior)
• Independent chat history with message persistence and ratings

16 tests across 6 categories using 8 rule-based metrics (keyword coverage, refusal accuracy, citation format, language match, etc.) — no LLM judge involved.

| Category | Pass Rate | Avg Score | |---|---|---| | Fact Extraction (VI + EN) | 5/5 | 0.93 | | Table Data | 2/3 | 0.83 | | Cross-Document Reasoning | 2/2 | 0.89 | | Anti-Hallucination | 3/3 | 1.00 | | Multi-turn History | 2/2 | 0.87 | | Citation Accuracy | 1/1 | 0.85 | | Overall | 15/16 | 0.89 — EXCELLENT |

30 auto-generated Q&A pairs evaluated by Gemini 2.0 Flash as RAGAS judge. Same questions tested on both models:

| Metric | gemma3:12b (local) | gemini-2.5-flash (cloud) | Winner | |---|---|---|---| | Overall score | 0.832 | 0.846 | Gemini | | Pass rate | 25/30 (83%) | 26/30 (87%) | Gemini | | Faithfulness | 0.749 | 0.812 | Gemini (+0.063) | | Factual correctness | 0.773 | 0.749 | gemma3 (+0.024) | | Context recall | 0.833 | 0.833 | Tie | | Table extraction | 0.697 | 0.905 | Gemini (+0.208) | | Avg latency | 3076ms | 3283ms | gemma3 (-207ms) |

| Aspect | Status | Detail | |---|---|---| | Anti-hallucination | :green_circle: Strong | Perfect refusal on out-of-scope questions | | Citation format | :green_circle: Strong | 100% correct format across all tests | | Cross-doc reasoning | :green_circle: Strong | Successfully synthesizes across multiple sources | | Table parsing | :yellow_circle: Model-dependent | gemma3 fails complex tables; Gemini handles well | | Language consistency | :yellow_circle: Model-dependent | gemma3 occasionally responds in wrong language | | Retrieval coverage | :redcircle: Weak | 5 cases with contextrecall = 0 (specific facts missed by retrieval) | | Faithfulness | :red_circle: Weak | 4 FAIL cases — LLM adds unsupported details when elaborating |

Full evaluation methodology and per-sample results: ragevaluation_report.md

Upcoming model benchmarks on the same 30 RAGAS test suite:

| Model | Type | Status | |---|---|---| | gemma3:12b | Local (Ollama) | :whitecheckmark: Done | | gemini-2.5-flash | Cloud (Google AI) | :whitecheckmark: Done | | qwen3.5:4b | Local (Ollama) | :hourglass: Planned | | qwen3.5:9b | Local (Ollama) | :hourglass: Planned | | gemini-3.1-flash-lite | Cloud (Google AI) | :hourglass: Planned |

Goal: compare cost-efficiency (local 4B/9B) vs cloud quality across faithfulness, table extraction, and multilingual consistency.

| Technology | Purpose | |---|---| | FastAPI | Async web framework with SSE streaming | | SQLAlchemy 2.0 | Async ORM with PostgreSQL (asyncpg) | | ChromaDB | Vector store — cosine similarity, per-workspace collections | | LightRAG | Knowledge graph — entity extraction, multi-hop queries | | Docling / Marker | Document parsing — PDF, DOCX, PPTX, HTML with structural extraction (switchable via config) | | sentence-transformers | BAAI/bge-m3 embeddings + BAAI/bge-reranker-v2-m3 reranking | | google-genai | Gemini API — chat, vision, function calling, extended thinking | | ollama | Local LLM — tool calling via prompt tags, multimodal support |

| Technology | Purpose | |---|---| | React 19 + TypeScript 5.9 | UI framework with strict typing | | Vite 7 | Dev server and production bundler | | TailwindCSS 4 | Utility-first styling with dark / light theme | | Zustand 5 | Lightweight state management | | React Query 5 | Async data fetching, caching, and mutations | | Framer Motion 12 | Layout animations, transitions, staggered entrances | | react-markdown + KaTeX | Rich markdown with LaTeX math rendering | | Lucide React | Icon library |

| Technology | Purpose | |---|---| | PostgreSQL 15 | Document metadata, chat history, workspace config | | ChromaDB | Vector embeddings (HTTP client, containerized) | | LightRAG | File-based KG (NetworkX + NanoVectorDB — no extra services) | | Docker Compose | Full-stack deployment (4 containers) | | nginx | Production frontend serving + API/SSE reverse proxy |

| Resource | Minimum | Recommended | |---|---|---| | RAM | 4 GB | 8 GB+ | | Disk | 5 GB | 10 GB+ | | Python | 3.10+ | 3.11+ | | Node.js | 18+ | 22 LTS | | Docker | 20+ | Latest |

Copy .env.example and configure:

cp .env.example .env

Required

| Variable | Description | |---|---| | GOOGLEAIAPI_KEY | Google AI API key (required for Gemini provider) |

LLM

| Variable | Default | Description | |---|---|---| | LLM_PROVIDER | gemini | gemini or ollama | | LLMMODELFAST | gemini-2.5-flash | Model for chat and KG extraction | | LLMTHINKINGLEVEL | medium | Gemini 3.x thinking: minimal / low / medium / high | | LLMMAXOUTPUT_TOKENS | 8192 | Max output tokens (includes thinking) | | OLLAMA_HOST | http://localhost:11434 | Ollama server URL | | OLLAMA_MODEL | gemma3:12b | Ollama model name |

KG Embedding

| Variable | Default | Description | |---|---|---| | KGEMBEDDINGPROVIDER | gemini | gemini, ollama, or sentence_transformers | | KGEMBEDDINGMODEL | text-embedding-004 | Model name (provider-specific) | | KGEMBEDDINGDIMENSION | 3072 | Embedding dimension (must match model) |

RAG Pipeline

| Variable | Default | Description | |---|---|---| | NEXUSRAGEMBEDDINGMODEL | BAAI/bge-m3 | Embedding model (1024-dim) | | NEXUSRAGRERANKERMODEL | BAAI/bge-reranker-v2-m3 | Cross-encoder reranker | | NEXUSRAGVECTORPREFETCH | 20 | Candidates before reranking | | NEXUSRAGRERANKERTOP_K | 8 | Final results after reranking | | NEXUSRAGENABLEKG | true | Enable knowledge graph extraction | | NEXUSRAGDOCUMENTPARSER | docling | Document parser: docling (default) or marker (lighter, better math) | | NEXUSRAGMARKERUSE_LLM | false | Enable LLM-enhanced mode for Marker (better tables, equations) | | NEXUSRAGENABLEIMAGE_EXTRACTION | true | Extract images from documents | | NEXUSRAGENABLEIMAGE_CAPTIONING | true | LLM-caption images for search | | NEXUSRAGKGLANGUAGE | Vietnamese | KG extraction language |

| Method | Endpoint | Description | |---|---|---| | GET | /workspaces | List all workspaces | | POST | /workspaces | Create workspace | | PUT | /workspaces/{id} | Update workspace | | DELETE | /workspaces/{id} | Delete workspace + all data |

| Method | Endpoint | Description | |---|---|---| | POST | /documents/upload/{workspaceid} | Upload file (supports custommetadata) | | GET | /documents/{id}/markdown | Get parsed content | | GET | /documents/{id}/images | List extracted images | | DELETE | /documents/{id} | Delete document |

| Method | Endpoint | Description | |---|---|---| | POST | /rag/query/{workspaceid} | Hybrid search (supports metadatafilter) | | POST | /rag/chat/{workspaceid}/stream | Agentic streaming chat (SSE) (supports metadatafilter) | | GET | /rag/chat/{workspace_id}/history | Chat history | | POST | /rag/process/{document_id} | Process document | | GET | /rag/graph/{workspace_id} | Knowledge graph data | | GET | /rag/analytics/{workspace_id} | Full analytics |