jgouviergmail
LIA-Assistant
Python

Smart multi-agent conversational assistant with LangGraph orchestration, Human-in-the-Loop, enterprise-grade observability, and full i18n support (6 languages)

Last updated Aug 8, 2026
37
Stars
4
Forks
11
Issues
+1
Stars/day
Attention Score
49
Language breakdown
Python 77.0%
TypeScript 20.9%
CSS 0.7%
Shell 0.6%
PowerShell 0.4%
JavaScript 0.3%
โ–ธ Files click to expand
README

LIA โ€” Your life. Your AI. Your rules.

LIA

Smart multi-agent conversational assistant with LangGraph orchestration, Human-in-the-Loop, enterprise-grade observability, and full i18n support (6 languages)

If you find my project and work valuable, I would be grateful for a star on GitHub. Thank you !

Try LIA    GitHub Stars

Python 3.12+ Node.js 24 LTS FastAPI Next.js 16 LangGraph LangChain 6 languages 360ยฐ technical audit: 8.3/10 on the normalized 24-area grid โ€” full public report License Ask DeepWiki

Features โ€ข Admin & Monitoring โ€ข Quick Start โ€ข Architecture โ€ข Documentation โ€ข Contributing

Version 1.29.0 โ€” LIA installs on your machine, shows itself without lying, and knows what it is allowed to spend (ADR-215 โ†’ ADR-218). Four decisions answering four questions that had no answer: how does someone else run this, how do they see it first, how much can one instance spend, and what can an operator switch off without redeploying. A guided installer (./install.sh) builds the images from your own checkout โ€” the v1 default, because an unqualified prebuilt artifact has no place in an installation. Prebuilt mode exists but accepts only repository@sha256:โ€ฆ references from a manifest marked qualification="passed": a mutable tag is never an installer input. Secrets enter through stdin only โ€” one JSON document creates the admin (through the real password authority, no default password) and encrypts the provider keys in a single transaction; nothing goes through argv, nothing lands in resume state. Reference data applies in one transaction, one psql, ONERRORSTOP=1 with a blocking verification file and a marker written in the same transaction โ€” a half-seeded install does not exist. And /ready is never sufficient: a secret-free verifier checks the single Alembic head, the exact marker, reference-data postconditions, an active admin, decryptable keys, and provider coverage on the post-seed effective configuration โ€” the one the first message will actually use. An instance now knows its daily budget: per-user limits bounded what one account consumes and nothing bounded what a deployment spends, because N accounts ร— their quota is unbounded. A daily UTC ledger takes each run's cost through an atomic UPSERT with column arithmetic inside the very transaction that persists the token summary โ€” measured in-container, three concurrent 0.30 โ‚ฌ runs give exactly 0.900000 โ‚ฌ and run_count = 3. Two bounds compose and the smaller wins: an operator may tighten what the deployment allows, never widen it. Where per-user limits fail open, an unknown instance spend fails closed โ€” at worst one message too many on one side, at worst the whole budget on the other โ€” and a dedicated error code carries a Retry-After to the next UTC midnight instead of the false "contact your administrator". Ten platform capabilities (dictation, speech, images, uploads, document spaces, web search, browsing, skills, MCP, telephony) now switch off from the admin panel in two seconds, each declaring the mode by which it is really enforced โ€” tools vanish from the planner's catalogue, a route refuses with a stable code, or an internal chokepoint cuts the source. Speech synthesis has no route at all: the first draft that called it route-enforced was wrong, and only checking the real wiring showed it. Three guards that recalculate found three faults green tests could not see: speech synthesis was billed and never counted against the ceiling; /auth/google/login was public and bypassed the newly mandatory terms acceptance while putting a real Google identity on a public instance; and eleven connector paths bound a real credential with no guard at all. Each guard was then deliberately broken to prove it goes red. 18,206 backend + 5,448 frontend tests, all six languages. โ€” 8 August 2026.


Table of Contents


Why LIA?

LIA solves the fundamental problems of today's AI assistants:

| Problem | LIA Solution | | ------------------------------- | -------------------------------------------------------------------------------------------- | | Unpredictable LLM costs | Real-time token tracking, budget alerts, 93% optimization | | Uncontrolled hallucinations | Human-in-the-Loop (HITL) with 6 approval levels | | Fragmented integrations | Unified multi-domain orchestration (20+ agents + MCP + sub-agents) | | Limited observability | 450+ Prometheus metrics, 26 Grafana dashboards (including a product-value cockpit), email alerting with runbooks, GeoIP analytics | | Inconsistent performance | Gemini embedding-001 with asymmetric task types, semantic routing with hybrid scoring |

Primary Use Cases

๐Ÿ“… "Find my meetings for tomorrow and send a reminder to all participants"
๐Ÿ“ง "Summarize my unread emails from this week that have attachments"
๐Ÿ‘ฅ "Update the companies of my contacts who work at startups"
๐Ÿ”” "Remind me tomorrow at 9am to call Marie for her birthday"

Try LIA Online

Try LIA

LIA is available as a hosted service at https://lia.jeyswork.com/ โ€” no installation required.

Interactive showroom โ€” lia.jeyswork.com/demo runs six guided synthetic missions, one per differentiating mechanism: orchestration under approval, proactivity, persistent memory, outbound calls, rich replies, and in-app configuration. Pick one, watch LIA read its sources, then approve, edit, or refuse each prepared change through the real approval UI โ€” and read LIA's closing reply rendered by the production rich-HTML pipeline. Everything is clearly labeled synthetic โ€” no account, model, or external service is contacted, and a proof drawer links every visible capability to its exact source. Self-hosting follows the Quick Start below โ€” manually, or through the guided ./install.sh (ADR-215); until its disposable clean-machine qualification gates pass, prebuilt-image installation stays locked behind a qualified release manifest.

Closed beta: Access is currently limited to a restricted number of users, at the administrator's discretion. To request an invitation, contact liamyassistant@gmail.com.

Built by an AI, Directed by a Human

"Speed comes from the AI. Quality comes from the framework."

Nearly 100% of this codebase was written by an AI, under human direction: a written engineering rulebook, blocking automated checks, systematic review, adversarial audits. The result is measured, not proclaimed:

| | | | | | ------------------------- | --------------------------------------- | --------------------------- | ----------------------------------------------------------------------- | | 39 functional domains | 548,000 lines of code (excl. tests) | 23,600+ automated tests | 217 ADRs | | 197 versions shipped | 6 languages, parity enforced in CI | 466 Prometheus metrics | 8.3/10 technical audit, 24 normalized areas |

  • The full story โ€” method, trade-offs, results and what remains to be done, weaknesses included: lia.jeyswork.com/story
  • The audit itself โ€” 24 normalized areas mapped to ISO/IEC 25010:2023, every score backed by executed evidence, 7 open worksites included, with the protocol and the full standalone report: docs/audit/

Screenshots

Dashboard โ€” Homepage with usage statistics
Dashboard โ€” Homepage with quick access, usage statistics, and personalized greeting

Chat โ€” Multi-agent conversation with debug panel
Chat โ€” Multi-agent conversation with real-time debug panel (right sidebar)

More screenshots

Chat โ€” Debug panel detail
Chat โ€” Debug panel: per-message routing, tool calls, token cost and reasoning timeline

Chat โ€” Interactive skill widgets
Chat โ€” Interactive skill widgets: maps, dashboards, calendars and mini-apps rendered inline

Settings โ€” Preferences (connectors, MCP, themes)
Settings โ€” Preferences: connectors, MCP servers, language, timezone, and themes

Settings โ€” Features (memory, interests, notifications)
Settings โ€” Features: LIA Style, long-term memory, interests, proactive notifications, scheduled actions, sub-agents, channels

Settings โ€” Long-term memory
Settings โ€” Long-term memory: pinned facts, automatic extraction, edit / delete / pin per memory

Settings โ€” Psyche Engine
Settings โ€” Psyche Engine: Big Five personality traits modulating the assistant's emotional responsiveness

Settings โ€” Administration panel
Settings โ€” Administration: LLM config, RAG Spaces, users, connectors, pricing, skills, voice, broadcast, debug

Settings โ€” One-click administration
Administration โ€” One-click simplicity: every admin action is accessible in a single click, no technical skills required

Settings โ€” LLM Configuration with multi-provider support
Administration โ€” LLM Configuration: 7 providers (OpenAI, Anthropic, Google Gemini, DeepSeek, Qwen, Perplexity, Ollama), per-node model selection

FAQ โ€” Searchable help center
FAQ โ€” Searchable help center with categorized Q&A sections


Features

Multi-Agent Intelligence (LangGraph 1.x)

  • 20+ Specialized Agents: Contacts, Emails, Calendar, Drive, Tasks, Reminders, Places, Routes, Weather, Wikipedia, Perplexity, Brave, Web Search, Web Fetch, Browser Control (with progressive screenshot streaming), Smart Home (Philips Hue), Context, Query + dynamic MCP agents
  • ReAct Execution Mode (ADR-070): Alternative to the pipeline โ€” the LLM iteratively reasons about tool outputs and decides next steps autonomously. User-toggleable preference, 4-node LangGraph architecture with native HITL support, timeout enforcement, cross-domain initiative via prompt engineering. Supports all tools including MCP and Skills
  • MCP (Model Context Protocol): Per-user external tool servers with OAuth 2.1, SSRF protection, structured items parsing, MCP Apps (interactive iframe widgets), Iterative Mode (ReAct) for complex servers โ€” a dedicated agent reads docs then calls tools correctly
  • Agent Initiative Phase: Post-execution cross-domain enrichment โ€” the assistant proactively verifies related information (e.g., weather shows rain โ†’ checks calendar for outdoor events). Prompt-driven, read-only, fully configurable
  • Skills (agentskills.io) with Rich Outputs: Open standard for expert instructions (SKILL.md), model-driven activation, progressive disclosure (L1/L2/L3), sandboxed scripts, marketplace import, auto-translated multi-language descriptions, ZIP download, admin management. Rich Skill Outputs (v1.16.8): skills can return interactive HTML frames (iframe srcDoc or external URL) and/or images in addition to text, via a simple JSON contract (SkillScriptOutput). Automatic theme & locale sync (theme switch propagates live to frames via postMessage), iframe auto-resize, CSP-sandboxed client-side interactivity (addEventListener, crypto.getRandomValues), bundled segno for QR codes. Seven built-in rich skills: interactive-map, weather-dashboard, calendar-month, qr-code, pomodoro-timer, unit-converter, dice-roller. Planner skill guard: multi-domain deterministic skills are protected from false-positive early clarification requests via domain overlap detection (haspotentialskillmatch). Built-in Skill Generator: create custom skills in natural language โ€” the assistant guides you through need analysis and archetype selection (the dialogue keeps its context across turns), then validates and installs the finished skill directly into My Skills, announced by name and immediately usable. Every import path (chat-generated or manual upload) goes through one hardened pipeline: strict name validation, zip-expansion caps, name-conflict rejection, atomic install with automatic rollback
  • Agentic Telephony (ADR-127): LIA places real outbound phone calls on your behalf via your own per-user ElevenLabs + Twilio connector (BYO โ€” zero cost on LIA's side). Every call is HITL-confirmed before dialing; the goal-driven voice agent greets the instant the line opens, resolves relative dates against a live temporal anchor, and hangs up when done. Privacy by capability: the call agent can only read free/busy availability โ€” never event titles or contents; no recording, no stored transcript. A strict mandate boundary forbids any expense or commitment beyond the objective (offers are captured with their price and deferred to you), and the asynchronous post-call summary must state every cost and flag every open point. Config self-heals: fingerprint-based lazy re-sync of the vendor agent, self-healing one-active-call guard (vendor status probe, deleted-conversation 404 handling), pinned thinking-free agent LLM, telephony-native ulaw8000 audio
  • AI Image Generation & Editing: Generate images from text prompts (gpt-image-1), edit existing images with natural language instructions. Multi-provider factory architecture, per-user quality/size preferences, cost tracking with DB-cached pricing, attachment-based storage with cascade cleanup
  • File Attachments (Images, PDF): Upload with client-side compression, configurable LLM vision analysis, PDF text extraction, strict per-user isolation
  • Response Grounding on Recent Entities (ADR-147): on a turn that produces no tool data, the response model is re-grounded on the most recent entities already in state (zero I/O, age-bounded, explicitly non-authoritative) instead of paraphrasing older prose โ€” and the prompt forbids inventing an entity attribute rather than admitting it is unknown
  • Semantic Routing: Binary classification with confidence scoring (high >0.85, medium >0.65)
  • Multi-Step Planning: ExecutionPlan DSL with dependencies and conditions
  • Parallel Execution: asyncio.gather for independent domains
  • Intelligent Context Compaction: LLM-based conversation history summarization when token count exceeds dynamic threshold (ratio of response model context window). Preserves identifiers (UUIDs, URLs, emails). /resume command for manual trigger. 4 HITL safety conditions prevent compaction during active approval flows
  • Scroll-up History Pagination: GET /conversations/me/messages exposes a keyset cursor (?before=<createdat>) with hasmore / nextcursor. The chat UI binds an IntersectionObserver on a top sentinel โ€” older pages prepend with id-based dedup, scroll position preserved via a shared wasPrependRef that skips the auto-scroll-to-bottom for that cycle. Conversations of any length stay fully reachable; the existing (conversationid, createdat DESC) composite index makes each page an index-only seek. Bounds env-tunable (CONVERSATIONHISTORYDEFAULTLIMIT / MAXLIMIT)

Psyche Engine โ€” Dynamic Emotional Intelligence

  • 5-Layer Psychological State: Big Five personality traits (permanent) โ†’ PAD mood space with 14 moods (hours) โ†’ 22 discrete emotions with cross-suppression (minutes) โ†’ 4-stage relationship progression (weeks) โ†’ curiosity/engagement drives (per-session)
  • Show, Don't Tell: Mood and emotions subtly influence word choice, sentence rhythm, energy level, and relational tone โ€” the assistant never declares "I'm feeling happy"
  • Emotional Avatar: Mood-responsive emoji with colored ring on each message. Historical avatars persisted per-message for reload consistency
  • Evolution Awareness: The assistant knows how its mood shifted since the last message, providing narrative continuity
  • 4-Chart Dashboard: Interactive recharts visualization of mood (PAD), emotions, relationship, and drives over time (24h to 90 days)
  • Education Guide: 7-section interactive documentation explaining every layer, with descriptive tables for 14 moods and 22 emotions
  • Customizable Temperament: Expressiveness (stoic โ†’ highly expressive) and stability (volatile โ†’ very stable) sliders. Soft reset (mood only) and full reset (everything) with explicit scope descriptions
  • Global Injection: Behavioral directives injected via template variables into all user-facing text generation (response, notifications, reminders, voice) within semantic XML blocks (<InnerState purpose="tone-calibration">)
  • Safety Guardrail: Explicit instruction prevents the LLM from projecting its own emotional state onto the user
  • Self-Report: Zero-cost emotion tracking via hidden <psyche_eval/> tag โ€” no additional LLM call

Voice: Input & Output

Voice Input (STT)

  • Push-to-Talk: Hold microphone button to speak, release to transcribe. Optimized for mobile (anti-long-press CSS, touch gesture handling)
  • Wake Word: Say "OK Guy" to activate hands-free recording. Sherpa-onnx WASM (Whisper Tiny.en) runs entirely in-browser โ€” no audio sent externally for wake word detection
  • Per-User Language: STT transcription uses the user's preferred language setting (Whisper Small, 99+ languages, fully offline)
  • Latency Optimized: Mic stream reuse, WebSocket pre-warming, parallel setup, cached AudioWorklet (~50-100ms wake-to-record)
Voice Output (TTS)

| Provider | Models | Cost | Latency (TTFA) | Notes | | --------------------------- | ------------------------ | ---------------------- | -------------- | --------------------------------------------------------- | | Edge TTS (Microsoft Neural) | edge-tts | Free | ~250 ms | Multilingual neural voices, free fallback | | OpenAI TTS | tts-1 / tts-1-hd | $15 / $30 per 1M chars | ~500 ms | 6 stable voices (alloy, echo, fable, onyx, nova, shimmer) | | ElevenLabs TTS | elevenmultilingualv2 | $100 / 1M chars | ~300 ms | High-quality multilingual, Voice Library access | | | eleventurbov2_5 | $50 / 1M chars | ~250 ms | Sweet-spot quality / latency | | | elevenflashv2_5 | $50 / 1M chars | ~75 ms | Ultra-low-latency for conversational agents |

  • Catalogue-driven (ADR-081): provider/model/voice are admin-controlled via Configuration LLM (LLM type voicetts). Voice + tuning live in providerconfig JSONB. No env vars to maintain across deployments.
  • Sentence streaming (ADR-082): TTS runs sentence-by-sentence pipelined with the LLM stream. First audio lands in ~1 s on chat mode (was ~5 s).
  • Per-message cost transparency: ๐Ÿ”Š N chars ยท โ‚ฌX.XXX badge on the assistant bubble (paid providers only โ€” Edge stays badge-free as it's $0).
  • Graceful degradation: missing API key on a paid provider transparently falls back to Edge with a structured warning log.
  • Persistent HTTP pool on ElevenLabs: keep-alive across sentences saves ~100โ€“300 ms TLS handshake per call.

FOR_EACH Iteration Pattern

# DSL Syntax
ExecutionStep(
    toolname="sendemail",
    foreach="$steps.getcontacts.contacts",
    foreachmax=10
)
  • Per-parent correlation identity: an enrichment fetched for one iteration belongs to that iteration โ€” results keyed on content alone (weather โ†’ place + day) used to collide between two parents sharing them, silently costing one of them its data; ids are now derived per parent and stay stable across replays and resumed checkpoints
  • HITL Thresholds: Mutations >= 1 trigger mandatory approval
  • Bulk Operations: Send emails, update contacts, mass deletions

Smart Services (Token Savings 89%)

| Service | Role | Optimization | | --------------------- | ------------------------ | ---------------------- | | QueryAnalyzerService | Routing decision | LRU Cache | | SmartPlannerService | ExecutionPlan generation | Pattern Learning | | SmartCatalogueService | Tool filtering | 96% token reduction | | PlanPatternLearner | Bayesian learning | Bypass >90% confidence |

Google Integrations (OAuth 2.1 + PKCE)

  • Gmail: Search, read, send, reply, trash
  • Contacts: Fuzzy search, list, details (14+ schemas)
  • Calendar: Search, create, update events
  • Drive: Search, file/folder listing
  • Tasks: Full CRUD with completion

Apple iCloud Integrations

  • Apple Mail: Search, read, send, reply, forward, trash (IMAP/SMTP)
  • Apple Calendar: Search, create, update, delete events (CalDAV)
  • Apple Contacts: Search, list, create, update, delete (CardDAV)

Microsoft 365 Integrations (OAuth 2.0 + PKCE)

  • Outlook: Search, read, send, reply, forward, trash (Graph API)
  • Calendar: Search, create, update, delete events (calendarView)
  • Contacts: Search, list, create, update, delete
  • To Do: Full CRUD with completion (task lists + tasks)
  • Multi-tenant: Personal accounts (outlook.com) and business accounts (Azure AD) via tenant=common

3-Way Mutual Exclusivity

  • Only one provider per functional category (email, calendar, contacts, tasks)
  • 3 supported providers: Google, Apple, Microsoft
  • Activating a new provider automatically deactivates the active competitor

Smart Home โ€” Philips Hue

  • Voice-controlled lighting: Turn lights on/off, adjust brightness and colors via natural language
  • Room & scene management: Control entire rooms or activate predefined scenes ("dim the living room", "activate movie mode")
  • Local or cloud connection: Connect via local bridge IP or Philips Hue cloud API
  • Feature flag: PHILIPSHUEENABLED=true to enable

Human-in-the-Loop (HITL)

| Type | Trigger | Severity | | ------------------- | -------------------------------------------------------------- | -------- | | Plan Approval | Destructive actions | CRITICAL | | Clarification | Detected ambiguity | WARNING | | Draft Critique | Email/Event review | INFO | | Destructive Confirm | Deletion of >= 3 items | CRITICAL | | FOR_EACH Confirm | Bulk mutations | WARNING | | Modifier Review | Review and approve AI-suggested modifications to draft content | INFO |

Note: the plan-approval level is currently auto-approved โ€” tool-level HITL supersedes it
(see ADR-106); the other five levels
interrupt execution and wait for the user.

Enterprise Observability

  • Prometheus: 425 custom metrics (agents, LLM, infrastructure)
  • Grafana: 26 production-ready dashboards
  • Langfuse: LLM-specific tracing with prompt versions
  • Loki: Structured JSON logs with PII filtering
  • Tempo: Distributed cross-service tracing
  • Probes: liveness (GET /health, always 200 while the process serves โ€” what Docker healthchecks poll) split from readiness (GET /ready, 503 unless PostgreSQL and Redis answer) โ€” ADR-115
  • Alerting: a 14-alert vital core (service/DB/Redis down, disk, container OOM, 5xx rate, SSE latency, backup failure, public-endpoint & TLS-certificate probes, chain self-monitoring) evaluated by Prometheus and emailed by a dedicated Alertmanager โ€” unit-tested with promtool test rules, every alert linking its runbook โ€” ADR-119

Cost Tracking & Billing

| Type | Tracking | Export | | -------------- | ---------------------- | ------------ | | LLM Tokens | Per node, per provider | Detailed CSV | | Google API | Per endpoint, per user | Detailed CSV | | Aggregated | Per user, per period | CSV summary |

  • Google Maps Platform: Places, Routes, Geocoding, Static Maps
  • Dynamic Pricing: Admin UI for full LLM catalogue CRUD โ€” provider, 8 capability flags (max input/output tokens, tools, structured output, strict mode, streaming, vision, reasoning) and pricing per model, all stored in the database. Same surface for image generation models (provider + quality/size/pricing). Cross-worker cache invalidation via Redis Pub/Sub (ADR-063), live cross-sibling refresh in the frontend โ€” no code change, no redeploy
  • ContextVar Pattern: Implicit tracking without explicit parameter passing
  • Admin CSV Exports: Token usage, Google API usage, Consumption summary (all users or filtered by user)
  • User CSV Exports (v1.9.1): Personal consumption export in Settings > Features โ€” users export their own data only (user_id forced server-side, IDOR-safe)

Security & Compliance

  • OAuth 2.1: PKCE (S256), single-use state token
  • BFF Pattern: HTTP-only cookies, server-side Redis sessions (fixed 7-day lifetime, 30 with remember-me)
  • Strong authentication: WebAuthn passkeys (discoverable credentials, conditional UI on the login form, single-use challenges, clone detection, zero enumeration) + TOTP second factor (two-step login, matched-timestep anti-replay, 10 single-use backup codes revealed once); step-up re-authentication guards every sensitive action behind a 5-minute sudo window (typed 403 contract; password, code, passkey, or provider re-sign-in). Feature flag: MFA_ENABLED=true โ€” ADR-143
  • Device sessions: "My devices" lists every live session with deliberately bounded metadata (browser/OS families, truncated IP, coarse last-seen), per-device revocation and step-up-guarded revoke-others; a revoked session's SSE streams close within one keepalive tick; new-login push alerts with FCM device attestation โ€” ADR-144
  • Encryption: Fernet (credentials), bcrypt (passwords)
  • GDPR: Automatic PII filtering, pseudonymization, and full-account export (Art. 20): durable jobs build a ZIP (JSON + readable Markdown + uploaded files) from a total data classification where secret tables are unexportable by construction. Feature flag: ACCOUNTEXPORTENABLED=true โ€” ADR-145
  • Offline PWA: one unified service worker serves push and a branded 6-language offline page; /api/ is never cached โ€” ADR-146
  • Per-User Usage Limits: Token, message, and cost quotas (period/global) with 5-layer defense-in-depth enforcement, admin kill switch, real-time dashboard with WebSocket gauges. Feature flag: USAGELIMITSENABLED=true
  • Instance Daily Spend Ceiling: a durable UTC ledger caps what the whole deployment may spend in a day, not what one account consumes โ€” atomic UPSERT with column arithmetic inside the transaction that persists the run's token summary, so concurrent runs can never lose spend to a read-modify-write race. Two bounds compose (INSTANCEDAILYBUDGET_EUR and an admin setting) and the smaller wins. Unlike per-user limits, which fail open, an unknown instance spend fails closed; refusals carry a dedicated code and a Retry-After to the next UTC midnight โ€” ADR-216
  • Administrable Platform Capabilities: ten non-connector capabilities switch off from the admin panel with no redeploy, each declaring the mode by which it is really enforced โ€” planner catalogue exclusion, a route dependency refusing with a stable code, or an internal chokepoint for capabilities that have no route at all. Two boot guards recalculate the declaration against the live agent catalogue and the live routers โ€” ADR-217
  • Backups: Automated daily PostgreSQL dumps (pgdump sidecar, daily/weekly/monthly rotation, all .env-driven) with a tested one-command restore and a verification drill (task backup:verify) โ€” ADR-109, runbook in docs/runbooks/DATABASEBACKUP_RESTORE.md

MCP (Model Context Protocol)

  • Per-user external servers: Each user connects their own MCP servers (third-party tools)
  • Flexible authentication: None, API Key, Bearer Token, OAuth 2.1 (DCR + PKCE S256)
  • Enhanced security: HTTPS-only, SSRF prevention (DNS resolution + IP blocklist), encrypted credentials (Fernet)
  • Structured Items Parsing: Automatic JSON array detection into individual items with McpResultCard HTML
  • Auto-generated descriptions: LLM analysis of discovered tools to generate domain descriptions optimized for intelligent routing
  • Per-server rate limiting: Redis sliding window per server/tool
  • Feature flag: MCPUSERENABLED=true to enable per-user

Multi-Channel Messaging (Telegram)

  • Bidirectional Telegram: Full chat with LIA via Telegram (text, voice, HITL)
  • OTP Linking: Secure account-to-Telegram linking via 6-digit OTP code (single-use, 5min TTL, brute-force protection)
  • HITL Inline Keyboards: Approval/rejection buttons localized in 6 languages directly in Telegram
  • Voice Transcription: Telegram voice messages to STT (Sherpa Whisper) to text processing
  • Proactive Notifications: Reminders and interest alerts also sent via Telegram
  • Extensible Architecture: BaseChannelSender/BaseChannelWebhookHandler abstraction for future channels (Discord, WhatsApp)
  • Observability: 12 dedicated Prometheus RED metrics (latency, errors, volumes)
  • Feature flag: CHANNELS_ENABLED=true to enable

Autonomous Heartbeat โ€” Proactive Notifications

  • LLM-driven proactivity: LIA takes the initiative to inform you when relevant (weather, calendar, interests)
  • Multi-source aggregation: Calendar, Weather (with change detection), Tasks, Interests, Memories, Activity โ€” parallel fetch
  • 2-phase LLM decision: Phase 1 (structured output, cost-effective model) decides whether to notify, Phase 2 rewrites with user personality and language
  • Intelligent anti-redundancy: Recent history + cross-type dedup (heartbeat vs. interests) in the decision prompt
  • User control: Push notifications (FCM/Telegram) independently toggleable, configurable daily max (1-8), dedicated time windows (independent from interests)
  • Feedback loop: thumbs up/down on every notification card, persisted on the archived message so a verdict is given once, across devices
  • Weather change detection: Rain start/end, temperature drops, wind alerts โ€” truly actionable notifications
  • Feature flag: HEARTBEAT_ENABLED=true to enable

Interest Learning โ€” Admission Doctrine

  • A creation requires a named ground (ADR-166): stated passion, own practice, prior knowledge, or digging into the same subject across an exchange โ€” and the model must quote the user's words that carry it. Asking about a subject is a task, not a taste.
  • Six exclusion classes, stated as classes rather than examples: the subject of a request, a remark about the assistant, a third party's taste, something tried once, a daily action, anything the assistant introduced itself
  • Deduplication reads every status: a blocked subject cannot be re-created under a neighbouring name, renamed back into place, or deleted to free the slot; a dormant one is revived instead of duplicated
  • Cap on destructive actions: beyond EXTRACTIONMAXDELETESPERRUN (2), the batch's deletions are dropped and counted โ€” one replayed production window proposed 19
  • Anchored confidence floor (INTERESTEXTRACTIONMIN_CONFIDENCE, 0.75): the prompt's scale is tied to the ground it can name, so the written rule is enforceable
  • Measured, not asserted: scripts/measureextractionselectivity.py replays a production-derived battery plus a held-out one โ€” noise on negatives 0.50 โ†’ 0.00, recall 0.75 โ†’ 1.00, reproduced on two model providers

Scheduled Actions

  • Recurring actions: Schedule repetitive actions executed automatically (send emails, checks, reminders)
  • Timezone-aware: Correct timezone handling per user
  • Retry logic: Automatic retries on failure with back-off
  • Auto-disable: Automatic deactivation after N consecutive failures
  • Multi-channel integration: Result notifications via FCM, SSE, and Telegram
  • Feature flag: SCHEDULEDACTIONSENABLED=true to enable

Sub-Agents (F6)

  • Persistent specialized agents: Create sub-agents with custom instructions, skills, and LLM configuration
  • Read-only V1: Sub-agents perform research, analysis, and synthesis โ€” no write operations
  • Template-based creation: Pre-defined templates (Research Assistant, Writing Assistant, Data Analyst)
  • Invisible to user: The principal assistant orchestrates sub-agents and presents results naturally
  • Token guard-rails: Per-execution budget, daily budget, auto-disable after consecutive failures
  • Feature flag: SUBAGENTSENABLED=true to enable (default: false)

RAG Knowledge Spaces

  • Personal knowledge bases: Create spaces, upload documents in 15+ formats (PDF, DOCX, PPTX, XLSX, CSV, RTF, HTML, EPUB, and more), automatic chunking and embedding
  • Google Drive folder sync: Link Google Drive folders to spaces for automatic file vectorization with incremental change detection (new, modified, deleted). Feature flag: RAGSPACESDRIVESYNCENABLED
  • Hybrid search: Semantic similarity (pgvector cosine) + BM25 keyword matching with configurable alpha fusion
  • Response enrichment: RAG context automatically injected into assistant responses when active spaces exist
  • Full cost transparency: Embedding costs tracked per document and per query, visible in chat bubbles and dashboard
  • System knowledge spaces: Built-in FAQ knowledge base (250 Q/A across 24 sections) indexed from Markdown files (docs/knowledge/, 27 documents). isapphelpquery detection by QueryAnalyzer, RoutingDecider Rule 0 override, App Identity Prompt injection with lazy loading (zero overhead on normal queries). Auto-indexed at startup with SHA-256 hash-based staleness. Admin UI for reindex and staleness monitoring. ADR-058
  • Admin reindexation: Full reindex when embedding model changes, with Redis mutual exclusion and automatic dimension ALTER. System spaces have independent reindex via admin API
  • Observability: 17 Prometheus metrics (14 user + 3 system), dedicated Grafana dashboard
  • Feature flags: RAGSPACESENABLED=true (user spaces), RAGSPACESSYSTEM_ENABLED=true (system FAQ spaces)

Personal Journals (Carnets de Bord) โ€” Stratified consciousness

  • Introspective notebooks: The assistant maintains thematic journals (self-reflection, user observations, ideas & analyses, learnings) written in first person, colored by its active personality
  • Four abstraction levels: Each entry carries a level โ€” L0 raw observation, L1 operational directive (WHENโ†’DO BECAUSE), L2 transversal pattern, L3 portrait facet. L2/L3 are produced exclusively at consolidation through active topic clustering (ADR-079)
  • Epistemic status: confidence โˆˆ {low, medium, high} plus evidencecount and contradictioncount counters per entry. The journal distinguishes hypotheses still in test from observations validated across many turns
  • Deferred self-evaluation T โ†’ T+1: MessagesState.injectedjournalids carries IDs across turns; the post-conversation extractor sees the previous turn's directives + the current user reaction, signals evidence_outcome="evidence" | "contradiction", and the service atomically increments the counters. Zero added LLM cost (same extractor call, enriched prompt). Anti-hallucination layer 4: LLM never writes absolute counter values.
  • Dual trigger: Post-conversation extraction (fire-and-forget) + periodic consolidation (APScheduler, 4โ€“12 h cooldown)
  • Gemini dual-vector embeddings: gemini-embedding-001 (1536d) โ€” one vector on title+content, one on searchhints keywords. Search uses LEAST(distcontent, distkeyword) per row to bridge the assistant's introspective vocabulary and the user's vocabulary (ADR-069)
  • Ambient diffusion of the user-model portrait: Consolidation produces, in the same LLM call, a portraitfull (~200 tokens) for conversation/planner and a portraitbrief (~60 tokens) diffused across 6 secondary flows (ReAct setup, interest proactive, reminder notification, voice, heartbeat, fallback sync+async). Standalone builder buildjournalusermodelblock(userid, format, flow) mirrors buildpsychepromptblock.
  • Three corrective levers on the portrait (never directly editable): edit L3 source entries, POST /journals/portrait/feedback (free text โ†’ L0 user_correction + synchronous re-consolidation), POST /journals/consolidate (manual, bypasses cooldown).
  • Prompt-driven lifecycle: The assistant manages its own journals โ€” no hardcoded auto-archival. Mandatory pairwise dedup at consolidation STEP 1, classification audit, active L1โ†’L2 clustering at STEP 5
  • Heartbeat integration: Journal entries enrich proactive notifications via dynamic second-pass query built from aggregated context. The compiled portrait brief is also injected so the notification voice is aligned with the same user model used by conversation
  • Full user control: Enable/disable (data preserved), consolidation toggle, conversation history analysis (with cost warning), 4 configurable numeric settings, group-by Theme/Level toggle, filter "show only entries never used", full CRUD in Settings (level + confidence editable)
  • 4-layer anti-hallucination: prompt guidance with ID reference tables, field_validator on UUIDs, known-ID filtering in extraction and consolidation, atomic counter increments
  • 11 Prometheus metrics: journalentriestotal{action,theme,source}, journalevidencetotal{outcome}, journalconsolidationpromotionstotal{fromlevel,tolevel}, journalleveldistribution{level}, journalportraitpresenttotal{flow,format}, journalportraitagehours, journalportraitfeedbacktotal{outcome}, etc.
  • Debug panel: Dedicated "Personal Journals" section showing injection metrics AND background extraction results (CREATE/UPDATE/DELETE badges with theme/title/mood, even on partial updates where the LLM omits fields)
  • Cost transparency: Real token costs tracked via TrackingContext, visible in Settings and dashboard
  • GDPR: Account deletion scrubs the three portrait columns alongside entries; export endpoint includes the compiled portrait under a portrait key
  • Feature flags: JOURNALSENABLED=false (system), user-level toggle in Settings > Features. ADRs: ADR-057 โ†’ ADR-064 โ†’ ADR-069 โ†’ ADR-079

Health Metrics โ€” iPhone Shortcuts Batch Ingestion

  • Two token-authenticated endpoints (POST /api/v1/ingest/health/steps and /api/v1/ingest/health/heartrate): an iPhone Shortcut automation pushes daily batches of samples. Each sample carries its own ISO 8601 datestart / date_end โ€” UTC-normalized server-side and second-truncated to keep uniqueness stable.
  • Polymorphic single-table storage (healthsamples): one row per sample with a kind discriminator (heartrate | steps). Extending to spo2 / sleep / calories reduces to a new kind value โ€” no new table, no new endpoint.
  • Idempotent UPSERT (ON CONFLICT (userid, kind, datestart, date_end) DO UPDATE) using PostgreSQL's RETURNING (xmax = 0) trick to split insert vs update counts in a single round-trip. Re-sending the same batch is free โ€” last value wins.
  • Flexible body parser: accepts JSON array, NDJSON, {"data": [...]} envelope, and the iOS Shortcuts "Dictionnaire" wrapping ({"<ndjson_blob>": {}}) โ€” no contract pressure on the user's Raccourci authoring.
  • Per-user hashed tokens: SHA-256 digest stored, raw value (hm_xxx) returned once at generation, display prefix shown in Settings, individually revocable. Multiple tokens may coexist for rotation.
  • Mixed per-sample validation: out-of-range / malformed / missing-field / invalid-date samples are individually rejected with their 0-based index + reason, while valid siblings in the same batch persist.
  • Bucketed aggregation (hour / day / week / month / year): heart rate averaged (plus min / max), steps SUM-ed per bucket; gaps kept (has_data=False) so the UI displays honest curves.
  • Settings visualization: four-section panel (ingestion API + tokens, recharts line/bar charts with period average overlays, statistics, deletion by kind or full wipe).
  • GDPR-aware: deletion by kind (DELETE ?kind=...), full erasure (DELETE /all), ON DELETE CASCADE on the user FK.
  • Observability: bounded-cardinality Prometheus metrics (healthsamplesupserted_total{kind, operation}, validation rejections, rate-limit hits, auth failures, token lifecycle, deletions, latency histogram) + Grafana dashboard 21.
  • Guards: 60 req/h/token sliding-window rate limit (configurable), 1000 samples/batch cap (413 beyond).
  • Feature flag: HEALTHMETRICSENABLED=false (system). ADR-076 ยท Guide iPhone ยท Technical doc

Health Metrics โ€” Assistant Agent

  • Single healthagent with 7 hand-crafted tools: steps (summary, daily breakdown, baseline delta), heart rate (summary, baseline delta), cross-kind (overview, change detection). One agent โ†” one domain pattern, mirroring emailagent / event_agent.
  • timemin / timemax windowed queries: aggregation tools accept ISO 8601 bounds exactly like calendartools.searchevents_tool. The QueryAnalyzer resolves "this week" / "last month" into concrete date ranges, and the planner splits them across the two parameters.
  • Inlined figures in the LLM message: all factual data (totals, averages, per-day values) ship in the UnifiedToolOutput.message so the Response LLM surfaces them without reaching into structureddata (pattern from weathertools).
  • Extensible registry (HEALTH_KINDS): adding sleep / SpO2 / calories = one entry in kinds.py โ€” bounds, merge strategy, aggregation method, baseline kind. Service helpers iterate the registry so cross-kind logic stays generic.
  • Baseline & variation detection: rolling 28-day median with bootstrap โ†’ rolling mode switch after 7 days of data, tunable thresholds (HEALTHMETRICSVARIATION_* env vars).
  • Heartbeat / Memory / Journal integration: healthsignals source injected for proactive context; contextbiometric JSONB persists deltas and trends in memories (never raw values) when emotional weight crosses a threshold.
  • Per-day server-side rollup (ADR-148): baselines and variations read one aggregated row per day instead of every raw sample โ€” 50ร— cheaper, and the heartbeat no longer drops its health signals on half the ticks
  • Per-user opt-in: single healthmetricsagents_enabled toggle governs the four integrations (tool access, Heartbeat, memory extraction, journal injection). PATCH /auth/me/health-metrics-agents-preference.

MCP Apps โ€” Interactive Widgets

  • Sandboxed iframes via a CSP airlock (ADR-098): third-party widgets boot through a same-origin shell (public/widget-frame.html) served with its own permissive CSP, so external-CDN widgets (Excalidraw, โ€ฆ) work while the main app keeps a strict policy. Isolation is the iframe sandbox (opaque origin, no parent cookies/DOM), not the CSP; the shell is hardened by anti-abuse locks + frame-ancestors 'self'
  • JSON-RPC Bridge: Bidirectional communication between iframe app and chat via PostMessage JSON-RPC 2.0
  • Excalidraw Iterative Builder: Intent-based diagram generation via dedicated LLM calls (shapes + arrows) with cheat sheet injection for format accuracy. Runs under a dedicated MCP-step timeout family (300 s floor / 600 s ceiling, ADR-100) so complex diagrams are not cut off mid-generation
  • readme convention: MCP servers exposing a readme tool have their content auto-injected into the planner prompt
  • Auto-generated descriptions: LLM analysis of discovered tools for domain description optimized for routing
  • App-only tools: Tools with visibility: ["app"] filtered from the LLM catalogue (iframe only)

Personal CRM โ€” Relations

  • A 360ยฐ lens over people you deal with (ADR-176): open loops (commitments), phone calls and name-matching memories aggregated per person โ€” no new truth store, identity resolved by accent/case folding with the confidence stated (exact vs normalized, honesty over false precision)
  • Persisted favorites: star anyone in one tap โ€” the star survives its live signals expiring, leads the overview before the cap, and rides GDPR export/purge; idempotent PUT/DELETE with an optimistic, server-reconciled toggle
  • Readable at scale: stable-tint initial avatars, colored signal pills, distinct Favorites/Others bands with counts, a name filter past nine people, and a ๐Ÿค badge for relations who are also connected LIA users (read-only peers bridge)
  • First-class navigation destination: desktop nav + mobile menu (one shared table); the spaces page keeps its permanent one-click door through the chat indicator, which now always renders
  • Three read capabilities the assistant was missing (ADR-193): past calls, open commitments and relayed messages are now answerable in chat. Each lives in the domain whose catalogue had none โ€” a domain that can only write will push to write, which is how "when did I last call my wife?" became a plan to phone her and ask. All three project the SAME service the relationship card uses, so the tool and the card cannot disagree about who someone is, and each returns the exact total next to its page
  • Merging two relationships, manually and reversibly (ADR-193): folding decides who is literally the same spelling; it cannot know that a raw phone number and a name are one person. The user says so, once โ€” and sees what was merged, with a per-row undo. The alias table is flat (no chain to walk, no cycle writable) and the merge never touches the peer directory: a display decision must not redirect a message to another account
  • Facts about a named peer, injected rather than searched (ADR-193, opt-in): naming a connected person used to correct only the routing, so the assistant announced a lookup for facts already one query away. The three local blocks are injected โ€” never the connector-backed ones, because merely naming someone must not trigger an external call

Peer Connections โ€” Users of the Same Instance, Assistant to Assistant

  • Opt-in discovery by exact full name (accent/case-folded, never prefix search): a masked-email hint disambiguates homonyms, results carry the relationship status, and an empty profile name plainly means "unfindable"
  • Connection lifecycle from chat or settings: request with an optional context note, accept/decline in one click (chat quick-actions or the ยซ Connexions ยป settings section), removal notifies BOTH users through their assistants
  • Silent anti-harassment blocking: blocking ends the connection without notifying the other side โ€” blocked, unknown and cooldown targets answer the byte-identical 404 (no existence leak, ADR-180)
  • Assistant-to-assistant relay: "tell Marieโ€ฆ" produces an HITL-confirmed draft; the recipient's OWN assistant delivers it in its personality, memory and language, naming the sender โ€” whose assistant then confirms delivery; indirect speech is rephrased into direct address; LLM delivery cost is billed to the sender; quotas cap relays per day and per pair
  • Field-level read-only shares: calendar (free/busy or titled slots) and task titles โ€” nothing shared by default, both directions visible to each side, every access re-validated at read time and recorded in a retention-pruned access log. Reads target the calendar or task list the owner configured as their default, never a hardcoded primary, and the answer is rendered in the ASKING user's timezone with all-day entries kept apart from real busy hours (a birthday blocks nothing at 10:00 โ€” ADR-182)
  • Routing that knows who is a peer (ADR-182): the analyzer is given the user's accepted connections, so a named person is recognised as another USER of the instance rather than an address-book contact โ€” with a deterministic guard that adds the peer domain (never substitutes it) when the verdict stays on a confusable one. The correction is counted and logged without the names
  • GDPR-complete: account export and purge cover both sides of every pair; message content is scrubbed after delivery

Internationalization (i18n) โ€” 6 Languages

LIA is fully translated in 6 languages: English, French, German, Spanish, Italian, and Chinese.

  • Complete UI coverage: All interfaces, dialogs, notifications, error messages, FAQ, and landing page
  • HITL localized: Human-in-the-Loop approval prompts adapted per language
  • Proactive notifications: Heartbeat and reminders delivered in the user's language
  • Telegram: Inline keyboards and messages localized
  • Skills: Auto-translated descriptions in all 6 languages
  • react-i18next: Namespace-based translations with locales/{lang}/translation.json

Landing Page & Public Showcase

  • Animated hero chat demo: three rotating scenarios mirroring the real display modes โ€” HITL draft approval, rich HTML weather card + proactive cross-domain initiative, multi-agent Markdown reply โ€” with per-mode title-bar chips
  • Proof band: verifiable engineering numbers (agents, tools, providers, tests, ADRs, releases, audit score) sourced from the codebase (LANDING_STATS documents each origin)
  • Two-mode diagram: faithful LangGraph topology โ€” router fork, five numbered pipeline steps (human approval highlighted), ReAct reasonโ†’actโ†’observe loop, streaming convergence
  • /story field report (6 languages): how LIA is built โ€” method, trade-offs, operations, measured audit profile โ€” on the /whyโ€“/how guide pattern
  • SEO & OpenGraph: dynamically generated OG image, per-locale hreflang, JsonLd (WebSite, Organization, SoftwareApplication, breadcrumbs), llms.txt for AI crawlers
  • Public-route guard: the 401 handler's public-page list is pinned by a filesystem-completeness test โ€” a new public page missing from the list fails CI instead of ejecting anonymous visitors to /login
  • Authenticated redirect: automatic redirect to dashboard if already logged in

Administration & Monitoring

LIA includes a full-featured administration interface โ€” giving operators complete control and real-time visibility over the system without touching configuration files or the database.

Admin Dashboard

A web-based administration panel covering every operational aspect:

| Section | Capabilities | | ---------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | LLM Configuration | Model selection per node, provider parameters, temperature/token limits, prompt versions | | RAG Knowledge Spaces | Manage document spaces, embedding configuration, user reindex operations, system knowledge spaces (FAQ staleness, reindex) | | Personalities | Create and manage assistant personalities (tone, language, behavior rules) | | User Management | User accounts, roles, permissions, connector status overview | | Connector Management | Google/Apple/Microsoft OAuth status, token health, per-user provider activation | | Skills Management | Enable/disable skills, edit descriptions, translate in 6 languages, delete | | MCP Servers | Admin-level MCP server configuration, tool discovery, do


README truncated. View on GitHub
๐Ÿ”— More in this category

ยฉ 2026 GitRepoTrend ยท jgouviergmail/LIA-Assistant ยท Updated daily from GitHub