#On-device-ai
Showing 54 of 54 repositories tagged #on-device-ai, ranked by stars
Production ready toolkit to run AI locally
Run frontier LLMs and VLMs locally on Qualcomm devices across NPU, GPU, and CPU with a few lines of code
Low-latency AI engine for mobile devices & wearables
An AI-powered file management tool that ensures privacy by organizing local texts, images. Using Llama3.2 3B and Llava v1.6 models with the Nexa SDK, it intuitively scans, restructures, and organizes files for quick, seamless access and easy retrieval.
26m function call model that runs on incredibly small devices
Declarative way to run AI models in React Native on device, powered by ExecuTorch.
Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG
On-device LLM execution in React Native with Vercel AI SDK compatibility
Fine-tune LLMs on your Mac with Apple Silicon. SFT, DPO, GRPO, Vision, TTS, STT, Embedding, and OCR fine-tuning — natively on MLX. Unsloth-compatible API.
A practical lab for building, testing, and evaluating apps with Apple's Foundation Models framework.
Mobile-native local AI Agent framework for phones and edge devices, with an iOS runtime, fully offline local path, on-device models, native iOS Skills, and optional Mac Gateway inference
NativeMind: Your fully private, open-source, on-device AI assistant
NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.
TinyChatEngine: On-Device LLM Inference Library
Local-first, open-source AI assistant for your data. Unify tasks, notes, docs, photos, and bookmarks. Private, self-hosted, and extensible via APIs.
Single-file memory layer for AI agents, sub mili-second RAG on Apple Silicon. Metal Optimized On-Device. No Server. No API. One File. Pure Swift
⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
Sudoless Apple Silicon system monitor (native SwiftUI GUI) with ANE / Media Engine / memory-bandwidth tracking
Android Input Method Editor (IME) based on Whisper
On-device Neural Engine
PennyWise automatically reads transaction SMS messages and transforms them into organized financial data with on-device AI assistance. No manual entry, no cloud processing, complete privacy.
A ready-to-use, minimal app that converts any speech into text.
AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.
电子鹦鹉 / Toy Language Model
Build apps powered by on-device AI
Instant thought capture for macOS. One shortcut, post-it appears, type, close.
Open Dungeon — the first easy-to-use, fully local AI roleplay app. Story and inline scene images generated 100% on your machine (Gemma 4 QAT via Ollama(and others) + FLUX). No accounts, no API keys, no cloud.
On-device memory layer for AI agents. Claude Code, Hermes and OpenClaw. Hooks + MCP server + hybrid RAG search.
Private on-device AI chat for Android — runs any GGUF model locally via llama.cpp with ARM-optimised SIMD. Zero network permissions, encrypted settings, biometric lock, tamper detection.
True on-device AI for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM, Speech-to-Text and Image Generation — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.
A ternary, zero-heap tiny language model that runs inside a $2 microcontroller — bit-exact Python <-> C99 <-> Cortex-M3 (QEMU) parity. Apache-2.0.
On-device AI recipes for building private, real-time applications with production-grade voice, language, and vision SDKs. Ready-to-use, open-source code examples.
InferrLM - On-device AI for iOS & Android
Local-first AI photo culling for professional photographers — 6-axis rubric, XMP/IPTC export, Lightroom & Capture One ready.
Source code for my blog post series "On-device training with Core ML"
This is a web demo for camera-based PPG sensing (rPPG).
On-device AI meeting assistant — real-time transcription & LLM insights, 100% offline
On-device AI SDK for Flutter — LLM inference, vision, STT, TTS, image generation, embeddings, RAG, and function calling. Metal GPU on iOS/macOS.
Fully local voice AI for iOS
Simplified AI runtime integration for mobile app development
On-device meeting transcriber for macOS — auto-records Teams/Zoom/Webex, transcribes & separates speakers locally. No cloud. Open-source alternative to Otter/Granola/Fireflies.
Cactus Flutter plugin: Run AI locally in your Flutter apps
Open-source, patient-controlled health record app with on-device AI. Aggregates medical data from 52K+ providers via FHIR R4. Offline-first. Flutter.
Run ml models with both tflite & litert runtimes on edge devices
A practical lab for exploring Apple's Core AI framework, model assets, specialization, and on-device inference.
On-device AI for Expo apps. Run language models locally. No API keys, no cloud, just native intelligence.
Android native AI inference library, bringing text, image, video, STT, TTS inference
Edge-first AI memory engine. Rust core, DuckDB single-file, sub-10ms search, 6 language bindings.
258 KB WASM runtime for Needle a 26M-parameter tool-calling transformer. Runs in browser, Cloudflare Workers, and Node.js. No backend required.
On-device speech-to-text for macOS. Hold a hotkey → Whisper transcribes locally via CoreML → text auto-pastes into the focused app. 99 languages, free, open-source.
On-device shell command generator for macOS Tahoe. Uses Apple's 3B model with dynamic few-shot retrieval from 21k tldr examples.
Swift 6 agent SDK: type-safe tools, streaming, cloud + on-device inference via MLX on Apple Silicon
Image Classification, Image Captioning and LLM inference with LiteRT