#Llamacpp
Showing 60 of 108 repositories tagged #llamacpp, ranked by stars
Jan is an open source alternative to ChatGPT that runs 100% offline on your computer.
Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.
Unified framework for building enterprise RAG pipelines with small, specialized models
A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
Production ready toolkit to run AI locally
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Private & local AI personal knowledge management app for high entropy people.
A web interface for chatting with Alpaca through llama.cpp. Fully dockerized, with an easy to use API.
⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.
Low-latency AI engine for mobile devices & wearables
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web. Make your own persistent autonomous agent on top!
State of the Art Natural Language Processing
A C#/.NET library to run LLM (🦙LLaMA/LLaVA) on your local device efficiently.
The most no-nonsense, locally or API-hosted AI code completion plugin for Visual Studio Code - like GitHub Copilot but 100% free.
AGiXT is a dynamic AI Agent Automation Platform that seamlessly orchestrates instruction management and complex task execution across diverse AI providers. Combining adaptive memory, smart features, and a versatile plugin system, AGiXT delivers efficient and comprehensive AI solutions.
LSP-AI is an open-source language server that serves as a backend for AI-powered functionality, designed to assist and empower software engineers, not replace them.
RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.
Instant, controllable, local pre-trained AI models in Rust
Text-To-Speech, RAG, and LLMs. All local!
Open-source LLM/VLM load balancer and serving platform for self-hosting LLMs (and VLMs) at scale 🏓🦙 Alternative to projects like llm-d, Docker Model Runner, etc but with less moving parts and simple deployments built around ggml ecosystem. Runs on CPU and GPU.
Chat with your favourite LLaMA models in a native macOS app
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
The easiest way to use Ollama in .NET
Build production-ready LLM applications and advanced agents using Python, LangChain, and LangGraph. This is the companion repository for the book on generative AI with LangChain.
The TypeScript library for building AI applications.
WebAssembly binding for llama.cpp - Enabling on-browser LLM inference
Local AI app and inference engine for agents. Run open-weight LLMs locally — private, 100% offline on your computer.
LSP server leveraging LLMs for code completion (and more?)
Bonsai Demo
Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.
Running any GGUF SLMs/LLMs locally, on-device in Android
Llama 3+ inference in pure Java
End-to-end documentation to set up your own local & fully private LLM server on Debian. Equipped with chat, web search, RAG, model management, MCP servers, image generation, and TTS.
🌌 Give a soul to your favorite characters. Soul of Waifu is a desktop roleplay & AI companion app featuring Live2D/VRM avatars, voice chat & local LLM. Evolve together across immersive chat, RPG adventures, and your desktop.
ComfyUI-IF_AI_tools is a set of custom nodes for ComfyUI that allows you to generate prompts using a local Large Language Model (LLM) via Ollama. This tool enables you to enhance your image generation workflow by leveraging the power of language models.
🤖 TUI for LLMs
The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.
The most advanced, fully offline client-side AI suite on Android today.
Yet Another Language Model: LLM inference in C++/CUDA, no libraries except for I/O
Use Code Llama with Visual Studio Code and the Continue extension. A local LLM alternative to GitHub Copilot.
Static suckless single batch CUDA-only qwen3-0.6B mini inference engine
Go with your own intelligence - Go applications that directly integrate llama.cpp for local inference using hardware acceleration.
A simple "Be My Eyes" web app with a llama.cpp/llava backend
Local first semantic code search and chat | Leverage custom copilots with fine-tuning datasets from code in Alpaca, Conversational, Completion and Instruction format
Browser Operator - The AI browser with built in Multi-Agent platform! Open source alternative to ChatGPT Atlas, Perplexity Comet, Dia and Microsoft CoPilot Edge Browser
Kolosal AI is an OpenSource and Lightweight alternative to LM Studio to run LLMs 100% offline on your device.
The open creative AI workspace
llama.cpp with BakLLaVA model describes what does it see
AI-powered cybersecurity chatbot designed to provide helpful and accurate answers to your cybersecurity-related queries and also do code analysis and scan analysis.
Inference Vision Transformer (ViT) in plain C/C++ with ggml
AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
Build apps powered by on-device AI
llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2
High-performance lightweight proxy and load balancer for LLM infrastructure. Intelligent routing, automatic failover and unified model discovery across local and remote inference backends.
Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with multi-GPU tensor-split, MoE expert placement, measured flag tuning (AI Tune), hardware-matched HuggingFace downloads, and crash recovery. An Ollama alternative for multi-GPU rigs.
An Automated AI-Powered Prompt Optimization Framework
An AI-assisted coding agent that runs in your terminal and supports many providers and models.
JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon