#Localllama
Showing 12 of 12 repositories tagged #localllama, ranked by stars
Reliable model swapping for any local OpenAI/Anthropic compatible server - llama.cpp, vllm, etc
The easiest way to use Ollama in .NET
✨ Kubectl plugin to create manifests with LLMs
🏗️ Fine-tune, build, and deploy open-source LLMs easily!
[NeurIPS 2024] KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization
Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.
AubAI brings you on-device gen-AI capabilities, including offline text generation and more, directly within your app.
Social and customizable AI writing assistant! ✍️
Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with multi-GPU tensor-split, MoE expert placement, measured flag tuning (AI Tune), hardware-matched HuggingFace downloads, and crash recovery. An Ollama alternative for multi-GPU rigs.
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
A native macOS AI workspace that's actually yours — local-first, private, pro-AI and anti-big-tech. Own your assistant, don't rent it. GPLv3.
LLM RAG Application with Cross-Encoders Re-ranking for YouTube video 🎥