#Ggml
Showing 21 of 21 repositories tagged #ggml, ranked by stars
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
[Unmaintained, see README] An ecosystem of Rust libraries for working with large language models
llama and other large language models on iOS and MacOS offline using GGML library.
Go manage your Ollama models
INT4/INT5/INT8 and FP16 inference on CPU for RWKV language model
Calculate token/s & GPU memory requirement for any LLM. Supports llama.cpp/ggml/bnb/QLoRA quantization
This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose
Suno AI's Bark model in C/C++ for fast text-to-speech generation
Running any GGUF SLMs/LLMs locally, on-device in Android
Self-evaluating interview for AI coders
Real-time 3D full-body reconstruction from a single camera, Multiperson BVH output, Pure C++ runtime, ONNX + ggml, 70-joint skeleton with hands.
Port of MiniGPT4 in C++ (4bit, 5bit, 6bit, 8bit, 16bit CPU inference with GGML)
Large Language Models for All, 🦙 Cult and More, Stay in touch !
Inference Vision Transformer (ViT) in plain C/C++ with ggml
llama.cpp (GGUF LLMs) and llava.cpp (GGUF VLMs) for ROS 2
Booster - open accelerator for LLM models. Better inference and debugging for AI hackers
True on-device AI for Kotlin Multiplatform (Android, iOS, Desktop, JVM, WASM). LLM, Speech-to-Text and Image Generation — powered by llama.cpp, whisper.cpp and stable-diffusion.cpp.
🪶 Lightweight OpenAI drop-in replacement for Kubernetes
lcpp is a dart implementation of llama.cpp used by the mobile artificial intelligence distribution (maid)
pure go for stable-diffusion and support cross-platform.
A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp