#Memory-optimization
Showing 7 of 7 repositories tagged #memory-optimization, ranked by stars
Run larger LLMs with longer contexts on Apple Silicon by using differentiated precision for KV cache quantization. KVSplit enables 8-bit keys & 4-bit values, reducing memory by 59% with <1% quality loss. Includes benchmarking, visualization, and one-command setup. Optimized for M1/M2/M3 Macs with Metal support.
VS Code extension: Go struct layout, padding, reorder
A fast and memory-optimized string library for heavy-text manipulation in Python
Keep ChatGPT fast — Firefox & Chrome extension that trims DOM in long conversations
Flash weight streaming for MLX: run massive models larger than your RAM on Apple Silicon.
This code repository contains the code used for my "Optimizing Memory Usage for Training LLMs and Vision Transformers in PyTorch" blog post.
First open-source implementation of Google TurboQuant (ICLR 2026) -- near-optimal KV cache compression for LLM inference. 5x compression with near-zero quality loss.