#Sparse-attention

thu-ml

SpargeAttn

[ICML2025] SpargeAttention: A training-free sparse attention that accelerates any model inference.

lucidrains

native-sparse-attention-pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

HKUSTDial

flash-sparse-attention

Trainable fast and memory-efficient sparse attention

AarambhDevHub

aarambh-studio

🦀 Decoder-only LLM built from scratch in pure Rust using Candle — no Python, no PyTorch. Gated DeltaNet + sparse attention, fine-grained MoE, native video/document understanding, long-horizon tool agents, quantization-aware training. Scales: Tiny (25M) to Large (1.3B).

skylight-org

sparse-attention-hub

Advancing the frontier of efficient AI

Infini-AI-Lab

vortex_torch

Vortex: Programmable Sparse Attention for Agents as Algorithm Designers