moritztng

fltr

Rust

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

Last updated Jul 8, 2026

386

Stars

Forks

Issues

Stars/day

Attention Score

View on GitHub ↗ Open in github.dev

Topics

#cli #grep #grep-like #llama #llama-2 #llm #localllama #mistral #mixtral #mixtral-8x7b #operating-system #rust

Language breakdown

Rust 81.5%

Python 8.9%

Cuda 6.6%

Shell 3.1%

▸ Files click to expand

README

Like grep but for natural language questions. Based on Mistral 7B or Mixtral 8x7B.

Text file input tokens per second:

Nvidia RTX 3070 with 8GB memory

- Mistral 7B: ~52 tok/s - Mixtral 8x7B: ~28 tok/s

Intel I5-6500 with 8GB memory

- Mistral 7B: ~5 tok/s - Mixtral 8x7B: ~2 tok/s

Installation

Linux (x8664) & macOS (x8664 & arm64)

If nvidia driver that supports cuda 12.1 exists, it installs cuda version, else cpu version. Replace small with large to install Mixtral 8x7B. It's ~7GB or ~48GB.

curl https://raw.githubusercontent.com/moritztng/fltr/main/install.sh -o install.sh && bash install.sh small && export PATH=$PATH:~/Fltr

Quickstart

Add --large for Mixtral 8x7B.

fltr --file emails.txt --prompt "Is the following email spam? Email:" --batch-size 32

It will output all lines in the file where the answer is yes.

🔗 More in this category