LLMFarm

The app is temporarily unavailable in TestFlight and Appstore.

LLMFarm is an iOS and MacOS app to work with large language models (LLM). It allows you to load different LLMs with certain parameters.With LLMFarm, you can test the performance of different LLMs on iOS and macOS and find the most suitable model for your project.
Based on ggml and llama.cpp by Georgi Gerganov.

Features

• [x] MacOS (13+)
• [x] iOS (16+)
• [x] Various inferences
• [x] Various sampling methods
• [x] Metal (dont work on intel Mac)
• [x] Model setting templates
• [x] Restore context state
• [x] Apple Shortcuts
• [x] RAG

Inferences

• [x] LLaMA
• [x] Gemma
• [x] Phi
• [x] GPT2 + Cerebras
• [x] Starcoder(Santacoder)
• [x] Falcon
• [x] MPT
• [x] Bloom
• [x] StableLM-3b-4e1t
• [x] Qwen
• [x] Yi models
• [x] Deepseek models
• [x] Mixtral MoE
• [x] PLaMo-13B
• [x] Mamba
• [x] RWKV
• [x] GPTNeoX

Multimodal

• [x] LLaVA 1.5 models, LLaVA 1.6 models
• [x] BakLLaVA
• [x] Obsidian
• [x] ShareGPT4V
• [x] MobileVLM 1.7B/3B models
• [x] Yi-VL
• [x] Moondream

Sampling methods

• [x] Temperature (temp, tok-k, top-p)
• [x] Tail Free Sampling (TFS)
• [x] Locally Typical Sampling
• [x] Mirostat
• [x] Greedy
• [x] Grammar

Getting Started

You can find answers to some questions in the FAQ section.

Inference options

Models

Development

git clone --recurse-submodules https://github.com/guinmoon/LLMFarm