Open-source · Coding · 6 of 8 tools · updated Oct 17, 2026
Open-source AI tools to run LLMs locally
Ollama leads the open-source options to run LLMs locally at 99/100, with 183,530 GitHub stars. 6 of the 8 tools tracked for this job publish their source, so you can read it, fork it or self-host it. It is also the top-ranked tool for this job overall, open-source or not.
Same evidence score as the ranking — GitHub stars, Hacker News mentions, release activity, free access — recomputed nightly, never sold. See all 8 tools to run LLMs locally.
Top 3
evidence / 100
+ `ollama run llama3` is the simplest path to a local model, and most tools integrate with it.
− Hides llama.cpp tuning options, so throughput trails hand-tuned setups.
+ Multi-user accounts, RAG, and tool plugins make it a full private ChatGPT replacement.
− Docker-only deployment and a licence change in 2025 that restricts branding removal.
+ The engine underneath Ollama, LM Studio, and most local tooling, with the widest hardware support.
− Building and tuning flags yourself is a chore compared with wrappers.
All 6, side by side
| Tool | Score | Stars | Source | Days idle | Platforms |
|---|---|---|---|---|---|
| Ollama | 99 | 183,530 | ollama/ollama | 0 | mac, windows, linux, cli, api |
| Open WebUI | 98 | 155,940 | open-webui/open-webui | 0 | web, linux, mac, windows |
| llama.cpp | 97 | 131,447 | ggml-org/llama.cpp | 0 | mac, windows, linux, cli, api |
 Jan | 90 | 44,975 | menloresearch/jan | 0 | mac, windows, linux |
 vLLM | 80 | 93,962 | vllm-project/vllm | 0 | linux, cli, api |
 GPT4All | 78 | 77,362 | nomic-ai/gpt4all | 507 | mac, windows, linux |
Open source here means the tool's own source is public, not that a vendor published an SDK. Self-hosting cost is yours.
Head to head: Ollama vs Open WebUI Ollama vs llama.cpp Open WebUI vs llama.cpp