vLLM alternatives

+ Every open model ends up here, with Inference Providers routing to multiple backends.
− Serverless inference rate limits and cold starts make it unreliable for production.

+ `ollama run llama3` is the simplest path to a local model, and most tools integrate with it.
− Hides llama.cpp tuning options, so throughput trails hand-tuned setups.

+ Unified provider interface and React hooks make streaming chat UIs trivial.
− Major-version upgrades have required significant rewrites.

+ The engine underneath Ollama, LM Studio, and most local tooling, with the widest hardware support.
− Building and tuning flags yourself is a chore compared with wrappers.

+ One OpenAI-compatible endpoint for Anthropic, OpenAI, Google, and open models with automatic fallbacks.
− Adds a routing layer, so latency and rate limits depend on the upstream provider chosen.

+ Friendly GUI for browsing Hugging Face GGUF and MLX models with hardware fit estimates.
− The app itself is closed source, which matters to some privacy-minded users.

+ Self-hosted gateway with spend tracking, virtual keys, and fallbacks across providers.
− Huge dependency surface and frequent releases that occasionally break provider mappings.

+ Multi-user accounts, RAG, and tool plugins make it a full private ChatGPT replacement.
− Docker-only deployment and a licence change in 2025 that restricts branding removal.

+ Thousands of community models deployable via Cog, not just text models.
− Cold starts on less popular models can take a minute or more.

+ Claude models lead on agentic coding and long-context tasks.
− Tighter rate limits on new accounts and no free tier beyond trial credits.