Which model API should I build on, or route between?

+ Serves many concurrent requests on GPUs far faster than llama.cpp-based tools.
− Linux and NVIDIA-centric; painful on Mac or consumer hardware.

+ Unified provider interface and React hooks make streaming chat UIs trivial.
− Major-version upgrades have required significant rewrites.

+ Every open model ends up here, with Inference Providers routing to multiple backends.
− Serverless inference rate limits and cold starts make it unreliable for production.

+ One OpenAI-compatible endpoint for Anthropic, OpenAI, Google, and open models with automatic fallbacks.
− Adds a routing layer, so latency and rate limits depend on the upstream provider chosen.

+ Self-hosted gateway with spend tracking, virtual keys, and fallbacks across providers.
− Huge dependency surface and frequent releases that occasionally break provider mappings.

+ Thousands of community models deployable via Cog, not just text models.
− Cold starts on less popular models can take a minute or more.

+ Claude models lead on agentic coding and long-context tasks.
− Tighter rate limits on new accounts and no free tier beyond trial credits.

+ Fastest generation speed available, often several hundred tokens per second.
− Limited model selection and context limits on the free tier.

+ Strong throughput on DeepSeek and Llama with serverless and on-demand GPU options.
− Smaller community and documentation than Together or Hugging Face.

+ Broad open-model catalogue plus fine-tuning and dedicated GPU endpoints.
− No proprietary models, so you cannot route to GPT or Claude.