If LiteLLM is the open-source, self-hosted default for routing LLM traffic, the main alternatives each lean a different way: Portkey adds governance and guardrails, Kong AI Gateway fits enterprises already running Kong, and Cloudflare AI Gateway is the managed, ecosystem-native option. This compares all four — and clears up where vLLM and Ollama actually fit, because they’re not gateways at all.
Quick Verdict
- Pick LiteLLM if you want an open-source gateway you self-host and fully own.
- Pick Portkey if governance, guardrails, and deep observability are priorities.
- Pick Kong AI Gateway if you already operate a Kong API mesh.
- Pick Cloudflare AI Gateway if your app lives in the Cloudflare ecosystem and you want zero ops.
AI Gateways Compared at a Glance
| Gateway | Self-hosted? | Open-source? | Guardrails | Ops overhead | Best for |
|---|---|---|---|---|---|
| LiteLLM | Yes | Yes (MIT) | Basic | Low | Self-hosted, open-source teams |
| Portkey | Limited | Core open + managed | Strong (semantic caching, guardrails) | Low (managed) | Governance & observability |
| Kong AI Gateway | Yes | Core open | Via plugins | High | Existing Kong / enterprise |
| Cloudflare AI Gateway | No | No | Platform features | Near-zero | Cloudflare-ecosystem apps |
LiteLLM vs Portkey
Kong AI Gateway brings LLM routing into Kong’s mature API-management platform, with a strong plugin ecosystem, SSO, and capabilities like PII redaction. That power comes with weight: it’s heavier to operate and assumes Kong infrastructure underneath. LiteLLM is the lighter, simpler, more self-contained option. The honest rule of thumb: choose Kong if you already run a Kong API mesh and want LLM traffic governed the same way; otherwise LiteLLM is far less to operate.
LiteLLM vs Cloudflare AI Gateway
Cloudflare AI Gateway is fully managed with near-zero operational overhead, adding caching and analytics in front of your providers — and it shines when your application already lives in the Cloudflare ecosystem. The trade-off is the familiar managed one: you give up some data control in exchange for not running anything. LiteLLM is the opposite choice — you operate it, and in return the full data path stays in your infrastructure.
Where vLLM and Ollama Fit (They’re Not Gateways)
This is the comparison people most often get wrong. vLLM and Ollama are not gateways — they’re inference engines and runtimes that actually run the models. A gateway like LiteLLM sits in front of them: it routes a request to a vLLM or Ollama backend exactly as it would to a cloud provider. So “LiteLLM vs vLLM” or “LiteLLM vs Ollama” is a category error; the real pattern is using them together — a gateway for routing and control, a runtime for the inference. If self-hosted inference is your goal, the linked Ollama guides are the place to start.
Which AI Gateway Should You Choose?
- Open-source and self-hosted → LiteLLM.
- Governance, guardrails, observability → Portkey.
- Already running Kong / enterprise needs → Kong AI Gateway.
- Cloudflare-ecosystem app, zero ops → Cloudflare AI Gateway.
- Running your own models → pair LiteLLM (gateway) with vLLM or Ollama (runtime).
How to Self-Host LiteLLM on a VPS
Of these, LiteLLM is the most self-host-friendly — a CPU-bound proxy with a small PostgreSQL database that runs comfortably on a modest virtual private server. A VPS gives you root access for Docker, full data control, and EU data-residency options; if you also want to self-host the models, you can pair the gateway with a GPU instance running vLLM or Ollama. Contabo’s Core VPS line offers strong RAM-per-Euro value for the gateway, with GPU options available for inference. See the linked Docker setup guide to deploy.
FAQ: LiteLLM vs Other AI Gateways
It depends on what you need. Portkey is the closest alternative if you want governance and guardrails as a managed control plane. Kong AI Gateway suits enterprises already on Kong, and Cloudflare AI Gateway fits managed, ecosystem-native setups. For open-source self-hosting specifically, LiteLLM remains the leading choice.
Neither is universally better. LiteLLM is better if you want an open-source gateway you self-host and own. Portkey is better if you want managed governance, guardrails, semantic caching, and observability without running the infrastructure. The choice is mainly about delivery model: self-hosted ownership versus a managed control plane.
No. vLLM is an inference engine that runs models, while LiteLLM is a gateway that routes requests to model backends. They operate at different layers and are typically used together — LiteLLM in front, routing to a vLLM backend. So vLLM complements LiteLLM rather than replacing it.
No. Ollama is a local runtime for running models on your own hardware, not a gateway. A gateway such as LiteLLM can sit in front of Ollama and route requests to it alongside other providers. If you want local inference, you’d use Ollama as a backend behind your gateway, not instead of it.
