Direct answer
What is an AI gateway?
An AI gateway sits between an application and one or more model APIs. It can centralize authentication, expose compatible request formats, select eligible routes, measure usage, and enforce access or spend boundaries. It does not make different models equivalent or remove the need for application-level evaluation and error handling.
Last reviewed 2026-08-15 · Editorial review: KeepRouter Editorial
The request path
Application -> AI gateway -> eligible model route -> model maker
<- normalized response and measured usage <-The application authenticates to the gateway and names a model or routing policy. The gateway validates the key, checks route compatibility and commercial rules, forwards the request, then returns a client-facing response and usage record. A managed gateway may also provide one billing relationship for a published catalog; a self-hosted gateway usually connects credentials your team already owns.
What is commonly centralized
- API keys, model allowlists, rate or spend controls;
- compatible OpenAI- or Anthropic-shaped routes;
- canonical model IDs and route eligibility;
- request status, token counts, latency, and cost attribution;
- bounded retry or failover between eligible routes for the same public model.
What is not safely universal
Tool semantics, reasoning fields, image input, context limits, structured output, safety behavior, regional availability, and response quality vary by model and route. “Compatible” describes a client contract, not identical capability. A gateway also cannot make a side-effecting agent retry safe unless the application provides idempotency.
Managed, self-hosted, or direct
Use a managed gateway when you want a hosted access and billing layer with low operational overhead. Use a self-hosted gateway when infrastructure control, custom policy, and bring-your-own provider credentials justify operating the proxy and its data stores. Use direct APIs when one provider-specific feature or contract is the primary requirement.
When you probably need one
An AI gateway becomes useful when several services share model access, teams need attributable keys and spend limits, model evaluation is frequent, or compatible clients must reach more than one maker. A single small workload with one stable provider may be simpler as a direct integration.
For a concrete decision sequence, use how to choose an AI gateway. The managed versus self-hosted guide then makes deployment, credential, data, and on-call ownership explicit.
Frequently asked questions
Is an AI gateway the same as an API gateway?
It applies API-gateway controls to model-specific concerns such as model IDs, tokens, tool streams, and provider routes.
Does an AI gateway improve answer quality?
Not automatically. Quality depends on model, prompt, context, tools, and your evaluation.
Can it hide every provider difference?
No. It can normalize common request shapes, but model-specific features and behavior still differ.
Should I manage or self-host the gateway?
Choose managed for less operations and self-hosted for deeper infrastructure and credential control.