ai-gateway cost-optimization latency-control vercel openai-gemini

AI Gateway adds service tier routing for latency-cost tradeoffs

Set `serviceTier` in `providerOptions.gateway` to route requests between `default`, `priority` (~1.8-2x cost), or `flex` (~0.5x cost) tiers; billing auto-adjusts based on actual tier used.

Summary

Developers can now optimize request handling without code restructuring—interactive endpoints get priority queuing while background jobs use cheaper capacity. Tier configuration is unified across OpenAI, Gemini, and other providers, reducing boilerplate.

Why it matters

Implementation verdict

This replaces manual provider-specific configuration for throughput/latency tuning. Requires single line addition to `providerOptions` in AI SDK or compatible APIs. Production-ready now; best-effort fallback to default tier on capacity constraints means no failure risk. Worth testing immediately if you have mixed latency requirements.

Sources

1.Service tiers let you optimize for latency, throughput, and cost per request to match your use case
2.Pick a faster tier for interactive workloads (less queueing, higher token throughput), or a lower cost tier for background jobs that can tolerate more latency
3.Set `serviceTier` under `providerOptions.gateway`
4.Cost relative to default: ~1.8-2x default pricing for priority, ~0.5x default pricing for flex
5.Service tier is serviced on a best-effort basis: if a tier can't be applied, the request runs on the default tier at the default rate

Dev Signal

Get briefs like this in your inbox — free, every weekday.

100+ sources compressed into one 4-minute read. Ranked, cited, implementation-ready.

Read the full issue →All briefs