routing model-selection cost-optimization inference-gateway offline-scoring

Route prompts offline by complexity, skip model calls

Wayfinder scores prompt structure deterministically (0–1.0 complexity) without an API call, routing cheap queries to local models and hard ones to expensive tiers, replacing learned routers that add latency and cost to the routing decision itself.

Summary

Eliminates the cost and latency tax of calling a classifier or LLM to decide which model to use—the routing decision is now free and reproducible. Developers pay top-tier prices only for prompts that actually need them, not for summarization or typo fixes.

Why it matters

Implementation verdict

Replaces RouteLLM and hosted routers (NotDiamond, OpenRouter Auto) with a local-first alternative. Requires two tiers (local + cloud), OpenAI-compatible endpoints, and a single TOML config. Ready now—zero-install CLI demo available. Honest tradeoff: wins on structural complexity (length, code, lists), loses on pure-semantic hard cases ('what is the 100th prime number?'). Worth trying if you're already multi-tier; skip if prompts are semantically subtle.

Sources

1.deterministic, sub-millisecond, and entirely offline — no API key, no network, no model call to make it
2.Cheap prompts stay local and hard ones go to the expensive model, so you stop paying top-tier prices for "summarize this" and "fix my typo."
3.Most routers decide by calling a model: a trained classifier, an LLM judge, or a hosted API. That adds latency, cost, and randomness to the exact step meant to save you money.
4.a double-blind test showed the lift doesn't generalize (it caught ~20% of unseen hard prompts and lost to a plain word-count baseline)
5.pip install wayfinder-router

Dev Signal

Get briefs like this in your inbox — free, every weekday.

100+ sources compressed into one 4-minute read. Ranked, cited, implementation-ready.

Read the full issue →All briefs