For two years, enterprise AI strategy relied mostly on running a bake-off. They spent efforts testing three or four frontier models on sample tasks. Then, standardized the clear winner to negotiate a volume deal.

That playbook is perfect when models change once a year. 2026 was different. Anthropic released 5 Opus models in under 8 months. Google updated 3.7 Flash in just 3 weeks with 3.8 Flash. Mistral also replaced its core tiers in under 1 year.

Routing is now a major enterprise dev tool. Data is now distributed across every major model which has prioritized system over developer for model selection. Cursor launched Cursor Router in July 2026 for its Teams and Enterprise customers. The developer is no longer needed to choose a model; a classifier inspects coding requests and sends it to the appropriate model for that task. The company revealed 60% of its users are working with a single daily-driver model. Its router was trained on 600,000+ live coding requests only to be A/B tested by more than a million. Cursor claimed it reduced 60% cost.

This shift is at the center of this article. The core difference between model picking and model routing lies in what each is asking: the first asks “Which model is best?” and the second asks, “Which model is best for this request, right now, at this cost?”

The second question becomes more relevant when models multiply, prices move, and availability changes quickly.

Four reasons single-model picking breaks

1. No long-term clear winner

Each month a new model emerges as the winner only to be displaced by new one quickly. Major labs launch new tiers, new reasoning modes and new price points multiple times. This reduces the shelf-life of bake-off results, sometimes, shorter than most procurement cycles. By the time companies sign contract, the model becomes outdated. By Perplexity’s count, major AI companies introduced 46 new models in 2025 alone.

Enterprises have noticed this problem. Andreessen Horowitz did a survey last year. The 2025 enterprise survey found 37% of respondents were using five or more models, which was 29% a year earlier. Model differentiated by use case was one of the key factors and not just to hedging against lock-in.

2. One model for all is not a solution

Dataiku’s 2026 CIO research revealed 93% of CIOs are in favor of LLMs specific to use cases because that improve performance. This, however, has a drawback as the selection requires continual evaluation and switching. 

CIOs prefer model switching because models come with some inherent lacks. A model that scores high in multi-file refactoring may fail in writing a commit message. Selecting one model could translate into overpaying on easy work and underperforming on hard work, or both.

3. Government interventions could affect availability

Take Anthropic’s case for instance. In June 2026, Anthropic released Claude Fable 5 and Claude Mythos 5. On 12 June, the US Department of Commerce raised some objections and Anthropic suspended access to both. Again, they lifted the checks on 30 June, and began restoring access on 1 July, as widely reported.

Anthropic brought back the models within weeks. But the lesson had a huge impact. It was a warning for enterprises that access to models can change due to reasons that have nothing to do with performance or price. Hard-wired single models are in the worst mix. Teams with a routing layer and tested fallbacks could shift traffic.

4. Routine work at frontier prices

The majority of requests in most enterprise workloads are simple. They involve tasks like classify, summarize, extract, format. If teams decide to send them to the most capable model, then it becomes like routing every internal email via legal review. It is effective but it costs more than it should.

Things have become worse with more companies leaning towards agents. One task can initiate threads for dozens of model calls. The model choice turns poor when teams multiply threads opened by all the agents and users. It affects the budget line.

What is model routing

A model router is a decision layer that exists between the application and the models. Generated requests must pass through it. It uses task type, complexity, cost, latency, policy or provider health as parameters to decide which model should handle the request.

Routing is not singular but a combined technique:

Strategy	How it decides	Best for	Trade-off
Rule-based	Fixed rules by task, team or data class	Predictable workloads, compliance boundaries	Rules go stale as models change
Cost or budget-aware	Token count, budget thresholds	High-volume, thin-margin traffic	Ignores quality unless paired with evals
Learned classifier	A trained model scores difficulty and picks a tier	Mixed-difficulty traffic	Adds latency; needs training data
Cascade	Try a cheaper model first, escalate if output fails a check	Tasks with a clear pass or fail test	Double cost when escalation is frequent
Fallback and failover	Switch provider on errors, limits or outages	Business continuity	Only works if fallbacks are tested

The evidence so far

UC Berkeley’s RouteLLM study has shown that learnt routers can reduce costs by more than half in some scenarios, without losing quality of response. Routing inside a single model family has saved AWS as much as 30%.

The main platforms have migrated, too. Microsoft Foundry provides a managed model router with a cost mode, a balanced mode, and a quality mode. Google’s Vertex AI provides a model optimizer with identical options. In July 2026, Cursor shipped request-level routing for coding agents at enterprise scale.

Adoption is following. VentureBeat’s September 2026 Pulse survey found 31% of organizations using agent orchestration are already sending repetitive, high-volume steps to cheaper models to keep spending in check. It was the single most frequently cited token-cost control.

Where routing fits in an agent architecture

Routing is operating at the same time on three levels. At the request level, a gateway selects a model per call. Different levels used for planner and worker responsibilities at the agent level; strong model plans, cheaper models follow well defined steps. At the platform level policy defines which teams, data classes and geographies have access to which providers.

Enterprises that are maximizing the benefits are combining all three routing designs.

The hidden costs of routing

Routing is not a switch. Teams that turn it on and walk away generally find that the hard way.

Swapping models is rarely a config change

The same VentureBeat study questioned organizations what occurred when they swapped the model inside a production AI app. 59% said performance decreased down and they had to rebuild the program or revert to the original model after doing this. Only 19% claimed it took almost no engineering work to swap. Half had been there a few weeks or longer or stopped before finishing.

Why is that? A production AI system is more than just a model. It’s prompts and tools and agent loops and tests, all calibrated to the behavior of one model. Make the model different and the assumptions go away.

A router model changes according to request all the time. If one swap leads to this much rework, a router with guardrails mitigates the risk.

Quiet quality drift

The greatest routing failure is invisible. The router forwards a request that required deep reasoning to a cheaper model. The answer looks good on the surface. But there’s something subtly wrong. Nobody knows till a consumer complains.

As various observers pointed out with the release of Cursor, the next competitive battle for routers will be how well they can avoid downgrading requests that actually needed the more powerful model. Only your own evals are a reliable check.

Opacity and control

Managed routers choose the model for you, but often without much visibility into why. That works well for low-stakes tasks. It becomes an issue when you have to tell an auditor what model was responsible for a regulated decision. Log routing decisions every request: which model, why, at what cost.

Latency, caching and overhead

Rule based routing adds almost nothing. Using classifier-based routing can add a few hundred millisecond per request. Switching models mid-conversation can also interrupt prompt caching, diminishing some of the savings. Measure the net, not the headline figure.

Five things to build before you route

Routing is rewarding when the basis is there. If you don’t, you’re just gambling quicker.

1. A model abstraction layer. Decouples prompts, tools and business logic from any provider’s API. If changing a model entails rewriting application code, you are not ready to route.
2. Task-level eval sets. Real samples of each main task type with known satisfactory output. Each model in the routing pool is scored against them. Each new model is graded prior to joining. This is what makes “seems fine” into “proven fine.”
3. A governed gateway. One control point per model invocation, controlling which teams, data classes and regions can invoke which providers. Routing without policy is simply shadow AI with superior economics.
4. Per-request telemetry. Model, reason, latency, tokens, cost, and outcome. Without it, you can’t determine if routing is saving you money or quietly decreasing quality.
5. Tested fallbacks. “Evaluated at least one alternative model per critical task and ready. Anthropic incident in June 2026 revealed why a fallback is crucial.

When picking still wins

For most mixed workloads, routing is the correct default. It’s not the answer every time.

Narrow, high-stakes tasks. Where a regulatory decision is driven by a single workflow, a single validated model with fixed behavior may be easier to evidence and defend than a dynamic mix.

Latency-critical paths. A classifier step can cost more than it saves when every millisecond counts. Rule-based routing or just one fast model might be the preferable call.

Early-stage products. When production is low and requirements are changing rapidly, the savings may not justify the engineering effort. Route later, abstract the model layer and choose well.

Even then, the abstraction and the evals still count. Selecting a model doesn’t imply you can’t alter it.

The bottom line

Model selection was a buying decision. Model routing is engineering feature.

That alters who owns it, and how it is calculated. The question is no longer “Did we pick the right model?” “It’s, ‘Is every request going to the cheapest model that can pass our quality bar, and can we prove that?’

Three questions to ask your team (or your engineering partner) this quarter:

• If our primary model vanished tomorrow, how quickly are we back in production?
• What % of our model budget is on work that a cheaper model can do?
• Are there evals that would capture a routing decision quietly degrading quality?

If you get answers like “weeks”, “we don’t know” and “no”, you’re still picking. The firms that are ahead in 2026 are gone.

The enterprises pulling ahead in 2026 have moved on.

Sources

• Cursor Router launch coverage, VKTR, July 2026
• Cursor Router: what the July 2026 launch means for teams
• How enterprises are building and buying generative AI, 2025, Andreessen Horowitz
• When AI model choices become a switching problem, Dataiku, 2026
• Anthropic restores Fable and Mythos after the US lifts export controls, CoinDesk, July 2026
• VB Pulse Agentic Orchestration Report, VentureBeat, September 2026
• Enterprise AI gateways for multi-model routing, including RouteLLM findings
• LLM router strategies and overhead, including AWS routing measurements