Optiak Docs
Gateway

Smart Router

Let Optiak choose the model for each request to lower model costs while keeping quality.

Smart Router chooses a model for each request from the models your application can use. Simple requests go to less expensive models, and harder ones go to more capable models, so you pay for a large model only when a request needs it.

Required role

Organization AdminWorkspace AdminProject AdminApp Admin

Applies to

LLM requests
Admins of an application turn on Smart Router and choose its strategy. The Organization Admin decides which models are available to it, through the organization's model catalog.

Why use it

Most applications send one model a mix of requests. Many of them don't need the most capable model. Smart Router makes that choice for each request, so the application doesn't have to.

  • Lower model costs. In our evaluations of single-request workloads, Smart Router kept accuracy close to that of the strongest model in its pool at about 30% lower cost. See Benchmarks.
  • No routing logic in your code. Send optiak/smart as the model. To change how Smart Router decides, change the strategy in the console, not the code.
  • The same controls as any request. Smart Router chooses only from models that the organization and the application allow, and that meet your data governance rules.

Smart Router fits best in single-request use: chat, question answering, classification, extraction and summarization. For multi-step agents, results depend on the task and on the models in the pool, so benchmark it on your own workload before you use it in production. For agents with heavy tool use, or for tasks that need the most capable model on every step, call that model directly.

How it works

To use Smart Router, send optiak/smart as the model. The short form smart also works.

{
  "model": "optiak/smart",
  "messages": [
    { "role": "user", "content": "Summarize this ticket in two sentences." }
  ]
}

For each request, Smart Router reads the conversation, predicts which eligible model gives a good answer for the request, and selects one based on the application's strategy. The Gateway then sends the request to that model, like any other request. The application's data governance rules, guardrails, inference defaults and fallback models apply as usual. The response has the same shape as the response of a direct model request.

Each request is routed on its own, so turns of the same conversation can go to different models. The application's Requests tab shows a Smart routing step in each request's trace.

You can use Smart Router and direct model IDs in the same application. The model field of each request decides which one is used.

A request that sends optiak/smart is rejected with a 400 error when Smart Router is off for the application, or when your organization's plan doesn't include it.

Strategies

The strategy sets the trade-off between cost and quality for the application's optiak/smart requests.

StrategyBehaviorUse it for
Balanced (default)Balances cost and quality.Most workloads. Start here.
CostPrefers less expensive models, accepting a small loss of quality on some requests.High-volume or simple workloads where cost matters most.

Requests with images or files don't use the strategy. Smart Router ranks the eligible models by quality benchmarks for that kind of input and picks the highest-ranked one. The same happens if the routing service is unavailable: the request goes to the strongest eligible model.

Eligible models

Smart Router chooses only from models that meet all of these conditions:

  • The model has the Smart routing badge in the model catalog. Use the Smart routing filter to list these models.
  • The model is enabled for the organization and the application. See Models and Routing.
  • The model meets your organization's data governance rules, such as data residency and no-training.
  • The model supports what the request uses, such as tool calls, structured output, reasoning or image input.

To see the candidates for an application, open the application, then Configuration, then Savings. The Eligible models list on the Smart Router tab shows the models with the badge that the application has enabled. For each request, Smart Router can exclude more of them: models that break your data governance rules, and models that don't support what the request uses.

How to choose the model pool

The eligible models are Smart Router's pool. Two or three models usually work best: one large model for hard requests and one or two much less expensive ones for simple requests.

  • Mix model sizes. Smart Router saves cost by sending simple requests to a smaller model. A pool of similar models leaves little to choose between.
  • Keep the pool small. Each turn is routed on its own, so the more models in the pool, the more often a conversation can switch between them. Each switch loses the provider's prompt cache from earlier turns, which adds cost and latency to long conversations. With two or three models, consecutive turns switch less often and reuse the cache more.
  • Use the latest version of each model. When a model family releases a new version, enable it and disable the older one. Two versions of the same model are similar models: they take up room in the pool and add switches without giving Smart Router a real choice.
  • Check what else the application enables. Any model with the Smart routing badge that the application enables joins the pool, even if you enabled it for direct calls. Review the Eligible models list after you change the application's models.

Benchmarks

Smart Router is trained and evaluated on models of different sizes from several providers. We benchmarked it against sending every request to a single model, on single requests, such as questions with known answers, and on multi-step agents that use tools.

WorkloadBalancedCost
Single requestsAccuracy close to the strongest model, at a lower costLarger savings, with a small loss of accuracy on some workloads
Multi-step agentsCan match a single large model, depending on the taskLower cost, especially when routing between a large model and a much less expensive one

What this means for your application:

  • Use it for single requests. For chat, question answering, classification or extraction, you can expect accuracy close to your strongest model at about 30% lower cost.
  • For agents, enable one large model and one small one. A large model paired with a much less expensive one keeps most of the large model's results at a lower cost. See How to choose the model pool.
  • Compare with your least expensive model. For some workloads, it's nearly as accurate as routing, at a fraction of the cost.
Benchmark Smart Router on your own workload first. Results depend on your requests and on the models you enable, so ours may not carry over. Before you route production traffic, run a representative sample of your requests with optiak/smart and with the model you use today. Compare quality and cost per request in Observability. This matters most for agents.

Set it up

  1. Make eligible models available. An Organization Admin enables models with the Smart routing badge in the organization's model catalog. See Models and Routing.
  2. Turn on Smart Router for the application. It's off by default. In the console, open the application, then Configuration, then Savings. On the Smart Router tab, turn on Enable smart routing and choose an Optimization strategy. Check the Eligible models list. If it shows a No eligible models warning, enable at least one model with the badge for the application; until then, optiak/smart requests fail unless the application has fallback models.
  3. Send optiak/smart as the model in the requests you want to route.
Turning on Smart Router doesn't route your existing requests. Smart Router routes only the requests that send optiak/smart as the model. Requests that name a model keep going to that model. To route traffic, change the model value in your code.

Limits and behavior

  • Custom provider models are not eligible. Call them by their model ID.
  • Provider-specific request content can't be routed between providers: provider-specific tools, encrypted reasoning, and conversation state stored at a provider. Send these requests to a specific model.
  • Prompt caching: when consecutive turns go to different models, the provider's prompt cache from earlier turns isn't reused. This can reduce savings for long conversations. A small pool limits how often this happens; see How to choose the model pool.

For error details, see Using the API.

Copyright © 2026