Optiak Docs
Observability

Metrics

The definition of every metric in the console, and how Optiak calculates it.

This page defines every number on the Dashboard and on an application's Overview, so you know what each one counts and how Optiak calculates it. The metric names match the labels in the console.

How metrics are calculated

These rules apply to every metric on the Dashboard and on an application's Overview.

RuleWhat it means
PeriodYou select Today, 7d, 30d or Custom. Days start at midnight in the Europe/Madrid time zone.
ComparisonThe console compares each figure with the same figure for the period of the same length immediately before. Change % is the change divided by the earlier value. If the earlier value is 0, the console shows no Change %.
RequestOne API request from an application to the Gateway. Failed and blocked requests count.
Provider callOne attempt to send a request to a provider. A retry or a fallback adds a call. A request that a guardrail blocks has no call.
Empty valuesWhen a rate, an average or a percentile has no data, the console shows no value. It does not show 0.

Charts and tables

Each section shows one metric at a time. You can show it as a chart or as a table.

  • Grouping. You can split a metric by Application, Project, Provider, Model, Tool or Reason. Total shows the metric with no split. Each metric offers only the groupings that apply to it.
  • Bucket. The bucket is the time step of the chart: hour, day, week or month. The console picks the smallest step that gives 62 points or fewer. Weeks start on Monday. The first and the last bucket can be partial.

When the rows and the total differ

Grouped by Provider or Model, the rows of some metrics do not add up to the total.

It affects Inference requests, Successful requests, Errored requests and Failed requests. Average cost per request and Success rate divide by these counts, so their Provider and Model rows are affected too. With every other grouping, the rows of a count, a token sum or Spend add up to the total.

Two causes change the sum of the rows:

  • Fallback or retry. One request reaches more than one provider or model, and counts once under each. The rows add up to more than the total.
  • Blocked before any provider call. The request counts in the total and in no row. The rows add up to less than the total.

Both causes can occur in the same period and cancel out. The rows then match the total, but they still count some requests twice and leave others out.

  1. Request A OpenAI1 provider call
  2. Request B OpenAIfallback Anthropic2 provider calls
  3. Request C Guardrail blocks it0 provider calls
Three requests, and the provider calls each one makes

Fallback

Request B counts under OpenAI and under Anthropic.

ProviderInference requests
Total1
OpenAI1
Anthropic1
Sum of the rows2 ≠ 1

Block

Request A counts under OpenAI. Request C counts in no row.

ProviderInference requests
Total2
OpenAI1
Sum of the rows1 ≠ 2

Both

Requests A, B and C in one period.

ProviderInference requests
Total3
OpenAI2
Anthropic1
Sum of the rows3 = 3
In the third case the rows match the total, but request B counts twice and request C counts in no row.

Usage

MetricDefinitionHow it's calculatedUnitWhere it appears
Inference requestsInference requests from applications to the Gateway. Failed and blocked requests count.Count of requests. See when the rows and the total differ.CountDashboard: summary card and chart. Projects list, for the last 30 days.
Input tokensPrompt tokens sent to providers.Sum of input tokens of every provider call, retries included. The sum includes tokens that the provider reads from or writes to its prompt cache.TokensDashboard: summary card and chart
Output tokensTokens that providers generated.Sum of output tokens of every provider call. The sum includes reasoning tokens.TokensDashboard: summary card and chart
Total tokensAll tokens of every provider call.Input tokens + output tokens.TokensDashboard: chart
Reasoning tokensOutput tokens that a model used to think before it answered.Sum of reasoning tokens that providers report. Some providers report 0.TokensDashboard: chart
Cache hit rateShare of input tokens that providers read from their prompt cache.Cached input tokens ÷ input tokens.%Dashboard: summary card and chart

Cost

MetricDefinitionHow it's calculatedUnitWhere it appears
SpendWhat the provider calls cost at list price.Sum of the price of every priced provider call, retries included. An unpriced call adds 0.USDDashboard: summary card and chart
Average spend per daySpend spread evenly over the period.Spend ÷ days in the period. A day with no spend still counts.USDDashboard: summary card
Average cost per requestWhat one request costs on average.Spend ÷ requests. Requests that cost nothing count. By provider or model, see when the rows and the total differ.USDDashboard: summary card and chart
Blended price per million tokensThe average price of one million tokens across all models.Spend × 1,000,000 ÷ tokens of priced calls. The divisor leaves out unpriced calls.USDDashboard: summary card and chart
USD is the standard currency. Costs are reported in USD because this is the currency providers use to price their models.

Where prices come from

Optiak prices each provider call when the request runs. It uses the price of the model that served the call:

  • Built-in models use the provider's list price. The price includes the cache read and cache write rates, the long-context rates, and the priority and flex tiers where the provider has them.
  • Custom provider models use the prices that an Organization Admin sets under Pricing (optional): input and output, plus cache read and cache write if set. A cache price left blank uses the input price. A price of 0 makes cached tokens free. See Custom providers.
Spend is an estimate. It uses list prices, so it is not your provider invoice. Spend does not include discounts, credits or committed-use prices. A call to a model with no price is unpriced. An unpriced call adds 0 to Spend. So when some models have no price, Spend is lower than the list-price cost.

How a change splits into volume, mix and rate

In the Table view, each row shows its part of the group's change over the previous period, and splits that change into key drivers:

ColumnMeaning
Share of changeHow much of the group's whole change this row caused, in percentage points. The rows add up to the group's Change %, except when the rows and the total differ.
VolumeThe group's total tokens changed. This row kept its share.
MixRows gained or lost share. The group's total stayed the same.
RateThe row's own price for each token changed.
EnteredThe row was at 0 in the earlier period. This column holds its whole change.
LeftThe row is at 0 in the later period. This column holds its whole change.

Volume-Mix-Rate is a standard decomposition used in financial and operational analytics to decompose variance in costs into distinct drivers: overall quantity changes (Volume), changes in product or customer composition (Mix), and changes in unit costs (Rate). Here these concepts are specialized to the cost of requests to model providers.

Worked example: why Spend changedA change of +$700, split by provider, with every number shown.OpenClose
Why Spend changed, by providerBefore$1,000The continuing providers used more tokens in total.Volume+$659Tokens moved from OpenAI to the cheaper Anthropic.Mix−$89Anthropic's price for each token went up.Rate+$80Mistral had no spend in the earlier period.Entered+$100Cohere has no spend in the later period.Left−$50After$1,700
Example numbers. Spend went from $1,000 to $1,700. The five parts add up to the +$700 change.

Spend grouped by Provider. M means million tokens. Amounts are rounded to whole dollars.

ProviderTokensPrice per 1M tokensSpendVolumeMixRateEnteredLeft
OpenAI50M → 60M$12 → $12$600 → $720+$358−$238$0
Anthropic50M → 110M$7 → $8$350 → $880+$301+$149+$80
Mistral0 → 20M$5$0 → $100+$100
Cohere10M → 0$5$50 → $0−$50
Total110M → 190M$1,000 → $1,700+$659−$89+$80+$100−$50
  • Volume +$659. OpenAI and Anthropic used 100M tokens before and 170M after. That growth, at each provider's share and price, adds $659.
  • Mix −$89. OpenAI's share of tokens fell from 50% to 35%, and Anthropic's grew from 50% to 65%. Anthropic is cheaper, so the move lowers Spend. OpenAI loses $238, and Anthropic gains $149.
  • Rate +$80. Anthropic's price went from $7 to $8 per 1M tokens, on 80M tokens on average. OpenAI's price did not change, so its Rate is $0.
  • Entered +$100. Mistral had no spend before, so all of its $100 is Entered.
  • Left −$50. Cohere has no spend now, so all of its $50 is Left.

Volume, Mix, Rate, Entered and Left add up to the row's Share of change.

Average cost per request and Blended price per million tokens are rates, so they have no Volume column. Their change splits into these parts:

  • Rate: the row's own rate changed. Its weight stayed the same.
  • Mix: weight moved from some rows to other rows. The rates stayed the same.
  • Entered and Left: the row had no weight in one of the two periods. Optiak calculates this part from the difference between the row's rate and the group's average rate.

The weight of a row is the divisor of its rate. It is requests for Average cost per request, and priced tokens for Blended price per million tokens.

Worked example: why Average cost per request changedA change of +$0.06, split by provider. A new provider can lower a rate.OpenClose
Why Average cost per request changed, by providerBefore$0.97Anthropic's cost per request went up.Rate+$0.09Requests moved toward providers that cost less than the average.Mix−$0.03Mistral joined, and it costs less than the average.Entered−$0.05Cohere left, and it cost less than the average.Left+$0.05After$1.03
Example numbers. The average went from $0.97 to $1.03. The four parts add up to the +$0.06 change. The axis starts at $0.80.

Average cost per request grouped by Provider. The weight of each row is its share of requests. The group's average cost per request over the two periods is $1.00.

ProviderRequestsCost per requestShare of requestsRateMixEnteredLeft
OpenAI50 → 80$1.20 → $1.2050% → 40%$0.00−$0.02
Anthropic40 → 100$0.80 → $1.0040% → 50%+$0.09−$0.01
Mistral0 → 20$0.500% → 10%−$0.05
Cohere10 → 0$0.5010% → 0%+$0.05
Total100 → 200$0.97 → $1.03+$0.09−$0.03−$0.05+$0.05
  • Rate +$0.09. Anthropic's cost per request went from $0.80 to $1.00. Its average share is 45%, so the average rises by $0.09. OpenAI's cost did not change, so its Rate is $0.00.
  • Mix −$0.03. OpenAI costs more than the $1.00 average, and its share fell from 50% to 40%: −$0.02. Anthropic's mean cost, $0.90, is below the average, and its share grew: −$0.01.
  • Entered −$0.05. Mistral costs $0.50, half the average, and now has 10% of requests. A cheap new row pulls the average down.
  • Left +$0.05. Cohere cost $0.50 and had 10% of requests. A cheap row that leaves pushes the average up.

The other metrics show only Share of change. Percentiles show no split.

Reliability

When you group by Provider or Model, the counts below are provider calls, not requests. See when the rows and the total differ.

MetricDefinitionHow it's calculatedUnitWhere it appears
Successful requestsRequests that succeeded.Count of successful requests. By provider or model: successful calls.CountDashboard: summary card and chart
Errored requestsRequests that ended in an error. Blocked requests are not errors.Count of errored requests. By provider or model: errored calls.CountDashboard: chart
Failed requestsRequests that did not succeed.Errored requests + blocked requests. By provider or model: errored calls.CountDashboard: summary card and chart
Success rateShare of requests that succeeded, without the requests that a guardrail blocked. A block shows that the policy works. It is not a failure.Successful requests ÷ (successful requests + errored requests). Blocked requests are not in the divisor. By provider or model: over calls.%Dashboard: summary card and chart
End-to-end latency p50The median time from request to response. Half of the requests are faster.50th percentile of request duration. The percentile leaves out requests with no measured time.msDashboard: chart
End-to-end latency p95The time that 95% of requests stay under.95th percentile of request duration. The percentile leaves out requests with no measured time.msDashboard: summary card and chart
Provider latency p50The median time a request spent in provider calls.50th percentile of the provider time per request, retries added. The percentile leaves out requests with no provider call.msDashboard: chart
Provider latency p95The provider time that 95% of requests stay under.95th percentile of the provider time per request, retries added. The percentile leaves out requests with no provider call.msDashboard: chart
Top failure reasonsThe most frequent reasons why requests failed.Count of requests with a failure reason, grouped by reason. Blocked requests count.CountDashboard: list

Governance & Security

These metrics come from Guardrails. One request can run several checks.

MetricDefinitionHow it's calculatedUnitWhere it appears
Total evaluationsGuardrail checks that ran.Count of checks.CountDashboard: summary card and chart
Blocked requestsRequests that a guardrail stopped.Count of blocked requests. A request that several checks block counts once.CountDashboard: chart
Blocked rateBlocked requests for each guardrail check that ran.Blocked requests ÷ checks that ran. The divisor is checks, not requests.%Dashboard: summary card and chart
Guardrail added latencyThe average time of one guardrail check.Total check time ÷ checks. A check with no measured time counts as 0 ms.msDashboard: summary card and chart
Audit detection rateShare of checks that found a problem.Checks that detected, blocked, redacted or replaced ÷ checks that did not end in an error.%Dashboard: summary card and chart
Violations by applicationChecks that found a problem, for each application.Count of checks that detected, blocked, redacted or replaced, grouped by application.CountDashboard: table

Data Integrations

These metrics come from the retrieval engine in Data Integrations. One retrieval is one search. One request can run several retrievals.

MetricDefinitionHow it's calculatedUnitWhere it appears
Average chunks per requestDocuments that retrieval added to one request, on average.Documents retrieved ÷ requests that used retrieval.CountDashboard: summary card and chart
Retrieval failuresRequests where retrieval failed.Count of requests with a failed retrieval. A request counts once. No value if no retrieval ran.CountDashboard: summary card and chart
Tokens contributed by retrieved chunksTokens of the documents that retrieval added to prompts.Sum of document tokens. No value if no retrieval ran.TokensDashboard: summary card and chart
Retrieval added latencyThe average time of one retrieval.Total retrieval time ÷ retrievals. A retrieval with no measured time counts as 0 ms.msDashboard: summary card and chart

Tools

These metrics come from tool calls through the MCP Gateway. See Tools. A tool call is an MCP tools/call request. A tool list request is an MCP tools/list request.

MetricDefinitionHow it's calculatedUnitWhere it appears
MCP tool callsRequests to run a tool.Count of tools/call requests. The count includes calls that ended in an error before the tool ran.CountDashboard: summary card and chart
MCP tool list requestsRequests that ask a server for its tools. They run no tool.Count of tools/list requests.CountDashboard: chart
Successful MCP tool callsTool calls that completed without an error.Count of successful tool calls.CountDashboard: chart
Failed MCP tool callsTool calls that ended in an error.Count of failed tool calls. The count includes calls to an unknown tool and calls that went past the time limit.CountDashboard: summary card and chart
MCP tool call success rateShare of tool calls that succeeded.Successful MCP tool calls ÷ MCP tool calls. The divisor leaves out tool list requests.%Dashboard: summary card and chart
MCP tool latency p50The median time a tool ran.50th percentile of tool run time per call. The percentile leaves out calls where no tool ran.msDashboard: chart
MCP tool latency p95The tool run time that 95% of calls stay under.95th percentile of tool run time per call. The percentile leaves out calls where no tool ran.msDashboard: summary card and chart
Top MCP failure categoriesThe most frequent causes of failed MCP tool calls.Count of failed MCP tool calls, grouped by the tool's failure category. A call with no category counts under its request failure reason.CountDashboard: list

Application overview

An application's Overview shows the Dashboard sections for that application. The Overview uses the metric definitions on this page, with the data of that application only. Where a table says Dashboard, the Overview shows the same card or chart. The Overview has no Application or Project grouping.

  • Application: the Overview and Requests tabs of one application.
  • Budgets: spend limits for projects and applications.
Copyright © 2026