Concepts & pricing
The data model behind the graph, and the rules that make the dollars match your invoice.
The span graph
Every run is a trace; everything inside is a typed span with a parent — that's what makes cost attributable rather than just loggable:
| Kind | What it prices |
|---|---|
trace | the run itself (carries feature/customer/team/session dims) |
agent_step | a logical step; descendant costs roll up to it (by_step) |
llm_generation | a model call — the usual cost driver |
tool_call / retrieval / embedding | tools, RAG lookups, embedding calls |
cache_read / cache_write / retry | explicit cache/retry accounting when you want it |
The versioned price table
Costs are computed from a versioned, immutable price table shipped with the SDK (current: OpenAI GPT-5.x/4.x/o-series + embeddings, Anthropic Claude 4.x/5 families, and $0 entries for local Ollama models). Every span records the table version it was priced with — a point-in-time audit trail for finance. Provider price change? New table version; history is never silently repriced.
from stepcost import register_custom_model
from stepcost.models import ModelPricing, Provider
from decimal import Decimal
table = register_custom_model("my-finetune", ModelPricing(
provider=Provider.OTHER,
input_per_1m=Decimal("1.50"), output_per_1m=Decimal("6.00"),
))
cc = StepCost(project="my-app", price_table=table)
Invoice-grade extraction rules
Most tools multiply prompt_tokens × input_rate and call it a day. Providers
don't bill that way. StepCost's extractors implement the actual billing semantics:
| Line item | Rate | Detail |
|---|---|---|
| Uncached input | 1x input | cached tokens are subtracted from OpenAI's prompt_tokens (they're a subset) |
| Cache read | ~0.1x input | OpenAI cached_tokens · Anthropic cache_read_input_tokens |
| Cache write (5m TTL) | 1.25x input | Anthropic cache_creation, 5-minute TTL |
| Cache write (1h TTL) | 2x input | tracked separately — collapsing the TTLs undercounts 1h traffic by 37.5% |
| Reasoning | output rate | split from visible output (OpenAI o-series/Responses API) |
| Visible output | 1x output | completion minus reasoning — no double count |
| Embeddings | per-model | embeddings responses extract as embedding_tokens |
ValueError
instead of extracting zeros. A model missing from the price table logs a warning once and shows
as "unpriced spans" in every report. All arithmetic is Decimal end-to-end — no float drift.
Waste signals
Heuristic flags over the span graph, each with an estimated dollar leak — deliberately
flags, not prescriptions: missing_cache, retry_loop,
oversized_context, model_oversized. Estimates are over observed
traffic; use --multiplier to project. The missing-cache estimate accounts for the
first call's cache-write premium, and traffic already reading or writing cache is never
flagged.
Drift & coverage
With stepcost sync, every report compares the SDK's
ledger against the provider's, per day: drift audits pricing accuracy
(gate: ≤2% of the invoice; measured 0.0016% on a live Anthropic run, July 2026); coverage exposes spend the SDK never saw.
The provider knows what you'll pay; the SDK knows why — each audits the
other's blind spot.
Privacy & performance
- Metadata-only by default (
PayloadCapture.NONE): token counts, model names, and your business dimensions. Never prompt or response content. - Fully offline: local SQLite (or stdout) sink; no account, no cloud, no network calls except the ones your agent already makes.
- <5 ms p99 span-emit overhead (benchmarked at 20k samples; p50 ~25 µs).
- Spans persist at trace and process exit; sink failures re-queue; async-safe parenting.