The Token Tax

If you are running an agentic coding harness in production today, you are likely paying a "token tax" that has nothing to do with intelligence — and everything to do with architecture. You have read the headlines about "up to 90% cost reduction," but those numbers often hide the most critical variable: which model you are actually running, and how much of what you are sending it is waste.

The same agent, the same workload. The difference is not a magic prompt trick or a fleeting discount. It is the result of combining frontier-tier model selection with a token-efficient agentic harness architecture.

If your reaction is "I should just switch to a cheaper model," hold that thought. Model selection is the foundation — but it unlocks only about 20% of the savings on its own. The remaining 80% comes from an architecture that stops burning tokens you never needed to send. This article is that architecture.

A) The Agentic Token Tax: Why Costs Are Exploding

The root cause is structural. A typical chatbot query uses 2,000 to 4,000 tokens. An agentic workflow — planning, tool calls, retrieval, validation, retries, and self-correction — can easily consume 50,000 to 500,000 tokens for a single user task.

Agents do not stop after one call. They loop. And every loop re-transmits the full conversation history, system prompts, and tool definitions. A 10-step debugging session that starts with 5,000 input tokens can balloon to 50,000+ by the final step as context accumulates.

At Opus 4.7 pricing ($5/MTok input), a single 10-step session with escalating context costs roughly $1.38 in input tokens alone (average 27,500 tokens × 10 steps = 275K tokens). With smart context management — summarizing previous steps, dropping irrelevant files, resetting context between phases — the same session might consume 80K tokens total, costing $0.40. That is a 3.5x reduction from a single optimization.

Andrej Karpathy, speaking at AI Ascent 2026, put it bluntly: roughly 90% of what developers spend on AI coding assistance is wasted — not on bad outputs, but on unnecessary token consumption. He called the fix "context engineering" — the discipline of curating exactly what enters the context window and, equally important, what stays out.

Karpathy's sentence, widely quoted across developer media since June, was this: "Context engineering is the new vibe coding." Every cost-cutting article on Dev-X is downstream of that sentence. The surface fix is "switch to a cheaper model." The actual lever is upstream: stop sending tokens you did not need to send.

B) The Model Selection Multiplier: Why the Chinese AI Labs Changed the Math

Every article about agentic harnesses assumes you are optimizing within Western pricing. But the Chinese AI labs have permanently restructured the cost curve — and they have not just cut prices; they have maintained frontier-tier performance.

In April 2026, DeepSeek launched V4-Pro: a 1.6 trillion parameter Mixture-of-Experts model that scores 80.6% on SWE-bench Verified. Its output pricing is $0.87 per million tokens — compared to Claude Opus 4.7 at $25. That is a 29x difference on every output token.

The obvious objection: "DeepSeek is not as good as Claude." Here are the facts. On SWE-bench Verified, Claude Opus 4.7 scores 87.6%. DeepSeek V4-Pro scores 80.6%. That is a real gap — 7 percentage points on a demanding coding benchmark. But consider:

• The gap narrows to zero on most task-specific coding benchmarks when combined with a good harness. The Endor Labs agentic harness experiment (2026) demonstrated that harness architecture alone shifts a model's coding performance by 12.8 percentage points — larger than the gap between Opus and V4-Pro.
• For 70% of agentic coding turns — file reads, syntax fixes, single-line edits, formatting — the quality difference between any two frontier models is imperceptible.
• At a 29x cost differential, you can run DeepSeek V4-Pro as the default, reserve Claude Opus for the hardest 10% of turns, and still come out ahead on both cost and quality.

This is not about "cheap alternatives." This is about frontier-tier models at a fraction of the cost. DeepSeek V4-Pro, Kimi K2.6 (excellent for long-context coding agents with 200-300 sequential tool calls), and GLM-5.1 are not budget options — they are top-tier models that happen to be priced for scale. On coding benchmarks that matter for agentic harnesses, the gap between the cheapest and most expensive option in this table is often smaller than the gap between two different harness architectures running the same model.

C) The Five Stackable Levers: Building a Token-Efficient Harness

Model selection gets you to 1/29th the cost. The remaining 95% reduction — from $87 to ~$30 — requires an architecture that compounds savings across every layer of the agentic loop.

The key insight: these levers stack multiplicatively, not additively. A 40% saving on input tokens combined with an 80% saving on routed turns is not 120% — it is a compound reduction that turns a $1,750 bill into $30.

Lever 1: Prompt Caching — 40-90% on input tokens

Prompt caching is the highest-ROI single lever for agents with a stable system prompt or large reference context. Anthropic's implementation returns cached tokens at one-tenth the price of fresh input tokens (a 90% discount per token) once a prefix is cached. DeepSeek's implementation is even more aggressive at $0.0036/MTok cached.

How to implement it properly:

• Deterministic tool ordering: Sort tools alphabetically by ID. Some gateways re-sort per session, silently invalidating the cache — a common and expensive bug.
• Frozen boilerplate: Keep system-prompt boilerplate static. Inject team-specific instructions as a separate cache block downstream, not by editing the prefix.
• Ephemeral context blocks: Promote project context to a cache_control: ephemeral block so it cools at 5 minutes of inactivity rather than between turns — preventing unnecessary re-caching on closely-spaced requests.

Lever 2: MCP Tool Compilation — 50-70% on tool tokens

When an agent has access to dozens of MCP servers, each with dozens of tools, the model spends most of its context budget merely reading tool definitions it will never use. Instead of advertising N individual tool definitions and round-tripping each invocation, compile MCP tools into a Python module exposed as a single high-level tool, execute_python(code). The model writes Python that calls compiled functions; the gateway sandboxes execution.

The Bifrost Code Mode benchmark (Maxim, May 2026): At 508 tools across 16 MCP servers, Code Mode reduced input tokens from 75.1M to 5.4M — a 92.8% reduction — while maintaining a 100% pass rate. Estimated cost dropped from $377 to $29. The savings compound as tool count grows, because classic MCP loads every tool definition on every request, while Code Mode's cost is bounded by what the model actually reads.

What this looks like in practice: A team running 10-15 MCP servers typically sees 25-45% token savings on the subset of tools that can be cleanly compiled. Implement a held-out evaluation gate: promote a tool to compile-mode only if it scores within 1% of tool-mode on task completion and code-correctness rubrics.

Lever 3: Semantic Caching of Tool Results — 10-30% on tool-call subset

When the same filesystem.read returns the same file in two sessions an hour apart, the model re-serializes the same payload twice. A semantic cache keyed on tool name plus a content-aware hash returns the previous payload without round-tripping the MCP server.

Real-world performance: The hit rate stabilizes around 35-55% within a week of real usage in teams running agentic coding harnesses. Importantly, this compounds with Lever 1: cached tool responses become part of a cacheable prefix, amplifying the savings from prompt caching. The implementation cost is low — a Redis-backed cache with TTL-based invalidation — and the ROI materializes within days, not weeks.

Lever 4: Model Right-Sizing / Routing — 80-95% on routed turns

A coding session has many simple turns — file reads, syntax fixes, formatting, single-line edits — that cheaper models handle indistinguishably from frontier models. The standard distribution in production agentic harnesses follows a 70/20/10 pattern:

The routing strategy in practice: Use DeepSeek V4-Flash ($0.14/$0.28) for the 70% — it handles classification, extraction, and filtering tasks at a fraction of frontier cost. Route drafting and code generation to Kimi K2.6 ($0.95/$4.00) or a mid-tier model. Reserve Claude Opus or DeepSeek V4-Pro exclusively for the 10% that genuinely requires frontier reasoning — architecture decisions, multi-step debugging, final code review.

Teams that implement this routing layer report effective blended costs of $0.80-$1.50 per million tokens — versus $5.00/$25.00 if every call goes to Opus. That is a 10-20x improvement on the turns that matter least.

Lever 5: Context Window Pruning — 15-25% on long sessions

Agents accumulate conversation history, tool results, and document chunks that grow unbounded across turns. The Agentic Token Reducer (ATR) pattern addresses this by breaking context into five artifact modes served at the cheapest representation that preserves task utility:

• Pointer (~20 tok): "Where is this file?" — path, digest, symbol references
• Map (~50 tok): "What does it export?" — imports, exports, public API surface
• Delta (~100 tok): "What changed?" — hunks from previous read
• Slice (~200 tok): "Show me lines 10-25" — targeted excerpt with line numbers
• Full (~800 tok): "I need everything" — complete content for editing

A real-world benchmark of the ATR pattern showed an agent consuming 42,646 tokens vs. 118,622 without — a 64% reduction — with identical output quality and 54% fewer tool calls. The mechanism is simple: most agent reads do not need the full file. A pointer, map, or delta suffices for planning, exploration, and impact analysis. Only the final edit step needs the full content.

D) The Complete Math: From $1,750 to $30

These five levers compound. Each one reduces the token base that the next one operates on, creating a multiplicative — not additive — effect.

That is a 98% total reduction from the unoptimized baseline. The "up to 90%" figures you see elsewhere assume you are optimizing within Western pricing. If you start with DeepSeek V4-Pro or Kimi K2.6 as the default, the math becomes almost absurd: a full agentic coding harness for less than the cost of a single ChatGPT Plus subscription.

E) Building the Harness: A Practical Starting Point

The architecture for a token-efficient agentic harness follows a specific pattern. Here is the minimal viable stack — implement these layers in order, and stop when your cost is acceptable:

1. Gateway Layer: Centralize routing policies, semantic caching, budget enforcement, and observability. LiteLLM or a custom gateway works. Instrument traces to log cost and latency per span before you optimize anything — measure first, cut second.
2. Router: Classify each request by complexity before the call. Route simple sub-tasks to smaller/cheaper models; escalate only when confidence is low or the task matches known complexity thresholds. Start with a hardcoded classification table (70%→Flash, 20%→Kimi, 10%→Pro) before building a learned classifier.
3. Caching Layer: Enable prompt caching for stable system-prompt prefixes. Implement semantic caching for repeated tool responses. This is the highest-ROI single change — implement first, optimize the rest later.
4. Model Pool: Maintain a pool of models across tiers (Flash, Mid, Frontier) with automatic fallback chains. Use DeepSeek V4-Flash as the default, Kimi K2.6 for drafting, and reserve Claude Opus or DeepSeek V4-Pro for the hardest cases.
5. Context Pruning: Implement the ATR pattern — pointer/map/delta/slice/full — for all file reads. This alone yields 64% fewer tokens on content reads with no quality degradation.

The biggest mistake teams make is standardizing too early on one model or one agent framework. In 2026, model quality, pricing, policy access, and tool capabilities are changing too quickly. A better strategy is to standardize the evaluation and governance layer — task taxonomy, approved tools, cost logging, review requirements, and fallback options — and let individual agent configurations compete under those rules.

The Bottom Line

The "98% token tax" is not a bug. It is the result of defaulting to the most expensive option, sending everything as context, and never measuring what you are actually paying for.

Karpathy's insight was that context engineering matters more than model selection. The data bears this out: a well-designed harness around a frontier Chinese model outperforms a bad harness around Claude Opus — and costs 98% less to operate. The model is the foundation, but the harness is where the leverage lives.

The moat is not the model. It is the architecture that orchestrates it.

Here is a practical next step: start with measurement. Log every agent turn — model used, tokens consumed, cache hits, routing decision — for one week. You will likely find that 70% of your spend goes to context you could eliminate with the first two levers alone. Implement prompt caching and model routing. That gets you 80% of the way. The remaining 20% — MCP compilation, semantic caching, context pruning — is optimization, not transformation. Full 98% results typically require the complete stack and 60-90 days of tuning.

References

• DeepSeek. (2026, April). DeepSeek V4 Pro API Pricing. DeepSeek API Docs. api-docs.deepseek.com
• Anthropic. (2026, April). Claude Opus 4.7 Pricing. Claude Platform Docs. platform.claude.com
• DeepSeek. (2026, April). DeepSeek V4 Pro Evaluation Results. Hugging Face. huggingface.co/deepseek-ai/DeepSeek-V4-Pro
• Anthropic. (2026, April). Introducing Claude Opus 4.7. Anthropic Blog. anthropic.com
• Maxim AI. (2026, May). Code Mode and the Architecture of Token-Efficient MCP Agents. Bifrost Blog. getmaxim.ai
• Bifrost. (2026). MCP Code Mode Benchmark Report. GitHub — maximhq/bifrost-benchmarking. github.com/maximhq/bifrost-benchmarking
• Agentic Token Reducer. (2026). ATR: Agentic Context Management. GitHub — timothydillan/agentic-token-reducer. github.com/timothydillan/agentic-token-reducer
• Karpathy, A. (2026, June). Context Engineering talk at AI Ascent 2026. Multiple developer media summaries. bittide.aicompass.dev
• Moonshot AI. (2026, April). Kimi K2.6: Next-Generation Multimodal Model. Kimi Blog. platform.kimi.ai
• Zhipu AI. (2026, April). GLM-5.1 API Pricing. Z.AI Developer Documentation. docs.z.ai