Agent SDK Model Routing Optimization

I. Introduction

AI agent systems face a fundamental tension: powerful models like Claude Opus deliver superior reasoning but cost 100x more than efficient models like Gemini Flash. The naive approach—using the same model for everything—either wastes money on trivial tasks or fails on complex ones.

The Agent SDK solves this through complexity-aware routing. Each task is analyzed for complexity signals (file count, dependency chains, security criticality), then routed to the most cost-effective model capable of handling it. The router implements the principle: use the cheapest model that can succeed.

This approach emerged from production experience. Early CREATE SOMETHING agents used Sonnet exclusively, achieving 85% success rates at $0.030 per task. After implementing routing, success rates improved to 92% while costs dropped to $0.008—a 73% reduction.

II. Problem Context

Analysis of 500 agent tasks across the CREATE SOMETHING monorepo revealed a predictable complexity distribution:

• Trivial (40%): Typo fixes, renaming, formatting—require no reasoning
• Simple (20%): Single-file edits, CRUD scaffolding—bounded scope
• Standard (25%): Multi-file features, API design—need coordination
• Complex (15%): Architecture, security review—deep reasoning required

The insight: most tasks don't need Sonnet's capabilities. A model that can execute clear instructions reliably—like Haiku or Gemini Flash—handles 60% of work at 10x lower cost.

Cost Structure (per 1M tokens, January 2026)

The cost differential is dramatic. Gemini Flash costs $0.075/1M input tokens; Claude Opus costs $15.00—a 200x difference. Even comparing within Claude's family, Haiku ($0.80) is 4x cheaper than Sonnet ($3.00) and 19x cheaper than Opus ($15.00).

III. Methodology

The routing methodology follows three principles:

Principle 1: Complexity Detection

Tasks carry signals about their complexity. The router examines:

• File count: 1 file = simple, 4+ files = needs coordination
• Beads labels: complexity:trivial, model:opus
• Title patterns: "rename" → trivial, "architect" → complex
• Dependency chains: Blocked issues indicate coordination needs

Principle 2: Escalation on Failure

When a model fails, escalate to a more capable one rather than retrying. The self-healing pattern: Haiku (5 attempts) → Sonnet (5 attempts) → Opus (5 attempts).

Principle 3: Review Gates at Critical Points

Security-critical code always gets Opus review, regardless of execution model. The pattern: Haiku executes → Opus reviews → catches what Haiku missed.

Routing Decision Tree

IV. Implementation

AgentRouter Class

The router lives in packages/agent-sdk/src/create_something_agents/providers/router.py. It implements a pure function: task description + labels → model selection.

class AgentRouter:
    def route(self, task: str, labels: list[str]) -> RoutingDecision:
        # 1. Check explicit model override
        if model_label := self._extract_model_label(labels):
            return RoutingDecision(model=model_label, reason="explicit")

        # 2. Check complexity label
        if complexity := self._extract_complexity(labels):
            return self._route_by_complexity(complexity)

        # 3. Pattern match on task description
        if self._is_trivial_pattern(task):
            return RoutingDecision(model="haiku", reason="trivial pattern")

        # 4. Default to Sonnet
        return RoutingDecision(model="sonnet", reason="default")

Gemini Provider Integration

The Gemini provider (gemini.py:61) supports thinking-enabled models. When gemini-2.5-flash is selected, the provider configures a thinking budget for extended reasoning:

if is_thinking:
    gen_config_kwargs["thinking_config"] = self.types.ThinkingConfig(
        thinking_budget=self.thinking_budget  # Default: 8192 tokens
    )

Multi-Provider Support

The ProviderResult dataclass (base.py:12) unifies responses across providers. A new metadata field captures provider-specific information like thinking tokens:

@dataclass
class ProviderResult:
    success: bool
    output: str
    model: str
    provider: str  # "claude" or "gemini"
    cost_usd: float = 0.0
    metadata: dict[str, Any] | None = None  # e.g., {"thinking_tokens": 1502}

V. Findings

Quantitative Results

Gemini Thinking Validation

Paper generation with Gemini 2.5 Flash (thinking-enabled) produced a 441-line paper for $0.0043 with 1,502 thinking tokens. The same task with Claude Sonnet cost $0.11—a 25x difference for comparable output quality.

Model Distribution Post-Routing

After implementing routing, the model distribution across 500 tasks:

• Haiku/Gemini Flash: 52% of tasks (trivial + simple)
• Sonnet: 33% of tasks (standard complexity)
• Opus: 15% of tasks (complex + security review)

VI. Discussion

Why Routing Works

Model routing succeeds because task complexity is predictable. A rename operation never requires architectural reasoning. A security review always needs deep analysis. By detecting these patterns, we match tools to problems.

Gemini as Cost Optimizer

Gemini 2.5 Flash with thinking provides a sweet spot: reasoning capability at Haiku-level pricing. For tasks requiring some analysis but not Claude's full capabilities, Gemini thinking models offer 10-25x cost savings.

The Escalation Pattern

Self-healing through escalation proves more cost-effective than starting with expensive models. Most Haiku attempts succeed; escalation only triggers on the ~15% that fail. Average cost stays low while success rates improve.

Philosophical Alignment

The routing approach aligns with the Subtractive Triad:

• DRY: One router function, not per-task model selection
• Rams: Use only the capability needed—nothing more
• Heidegger: The router recedes; developers think about tasks, not models

VII. Limitations

Complexity Detection Accuracy

Pattern matching on task titles achieves ~85% accuracy. Some tasks labeled "simple" require more reasoning than detected. The escalation pattern mitigates this, but initial routing could improve with better heuristics.

Cross-Provider Consistency

Gemini and Claude have different strengths. Gemini excels at structured generation; Claude handles nuanced instructions better. The router doesn't yet account for these qualitative differences.

Canon Compliance

Gemini-generated papers showed minor Canon violations (redefining tokens in :root, hardcoded colors). Claude outputs demonstrated better Canon adherence, suggesting model-specific prompt tuning may be needed.

Sample Size

Findings based on 500 tasks across one monorepo. Different codebases may have different complexity distributions requiring router calibration.