Workload Benchmark & Cost Model
Multi-Turn Support Agent LLM Cost Calculator
A 5-turn live support conversation re-sends the system/policy prefix (2,500 tokens) plus accumulating chat history (800 Turn-1 query, growing by 300 user follow-up + 350 assistant reply = 650 tokens/turn) across 1,500 sessions/day (7,500 total turns/day).
Support Agent (Multi-Turn) Cost Comparison (20 models)
100% Client-Side · Zero Prompt TransmissionAvg input/turn: 4,600 tok · Daily turns: 7,500 · Registry verified: 2026-11-12
| Model & Host | Active Tier & Rules | Eff. In / Out ($/1M) | Cache Read ($/1M) | Daily | Monthly (30d) | $/1K Turns | Savings vs Sync |
|---|---|---|---|---|---|---|---|
LOWESTGPT-6 Luna OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $0.100 $0.500 | $0.0100 | $2.66 | $79.69 | $0.354 | -44.2% $63/mo saved |
Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 100K) | $0.100 $0.500 | $0.0100 | $2.66 | $79.69 | $0.354 | -44.2% $63/mo saved |
Mistral Small 4 (2603)Open-Weight Mistral·Mistral La Plateforme·256K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 256K) | $0.150 $0.600 | $0.0150 | $3.56 | $106.82 | $0.475 | -47.3% $96/mo saved |
GPT-OSS 120B (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 131K) | $0.150 $0.600 | $0.0750 | $4.98 | $149.34 | $0.664 | -26.3% $53/mo saved |
DeepSeek Flash (V4.1-Flash-1017)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.300 $1.200 | $0.0060 | $6.55 | $196.63 | $0.874 | -51.4% $208/mo saved |
DeepSeek V4.1 Flash (Together AI)Open-Weight Together AI·Together AI Serverless (FP8)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.300 $1.200 | $0.0060 | $6.55 | $196.63 | $0.874 | -51.4% $208/mo saved |
GPT-OSS 120B (Together AI)Open-Weight Together AI·Together AI Serverless (MXFP4)·131K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 131K)
| $0.150 $0.600 | — | $6.75 | $202.50 | $0.900 | 0.0% |
GPT-OSS 120B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·128K max in Verified 2026-11-12Official Docs ↗ | US Standard (<= 128K)
| $0.150 $0.600 | — | $6.75 | $202.50 | $0.900 | 0.0% |
Llama 4 Maverick 17B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·1000K max in Verified 2026-11-12Official Docs ↗ | US Standard (<= 1M)
| $0.240 $0.970 | — | $10.83 | $324.79 | $1.444 | 0.0% |
Mistral Large 4Open-Weight Mistral·Mistral La Plateforme·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.700 $2.100 | $0.0700 | $14.78 | $443.36 | $1.971 | -50.2% $447/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Cache OnPromo ends 2026-12-31 Intro Promo (<= 1M, thru 2026-12-31)
| $0.750 $3.750 | $0.0750 | $24.48 | $734.27 | $3.263 | -31.5% $337/mo saved |
DeepSeek V4 Pro (0813)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $1.320 $3.960 | $0.0440 | $25.79 | $773.69 | $3.439 | -53.9% $904/mo saved |
Qwen3.8 27B Preview (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 131K)
| $0.800 $4.000 | — | $38.10 | $1,143.00 | $5.080 | 0.0% |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $2.000 $10.000 | $0.1000 | $50.77 | $1,522.97 | $6.769 | -46.7% $1,335/mo saved |
Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $2.000 $10.000 | $0.1000 | $50.77 | $1,522.97 | $6.769 | -46.7% $1,335/mo saved |
Claude Sonnet 5.5 (Amazon Bedrock) AWS Bedrock·Amazon Bedrock (Cross-Region Inference)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Global CRIS (<= 1M) | $2.000 $10.000 | $0.1000 | $50.77 | $1,522.97 | $6.769 | -46.7% $1,335/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Cache OnPromo ends 2026-11-30 Intro Promo (<= 1M, thru 2026-11-30)
| $2.000 $10.000 | $0.1000 | $63.51 | $1,905.24 | $8.468 | -33.3% $952/mo saved |
Gemini 3.1 Pro Preview Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 200K)
| $2.000 $12.000 | $0.2000 | $70.73 | $2,121.93 | $9.431 | -29.6% $893/mo saved |
Claude Fable 5.2 Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $8.000 $40.000 | $0.2000 | $198.34 | $5,950.13 | $26.445 | -47.9% $5,480/mo saved |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $10.000 $50.000 | $1.0000 | $265.64 | $7,969.23 | $35.419 | -44.2% $6,318/mo saved |
Budgeting & Tokenizer Disclaimer: Estimates assume USD list pricing verified on 2026-11-12. Actual token counts vary across provider tokenizers (for example, Claude 4.7+ uses a newer tokenizer that generates ~30% more tokens on identical prose than legacy models), and reasoning models bill hidden thinking tokens as output tokens.
Token Anatomy & Scaling Assumptions
- Static Prefix (2,500 tokens): Brand tone policy, refund rules, escalation guardrails, and support tool schemas.
- Turn-by-Turn Input Growth: Turn 1 sends 3,300 input tokens (2,500 + 800); Turn 2 sends 3,950; Turn 3 sends 4,600; Turn 4 sends 5,250; Turn 5 sends 5,900—totaling 23,000 input tokens per session (4,600 average input tokens/turn).
- Output Per Turn (350 tokens): Helpful support response and/or tool-call arguments (1,750 output tokens across 5 turns).
- Daily Scale: 1,500 sessions/day × 5 turns = 7,500 API turns/day over 16 active hours/day (225,000 turns/month).
Worked Note: For 1,500 5-turn sessions/day (4,600 avg input/turn, 350 output/turn, 75% hit, 12 writes/hr over 16h), Claude Sonnet 5.5 costs $50.77/day ($1,522.97/mo) with caching vs. $2,857.50/mo uncached—saving $1,334.53/mo (46.7%). Switching to a lightweight tier like GPT-6 Luna ($0.10/$0.50, $0.01 read, $0.125 30m write) drops the same 225,000-turn monthly load to $2.66/day ($79.69/mo vs. $142.88/mo uncached).
Caching & Batch Strategy
Multi-turn agents live or die by prompt caching because Turns 2–5 re-send prior turns verbatim as a growing prefix. At 75% cache hit rate and 12 cold writes/hour across 16 hours, caching cuts monthly costs by ~44%–47%. Watch Gemini 4 Carbon's explicit-cache absence and implicit multi-turn rule: a turn must add at least 2,050 new tokens to extend the cached prefix—otherwise the trailing 2,050 tokens are billed uncached.
Live customer chat requires real-time streaming responses, so Batch API is disabled by default. Batch is only applicable to post-chat QA auditing, sentiment tagging, or CSAT classification.