Workload Benchmark & Cost Model
RAG Q&A LLM Cost Breakdown & Model Comparison
Production RAG Q&A pairs a stable system, citation, and tool-schema prefix (1,800 tokens) with per-query retrieved chunks and user questions (3,200 dynamic tokens) to produce concise grounded answers (450 output tokens) at 2,000 requests/day.
RAG Q&A Cost Comparison (20 models)
100% Client-Side · Zero Prompt TransmissionAvg input/turn: 5,000 tok · Daily turns: 2,000 · Registry verified: 2026-11-12
| Model & Host | Active Tier & Rules | Eff. In / Out ($/1M) | Cache Read ($/1M) | Daily | Monthly (30d) | $/1K Turns | Savings vs Sync |
|---|---|---|---|---|---|---|---|
LOWESTGPT-6 Luna OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $0.100 $0.500 | $0.0100 | $1.18 | $35.50 | $0.592 | -18.4% $8/mo saved |
Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 100K) | $0.100 $0.500 | $0.0100 | $1.18 | $35.50 | $0.592 | -18.4% $8/mo saved |
Mistral Small 4 (2603)Open-Weight Mistral·Mistral La Plateforme·256K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 256K) | $0.150 $0.600 | $0.0150 | $1.63 | $48.81 | $0.813 | -20.3% $12/mo saved |
GPT-OSS 120B (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 131K) | $0.150 $0.600 | $0.0750 | $1.81 | $54.32 | $0.905 | -11.3% $7/mo saved |
GPT-OSS 120B (Together AI)Open-Weight Together AI·Together AI Serverless (MXFP4)·131K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 131K)
| $0.150 $0.600 | — | $2.04 | $61.20 | $1.020 | 0.0% |
GPT-OSS 120B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·128K max in Verified 2026-11-12Official Docs ↗ | US Standard (<= 128K)
| $0.150 $0.600 | — | $2.04 | $61.20 | $1.020 | 0.0% |
DeepSeek Flash (V4.1-Flash-1017)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.300 $1.200 | $0.0060 | $3.18 | $95.41 | $1.590 | -22.1% $27/mo saved |
DeepSeek V4.1 Flash (Together AI)Open-Weight Together AI·Together AI Serverless (FP8)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.300 $1.200 | $0.0060 | $3.18 | $95.41 | $1.590 | -22.1% $27/mo saved |
Llama 4 Maverick 17B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·1000K max in Verified 2026-11-12Official Docs ↗ | US Standard (<= 1M)
| $0.240 $0.970 | — | $3.27 | $98.19 | $1.637 | 0.0% |
Mistral Large 4Open-Weight Mistral·Mistral La Plateforme·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $0.700 $2.100 | $0.0700 | $6.96 | $208.87 | $3.481 | -21.7% $58/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Promo ends 2026-12-31 Intro Promo (<= 1M, thru 2026-12-31)
| $0.750 $3.750 | — | $10.88 | $326.25 | $5.438 | 0.0% |
Qwen3.8 27B Preview (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 131K)
| $0.800 $4.000 | — | $11.60 | $348.00 | $5.800 | 0.0% |
DeepSeek V4 Pro (0813)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $1.320 $3.960 | $0.0440 | $12.86 | $385.78 | $6.430 | -23.3% $117/mo saved |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $2.000 $10.000 | $0.1000 | $23.36 | $700.76 | $11.679 | -19.5% $169/mo saved |
Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $2.000 $10.000 | $0.1000 | $23.36 | $700.76 | $11.679 | -19.5% $169/mo saved |
Claude Sonnet 5.5 (Amazon Bedrock) AWS Bedrock·Amazon Bedrock (Cross-Region Inference)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Global CRIS (<= 1M) | $2.000 $10.000 | $0.1000 | $23.36 | $700.76 | $11.679 | -19.5% $169/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Promo ends 2026-11-30 Intro Promo (<= 1M, thru 2026-11-30)
| $2.000 $10.000 | — | $29.00 | $870.00 | $14.500 | 0.0% |
Gemini 3.1 Pro Preview Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 200K)
| $2.000 $12.000 | — | $30.80 | $924.00 | $15.400 | 0.0% |
Claude Fable 5.2 Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M) | $8.000 $40.000 | $0.2000 | $92.82 | $2,784.70 | $46.412 | -20.0% $695/mo saved |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 272K) | $10.000 $50.000 | $1.0000 | $118.32 | $3,549.72 | $59.162 | -18.4% $800/mo saved |
Budgeting & Tokenizer Disclaimer: Estimates assume USD list pricing verified on 2026-11-12. Actual token counts vary across provider tokenizers (for example, Claude 4.7+ uses a newer tokenizer that generates ~30% more tokens on identical prose than legacy models), and reasoning models bill hidden thinking tokens as output tokens.
Token Anatomy & Scaling Assumptions
- Cacheable Prefix (1,800 tokens): System instructions, grounding/citation rules, JSON output guidelines, and fixed few-shot examples.
- Dynamic Input (3,200 tokens): Top-k vector/hybrid retrieved document chunks plus the user's raw query—changes on every request and cannot be cached across distinct questions.
- Completion Output (450 tokens): Grounded answer with inline source citations (plus thinking tokens on reasoning-enabled models).
- Daily Scale: 2,000 requests/day across 16 active hours/day (60,000 requests/month on a 30-day basis).
Worked Note: Under default RAG Q&A parameters (1,800 prefix / 3,200 dynamic / 450 output, 2,000 req/day, 85% hit, 12 writes/hr over 16h), Claude Sonnet 5.5 ($2.00 in / $10.00 out / $0.10 read / $2.50 5m write) and GPT-6.1 Sol ($2.00 in / $10.00 out / $0.10 read / $2.50 30m write) both total $23.36/day ($700.76/mo) vs. $870.00/mo uncached—saving $169.24/mo (19.5%). Routing the same traffic to Claude Haiku 5.5 ($0.10 in / $0.50 out / $0.01 read / $0.125 5m write) drops the bill to $1.18/day ($35.50/mo vs. $43.50/mo uncached).
Caching & Batch Strategy
With an 1,800-token static prefix, this preset exceeds OpenAI's 1,024-token minimum, Anthropic's 512-token minimum (5.x models), and Mistral's 64-token block minimum—while falling short of Gemini's 4,096-token implicit/explicit threshold. At an 85% cache hit rate and 12 cold writes/hour (192 writes/day), cache reads drop to 0.02x–0.10x input price depending on provider.
Live user-facing RAG Q&A requires sub-second to few-second latency, so Batch API (up to 24h turnaround) is turned off by default. Use Batch only for offline RAG evaluation suites or overnight knowledge-base indexing QA runs.