Workload Benchmark & Cost Model
Long-Context Document Summarization & Batch API Cost Guide
Bulk document summarization processes large unique payloads (45,000 dynamic tokens per PDF/contract) with a lightweight instruction header (600 prefix tokens) and structured summaries (1,200 output tokens) at 300 documents/day—making asynchronous Batch API the primary cost lever.
Document Summarization Cost Comparison (20 models)
100% Client-Side · Zero Prompt TransmissionAvg input/turn: 45,600 tok · Daily turns: 300 · Registry verified: 2026-11-12
| Model & Host | Active Tier & Rules | Eff. In / Out ($/1M) | Cache Read ($/1M) | Daily | Monthly (30d) | $/1K Turns | Savings vs Sync |
|---|---|---|---|---|---|---|---|
LOWESTClaude Haiku 5.5 Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache OnBatch -50% Standard (<= 100K) | $0.050 $0.250 | $0.0050 | $0.77 | $23.20 | $2.578 | -50.0% $23/mo saved |
GPT-6 Luna OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 272K)
| $0.050 $0.250 | — | $0.77 | $23.22 | $2.580 | -50.0% $23/mo saved |
Mistral Small 4 (2603)Open-Weight Mistral·Mistral La Plateforme·256K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 256K)
| $0.075 $0.300 | — | $1.13 | $34.02 | $3.780 | -50.0% $34/mo saved |
GPT-OSS 120B (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 131K)
| $0.075 $0.300 | — | $1.13 | $34.02 | $3.780 | -50.0% $34/mo saved |
GPT-OSS 120B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·128K max in Verified 2026-11-12Official Docs ↗ | Batch -50% US Standard (<= 128K)
| $0.075 $0.300 | — | $1.13 | $34.02 | $3.780 | -50.0% $34/mo saved |
Llama 4 Maverick 17B (Amazon Bedrock)Open-Weight AWS Bedrock·Amazon Bedrock (US Regions)·1000K max in Verified 2026-11-12Official Docs ↗ | Batch -50% US Standard (<= 1M)
| $0.120 $0.485 | — | $1.82 | $54.49 | $6.054 | -50.0% $54/mo saved |
GPT-OSS 120B (Together AI)Open-Weight Together AI·Together AI Serverless (MXFP4)·131K max in Verified 2026-11-12Official Docs ↗ | Batch -0% Standard (<= 131K)
| $0.150 $0.600 | — | $2.27 | $68.04 | $7.560 | 0.0% |
DeepSeek Flash (V4.1-Flash-1017)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M)
| $0.300 $1.200 | $0.0060 | $4.53 | $135.92 | $15.102 | -0.1% $0/mo saved |
DeepSeek V4.1 Flash (Together AI)Open-Weight Together AI·Together AI Serverless (FP8)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache OnBatch -0% Standard (<= 1M)
| $0.300 $1.200 | $0.0060 | $4.53 | $135.92 | $15.102 | -0.1% $0/mo saved |
Mistral Large 4Open-Weight Mistral·Mistral La Plateforme·1000K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 1M)
| $0.350 $1.050 | — | $5.17 | $154.98 | $17.220 | -50.0% $155/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Batch -50%Promo ends 2026-12-31 Intro Promo (<= 1M, thru 2026-12-31)
| $0.375 $1.875 | — | $5.81 | $174.15 | $19.350 | -50.0% $174/mo saved |
Qwen3.8 27B Preview (Groq)Open-Weight Groq·Groq LPU·131K max in Verified 2026-11-12Official Docs ↗ | Standard (<= 131K)
| $0.800 $4.000 | — | $12.38 | $371.52 | $41.280 | 0.0% |
Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache OnBatch -50% Standard (<= 1M) | $1.000 $5.000 | $0.0500 | $15.47 | $463.99 | $51.555 | -50.0% $465/mo saved |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 272K)
| $1.000 $5.000 | — | $15.48 | $464.40 | $51.600 | -50.0% $464/mo saved |
Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Batch -50%Promo ends 2026-11-30 Intro Promo (<= 1M, thru 2026-11-30)
| $1.000 $5.000 | — | $15.48 | $464.40 | $51.600 | -50.0% $464/mo saved |
Gemini 3.1 Pro Preview Google·Google AI Studio / Gemini API·1049K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 200K)
| $1.000 $6.000 | — | $15.84 | $475.20 | $52.800 | -50.0% $475/mo saved |
DeepSeek V4 Pro (0813)Open-Weight DeepSeek·DeepSeek First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Standard (<= 1M)
| $1.320 $3.960 | $0.0440 | $19.46 | $583.81 | $64.867 | -0.1% $1/mo saved |
Claude Sonnet 5.5 (Amazon Bedrock) AWS Bedrock·Amazon Bedrock (Cross-Region Inference)·1000K max in Verified 2026-11-12Official Docs ↗ | Cache On Global CRIS (<= 1M)
| $2.000 $10.000 | $0.1000 | $30.93 | $927.99 | $103.110 | 0.0% |
Claude Fable 5.2 Anthropic·Anthropic First-Party·1000K max in Verified 2026-11-12Official Docs ↗ | Cache OnBatch -50% Standard (<= 1M) | $4.000 $20.000 | $0.1000 | $61.86 | $1,855.93 | $206.214 | -50.0% $1,859/mo saved |
OpenAI·OpenAI First-Party·922K max in Verified 2026-11-12Official Docs ↗ | Batch -50% Standard (<= 272K)
| $5.000 $25.000 | — | $77.40 | $2,322.00 | $258.000 | -50.0% $2,322/mo saved |
Budgeting & Tokenizer Disclaimer: Estimates assume USD list pricing verified on 2026-11-12. Actual token counts vary across provider tokenizers (for example, Claude 4.7+ uses a newer tokenizer that generates ~30% more tokens on identical prose than legacy models), and reasoning models bill hidden thinking tokens as output tokens.
Token Anatomy & Scaling Assumptions
- Cacheable Prefix (600 tokens): Short extraction/summarization rubric shared across all documents.
- Dynamic Input (45,000 tokens): Unique PDF, legal brief, earnings transcript, or medical record—rarely repeated across requests.
- Completion Output (1,200 tokens): Executive summary, key entities, and risk bullets.
- Daily Scale: 300 documents/day across 24 hours/day (9,000 documents/month).
Worked Note: At 600 prefix / 45,000 dynamic / 1,200 output and 300 docs/day with Batch enabled, Claude Haiku 5.5 ($0.10/$0.50 base, 50% Batch stacking with 512-tok caching) costs $0.77/day ($23.20/mo vs. $46.44/mo sync uncached). Mistral Large 4 (50% Batch, caching disabled in Batch) costs $5.17/day ($154.98/mo vs. $309.96/mo). Gemini 3.8 Flash (promotional $0.75/$3.75 through 2026-12-31, 50% Batch, prefix < 4,096 so uncached) costs $5.80/day ($174.15/mo vs. $348.30/mo), jumping 2x on 2027-01-01.
Caching & Batch Strategy
Cache hit rate defaults to 10% because every document is unique. Furthermore, the 600-token instruction prefix is below OpenAI's 1,024-token and Gemini's 4,096-token minimums, so caching only activates on Anthropic 5.x (512-token min) and Mistral (64-token min). Caching matters here only if you run multiple distinct prompts against the *same* 45,000-token document.
Document pipelines are typically asynchronous, making Batch API ideal. However, verified Batch discounts vary sharply by provider: OpenAI, Anthropic, Mistral, and Gemini offer 50% off (with Gemini 3.1 Pro Preview excluding cache reads/storage from Batch discount), Bedrock gives 25% off on Mistral Large 2 and N/A on Claude 5.x, DeepSeek offers 50% Off-Peak scheduling instead of Batch, and Together AI charges 0% discount (standard rates) for serverless chat models.