Large language models (LLMs) do not charge per question or per user. They charge per token, and the price of a token depends on whether the model reads it or writes it. That one detail explains most AI bills that come in higher than expected.
This guide explains how token pricing works, shows what the main models cost as of November 2026, walks through a real cost calculation, and lists the five changes that cut the bill the most.
What a token is
A token is a small chunk of text: a short word, part of a longer word, a number or a punctuation mark. A typical English word is one or two tokens. Everything the model reads counts as input tokens, and everything it writes counts as output tokens.
Input is more than the user's question. It includes the system prompt (your instructions to the model), the conversation history, and any documents you attach for context. In a chatbot, that hidden context is often most of the input.
The four prices on every rate card
Each model has up to four prices, all quoted per million tokens (1M tokens).
Input: what you pay for the tokens the model reads.
Output: what you pay for the tokens it writes. Output usually costs five times as much as input, or more.
Cached input: a much lower price for input the provider has already processed recently, such as a system prompt you send with every request. It is typically about 10% of the input price.
Batch: OpenAI and Google offer 50% off when you send requests as a batch job and accept results within hours instead of seconds.
What the main models cost (November 2026)
| Model | Input | Cached input | Output |
|---|---|---|---|
| GPT-6 Sol (OpenAI) | $2.00 | $0.20 | $10.00 |
| GPT-6 Luna (OpenAI) | $0.10 | $0.01 | $0.50 |
| Claude Sonnet 5.5 (Anthropic) | $2.00 | $0.10 | $10.00 |
| Claude Haiku 5.5 (Anthropic) | $0.10 | $0.01 | $0.50 |
| Gemini 3.1 Pro (Google) | $2.00 | $0.20 | $12.00 |
| Gemini 3.5 Flash-Lite (Google) | $0.30 | $0.03 | $2.50 |
US list prices per 1M tokens, standard tier, prompts under 200K tokens. Check the official pages before budgeting: OpenAI, Anthropic, Google.
Two things stand out. Mid-tier models from all three providers now sit close to $2 in and $10 out. And the small models cost about 20 times less, which is why choosing the model is the biggest cost decision you make.
A worked example: a support chatbot
Take a customer support assistant with 10,000 conversations a day, six turns per conversation. Each turn sends 2,000 input tokens (system prompt, history and the question) and gets 300 output tokens back.
Over 30 days that is 1.8 million turns: 3.6 billion input tokens and 540 million output tokens.
| Scenario | Input cost | Output cost | Monthly total |
|---|---|---|---|
| Mid-tier model ($2 in / $10 out) | $7,200 | $5,400 | $12,600 |
| Same model, 1,500 of the 2,000 input tokens cached at $0.20 | $2,340 | $5,400 | $7,740 |
| Small model ($0.10 in / $0.50 out), no caching | $360 | $270 | $630 |
Caching alone saves 39%. A small model costs 95% less, if it answers well enough for the use case. Some providers also charge a one-time fee to write the cache.
Run your own numbers with our free AI cost simulator, which uses current list prices and shows the effect of caching, routing and batching.
Why the bill grows even as prices fall
Token prices keep dropping. In September 2026 OpenAI cut the price of its new GPT-6 models by 50% compared with the previous generation. Yet most companies see their AI bills rise, because usage grows faster than prices fall. The FinOps Foundation describes the pattern this way: each token gets cheaper, but tokens in aggregate do not.
Watch for three traps. Agents and reasoning models use many more tokens per task than a single chat reply. Long prompts can move you into a higher price tier: Gemini 3.1 Pro doubles its input price above 200,000 tokens, and OpenAI's GPT-6 models charge more above 272,000. And some low prices are introductory: Google's Gemini 3.8 Flash rates double on 1 January 2027.
Five ways to cut the tokens you pay for
1. Route each request to the smallest model that can handle it
Send simple tasks such as classification, extraction or short answers to a small model, and keep the large model for hard ones. The price gap is about 20 times.
2. Cache what does not change
Put stable content (instructions, policies, product catalogue) at the start of the prompt so the provider can cache it. In the example above, that cut the bill by 39%.
3. Send less context
Summarize long conversation histories, retrieve only the relevant passages of a document, and remove instructions nobody needs. Every token you do not send is a token you do not pay for.
4. Limit output length
Output tokens cost about five times more than input. Ask for short answers and set a maximum output length.
5. Batch what can wait
Reports, document processing and evaluations rarely need an answer in seconds. Batch jobs cost 50% less with OpenAI and Google.
Where FinOps fits
Token pricing is only half the job. FinOps for AI adds the other half: assign every API key and model deployment to a team, report AI cost per team every week, set anomaly alerts, and track cost per outcome (per resolved ticket, per document, per user) instead of cost per token.
Test your knowledge with the free FinOps for AI practice test, then go further with Mastering FinOps for AI, Level 1: 161 short animated lessons built on the FinOps Framework capabilities for AI.
Cloud Institute Canada is an independent training provider and is not affiliated with or endorsed by the FinOps Foundation.
LLM token cost FAQ
How much does 1 million tokens cost?
In November 2026, mid-tier models charge about $2 for 1 million input tokens and $10 to $12 for 1 million output tokens. Small models charge $0.10 to $0.30 for input.
Why is output more expensive than input?
The model reads input in one pass but writes output one token at a time, which uses more computing power per token.
How do I estimate my monthly LLM cost?
Multiply requests per month by input tokens and output tokens per request, then by each price per million. Add caching and batch discounts where they apply.
Are cheaper models good enough?
Often, for simple tasks. Test a small model on real examples of your use case and compare quality before switching.
Sources
OpenAI API pricing, October 2026, BenchLM
Claude API pricing 2026, AI Pricing Guru
Gemini API pricing 2026, Maximem
OpenAI cuts GPT-6 Sol and Luna API prices by 50%, Yahoo Finance
Token economics: the atomic unit of AI value, FinOps Foundation