What is chain-of-thought prompting?

The technique was named in a 2022 Google paper, “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” (NeurIPS 2022). Its few-shot examples showed the working, not just the answer. With eight such examples, a 540-billion-parameter model reached state-of-the-art accuracy on GSM8K, a benchmark of maths word problems, and the gains appeared only in sufficiently large models.

Reasoning models now do this on their own: they generate reasoning tokens before answering, billed as output and mostly hidden. OpenAI says prompting them to “think step by step” is unnecessary. Anthropic treats manual chain-of-thought as a fallback for when thinking is off, and warns that on Claude Fable 5.1, Fable 5, Opus 5.5, Opus 5 and Sonnet 5.5 a prompt asking the model to write out its reasoning may be declined.

The paper’s first figure asks: “The cafeteria had 23 apples. If they used 20 to make lunch and bought 6 more, how many apples do they have?” With a standard example in the prompt, the model answered 27, which is wrong. With an example that showed its working, it wrote out the steps and reached 9.

The working costs tokens. With o200k_base, the short answer is 8 tokens and the worked one 55, about 7 times as many. Output is the slow, expensive side of a request: OpenAI’s latency guide says cutting half your output tokens may cut about half your latency.

Standard prompting:
A: The answer is 27.

Chain-of-thought prompting:
A: The cafeteria had 23 apples originally. They used 20 to make lunch. So they had 23 - 20 = 3. They bought 6 more apples, so they have 3 + 6 = 9. The answer is 9.

On models without built-in reasoning, asking for the working can still improve multi-step answers, and the steps make mistakes easier to spot. On reasoning models, spend the effort on the reasoning setting instead (OpenAI’s reasoning.effort, Anthropic’s effort and thinking options) and keep prompts plain.

Either way you pay for every reasoning token, so use it where a task has real steps, such as maths, planning or debugging, not for lookups or classification.

Chain-of-thought prompting vs reasoning models

Chain-of-thought is a prompt technique: the steps appear in the visible answer. A reasoning model is trained to think before answering, whatever the prompt says, and its reasoning tokens are usually hidden but still billed.

Chain-of-thought vs few-shot prompting

Chain-of-thought can be few-shot (examples that include the working, as in the original paper) or a plain instruction to reason first. Plain few-shot prompting only shows inputs and answers.

• Free toolAI Token Counter→Count tokens for GPT, Claude, Gemini, DeepSeek, Qwen and more.
• Free toolLLM API Cost Calculator→Estimate per-request, daily and monthly API costs.
• Guide · 10 min readHow to choose a model for coding agents: the criteria that matterFind the best model for a coding agent by tool-use reliability, context, speed, price per task and where it runs, with live prices for tool-capable models.

• Reasoning tokensReasoning tokens are the tokens a reasoning model generates while it works through a problem before writing its answer, billed as output tokens even though the API hides them or returns only a summary.
• Few-shot promptingFew-shot prompting is giving a model a handful of worked input-and-output examples inside the prompt, so it copies the pattern for a new input without any retraining.
• HallucinationA hallucination is a confident, plausible-sounding statement from a language model that is false or unsupported by its input, such as an invented citation, date, function or quote.
• TokenA token in AI is the unit of text a language model reads and writes, usually a whole word, part of a word or a punctuation mark, which the model sees only as a number from its vocabulary.

All 40 terms in the AI glossary