Chain-of-Thought Prompting: The Google Paper That Made LLMs Better at Math

Updated
9 min read
Chain-of-Thought Prompting: The Google Paper That Made LLMs Better at Math
TL;DR

Wei et al. 2022 showed that prompting LLMs to show their reasoning steps - chain-of-thought - dramatically improves performance on arithmetic and logical reasoning tasks.

The Arithmetic Problem

In 2022, large language models were already impressive at many tasks, but math was a persistent weakness. A model could explain calculus but fail a simple word problem requiring multiple steps. The issue was not knowledge - it was reasoning process. Standard prompting asked for the answer directly, giving the model no opportunity to work through intermediate steps.

The Chain-of-Thought Discovery

The Wei et al. paper (arXiv:2201.11903) demonstrated a simple fix: include worked examples that show the reasoning process, not just the final answer. Instead of:

Q: Roger has 5 tennis balls. He buys 2 more cans of 3 balls each. How many does he have? A: 11

Use:

Q: Roger has 5 tennis balls. He buys 2 more cans of 3 balls each. How many does he have? A: Roger starts with 5 balls. 2 cans × 3 balls = 6 balls. 5 + 6 = 11. The answer is 11.

This forces the model to externalize its reasoning chain, and the improvement was dramatic.

Zero-Shot CoT: "Think Step by Step"

Kojima et al. showed that you do not even need few-shot examples. Simply appending "Let's think step by step" to the prompt elicits chain-of-thought reasoning from sufficiently large models. This works because LLMs have seen problem-solving text in training and can mimic the style when cued.

Emergence at Scale

The critical finding was that CoT is an emergent ability - it only helps models above approximately 100B parameters. Smaller models get worse with CoT prompting because they generate plausible-sounding but incorrect reasoning chains. This was one of the first documented cases of emergent capabilities appearing at scale.

Benchmark Improvements

On GSM8K (grade school math word problems), GPT-3 175B with standard prompting solved 17.9% of problems. With 8-shot chain-of-thought prompting, the same model solved 58.1%. On AQUA-RAT (algebraic word problems) and SVAMP (math with linguistic variations), improvements were similarly large.

Self-Consistency Decoding

A follow-up paper (arXiv:2203.11171) showed that generating 40 different reasoning chains and taking the majority vote on the final answer improves accuracy further. Different chains may take different paths but often reach the same correct answer. Incorrect chains are more likely to disagree with each other.

python
from openai import OpenAI

client = OpenAI()

def chain_of_thought(problem: str, n_samples: int = 10) -> str:
    prompt = f"{problem}
Let's think step by step."
    responses = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": prompt}],
        n=n_samples,
        temperature=0.7,
    )
    # Extract final answers and take majority vote
    answers = [r.message.content.split("answer is")[-1].strip()
               for r in responses.choices]
    return max(set(answers), key=answers.count)

Modern Implementation

CoT is now standard in production prompts for reasoning tasks. OpenAI's o1/o3 models internalize chain-of-thought as part of their training rather than relying on prompting. Anthropic builds extended thinking into Claude using similar principles.

Further Reading

#chain-of-thought#prompting#reasoning#few-shot#google

One AI engineering post, weekly

LLM benchmarks, prompt techniques, and token-cost breakdowns, delivered to your inbox.

What should we test or explain next?

Comments

Have a question or something to add? Sign in to join the discussion.

Mahmudul Haque Qudrati — CEO & ML Engineer at Pristren

Mahmudul Haque Qudrati

CEO & ML Engineer

Software architect, full-stack engineer, and machine learning practitioner. Founder & CEO of Pristren, CTO at Venture Kick-backed PIN Sport Application GmbH (Basel, Switzerland), Software Development Lead at DailyFrame GmbH (building Dimetra patient transport software), and creator of Zlyqor. Focused on custom workflow automation, scalable web and mobile applications, and production AI/ML systems.