OpenAI o3 and o3-mini: The Next Generation of Reasoning Models

Updated
7 min read
OpenAI o3 and o3-mini: The Next Generation of Reasoning Models
TL;DR

OpenAI o3 scores 87.5% on ARC-AGI and 96.7% on AIME 2024. Here's what compute-optimal inference scaling means and how to use o3-mini cost-effectively.

ARC-AGI: The Benchmark That Stumped AI

The ARC-AGI benchmark tests abstract reasoning using visual pattern puzzles that humans solve easily but AI has historically failed at. Prior to o3, the best models scored around 32% (o1) and 34% (Claude 3.5).

o3 achieved 87.5% on the semi-private ARC-AGI evaluation - a jump so dramatic it made headlines and prompted the ARC Prize team to call it "a genuine breakthrough."

Benchmark Scores

Click any column to sort
4 rows
ARC-AGI87.5%32.0%5.0%
AIME 202496.7%74.4%13.4%
SWE-Bench71.7%48.9%38.0%
GPQA Diamond87.7%78.3%53.6%

The SWE-Bench score of 71.7% is particularly significant - it means o3 can resolve nearly 3 in 4 real GitHub issues autonomously.

Compute-Optimal Inference Scaling

Unlike standard models where inference cost is fixed, o3 uses test-time compute scaling: you can tell it how much "thinking budget" to spend. More thinking = higher accuracy = higher cost.

o3-mini offers three compute tiers:

  • Low: Fast, cheap, similar to o1-mini
  • Medium: Balanced quality/cost (recommended default)
  • High: Maximum accuracy, higher latency and cost
python
from openai import OpenAI

client = OpenAI()

# o3-mini with medium reasoning effort
response = client.chat.completions.create(
    model="o3-mini",
    reasoning_effort="medium",
    messages=[
        {"role": "user", "content": "Solve: if 3x + 7 = 22, what is x^2 + 2x?"}
    ]
)
print(response.choices[0].message.content)

When to Use o3 vs o1

Use o3/o3-mini for:

  • Mathematical proofs and competition-level problems
  • Complex multi-step coding tasks (algorithm design, debugging)
  • Scientific reasoning and STEM research assistance
  • Tasks where accuracy matters more than cost

Use o1 for:

  • Tasks where o3's performance gains don't justify the cost premium
  • Existing pipelines already tuned for o1 behavior
  • When you need faster time-to-first-token

API Access

o3 and o3-mini are available through the OpenAI API. Access for o3 (full) is currently prioritized for Tier 4-5 API users. o3-mini has broader availability.

python
# Full o3  -  highest capability, highest cost
response = client.chat.completions.create(
    model="o3",
    messages=[{"role": "user", "content": "Design a Byzantine fault-tolerant consensus algorithm."}],
    max_completion_tokens=8000,
)

Cost Considerations

o3 is significantly more expensive than GPT-4o. For most production applications, o3-mini at medium reasoning effort hits the best accuracy/cost balance. Run a cost analysis before deploying o3 at scale - the quality gains are real, but so is the bill.

Summary

o3 represents a step-change in AI reasoning capability, particularly for math, science, and complex coding. o3-mini with configurable compute tiers makes this capability accessible at reasonable cost. Track benchmark results and access details at the ARC Prize blog.

#o3#openai#reasoning#arc-agi#math

One AI engineering post, weekly

LLM benchmarks, prompt techniques, and token-cost breakdowns, delivered to your inbox.

What should we test or explain next?

Comments

Have a question or something to add? Sign in to join the discussion.

Mahmudul Haque Qudrati — CEO & ML Engineer at Pristren

Mahmudul Haque Qudrati

CEO & ML Engineer

Software architect, full-stack engineer, and machine learning practitioner. Founder & CEO of Pristren, CTO at Venture Kick-backed PIN Sport Application GmbH (Basel, Switzerland), Software Development Lead at DailyFrame GmbH (building Dimetra patient transport software), and creator of Zlyqor. Focused on custom workflow automation, scalable web and mobile applications, and production AI/ML systems.