OpenAI o3 and o4-mini turn ChatGPT into a tool-using agent

OpenAI has introduced o3 and o4-mini, its new reasoning models. The significant change is not just that they spend more time thinking before responding: for the first time in this family, they can decide when and how to combine the tools available in ChatGPT, from web search to Python, file analysis, and image generation.

The change moves ChatGPT closer to becoming an agent capable of carrying out parts of a task on its own. Rather than limiting themselves to what they learned during training, the models can find current information, process it, check it, and present the result in a useful format.

Reasoning before choosing a tool

OpenAI’s reasoning models—the o-series—were designed to devote more compute to difficult problems. Until now, that capability had been more separate from tool use. o3 and o4-mini are trained to decide whether they also need to search the web, write code, examine a chart, or transform an image before responding.

OpenAI describes a workflow in which the model can search for information multiple times, review the results, detect that a piece of data is missing, and launch a new query. It can also write Python code to analyze that data, generate a chart, and explain its conclusions. The company gives a forecast of California’s electricity consumption during the summer compared with the previous year as an example.

The practical difference is significant: the model does not simply answer a question; it can organize a small research process. The tool stops being a button the user activates and becomes part of the system’s strategy.

o3 targets the most complex problems

OpenAI presents o3 as its most powerful reasoning model to date, with improvements in programming, mathematics, science, and visual perception. The company says it sets new state-of-the-art results on Codeforces, SWE-bench, and MMMU.

On SWE-bench, a test that measures the ability to solve real-world software problems, the evaluations use a fixed subset of 477 verified tasks. OpenAI also says that o3 makes 20 percent fewer major errors than o1 on difficult real-world tasks, based on evaluations conducted by external experts. The most notable improvements appear in programming, business consulting, and creative ideation.

The model is designed for questions whose answers require combining several areas of knowledge or are not obvious at first glance. In ChatGPT, the company says it typically completes these tasks in less than a minute, although the time depends on their complexity and the tools used.

o4-mini prioritizes speed and scale

o4-mini is the smaller, more efficient option. Its goal is to deliver advanced reasoning at a lower cost and with greater capacity for use, especially in mathematics, programming, and visual tasks. OpenAI says it outperforms o3-mini on non-STEM evaluations, as well as in data science.

In AIME 2024 and AIME 2025, mathematics tests for students, o4-mini achieves the best results among the models compared by OpenAI. With access to a Python interpreter, it reaches 99.5 percent accuracy on the first attempt in AIME 2025 and 100 percent agreement after eight attempts. o3 scores 98.4 percent and 100 percent, respectively.

These figures come with an important limitation: they cannot be directly compared with models that do not have access to tools. Part of the result comes precisely from the system’s ability to run calculations and check its answers with Python. It is a measure of the model’s capabilities within an equipped environment, not solely of its internal knowledge.

Images within the reasoning process

Another new feature is that o3 and o4-mini can integrate images into their problem-solving process. They can interpret a photo of a whiteboard, a textbook diagram, or a hand-drawn sketch, even when the image is low quality, rotated, or reversed.

They can also manipulate the image during analysis: rotating, enlarging, or transforming it to obtain a more useful perspective. This expands the types of tasks they can handle, from interpreting charts and diagrams to working with visual annotations and problems that combine text and images.

The phrase “thinking with images” does not mean that the model sees like a person or that its conclusions are infallible. It does describe a step beyond visual classification: the image becomes an active element in an analysis chain that can include searches, calculations, and further checks.

Training strengthens autonomous tool use

OpenAI says it trained both models through reinforcement learning, a technique in which the system receives signals about the quality of its decisions. In this case, it is not only taught how to use a tool, but also to decide when it adds value and how to chain it with others.

During the development of o3, the company observed that increasing the compute used in training and during the response phase continued to improve performance. According to OpenAI, it increased both reinforcement-learning training compute and reasoning time during inference by an order of magnitude, and continued to see improvements.

For users, the advance promises more verifiable and useful answers for open-ended tasks. For businesses, it points to a different architecture: rather than building a rigid workflow that connects a model to each tool, some of that coordination can be left to the model itself. In return, it increases the need for oversight, the cost of operations, and the risk that a poor initial decision will propagate through the entire chain.

o3 and o4-mini thus mark the transition from a chatbot that generates text to a system that conducts research, performs calculations, and produces results using multiple types of information. The value no longer depends only on what the model knows, but on whether it knows when to stop answering and start working.