OpenAI announced today (July 31) that improvements to its framework increased the GPT-5.6 Sol model’s ARC-AGI-3 score by 188%.
Note: ARC-AGI-3, launched by the ARC Prize team, is a new interactive reasoning benchmark and is currently recognized worldwide as one of the most rigorous and advanced interactive reasoning evaluations for measuring artificial general intelligence (AGI).
The benchmark completely breaks away from the static testing model of traditional AI “problem solving” and knowledge Q&A. Instead, it places AI directly into a completely unfamiliar “game environment” to evaluate whether it has human-like real-time exploration, learning, and adaptive capabilities.
Before optimization, GPT-5.6 Sol scored 7.8% on the ARC-AGI-3 benchmark, while the GPT-5.5 model scored only 0.4%.
After reviewing the results, OpenAI found that the official framework for the GPT-5.6 series contained 2 mechanisms that affected GPT-5.6 Sol’s performance:
First, private reasoning was discarded after every game action. This meant that GPT-5.6 Sol had to understand the game from scratch after each action. The model could still see records of past actions and brief notes, but not the plans, insights, and thought processes that led to those actions.
Second, the official framework used a rolling truncation window. As the history grew longer, earlier actions disappeared from the context. In other words, GPT-5.6 Sol not only could not retain its past reasoning, but also lost its memory of past actions.
To address these issues, OpenAI proposed two solutions: inference preservation and context compression.
For inference preservation, OpenAI reimplemented the ARC-AGI-3 testing framework using the Responses API. For GPT-5.6, simply passing in the previous response ID automatically preserves reasoning across tool calls and multi-turn interactions.
OpenAI said that with inference preservation enabled, GPT-5.6 no longer needs to reinterpret the game each round, reducing the time spent thinking before each action. By retaining its previous ideas, the model learns better over time and adopts more coherent strategies.
For context compression, the official ARC-AGI-3 framework discards the earliest messages once the conversation context exceeds 175000 characters. OpenAI said rolling truncation has 2 drawbacks: the model loses early observations and actions; and the model runs with a fuller context window for most of the task, which may slightly hurt performance.
With context compression enabled, GPT-5.6 Sol is better able to retain what it has learned about each game during longer tasks and achieves higher scores with fewer output tokens.
With the official harness, GPT-5.6 Sol scored 13.3% on ARC-AGI-3. After inference preservation and context compression were enabled, GPT-5.6 Sol’s score reached 38.3% while reducing the number of output tokens to one-sixth, representing a performance increase of 188.42%.
A harness, often referred to as harness engineering, is the complete system architecture wrapped around a large model to provide its operating environment, tool integrations, rule constraints, and feedback mechanisms.
If a powerful AI model is compared to a strong but easily uncontrollable horse, then the harness is its reins, saddle, and riding route. It does not change the model’s architecture itself, but determines how the model is safely and reliably controlled to complete complex work.
