An AI agent recently produced accepted final answers for 49 out of 50 research-level physics problems, a score of 98%. It is an eye-catching result. But to understand why it matters, we need to look beyond the percentage and ask what the researchers actually tested.
The study is called TheoPhysEval, and it appeared on arXiv on 9 October 2026. It covers eleven areas of theoretical physics, including quantum field theory, statistical mechanics, gravitation and cosmology. The strongest result came from GPT-6-Astra working through Codex, with access to computational tools and the web. So, what were these problems like?
They involved calculations connected to current research. The researchers selected questions with answers that could be checked reliably and, according to the authors, were not publicly available when the evaluation took place. This captures an important part of a physicist’s work: taking established ideas and techniques, adapting them to a particular situation, and working out the consequences.
That can be demanding. Knowing which equations are relevant is only the beginning. You may need to understand specialized literature, write a program, choose a numerical method, and check that the result makes sense. A mistake anywhere along the way can derail the calculation.
The questions themselves, however, were supplied by humans. The agents did not have to decide which questions were scientifically interesting or which research directions were worth pursuing. That distinction matters because physics research involves much more than obtaining an answer.
Imagine having a calculation available almost immediately. You still need to know why you wanted it, whether the assumptions are reasonable, and what the result tells you about the physical world. Faster calculations can expand the possibilities, but interpreting those possibilities remains essential.
The word “agent” also deserves a little explanation. These systems combine a language model with software tools. They can write and run code, inspect the output, and use it to continue working on a task.
TheoPhysEval tested two settings. One allowed web access. The other blocked outside internet access while keeping local computation available. In both, the agents worked without interactive human hints. With web access, GPT-6-Astra answered 49 problems correctly. Without it, the score was 46. Opus 5.5, working through Claude Code, reached 46 in both settings.
The small difference is interesting, but it does not settle the question of how much knowledge was memorized and how much was reconstructed during the calculation. It also does not show that web access is generally unimportant. There are several reasons to read the results carefully. For a start, the benchmark primarily checked final answers. A correct answer does not independently verify every intermediate step. In scientific work, we often need a calculation we can examine, reproduce and adapt, not just a number at the end.
The authors also describe cases where apparent AI mistakes turned out to expose errors in the human reference solutions. That is a useful reminder that checking scientific work goes both ways: neither the agent’s output nor the expected answer should be treated as automatically correct.
There is uncertainty in the scores, too. Each agent had only one full evaluation run per setting, and every problem represents two percentage points. Small gaps should therefore be interpreted cautiously. Possible prior data exposure is another limitation acknowledged by the authors.
TheoPhysEval joins earlier efforts such as TPBench, SciCode and CritPt. These benchmarks investigate different combinations of physics knowledge, reasoning and scientific coding. Their percentages cannot simply be lined up as though everyone took the same exam: the questions, tools and grading procedures differ.
The possibility I find most interesting is what increasingly capable computational assistance could make practical. A small research group might explore more variations of an idea. A physicist entering an unfamiliar area might get further with an initial calculation. Researchers might spend less time overcoming programming obstacles and more time deciding which results deserve attention.
Those are possibilities, not outcomes established by this benchmark. Making them useful will require careful validation, and people who understand enough physics to recognize when an apparently convincing calculation has gone wrong.
If demanding calculations become easier to attempt, choosing worthwhile questions and checking the answers could become even more important parts of scientific work.
I explore the experiment, the results and these limitations in my video, “Can AI Do Theoretical Physics? The 98% Benchmark Explained.” If you would like a visual walkthrough of what the agents did and what the score means, you can watch it here:
