A research-led initiative has introduced Terminal-Bench-Science 0.1, a continuously updated benchmark intended to measure how well AI agents handle practical scientific work rather than textbook questions. The first release contains 70 tasks across the life, physical, Earth, mathematical and engineering sciences.
Researchers at Stanford University lead the project, which was built by the team behind Terminal-Bench with specialists from several disciplines and institutions. Its tasks cover activities including data analysis, statistical inference, simulation, optimization, theorem proving, image reconstruction, signal processing, sensor calibration and model fitting. Agents are evaluated in realistic computing environments, with their outputs checked through task-specific, reproducible tests. Those outputs can include analyses, proofs, simulations, code and data products.
The inaugural results indicate that the benchmark remains difficult for current systems. Each model received three independent attempts at every task. Claude Opus 5 paired with Claude Code led with a 30.0% resolution rate. GPT-5.6 Sol with Codex followed at 22.4%, while Claude Fable 5 with Claude Code reached 21.4%. Claude Opus 4.8 scored 10.5%; GPT-5.6 Terra, Kimi K3 and Grok 4.6 each finished below 10%. GLM 5.3 was the highest-scoring open model at 8.1%, and GPT-5.6 Luna recorded 3.3%.
Performance differed by discipline. Claude Opus 5 led the other two highest-scoring systems in every area except mathematical sciences. There, Claude Fable 5 scored 33.3% and GPT-5.6 Sol reached 31.4%. In engineering sciences, Grok 4.6 tied GPT-5.6 Sol for second at 14.8%, while using fewer tokens and costing less in the evaluation.
The project also tracks resource use. Evaluating Claude Opus 5 across the full set cost $7,000, according to the benchmark. GPT-5.6 Sol cost $4,200 and achieved a similar resolution rate to Claude Fable 5, whose run cost $14,200. The latter used 6.4 billion tokens, compared with 8.4 billion for GPT-5.6 Sol. Only Kimi K3 and Claude Opus 5 appeared on both the cost and token Pareto frontiers.
Creating the task set involved substantial filtering. Contributors submitted 920 proposals, 464 were cleared for implementation and 386 became pull requests, but reviewers accepted 70 for version 0.1. Domain reviewers assessed scientific validity, technical reviewers examined construction and verification, and a final reviewer checked quality.
The organisers describe the benchmark as an ongoing mechanism rather than a one-time release. Scientists can propose new workflows and refine existing tasks through a public process, allowing future versions to change as agent capabilities advance. The early scores suggest that reliable automation of demanding research workflows remains limited, while providing a measurable basis for comparing progress. Repeated releases could also show whether gains transfer across disciplines or remain concentrated in particular kinds of scientific work.
