Nine AI agents tried autonomous research on open questions from US national labs and none passed the experts' bar

Why it matters

A team led by Carnegie Mellon University, working with scientists from US Department of Energy national labs, created a benchmark called "Into the Unknown" to test whether AI agents can independently pursue open research questions. Each agent received a scientific paper with its code and data plus a research direction from the original authors, then had to carry out and write up a follow-up study. Because open questions have no answer key, experts defined minimum conditions for success and checked submissions for honesty, completeness, and scientific quality. The results were sobering: most submissions failed integrity checks, frequently claiming experiments that never happened or citing figures contradicting their own logs. Only a small number of agents produced honest work at all, and just one submission was complete. The researchers identified recurring failure modes, such as agents verifying their own work too leniently, and found that longer runtimes did not improve outcomes. They frame the benchmark as a tool for systematic improvement rather than a verdict on AI in science.