Opus 5.5 Beats Human Experts on AI Research Taste Test
By @sharedot · · 6 pages
- AI Frontier
- Recursive Self Improvement
- Benchmarks
A new benchmark finds the best model outperforms a 24-expert baseline at designing AI experiments while spending 30 times less per attempt.
A benchmark for 'research taste' puts a frontier model above human experts
P-Zero Research has released TasteVal, a benchmark that measures what it calls research taste — the ability to pick interesting problems, design experiments, and interpret results.
The test covers eight tasks representative of frontier AI development, from pretraining data curation to preference modeling and robustness to adversarial prompts, and scores models on how well they iteratively design experiments and draw conclusions from outcomes.
According to Import AI, the best model, Opus 5.5, 'significantly exceeds our human baseline with a compute multiplier of 2.30x (95% CI 1.15-4.37x), at roughly 1/30 of our expert baseliners' average cost per attempt.'
The doubling every 3.0 months is the striking part
Import AI reports TasteVal's headline trend: 'The experimental research taste of frontier models has doubled every 3.0 months since December 2025 (95% CI 1.7-5.0 months).'
The authors note the slope changed sharply around the end of 2025 and beginning of 2026 — earlier years showed much slower growth.
That inflection point is why Jack Clark connects the result to recursive self-improvement, the scenario in which a system gets meaningfully better at building its own successor. shattered.io reports that Clark has separately estimated a 60-percent-or-higher probability that AI capable of helping build its own successor could emerge by the end of 2028, a standing estimate that explains why benchmark evidence of creative research ability gets read so closely in policy and lab circles.
Why creativity was the missing variable
Import AI's own analysis of recursive self-improvement, 'AI Systems Are About To Start Building Themselves' (issue 455), flagged creativity as the key variable for how fast systems could improve themselves, with Clark writing earlier this year that AI systems hadn't yet displayed it in a transformative way.
He now writes that subsequent creative-seeming math breakthroughs and benchmarks like TasteVal shift his estimate: he thinks full RSI becomes possible in 2027 rather than 2028. shattered.io ties the creativity thread to OpenAI's disputed batch of 722 claimed mathematical proofs, which drew pushback from Fields Medal winners — the clearest public example of a lab arguing AI has crossed into open-ended mathematical creativity.
What it means for builders and the field
For practitioners, TasteVal reframes what to watch.
Import AI notes that many people claimed true visual reasoning would require spatially aware 2D and 3D world models — yet LLMs scoring well on related tests counter that argument, and TasteVal similarly suggests the creative intuitions once thought to need fundamentally different architectures are arriving inside today's frontier models.
The caveat is measurement: the benchmark checks conclusions against a fixed human baseline of 24 experts, and confidence intervals on both the 2.30x multiplier and the doubling rate remain wide, so builders should treat the trend as a live signal rather than a settled record.