The development

What happened

OpenAI and Ligo introduced BioBasicsBench on 6 November: 100 tasks built from experimental evidence. The evaluation rewards analysis and recognizing insufficient evidence; it does not establish reliable autonomous laboratory work.

Key facts

  • At xhigh effort, OpenAI reports 66.7% mean reward for GPT-6 Astra versus 43.8% for GPT-5.6 Sol. These are average rubric credits, not percentages of experiments successfully completed.

  • Seven models and 37 model/effort settings were tested five times on each task: 18,500 results. Confidence intervals resample whole tasks, retaining repeated attempts together.

  • Tasks underwent scientific review. The announcement also documents failures involving controls, contradictory observations and counting; higher scores do not eliminate those errors.

The context

Why it matters

TideReach assessment: preserving ambiguous raw evidence makes evaluation more useful than checking polished answers alone. Any deployment still needs validation on the actual workflow and qualified scientific review. These publisher-reported results are not independent replication or clinical approval.