HYPE OR FACT? · CHECKED 7 NOVEMBER 2026
OpenAI BioBasicsBench: wet-lab agent eval with Ligo
OUR VERDICT
Partly factOur original verdict: “Fact that OpenAI published BioBasicsBench with Ligo (6 Nov 2026): 100 routine wet-lab tasks on raw gels, plates, microscopy and instrument exports, scored against expert rubrics. Partly fact on GPT-6 Astra 66.7% mean reward vs 43.8% for GPT-5.6 Sol at the 'xhigh' setting — those are OpenAI's figures. The post also shows where models still fail (wrong controls, overcounting colonies, assumptions overriding evidence).”
THE CLAIM
“BioBasicsBench measures how AI agents handle wet-lab science — 100 routine tasks scored against expert rubrics; GPT-6 Astra leads GPT-5.6 Sol on OpenAI's reported scores.”
What’s actually true
- Official OpenAI post dated 6 Nov 2026 describes BioBasicsBench with Ligo.
- 100 routine wet-lab tasks on raw gels, plates, microscopy and instrument exports.
- OpenAI reports GPT-6 Astra 66.7% mean reward vs 43.8% for GPT-5.6 Sol at 'xhigh' — company figures.
- Failure cases (wrong controls, overcounting colonies, assumptions overriding evidence) are documented in the post.
Sources
Cost
Free to read; model use billed separately if you reproduce tasks.
USE IT IF
Ops and research teams who need a reminder that agent 'done' on messy real-world inputs needs rubrics and human review — including outside biotech.
SKIP IT IF
Anyone treating the 66.7% / 43.8% scores as transferable guarantees on their own pipelines without rubrics.
How we knowChecked on 7 November 2026. Verdicts: Fact = the claim holds up; Partly fact = real, but with caveats or overstated parts; Hype = the headline claim isn't supported. Products change quickly, so check the maker's site before you buy. Our methods & sources ↗