OpenAI Graded 18,500 Wet-Lab Runs. The Failure Log Is the Story.

OpenAI published a wet-lab benchmark on November 6, and the useful part is not the leaderboard. It is the failure log. GPT-6 Astra, the strongest model they tested, reads a colony PCR gel, groups the bands correctly, then recommends the wrong lanes because it assumes a bigger PCR product means the cloning worked. The reference answer picks three different colonies. Run that mistake in a real lab and you purify the wrong plasmid.

The benchmark is BioBasicsBench, built with Ligo, a biotech company focused on enzyme design. It contains 100 tasks across five areas of routine wet-lab work: protein methods, imaging and cytometry, quantification and assays, nucleic acid methods, and molecular cloning. The inputs are not the polished figures you see in papers. They are what sits on the bench: agarose gels and Western blots on 31 tasks, protocols, lab notebooks and photographed handwritten notes on 27, raw flow cytometry files, microscopy z-stacks, videos of someone streaking a plate. Every task carries an expert-written rubric, and each rubric went through three reviews: a Ligo scientist who did not write the task, an external PhD-level wet-lab scientist, and a final internal pass.

OpenAI ran seven of its own models across 37 model-and-effort settings, five attempts per task, 18,500 graded results in total. At the highest reasoning effort:

Look at the last column, not the first. The score went up as the tokens went down. GPT-6 Astra earns 23 more points than GPT-5.6 Sol while emitting about half the output per task, and GPT-5.2 burned six times Astra's tokens for less than half the score. OpenAI's own interpretation is that native multimodal capability moved the result, not reasoning effort: Astra clears GPT-5.6 Sol even at low effort. The confidence intervals back part of that read. Astra's floor sits above GPT-5.6 Sol's ceiling, so that gap is solid. Astra and GPT-6.1 Sol overlap, so the two GPT-6 flagships are not separable on this suite. GPT-6 Luna lands below the previous Sol generation, which is a reminder that newer does not mean better at reading a gel. OpenAI's other new benchmark this week points the same way: on LifeVal's 169 everyday tasks, Astra manages 80.6 against GPT-5.6 Sol's 80.0, per BenchLM's weekly data. The generational jump shows up where images are involved, not everywhere.

Now the problem with the leaderboard. Every model in the evaluation is an OpenAI model. No Claude, no Gemini, nothing open-weight. Two benchmarks that landed the same week with cross-lab fields split the other way: on IBM Research's ScarfBench, Gemini 4 Argon leads at 75.2 while GPT-6 Astra sits at 65.4, and on Dwarkesh Patel's Clocktower Bench, Astra leads Claude Fable 5.1 by 264 rating points. BenchLM tracks both. No lab wins both of those, and neither included the field OpenAI skipped. So 66.7% is a family result. It establishes that Astra is the best OpenAI model at interpreting raw lab evidence. It cannot establish that Astra is the best model, because the models that beat it on IBM's suite never sat this exam.

Why publish it at all, then? Because the grading design is the transferable part, and it is the part most internal evals skip. Credit is assigned per rubric criterion, so a run can earn the measurement and still lose the conclusion. "The evidence cannot support a conclusion" is a scorable answer, not a penalty. OpenAI highlights one case where Astra refused to call cells dead off a brightness signal alone, because two dyes in the sample share a detection window, and asked for controls instead. That refusal is worth more than any leaderboard position if you build agents that touch real-world data.

The failure log is the other reason to read the release. OpenAI documented three ways its flagship gets routine work wrong, with annotated images. One: a familiar assumption overrides the controls. Astra sorts the colony PCR bands into the right visual groups, attaches the wrong identities, and nominates lanes 4, 6, 7 and 8 where the reference selects 5, 9 and 10. Two: discounting contradictions. Asked about vacuole contrast, it observes that focus flips the contrast, then keeps only the darker-vacuole claim in its final answer, dropping the lighter examples it had itself noted. Three: counting. Its marks fail to resolve close neighbors, skip visible colonies, and label empty agar; one plate comes out overcounted by roughly 12%.

That section reads like a spec for what wet-lab agents cannot do yet. It is also unusually honest for a vendor benchmark: the company selling the model published the annotated evidence of that model selecting the wrong colonies. The number to watch from here is not another OpenAI internal refresh. It is whether an independent lab picks up the task suite and runs a cross-vendor field through it, the way IBM and Dwarkesh Patel did with theirs. Until then, treat 66.7% as an internal progress marker, and treat the failure log as the requirements document.

Sources: Measuring how AI agents handle everyday wet-lab science (OpenAI, Nov 6) and BenchLM weekly, November 2 to 7, 2026.