Skip to content
Main Site News Console

Evaluating AI Agent Capabilities in Routine Wet-Lab Experiments

· OpenAI Translated
OpenAILLM

Evaluating AI agents on everyday wet lab science | OpenAI

November 6, 2026

Publication

Introducing BioBasicsBench: a benchmark measuring the ability of AI agents to interpret raw experimental data across 100 routine wet lab tasks.

Our mission is to ensure that artificial general intelligence benefits all of humanity. As part of this mission, we want to help scientists accelerate biological research—from understanding disease to developing new treatments. For AI agents to make meaningful contributions, they must be able to reliably interpret the experimental evidence that scientists encounter daily in wet labs (laboratories where biological samples are handled). Everyday wet lab work involves failed experiments, ambiguous images, and conflicting observations. Most existing benchmarks are built around curated charts and figures selected for scientific papers, which can overlook this ambiguity and the troubleshooting it entails. In partnership with Ligo, a biotechnology company focused on enzyme design, we are introducing BioBasicsBench, a new benchmark designed to measure an agent’s ability to handle routine wet lab analyses.

The 100 tasks in this benchmark test whether an agent can interpret raw experimental data, catch errors, and recognize when the evidence is insufficient to support a conclusion. Getting these tasks right is critical: misreading an image or overlooking an issue with controls meant to validate experimental results can lead scientists to draw false conclusions and waste time on experiments that fail to resolve the problem.

Under the “xhigh” setting, which allows for deeper reasoning per task, GPT-6 Astra achieved an average of 66.7% of available points, compared to just 43.8% for GPT-5.6 Sol. These results demonstrate progress in routine experimental analysis, though there is still substantial room for improvement in interpreting images, checking controls, and reasoning across experimental context.

A closer look at BioBasicsBench

BioBasicsBench uses raw experimental evidence spanning gel electrophoresis images, colony plates, microscopy images, and instrument readouts.

BioBasicsBench encompasses 100 tasks across five core areas of routine wet lab work: protein methods, imaging and cytometry, quantitative and analytical assays, nucleic acid methods, and molecular cloning. Tasks range from straightforward requests (such as reading the molecular weight of a DNA sample on a gel) to detailed analytical assays, but all represent routine work that a wet lab scientist should be able to perform reliably.

Scientists created these tasks using data from their own experiments, pairing each question with an expert-authored rubric. Each task underwent internal review (conducted by Ligo scientists who were not involved in authoring the task), external review (by PhD-level scientists actively working in wet labs), and a final internal review.

Agents are required to return structured responses containing the requested conclusions, measurements, or recommendations, along with supporting reasoning where needed. Responses are scored against individual criteria within the rubric, allowing the benchmark to distinguish completely correct conclusions from partially correct analyses. Most tasks assess multiple skills simultaneously, including biological knowledge, interpreting images and other data, error detection, and protocol understanding.

Tasks by domain and assessed skills

100 unique tasks · Each task can assess multiple domains

“Unique tasks” counts each task once. Skill columns tally tasks carrying the corresponding skill tag. These categories overlap: a single task may appear in multiple columns, so skill counts should not be summed to get the total number of tasks. Colors identify domains; darker shading indicates a higher proportion of tasks in that domain testing a given skill.

Input data preserves the forms of evidence scientists work with daily: gel and blot images, plate photos, microscopy stacks, sequencing reads, plate reader exports, and bench protocol videos. A single task may include multiple data types. Reference materials (such as ladder charts and plasmid maps) are provided to aid in interpreting experimental evidence.

What is provided to the agent

Data typeTasksWhat’s included
Gels & blots31Agarose DNA gels, SDS-PAGE gels, Western blots
Protocols, notes & specs27Protocols and standard operating procedures (SOPs), lab notebooks and run logs, photos of handwritten notes, product catalogs, instrument specifications, published methods
Tables & instrument exports20CSV and Excel exports, well layouts and manifests, raw flow cytometry (FCS) files
Bench, apparatus & reagent photos18Lab benches, electrophoresis tanks, reagent and culture tubes, PCR plates, tube racks, pH strips, concentrators, refrigerators, liquid handler decks
Microscopy14Raw brightfield and fluorescence images, multichannel image sets, z-stacks, time-lapse videos
Instrument screens & plots13Screen photos from flow cytometers, microscopes, NanoDrops, plate readers, and balances; exported standard curves, chromatograms, and data plots
Sequencing files12Raw long-read FASTQ files, FASTA sequences, GenBank maps
Protocol videos7Streak plating, inoculations, serial dilutions, SDS-PAGE tank and incubator setups, balance measurements
Agar plates & colonies7Transformation plates, blue-white screening, antibiotic disc diffusion assay plates
Reference sheets21Ladder charts, plasmid maps, pH color charts (supplementary materials)

Number of tasks containing each data type. Categories overlap; reference sheets serve as supplementary materials.

Scientists authored these tasks using data from their own experiments, pairing experimental evidence with questions and expert-developed rubrics. Each task underwent internal review by non-authoring Ligo scientists, external review by PhD-level scientists working in wet labs, and a final internal review. Reviewers inspected the prompts, experimental evidence, rubrics, author notes, and scored model outputs.

Model responses are graded against the task’s rubric, with points awarded across individual criteria. Depending on the task, these criteria evaluate conclusions, numerical results, supporting reasoning, or the identification of insufficient evidence.

Results

We evaluated…