Results
100 questions
| Agent | Correctness | Resource use | |||||
|---|---|---|---|---|---|---|---|
| Biomni | 0.78 | 0.76 | 243.65 | 3.70 | 5.62 | 0.116 | 0.148 |
| BixBench-Agent | 0.76 | 0.76 | 40.76 | 4.56 | 6.65 | 0.128 | 0.168 |
| BioMedAgent | 0.73 | 0.72 | 167.57 | 10.37 | 13.21 | 0.412 | 0.565 |
| STELLA | 0.71 | 0.74 | 367.81 | 6.26 | 20.23 | 0.247 | 0.347 |
| AutoBA | 0.34 | 0.39 | 72.38 | 34.41 | 27.10 | 0.247 | 0.726 |
| BioMaster | 0.05 | 0.11 | 68.92 | 14.81 | 7.77 | 0.267 | 5.340 |
Performance by question
| Question | AutoBA | BioMaster | Biomni | BixBench-Agent | STELLA | BioMedAgent |
|---|---|---|---|---|---|---|
| BL-0004 | Fail | Fail | Fail | Fail | Fail | Fail |
| BL-0009 | Fail | Fail | Fail | Fail | Fail | Fail |
| BL-0019 | Fail | Fail | Pass | Fail | Fail | Fail |
| BL-0022 | Fail | Fail | Fail | Fail | Fail | Fail |
| BL-0025 | Fail | Fail | Fail | Fail | Fail | Pass |
| BL-0028 | Fail | Fail | Pass | Fail | Pass | Pass |
| BL-0116 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0126 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0134 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0137 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0145 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0149 | Pass | Pass | Pass | Pass | Pass | Pass |
| BL-0152 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0155 | Fail | Fail | Pass | Pass | Fail | Pass |
| BL-0173 | Fail | Fail | Pass | Pass | Pass | Fail |
| BL-0175 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0177 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0179 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0189 | Pass | Fail | Pass | Pass | Pass | Pass |
| BL-0196 | Pass | Fail | Pass | Pass | Pass | Pass |
1–20 / 100
1 / 5