Papers, tools and guides on agents for research, checked on 6 October 2026. Every link was opened at its source. Numbers are quoted from the source.
I ran the open-source ones on our cluster and checked every output: see Tools tried.
Building effective agents (Anthropic, 2024). Start with one model call; add agent steps only when the simpler version measurably falls short.
Best practices for Claude Code (Anthropic docs). Give the agent a check it can run. Keep the rules file short. Use hooks for anything that must always happen.
Designing agentic loops (Simon Willison, 2025). Run unattended agents only in a sandbox, with limited credentials, on problems with clear success criteria. His definition: "An LLM agent runs tools in a loop to achieve a goal."
rewrites.bio (Seqera). Twelve principles for rewriting bioinformatics tools with AI: credit the original authors, match outputs exactly, say how AI was used, validate each small step against the original.
AI for Auto-Research: Roadmap and User Guide (Kong et al., 2026). A map of AI use across the research cycle. Conclusion: more automation can hide failure modes rather than remove them.
On the agent setups these papers tested, end-to-end analysis stayed hard, and several point at choosing methods and interpreting results as the weak steps. They do not tell you which step will fail in your task.
BixBench (FutureHouse, 2025). About 300 open-answer bioinformatics questions, with repeated runs per analysis. The two models tested (GPT-4o and Claude 3.5 Sonnet) reached at most 17% on open answers.
FlowBench (Kurjan and Cribbs, 2026). 23 models. Choosing a toolchain from the biological goal alone: 44 to 57%. Agents often "fix" a failing run so it exits cleanly while the data stay invalid.
BiomniBench (Qu et al., 2026). Scores the agent's whole working trail against expert rubrics. The harness around the model moved scores more than a model generation did.
ScienceAgentBench (Chen et al., 2024). 102 tasks from 44 papers. Best agent solved 32.4%.
LAB-Bench (Laurent et al., 2024). Over 2,400 questions on practical biology tasks: literature, databases, figures, sequences, protocols.
I told ChatGPT to continue, and it went too far (Ying Zhao, Nature career column, 2026). A researcher with her e-mail connected typed "please continue"; the chatbot e-mailed other researchers in her name and reported it afterwards. Her lesson: "capability is not permission." Part of the article is behind a login.
Hidden pitfalls of AI scientist systems (Luo, Kasirzadeh, Shah, 2025). Data leakage, metric misuse and post-hoc selection are often invisible in the final paper. Recommends requiring trace logs and code.
Evaluating Sakana's AI Scientist (Beel, Kan, Baumgart, 2025). 42% of experiments failed on coding errors; established ideas labelled as novel.
AI scientists fail without strong implementation capability (Zhu et al., 2025). The bottleneck is running the checks, not generating ideas.
The verification gap (Ding et al., 2026). Survey of 24 systems. Most release code; few release what is needed to check their claims.
Phantom references (Russinovich et al., 2026). About one in twenty NeurIPS and USENIX Security papers has at least two likely fabricated references.
Fabricated citations at NeurIPS 2025 (GPTZero, 2026). 100+ fabricated citations in 53 accepted papers.
The essay cites Bainbridge (1983) and Parasuraman and Riley (1997). These carry the same question into generative AI.
Ironies of generative AI (Simkute et al., 2024). Bainbridge's ironies applied to generative AI: "a shift in users' roles from production to evaluation", and "a tendency for automation to make easy tasks easier and hard tasks harder".
How AI impacts skill formation (Shen and Tamkin, Anthropic, 2026). Randomized: 52 people learning a new Python library. With AI they scored 17% lower on a quiz of concepts, code reading and debugging, and were not significantly faster. The high scorers asked for explanations, alone or alongside generated code.
The widening gap (Prather et al., ICER 2024). 21 novice programmers observed while coding with AI tools. Those who already knew what code they meant to write went faster and ignored bad suggestions. Those who struggled overestimated how well they did and "finished with an illusion of competence". Students in an intro course, not researchers with agents.
Endoscopist deskilling risk after exposure to AI in colonoscopy (Budzyń et al., Lancet Gastroenterology and Hepatology 2025). Observational, four centres in Poland. In colonoscopies done without AI, the adenoma detection rate fell from 28.4% to 22.4% in the three months after AI arrived. Other changes over those months could contribute.
Artificial intelligence and illusions of understanding in scientific research (Messeri and Crockett, Nature 2024). AI tools can leave scientists believing they understand more than they do: "a phase of scientific enquiry in which we produce more but understand less".
We are changing our developer productivity experiment design (METR, 2026). The follow-up to the 2025 trial in the essay. With 57 developers and 800+ tasks the data gave "an unreliable signal", partly because developers who did not want to work without AI stayed out of the study. METR reads the new estimate as a lower bound on the speedup.
The AI Scientist (Sakana AI, 2024; Nature 2026). Idea, code, experiments, paper, automated review. Machine learning only. Sakana lists inaccurate citations and duplicated figures among its limits. v2 uses tree search; one paper passed a workshop review.
AI co-scientist (Google, 2025). Agents generate, critique and rank hypotheses in a tournament. Tested on drug repurposing and resistance mechanisms.
Kosmos (Mitchener et al., 2025). Long runs over data and literature. Scientists judged 79.4% of report statements accurate.
Robin (Ghareeb et al., 2025). Literature and analysis agents; humans do the wet lab. Proposed a drug candidate for dry macular degeneration.
The Virtual Lab (Swanson et al., Nature 2025). An LLM "PI" runs meetings with specialist agents and a human. Designed 92 nanobodies; two improved binding to new SARS-CoV-2 variants.
Coscientist (Boiko et al., Nature 2023). Planned and ran chemistry on robotic hardware.
Agent Laboratory (Schmidgall et al., 2025). Literature, experiments, report, with optional human feedback. Quality improved markedly with it.
CellVoyager (Alber et al., 2025; Nature Methods 2026). Proposes and runs new analyses on published single-cell data in a notebook.
BioDiscoveryAgent (Roohani et al., 2024). Designs rounds of CRISPR screens by reasoning plus literature, without training a model.
Claude Code (Anthropic). Terminal, editor, desktop and web. Reads CLAUDE.md and AGENTS.md, skills, hooks.
Codex CLI (OpenAI). Terminal coding agent. Reads AGENTS.md.
Gemini CLI (Google). Open-source terminal agent with search grounding. Reads GEMINI.md. Has a free tier.
Aider (open source). Commits every change to git so it can be reviewed and reverted. Works with local models.
Cursor (Anysphere). Code editor with agents.
Jupyter AI (Project Jupyter). Connects agents to notebooks; edits and commands need approval.
PaperQA2 (FutureHouse, 2024). Literature agent that answers with citations. The FutureHouse platform offers it and three related agents.
Biomni (Huang et al., 2025). General biomedical agent with tools mined from 25 domains. Preprint.
Paper2Agent (Miao et al., 2025). Turns a paper and its code into tools an agent can call, with generated tests.
CellAgent (Xiao et al., 2024). Planner, executor and evaluator agents for single-cell analysis.
ChemCrow (Bran et al., 2024). GPT-4 with 18 chemistry tools.
Claude for Life Sciences (Anthropic, 2025). Connectors (PubMed, Benchling, 10x Genomics and others) and skills.
ai-research-skills (Wenyu Chiou). Seventeen installable skills from literature search to manuscript, tied to Zotero, Obsidian and NotebookLM, with a dated verification record per skill and a written list of where automation must stop. Its author says it was tested by one person, in water resources. I have not run it.
design-taste (Mustafa, 2026). Plain-text skills that stop an agent from producing the default AI look (purple gradients, centered hero, three cards, emoji icons), plus a pre-flight checklist to run before calling a page done. No service or account needed.
AGENTS.md (Linux Foundation). One instructions file many agents read.
Agent Skills (open standard). A folder with a SKILL.md; loaded only when a task matches.
Not included: several 2026 preprints seen in search but not read. Affiliations for some arXiv entries are from the author groups, not the abstract pages. Full notes: notes/lit_round_2026-10.md.
Source · text CC BY 4.0