Ian's Noesis Wiki

Properties1
tagsai, agi, scientific-method, open-source, overview

7 min read

AI Scientist (Autonomous AI Research)

Curated link hub for AI Scientist systems — AI agents that automate parts or all of the scientific research pipeline (ideation → experimentation → paper writing → review). Fast-moving area; this page collects core papers, official code, evaluations, and the surrounding agent-skills ecosystem. As of August 2026.

Core Systems & Papers

  • The AI Scientist — Sakana AI (2024). First end-to-end automated scientific discovery pipeline: LLM proposes ideas, writes and runs experiments, produces complete papers, and scores them with an automated reviewer. arXiv:2408.06292 — https://arxiv.org/abs/2408.06292
  • The AI Scientist-v2 — Sakana AI (2025). Adds agentic tree search for hypothesis exploration; achieved workshop-level (top-tier) results on ICLR 2025 workshop papers. arXiv:2504.08066 — https://arxiv.org/abs/2504.08066
  • Agent Laboratory — (2025). LLM agents as research assistants in a multi-agent workflow (planning → experimentation → analysis → writing) with human-in-the-loop checkpoints. arXiv:2501.04227 — https://arxiv.org/abs/2501.04227
  • Biomni — Stanford-led general-purpose biomedical AI agent (Huang et al., 2025). Its action-discovery agent mines tools, databases, and protocols from tens of thousands of publications across 25 biomedical domains, then combines LLM reasoning, retrieval-augmented planning, and code execution to compose biomedical workflows without fixed templates. The bioRxiv preprint reports generalization across tasks including causal gene prioritization, drug repurposing, rare-disease diagnosis, microbiome analysis, and molecular cloning, plus case studies generating experimentally testable protocols; it is a preprint and has not been peer reviewed. The official environment currently advertises 150 specialized tools, 59 databases, and 105 software packages — official site: https://biomni.stanford.edu · related paper: Huang et al., “Biomni: A General-Purpose Biomedical AI Agent,” bioRxiv (2025), DOI: 10.1101/2025.05.30.656746 — https://www.biorxiv.org/content/10.1101/2025.05.30.656746v1
  • Claude Science — Anthropic’s AI workbench for scientists, announced 30 June 2026 and released in beta for Claude Pro, Max, Team, and Enterprise users on macOS and Linux. It combines a generalist coordinating agent, 60+ curated scientific skills/connectors, user-created specialist agents, a reviewer agent for citations/calculations, native rendering of reproducible artifacts (including 3D protein structures, genome-browser tracks, and chemical structures), and access to local, SSH/HPC, or Modal GPU compute. Generated figures include their code, environment, plain-language provenance, and message history; the system can iteratively edit figures and manuscripts in natural language and fork sessions for comparison. Anthropic describes integrations with PubMed, Jupyter, R, and the NVIDIA BioNeMo platform, including BioNeMo tools/models such as Evo 2, Boltz-2, and OpenFold3; workflows span single-cell RNA-seq, CRISPR screens, protein structure prediction, cheminformatics, and epidemiology. The announcement reports user case studies from Manifold Bio, Allen Institute neuroscientist Jérôme Lecoq, and UCSF Brain Tumor Center epidemiologist Stephen Francis; these are vendor-reported deployments rather than independent evaluations. Official announcement: https://www.anthropic.com/news/claude-science-ai-workbench · product: https://claude.com/science
  • Accelerating scientific discovery with Co-Scientist — Google (2025; originally titled “Towards an AI Co-Scientist”). Gemini 2.0-based multi-agent co-scientist generating novel hypotheses and research proposals. arXiv:2502.18864 — https://arxiv.org/abs/2502.18864
  • Language Agents Achieve Superhuman Synthesis of Scientific Knowledge — Future House / PaperQA2 (2024). Retrieval-augmented language agents that outperformed human experts on scientific synthesis benchmarks. PDF — https://storage.googleapis.com/fh-public/paperqa/Language_Agents_Science.pdf
  • Augmenting Large Language Models with Chemistry Tools — ChemCrow (2024). Tool-augmented LLM agent for chemistry (synthesis, drug discovery); Nature Machine Intelligence. — https://www.nature.com/articles/s42256-024-00832-8
  • MIND: AI Co-Scientist for Material Research — (2026). LLM-driven framework for automated hypothesis validation in materials research, organizing discovery into hypothesis refinement, experimentation, and debate-based reasoning. arXiv:2604.13699 — https://arxiv.org/abs/2604.13699
  • Can AI Conduct Autonomous Scientific Research? — (2026, bioRxiv). Case study evaluating eight open-source AI frameworks (Agent Laboratory, AutoGen, BabyAGI, GPT Researcher, MOOSE-Chem2, SciAgents, SciMON, Virtual Lab) on two real research tasks. — https://www.biorxiv.org/content/10.64898/2026.01.05.697809v1.full
  • OmniScientist — Li et al. (2026, NUS/Oxford). End-to-end omni-modal AI scientist that works directly from heterogeneous raw evidence (images, signals, audio, video, 3-D, trajectories, tables, formulae, graphs) across 20+ disciplines. Perception layer + 3 agents (ideation, experiment, writeup) with code-enforced rigour/claim checks. 36 real-data cases, mean paper score 6.3; direct perception wins 85% of head-to-head vs. blind scalar-only variant. arXiv:2608.13558 — https://arxiv.org/abs/2608.13558 — See omniscientist

Official Code & Experiments

Evaluations & Critiques

  • Evaluating Sakana’s AI Scientist — Beel group (2025). Independent reproduction and critical evaluation of the AI Scientist pipeline. — https://isg.beel.org/pubs/2025-sakana-ai-scientist-reproduced.pdf
  • BiomniBench-AI4S — Omicverse (2026). Open benchmark and reference harness for horizontal comparison of seven AI-for-Science agents—OmicOS, Claude/CSSwitch, EvoScientist, Biomni OSS, synthetic-sciences/openscience, ai4s-research/open-science, and Wisp Science—on 50 BiomniBench-DA biomedical data-analysis tasks. The comparison holds the model (deepseek-v4-pro), task set, output contract, and rubric judge constant so score differences are attributed to the agent; a second Gemini 3.1 Pro judge checks ranking robustness. The published DeepSeek leaderboard reports OmicOS first (mean 0.77; 36/50 pass at score ≥0.70; 72% accuracy), followed by Claude/CSSwitch (0.68), EvoScientist (0.65), Biomni OSS (0.64), synthetic-sciences/openscience (0.63), ai4s-research/open-science (0.61), and Wisp Science (0.61). The dataset is gated, trajectories are released separately on Hugging Face, and the OmicOS runs are not externally reproducible because they depend on private repositories; treat the leaderboard as a documented comparative study rather than a fully independent replication. MIT-licensed repository — https://github.com/omicverse/BiomniBench-AI4S

Autonomous Research Repos

Agent Skills Ecosystem (tooling to make agents do research)

  • Codex Skill structure — OpenAI’s canonical SKILL.md format/sample (skill-creator) that the skills below follow — https://github.com/openai/codex/blob/main/codex-rs/skills/src/assets/samples/skill-creator/SKILL.md
  • Academic Research Skills — Claude Code skill suite: research → write → review → revise → finalize — https://github.com/Imbad0202/academic-research-skills
  • Academic Research Skills (Codex version) — Codex-native port of the above for human-in-the-loop academic workflows — https://github.com/Imbad0202/academic-research-skills-codex
  • RigorPilot Skills — research-first agent skills for deep-learning experiments: comparability, reproducible evidence, auditable changes while an agent reproduces/improves a repo — https://github.com/lllllllama/rigorpilot-skills
  • Scientific Agent Skills — K-Dense-AI’s open-source Agent Skills/Agent Plugins repository for turning compatible AI agents into scientific research assistants. The GitHub API description reports 165 validated skills and 100+ scientific databases spanning biology, chemistry, medicine, and drug discovery, compatible with Cursor, Claude Code, Codex, Pi, Antigravity, and the open Agent Skills standard; the current README badge reports 163 skills (version 2.65.0), so the repository’s headline counts are version-sensitive. The repository is MIT-licensed, has an associated methods paper (Scientific Agent Skills: A Library of Procedural Knowledge for Research Agents, arXiv:2609.00065), and packages reusable workflows for bioinformatics, cheminformatics, clinical research, imaging, scientific ML, literature synthesis, and related domains — https://github.com/K-Dense-AI/scientific-agent-skills
  • NVIDIA BioNeMo — NVIDIA’s open developer platform for AI-driven life-science research, providing datasets, open models, GPU-optimized libraries and tools, training/customization recipes, and optimized inference/deployment components. Its model ecosystem spans protein and RNA structure/design, small-molecule generation, property prediction, and related drug-discovery workflows; the organization notes that components are modular and hosted in dedicated repositories, with licenses varying by component (generally Apache 2.0 for code, CC BY 4.0 for data, and NVIDIA’s Open Model License for model weights). Anthropic identifies BioNeMo as the source of life-science models and libraries accessed by Claude Science, including Evo 2, Boltz-2, and OpenFold3 — https://github.com/NVIDIA-BioNeMo
  • Wisp Science — Zhou-Geng Xu’s open-source, local-first desktop AI research workbench, branded as “Workspace for Intelligent Scientific Practice.” It combines Python/R execution, persistent isolated kernels, OpenAI-compatible or Anthropic models plus Codex/Claude Code ACP integration, reusable SKILL.md workflows, MCP access to roughly 80 scientific databases (including PubMed and GEO), local/WSL/SSH/GPU runtimes, project explorations, and publication evidence capsules that preserve figures, runs, decisions, and manuscript revisions. It is distributed for Windows, macOS, and Linux; the repository is AGPL-3.0-only and has a Zenodo software record (v1.5.0, DOI 10.5281/zenodo.22009273) — https://github.com/xuzhougeng/wisp-science
  • ARIS (Auto-Claude-Code-Research-In-Sleep) — lightweight Markdown-only skills for autonomous ML research: cross-model review loops, idea discovery, experiment automation, no framework lock-in — https://github.com/wanshuiyin/Auto-claude-code-research-in-sleep

Biomedical Deep Research Agents

  • DeepEvidence — Wang et al. (2026, Nature Machine Intelligence). Deep research agent for biomedical evidence exploration and synthesis across 17 heterogeneous knowledge sources (KEGG, PubTator, ClinicalTrials.gov, ChEMBL, etc.). Multi-agent orchestrator–subagent architecture with BFRS (breadth-first) and DFRS (depth-first) research strategies and an incrementally constructed evidence graph. Outperforms Biomni, ToolUniverse, GPT-5, and Sonnet-4.5 across 4 open benchmarks and 7 new biomedical discovery tasks spanning drug discovery, preclinical research, clinical trials, and EBM. DOI: 10.1038/s42256-026-01266-0 — Code: https://github.com/RyanWangZf/BioDSA/tree/main/biodsa/agents/deepevidence — See deepevidence
  • sakana-ai — the lab behind The AI Scientist series
  • andrej-karpathy — creator of the autoresearch project
  • deepevidence — biomedical deep research agent for evidence synthesis across heterogeneous knowledge sources
  • omniscientist — omni-modal AI scientist working directly from raw evidence across 20+ disciplines
  • what-is-intelligence-book — broader question of whether prediction-based AI is genuinely intelligent, which underlies claims about AI scientists

Graph View

Backlinks