Empowering biomedical evidence exploration and synthesis with deep knowledge graph research

Authors: Zifeng Wang¹, Zheng Chen², Ziwei Yang³, Xuan Wang², Qiao Jin⁴, Yifan Peng⁵, Zhiyong Lu⁴, Jimeng Sun¹,⁶

Affiliations:

1. Keiji AI, Seattle, WA, USA
2. Institute of Scientific Information and Industrial Research, Osaka University, Osaka, Japan
3. Bioinformatics Center, Institute for Chemical Research, Kyoto University, Kyoto, Japan
4. Division of Intramural Research, National Library of Medicine, National Institutes of Health, Bethesda, MD, USA
5. Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, USA
6. School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA

Journal: Nature Machine Intelligence, 2026, 8(7):1142–1156 DOI: 10.1038/s42256-026-01266-0 PMID: 42569156 | PMCID: PMC13449190 Published: 2026-07-02 Code: https://github.com/RyanWangZf/BioDSA/tree/main/biodsa/agents/deepevidence Data: https://huggingface.co/datasets/zifeng-ai/DeepEvidence

Abstract

Compared with generic artificial intelligence agents, deep research agents perform longer-horizon reasoning and deeper literature exploration to investigate complex questions. Here we present DeepEvidence, a deep research agent for evidence exploration and synthesis across heterogeneous biomedical knowledge sources. DeepEvidence advances deep research through coordinated multi-agent collaboration combining breadth-first and depth-first research strategies to search, explore and aggregate evidence from multiple biomedical knowledge bases and literature. It also incrementally constructs an evidence graph of key entities and observations to support transparent tracking, attribution and validation of the research process. DeepEvidence substantially outperforms generic artificial intelligence agents across four open benchmarks. We further establish seven benchmark tasks spanning major stages of biomedical discovery, including drug discovery, preclinical experimentation, clinical trial development and evidence-based medicine. DeepEvidence demonstrates substantial improvements in systematic evidence exploration and synthesis. These results highlight the potential of deep research agents to accelerate biomedical discovery and translational research.

System Overview

DeepEvidence is a multi-agent framework with an orchestrator agent that delegates tasks to specialized research agents. Two types:

• BFRS (Breadth-First Research Strategy): Performs multigraph queries to rapidly survey first-hop neighbourhoods (e.g., identifying drugs linked to a disease across disease–drug or gene–disease graphs).
• DFRS (Depth-First Research Strategy): Builds on entities surfaced by BFRS and performs deeper, multihop exploration (e.g., recursively tracing references across publications to construct comprehensive evidence graphs for systematic literature review).

Key design: BFRS/DFRS are not fixed BFS/DFS graph traversal — AI agents choose traversal steps autonomously. Agents can bridge across heterogeneous knowledge graphs by recognizing shared entities or inferring cross-graph relations.

Evidence Graph

• Incrementally constructed as a first-class memory structure
• Stores high-value normalized biomedical entities (genes, diseases, drugs, pathways, papers, findings)
• Each node/relation grounded with explicit provenance from primary literature or curated knowledge graphs
• Supports transparent traceability and iterative hypothesis refinement
• Quality: 100% provenance validity, 99.7% normalization accuracy, 0.6% duplication rate, ≥99% relation correctness, 93.3% claim alignment

Knowledge Sources (17 databases)

BioThings, ChEMBL, ClinicalTrials.gov, Gene Ontology, Human Phenotype Ontology, KEGG, NCBI Datasets, OpenFDA, Open Genes, ProteinAtlas, PubChem, PubTator, PubMed, Reactome, UMLS, UniProt.

Architecture

• Built with LangGraph (state-machine graphs)
• Orchestrator maintains state machine: tracks search budgets, manages evidence graph, coordinates BFRS/DFRS subagents
• All agents use code execution as primary action mechanism (Python sandbox)
• Unified modality-wise search tools aggregate multiple sources (e.g., search_genes_unified queries BioThings, KEGG, OpenTargets concurrently)
• Execution sandbox with action budgets, bounded retries, incremental processing

Benchmark Results

Open Benchmarks (all with GPT-5 as underlying LLM)

Benchmark	DeepEvidence	Biomni	ToolUniverse	Sonnet-4.5	GPT-5
HLE-Medicine	40.0%	20.0%	10.0%	3.3%	3.3%
LabBench-LitQA2	64.7%	48.9%	25.0%	—	—
SuperGPQA-Medicine-Hard	47.1%	40.7%	—	43.6%	—
TrialPanorama-EvidenceQA	96.0%	84.0%	—	88.0%	—

New Benchmark Tasks (7 total)

1. Target Identification (drug discovery): 68% accuracy vs Biomni 56%, ToolUniverse 40%, LLMs 42%
2. Target Validation (conflicting evidence): DeepEvidence strongest in reconciling evidence hierarchy, context, and temporal conflicts
3. Mechanism of Action & Pathway Reasoning (preclinical): 72% vs Biomni/ToolUniverse 44%, LLMs 52%
4. In vivo Metabolic Flux Response (preclinical): 80% vs Biomni 68%, LLMs 60%, ToolUniverse 52%
5. Sample Size Estimation (clinical trials): 68% vs Biomni 20%, ToolUniverse 32%, PubMed-LLM 24%
6. Drug Regimen Design (clinical trials): 52% vs Biomni 36%, ToolUniverse 28%, PubMed-LLM 20%
7. Surrogate Endpoint Discovery (clinical trials): F1 73.3% vs Biomni 60.9%, ToolUniverse 61.9%, LLMs 58.7%
8. Evidence Gap Discovery (EBM): gap detection 90.0%, recall@30 44.14% vs Biomni 50.0%/30.17%

Open-Ended Research Challenges (human expert evaluation)

• Target Prioritization: DeepEvidence 1.40 vs Biomni 1.15 (P<0.001), outperforming in 5/8 dimensions
• Translational Validation: DeepEvidence 1.64 vs Biomni 1.46 (P<0.05), outperforming in 6/8 dimensions

Key Ablation Findings

1. Directly exposing many tools via MCP is suboptimal (tool confusion); code execution mitigates this
2. Multi-agent decomposition alone doesn’t guarantee gains — vanilla Orchestrator+ReAct underperforms single-agent ReAct+MCP+CodeExec
3. Combining BFRS and DFRS yields best results — strategies are complementary

Limitations

• Incomplete knowledge source coverage
• Works with well-curated KBs (KEGG, GO) — needs adaptation for proprietary/incomplete sources
• No explicit probabilistic modelling or uncertainty-aware mechanisms for conflicting evidence
• BFRS/DFRS may not be optimal strategies
• Not yet expert-level on many tasks
• Text-only; no multimodal data (imaging, structural biology, proteomics)

Relationship to Other Paradigms

• NOT GraphRAG — does not preprocess heterogeneous data into a unified knowledge graph
• NOT traditional BFS/DFS over knowledge graphs
• Agents are “graph-inspired” deep researchers: LLM-driven agents operating under a graph-inspired exploration paradigm, iteratively expanding evidence through BFRS/DFRS while constructing an evidence graph

References

• Wang, Z. et al. (2026). Empowering biomedical evidence exploration and synthesis with deep knowledge graph research. Nature Machine Intelligence, 8(7), 1142–1156. DOI: 10.1038/s42256-026-01266-0
• Code: https://github.com/RyanWangZf/BioDSA/tree/main/biodsa/agents/deepevidence
• Data: https://huggingface.co/datasets/zifeng-ai/DeepEvidence