Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation
TLDR
Introduces IdeaGene-Bench, a benchmark for evaluating AI systems on scientific lineage reasoning and lineage-grounded idea generation across 10 domains.
Reasoning
The paper presents a novel, well-structured benchmark (IG-Bench) with curated lineage traces and genome objects, enabling systematic evaluation of AI's ability to reason about scientific inheritance and generate coherent ideas. However, the benchmark focuses narrowly on lineage reasoning rather than full scientific discovery, and the reported performance (27.3% for the strongest system) indicates significant limitations.
Read-first score
Read-first score 49.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 46.
Field roles
Rank sensitivity
Stability: volatile; rank range: 30.
Keyword Scores
Deep Analysis
Innovations
- IdeaGene framework: papers represented as minimal, typed, evidence-grounded Idea Genome objects, and GenomeDiff records aligning objects to capture inheritance, mutation, loss, external import, and novel insertion under six evolutionary dynamics.
- IG-Bench: a benchmark with 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 pairwise GenomeDiff records across 10 scientific domains.
- IG-Exam: 42 task types and 1,029 instances for closed-form lineage reasoning covering Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification.
- IG-Arena: evaluation of lineage-grounded idea generation using a lineage-conditioned Population-Evolution Score (PES) that checks inheritance, meaningful variation from nearby work, and selection value for future research.
- Identification of a compositional bottleneck: strongest LLM-based scientist reaches only 27.3% exact accuracy on lineage reasoning, and structured lineage context reshuffles rankings rather than helping all uniformly.
Methodology
IG-Bench is constructed around the IdeaGene framework, with curated Idea Genome objects and GenomeDiff records capturing evolutionary dynamics. The benchmark offers two evaluations: IG-Exam for closed-form lineage reasoning tasks and IG-Arena for generation, where proposals are scored with the Population-Evolution Score (PES) that measures inheritance, variation, and selection value. Experiments are conducted on 14 LLM-based scientists.
Key Results
The strongest LLM-based scientist achieves only 27.3% exact accuracy on lineage reasoning tasks, and providing structured lineage context reshuffles system rankings instead of uniformly improving all participants.