An AI Scientist that Doesn't Drift: Taste, Structure, and Falsifiable Findings in a Quadruped Navigation Research Loop

arXiv 2026 57.2 method

TLDR

An AI Scientist loop for quadruped navigation adds immutable experiment cards, specialized subagents, and a taste oracle to prevent drift; oracle changes research direction, not scores.

Reasoning

The paper presents a structured autonomous research loop with a controlled comparison to isolate the effect of a preference oracle, which is a strong methodological strength. However, it is limited to simulation and the abstract provides little detail on baselines, metrics, or broader applicability.

Read-first score

Read-first score 57.2, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 76.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 42%

63.3

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=experiment,metric

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 88.

Keyword Scores

AI scientist

automated scientific discovery

autonomous research agent

automated research

AI for scientific research

research automation

automated experimentation

scientific discovery agent

experiment design agent

literature review agent

survey generation

paper writing agent

Deep Analysis

Innovations

Immutable experiment card that pairs each iteration's prediction with its outcome under a fixed schema, preventing retroactive reinterpretation of falsified hypotheses
Specialised subagents restricted to mechanical roles in the research loop
kkanbu preference oracle, a typed knowledge graph capturing the user's research taste, as the sole component permitted to make subjective judgments

Methodology

The authors build an AI Scientist for studying generalization in quadruped robot navigation policies in simulation, augmenting Karpathy's autoresearch paradigm with an immutable experiment card, mechanical subagents, and a preference oracle (kkanbu). To isolate the oracle's effect, they run the identical loop twice across eleven research streams, with and without kkanbu.

Key Results

Neither arm drifts; both falsify about three-quarters of their hypotheses. The best trained policy comes from the oracle-less arm, while the oracle arm uniquely explores test-time adaptation, authors winning designs, and transfers lessons across streams that the other arm repeatedly re-derives.

Resources

Paper