Online Surrogate Repair: Decoupling High-Fidelity Feedback from Search Length in Closed-Loop Discovery

arXiv 2026 61.3 method

TLDR

Online surrogate repair uses sparse high-fidelity labels to update surrogates during long closed-loop searches, reducing oracle queries in synthetic and MADE discovery benchmarks.

Reasoning

Strengths include a novel method addressing high-fidelity feedback bottlenecks and strong empirical gains on MADE across multiple orchestrators. Weaknesses include reliance on synthetic/benchmark evaluation and limited visible details on real-world generality and robustness.

Read-first score

Read-first score 61.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 59.

Methodology quality 25%

100

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,experiment,metric,result

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 42%

49.2

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 67.

Keyword Scores

AI scientist

automated scientific discovery

AI for scientific research

scientific discovery agent

autonomous research agent

experiment design agent

research automation

automated research

automated experimentation

literature review agent

survey generation

paper writing agent

Deep Analysis

Innovations

Online surrogate repair (OSR), a closed-loop algorithm that uses sparse high-fidelity evaluations to update a surrogate during a longer agent search conducted mainly with inexpensive surrogate feedback.
Acquisition rules such as Q90-UCB and expected improvement that select accumulated proposals for high-fidelity evaluation based on decision-relevant regions rather than global surrogate fit.
A third feedback regime between fixed-surrogate operation and high-fidelity feedback after every episode, decoupling high-fidelity evaluation frequency from agent search length.

Methodology

OSR interleaves low-cost surrogate-guided agent search with occasional high-fidelity evaluations chosen by an acquisition rule from accumulated proposals; the resulting labels update the surrogate for subsequent episodes. It is evaluated in controlled synthetic environments and on the MADE benchmark, with controls receiving high-fidelity feedback after every episode and metrics including maximum regret and oracle-query counts.

Key Results

Improving global surrogate fit does not necessarily reduce maximum regret, while Q90-UCB and expected improvement substantially reduce regret by targeting high-fidelity evaluations toward decision-relevant regions. On MADE, per-episode high-fidelity controls require 6.36-7.23x more oracle queries to match Online EI under two LLM orchestrators and 10.27x more under Chemeleon+MLIP.

Limitations

Custom laboratory automation is infrastructure-intensive and costly.
Fixed surrogate replacement leaves persistent model errors that can be amplified by optimization.
No explicit limitations of the proposed OSR method are stated in the title/abstract.

Resources

Paper