Enabling AI Scientists to Recognize Innovation: A Domain-Agnostic Algorithm for Assessing Novelty

arXiv 2025 39.8 method

TLDR

A domain-agnostic algorithm (RND) for assessing novelty of research ideas, achieving SOTA cross-domain performance without expert labeling.

Reasoning

Strengths include a scalable test set creation method and domain-invariant novelty assessment with strong AUROC scores. Weaknesses: limited to novelty evaluation, not full automated discovery; only two domains tested.

Read-first score

Read-first score 39.8, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 29.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric,result

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Topical relevance 42%

24.2

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 32.

Keyword Scores

AI for scientific research

AI scientist

research automation

automated scientific discovery

automated research

scientific discovery agent

autonomous research agent

literature review agent

survey generation

automated experimentation

experiment design agent

paper writing agent

Resources

Paper