An Axiomatic Benchmark for Evaluation of Scientific Novelty Metrics

arXiv 2026 37.3 method

TLDR

Proposes an axiomatic benchmark to evaluate scientific novelty metrics, revealing that no existing metric satisfies all axioms consistently.

Reasoning

Strengths: Introduces a principled axiomatic framework for novelty evaluation, with empirical validation across multiple domains. Weaknesses: Limited to AI research domains; does not propose a new metric itself, only evaluates existing ones.

Read-first score

Read-first score 37.3, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 13.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric,result

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Topical relevance 42%

10.8

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 17.

Keyword Scores

AI for scientific research

AI scientist

automated research

research automation

automated scientific discovery

autonomous research agent

paper writing agent

scientific discovery agent

literature review agent

survey generation

automated experimentation

experiment design agent

Resources

Paper