Jr. AI Scientist and Its Risk Report: Autonomous Scientific Exploration from a Baseline Paper

arXiv 2025 73.7 method

TLDR

Jr. AI Scientist autonomously generates novel research papers from baseline papers by analyzing limitations, experimenting, and writing, achieving higher review scores.

Reasoning

Strengths include a well-defined workflow leveraging modern coding agents for complex implementations and evaluation on real conference papers. Weaknesses are the reliance on automated reviewers and author-led assessments, and limitations not fully detailed in the abstract.

Read-first score

Read-first score 73.7, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 92.

Methodology quality 25%

100

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=analysis,baseline,evaluation,experiment,result

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Topical relevance 42%

76.7

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code

Field roles

FrontierBridgeMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 21.

Keyword Scores

AI scientist

autonomous research agent

scientific discovery agent

automated scientific discovery

automated research

paper writing agent

research automation

automated experimentation

AI for scientific research

experiment design agent

literature review agent

survey generation

Deep Analysis

Innovations

Jr. AI Scientist: an autonomous AI scientist system that mimics a novice student researcher's workflow, from analyzing a baseline paper's limitations to formulating hypotheses, iteratively experimenting, and writing a paper.
Integration of modern coding agents to handle complex, multi-file implementations, enabling scientifically valuable contributions on real NeurIPS, IJCV, and ICLR works.
Comprehensive risk report identifying limitations and potential risks of current AI Scientist systems, based on author evaluation, Agents4Science reviews, and development experience.

Methodology

Jr. AI Scientist takes a baseline paper, analyzes its limitations, formulates novel hypotheses, and iteratively experiments using modern coding agents for complex multi-file code. It then writes a paper with results. Evaluation uses automated AI Reviewers (DeepReviewer), author-led assessments, and submissions to the Agents4Science venue.

Key Results

Papers generated by Jr. AI Scientist received higher review scores from DeepReviewer than those from existing fully automated systems. However, author evaluations and Agents4Science reviews revealed important limitations and risks.

Limitations

The system operates at a novice student researcher level and still requires human expertise for areas identified in the study.
Author evaluation and Agents4Science reviews uncovered important limitations, indicating risks in directly applying current AI Scientist systems.
Various risks were identified during development, as reported in the comprehensive risk analysis.

Resources

Paper