ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
TLDR
ResearchAgent uses LLMs to iteratively generate novel research ideas by defining problems, proposing methods, and designing experiments with reviewer feedback.
Reasoning
The paper presents a clear methodology combining LLMs with academic graphs and knowledge stores for idea generation, validated across multiple disciplines. Strengths include iterative refinement and human-aligned reviewing agents; weaknesses may include reliance on LLM quality and limited scope to idea generation rather than full automation.
Read-first score
Read-first score 57.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 76.
Field roles
Rank sensitivity
Stability: volatile; rank range: 100.
Keyword Scores
Deep Analysis
Innovations
- Automatic research idea generation (problem, method, experiment) from a core paper with iterative refinement via LLM reviewing agents
- Augmentation with relevant publications from an academic graph and entities from a knowledge store of shared concepts mined across papers
- Multiple LLM-based ReviewingAgents with human preference alignment and evaluation criteria elicited from human judgments via prompting
Methodology
Starting from a core scientific paper, ResearchAgent retrieves related publications and concept entities, then generates a research idea comprising a problem, method, and experiment. Multiple LLM-based reviewing agents, aligned with human preferences and prompted with human-judgment criteria, provide iterative feedback to refine the idea. The system is evaluated on publications across disciplines using human and model-based assessments.
Key Results
ResearchAgent produced ideas rated as novel, clear, and valid in both human and model-based evaluations across multiple scientific disciplines.