AISSISTANT: Human-AI Collaborative Review and Perspective Research Workflows in Data Science

arXiv 2025 52.1 method

TLDR

AIssistant is an open-source human-AI collaborative framework for generating scientific review and perspective papers in data science using LLM agents with human oversight.

Reasoning

The paper introduces a novel human-AI collaborative approach with multi-agent systems for review and perspective generation, demonstrating significant time savings (65.7%) through evaluation with human experts and LLM assessments. However, its scope is limited to review/survey papers rather than broader scientific discovery, and the evaluation relies on subjective assessments without comparison to fully autonomous baselines.

Read-first score

Read-first score 52.1, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 50.

Recency 8%

86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=baseline,evaluation,experiment,result

Topical relevance 42%

41.7

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=none

Field roles

FrontierMethodology anchor

Rank sensitivity

Stability: volatile; rank range: 26.

Keyword Scores

survey generation

paper writing agent

literature review agent

AI for scientific research

research automation

automated research

AI scientist

automated scientific discovery

autonomous research agent

scientific discovery agent

automated experimentation

experiment design agent

Deep Analysis

Innovations

First open-source agentic framework for Human-AI collaborative generation of scientific perspectives and review research in data science.
Multi-agent system with two workflows: Research Workflow (7 agents) and Paper Writing Workflow (8 agents), enabling human intervention throughout.
Integration of LLM-driven agents with external scholarly tools for augmented literature search.
Comprehensive evaluation using both human expert reviewers and LLM-based assessment following NeurIPS standards.
Demonstrated 65.7% time savings in human-AI interaction survey.

Methodology

The framework consists of two multi-agent systems: a Research Workflow with seven agents and a Paper Writing Workflow with eight agents, all LLM-driven and augmented with external scholarly tools. Human intervention is allowed throughout. Evaluation was done via human expert reviewers and LLM-based assessment following NeurIPS standards, plus a human-AI interaction survey measuring time savings.

Key Results

OpenAI o1 with chain-of-thought prompting and augmented Literature Search tools achieved the highest quality scores. A human-AI interaction survey showed 65.7% time savings.

Resources

Paper