Awesome Auto Research Hub 论文 · 数据集 · 项目

数据集大全

数据集

按表示方法和任务分组的数据集。

Preview for Kosmos
2025

Kosmos

Kosmos自主进行数据分析、文献搜索和假设生成循环,12小时内生成高准确度可追溯的科学报告。

AI
Preview for LABBench2
2026

LABBench2

LABBench2包含近1900项生物学任务,衡量AI真实能力,难度较前代显著提升。

AICLLG
Preview for AlphaResearch
2025

AlphaResearch

LLMs have made significant progress in complex but easy-to-verify problems, yet they still struggle with discovering the unknown. In this paper, we present \textbf{AlphaResearch}, an autonomous research agent designed to discover new algori...

CL
Preview for MLReplicate
2024

MLReplicate

提出MLReplicate基准,基于ICML 2025论文评估自主研究系统的ML可复现性,发现自动评审不可靠且成本与质量无关。

CLAICYLG
Preview for BioKGBench
2024

BioKGBench

BioKGBench通过声明验证和KGQA评估生物医学AI智能体的文献理解能力,引入KGCheck任务识别知识图谱中的事实错误。

CLAI
Preview for 1GC-7RC
2026

1GC-7RC

提出1GC-7RC基准,评估AI编码智能体在七项ML任务上的表现,揭示性能差异。

LGAICL
Preview for DeepSurvey-Bench
2026

DeepSurvey-Bench

提出DeepSurvey-Bench基准,评估自动生成科学综述的学术价值,弥补现有表面层次指标的缺陷。

AICL
Preview for SurveyGen-I
2025

SurveyGen-I

SurveyGen-I采用由粗到精检索、自适应规划和记忆引导生成,生成连贯且引用丰富的科学综述,在四个领域超越先前方法。

CLDLIR
Preview for CodeScientist
2025

CodeScientist

CodeScientist通过文章和代码块的遗传搜索实现半自动化科学发现,产出19项发现并经多维度评估。

CLAIIR
Preview for QMBench
2025

QMBench

介绍QMBench,一个用于评估大语言模型智能体在量子材料研究中应用凝聚态物理和DFT能力的基准。

mtrl-sciAI
Preview for MIR
2025

MIR

There has been a surge of interest in harnessing the reasoning capabilities of Large Language Models (LLMs) to accelerate scientific discovery. While existing approaches rely on grounding the discovery process within the relevant literature...

AICL
Preview for MegaScience
2025

MegaScience

提出开源科学推理数据集TextbookReasoning和MegaScience,提升AI科学家训练效果,超越现有数据集。

CLAILG
Preview for PaperSearchQA
2026

PaperSearchQA

使用RLVR训练搜索代理在科学论文中推理,发布生物医学语料库和QA数据集。

LGAICLIR
Preview for SurveyLens
2026

SurveyLens

SurveyLens是首个多学科感知的自动综述生成基准,评估了10个学科上的11个系统。

CL
Preview for MLRC-Bench
2025

MLRC-Bench

提出MLRC-Bench基准,用客观指标评估语言智能体在机器学习研究挑战中的表现,显示显著差距。

AI
Preview for LABBench2
2024

LABBench2

LABBench2包含近1900项生物学任务,衡量AI真实能力,难度较前代显著提升。

AI
Preview for AstroVisBench
2025

AstroVisBench

AstroVisBench评估LLM在天文科学计算与可视化中的表现,发现显著性能差距。

CLIMLG
Preview for PRL-Bench
2026

PRL-Bench

PRL-Bench利用《物理评论快报》100篇论文评估LLM的端到端物理研究能力,表现有限。

LGAIdata-an
Preview for SGSimEval
2025

SGSimEval

SGSimEval是一个结合人类偏好和相似性的多维度自动综述生成评估基准。

CLAIIR
Preview for OpenBioRQ
2026

OpenBioRQ

OpenBioRQ: 12,553个未解生物医学问题,测试代理引用忠实性,发现15.9%错误链接。

Preview for Meow
2025

Meow

Meow是一个元数据驱动的端到端大纲写作框架,用于自动学术综述生成,采用整理的数据集和两阶段训练。

CLAI
Preview for PaperBanana
2026

PaperBanana

PaperBanana利用VLM和图像模型自动生成可发表的学术插图,并基于NeurIPS 2025构建基准。

CLCV
Preview for 进化微调
2026

进化微调

提出进化微调(EFT),教会LLM跨优化任务演化解决方案,实现跨任务泛化。

Preview for DeepScholar-Bench
2025

DeepScholar-Bench

Derived from paper: DeepScholar-Bench: A Live Benchmark and Automated Evaluation for Generative Research Synthesis

Preview for LitSearch
2024

LitSearch

Derived from paper: LitSearch: A Retrieval Benchmark for Scientific Literature Search

Preview for GoodPoint
2026

GoodPoint

提出GoodPoint,一种利用作者回复生成建设性、可操作反馈的训练方法,达到最先进水平。

AICL
Preview for ReportBench
2025

ReportBench

Derived from paper: ReportBench: Evaluating Deep Research Agents via Academic Survey Tasks

Preview for SciReplicate-Bench
2025

SciReplicate-Bench

Derived from paper: SciReplicate-Bench: Benchmarking LLMs in Agent-driven Algorithmic Reproduction from Research Papers

Preview for SciCoQA
2026

SciCoQA

SciCoQA数据集评估LLM检测论文-代码差异的能力,揭示自动化质量保证的重大差距。

CLAI
Preview for AutoTrainess
2026

AutoTrainess

AutoTrainess通过智能体-计算机接口自动化语言模型后训练,在PostTrainBench上超越纯CLI基线。

Preview for SciCode
2024

SciCode

Derived from paper: SciCode: A Research Coding Benchmark Curated by Scientists

Preview for MLAgentBench
2023

MLAgentBench

Derived from paper: MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation

Preview for DeepCode
2025

DeepCode

DeepCode 自主从科学论文合成代码,在 PaperBench 上超越人类专家和商业代理。

Preview for AutoResearchBench
2026

AutoResearchBench

Derived from paper: AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery

Preview for ResearchClawBench
2026

ResearchClawBench

Derived from paper: ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Research

Preview for SciFlow-Bench
2026

SciFlow-Bench

Derived from paper: SciFlow-Bench: Evaluating Structure-Aware Scientific Diagram Generation via Inverse Parsing

Preview for SciNetBench
2026

SciNetBench

Derived from paper: SciNetBench: A Relation-Aware Benchmark for Scientific Literature Retrieval Agents

Preview for AbGen
2025

AbGen

Derived from paper: AbGen: Evaluating Large Language Models in Ablation Study Design and Evaluation for Scientific Research

Preview for ResearchBench
2025

ResearchBench

Derived from paper: ResearchBench: Benchmarking LLMs in Scientific Discovery via Inspiration-Based Task Decomposition

Preview for ClaimCheck
2026

ClaimCheck

Derived from paper: ClaimCheck: How Grounded are LLM Critiques of Scientific Papers?

Preview for EvoMaster
2026

EvoMaster

EvoMaster是领域无关、自我演化的智能体框架,用于可扩展的智能体科学,在四个基准上取得最先进结果。

Preview for PaperMind
2026

PaperMind

Derived from paper: PaperMind: Benchmarking Agentic Reasoning and Critique over Scientific Papers in Multimodal LLMs

Preview for LiveIdeaBench
2024

LiveIdeaBench

Derived from paper: LiveIdeaBench: Evaluating LLMs' Scientific Creativity and Idea Generation with Minimal Context

Preview for HindSight
2026

HindSight

Derived from paper: HindSight: Evaluating LLM-Generated Research Ideas via Future Impact

Preview for IDRBench
2026

IDRBench

Derived from paper: IDRBench: Interactive Deep Research Benchmark

Preview for PseudoBench
2026

PseudoBench

Derived from paper: PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

Preview for ScholarGym
2026

ScholarGym

Derived from paper: ScholarGym: Benchmarking Large Language Model Capabilities in the Information-Gathering Stage of Deep Research

Preview for AIBuildAI-2
2024

AIBuildAI-2

一种具有分层外部知识的知识增强智能体,用于自动构建AI模型,解决了静态LLM知识的局限性。

Preview for jablonkagroup/rise_ai_scientists
2026

jablonkagroup/rise_ai_scientists

Corral – Rise of AI Scientists Bibliometric evidence for the rise of AI scientists in chemistry and materials science relative to general AI for chemistry 📋 Dataset Summary This dataset is part of the Corral collection accompanying the paper AI scientists produce results without reasoning scientifically. It contains the bibliometric evidence for the growing relevance and impact of AI scientists in chemistry and materials science compared to general AI… See the full description on the dataset page: https://huggingface.co/datasets/jablonkagroup/rise_ai_scientists.

task_categories:text-classificationtask_ids:multi-class-classificationannotations_creators:machine-generatedlanguage_creators:expert-generatedlanguage_creators:machine-generatedmultilinguality:monolingual
Preview for carps
2025

carps

Hyperparameter Optimization (HPO) is crucial to develop well-performing machine learning models. In order to ease prototyping and benchmarking of HPO methods, we propose carps, a benchmark framework for Comprehensive Automated Research Perf...

LG