Awesome Auto Research Hub Papers · Datasets · Projects
← Back to papers

DeepCode: Open Agentic Coding

arXiv 2025 41.6 method

TLDR

DeepCode autonomously synthesizes code from scientific papers, outperforming human experts and commercial agents on PaperBench.

Reasoning

The paper introduces a novel framework for automated code synthesis from documents, addressing context bottlenecks with four information operations. Strengths include strong empirical results and a principled approach; weaknesses are the narrow focus on coding rather than full scientific discovery.

Read-first score

Read-first score 41.6, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 44.

Recency 8%
86.7

Uses a gentle age decay so recent papers surface without erasing older foundations. 2025

Methodology quality 25%
50

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=benchmark,evaluation,metric

Reproducibility 25%
46

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=code,repository

Topical relevance 42%
25

Matches configured research keywords against title, abstract, tags, and analysis text. matched=7

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 20.

Keyword Scores

research automation
7
autonomous research agent
6
AI for scientific research
6
AI scientist
5
automated research
5
automated scientific discovery
4
scientific discovery agent
4
automated experimentation
2
experiment design agent
2
literature review agent
1
survey generation
1
paper writing agent
1

Deep Analysis

Innovations

  • Fully autonomous framework that treats repository synthesis as a channel optimization problem
  • Source compression via blueprint distillation
  • Structured indexing using stateful code memory
  • Conditional knowledge injection via retrieval-augmented generation
  • Closed-loop error correction

Methodology

DeepCode is an autonomous agentic framework that manages information flow by treating repository synthesis as channel optimization. It orchestrates four operations: blueprint distillation for source compression, stateful code memory for structured indexing, retrieval-augmented generation for conditional knowledge injection, and closed-loop error correction. The framework is evaluated on the PaperBench benchmark for document-to-codebase synthesis.

Key Results

DeepCode achieves state-of-the-art performance on PaperBench, decisively outperforming commercial agents Cursor and Claude Code and surpassing PhD-level human experts from top institutes on key reproduction metrics.

Tags