Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills

arXiv 2026 53.9 method

TLDR

DisCo distills GitHub repositories into reusable verified skills for ML research agents, significantly improving benchmark performance across multiple suites.

Reasoning

The paper introduces a practical method for capturing operational knowledge from repositories and shows strong empirical gains on several research benchmarks. However, the abstract is truncated and lacks details on skill verification, generalization, and limitations.

Read-first score

Read-first score 53.9, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 62.

Recency 8%

100

Uses a gentle age decay so recent papers surface without erasing older foundations. 2026

Topical relevance 42%

51.7

Uses existing LLM keyword relevance scores normalized to 0-100. AI scientist,automated scientific discovery,autonomous research agent,automated research,literature review agent,survey generation,automated experimentation,experiment design agent,AI for scientific research,paper writing agent,research automation,scientific discovery agent

Methodology quality 25%

Screens visible abstract and analysis fields for experiment, dataset, baseline, metric, and limitation evidence. markers=evaluation

Reproducibility 25%

Screens links and visible text for paper, code, dataset, artifact, and repository signals. pdf=True; code=False; dataset=False; markers=github,repository

Field roles

Frontier

Rank sensitivity

Stability: volatile; rank range: 66.

Keyword Scores

autonomous research agent

research automation

automated research

AI for scientific research

AI scientist

automated scientific discovery

automated experimentation

scientific discovery agent

experiment design agent

literature review agent

survey generation

paper writing agent

Deep Analysis

Innovations

Introduces operational knowledge as the missing layer of domain-specific know-how that separates knowing a method from making it work.
Presents DisCo, a skill-powered research agent that creates skills and uses them during research.
Distills repository knowledge in two complementary forms: task-agnostic condensation of widely used repositories and task-oriented generation of skills for concrete tasks.
Produces the AREX-Skill Library, with 5,000+ verified skills distilled from 1,000 widely used ML repositories and organized into 20 areas and 178 capability families.

Methodology

DisCo distills operational knowledge from GitHub repositories into compact, verified skills. Task-agnostic distillation condenses 1,000 widely used ML repositories into the AREX-Skill Library, while task-oriented distillation produces skills needed for concrete tasks. Evaluation compares a skill-equipped research agent against the same agent without skills using a fixed GPT-5.5 backbone, research harness, and downstream execution budget on MLE-bench, PaperBench, FrontierCS, and PassNet.

Key Results

With the GPT-5.5 backbone, research harness, and downstream execution budget held fixed, the skill-equipped agent scores 134.3% higher on MLE-bench, 34.4% higher on PaperBench, 9.2% higher on FrontierCS, and 14.0% higher on PassNet than the same agent without skills. The AREX-Skill Library contains 5,000+ verified skills distilled from 1,000 widely used ML repositories and organized into 20 areas and 178 capability families.

Resources

Paper