MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering
TLDR
Introduces MLExpert, a 35B Mixture-of-Experts agentic model trained with hierarchical SFT and Windowed Interaction RL for autonomous long-horizon ML engineering using Kaggle-MLE Factory.
Reasoning
The paper proposes a novel post-training method (WIRL) for autonomous ML engineering agents, addressing long-horizon experimentation with outcome-guided supervision. However, the abstract lacks concrete experimental results, baselines, or limitations, making empirical validation unclear.
Read-first score
Read-first score 53.5, weighted from topical fit, citation, graph, method, reproducibility, and recency signals. Original total remains 51.
Field roles
Rank sensitivity
Stability: volatile; rank range: 41.
Keyword Scores
Deep Analysis
Innovations
- Post-training method for autonomous long-horizon machine learning engineering optimization
- MLExpert, a 35B Mixture-of-Experts agentic model trained with the proposed method
- Kaggle-MLE Factory for constructing executable tasks and an interaction harness for model-directed experimentation
- Hierarchical agentic SFT using outcome-guided supervision to transfer general tool-use and coding capabilities to MLE
- Windowed Interaction RL (WIRL), which groups multi-turn decisions between successive experiment results into execution windows with shared outcome-based rewards
- Asynchronous collection and policy correction enabling policy updates before a trajectory ends
Methodology
MLExpert is a 35B Mixture-of-Experts agentic model trained with a proposed post-training method for autonomous long-horizon MLE optimization. The authors construct executable tasks and a model-directed experimentation harness via the Kaggle-MLE Factory, then apply hierarchical agentic SFT with outcome-guided supervision and Windowed Interaction RL (WIRL), which assigns shared outcome-based rewards to decision windows between experiment results and supports asynchronous policy correction. Evaluation on MLE-Bench Lite uses average medal rate, with comparisons to the base model and larger frontier models Kimi-K3 and GLM-5.3, plus further tests on NatureBench Lite, agent scaffold optimization, and algorithm optimization.
Key Results
On MLE-Bench Lite, MLExpert improves the average medal rate from 37.9% for the base model to 72.7%, achieving performance comparable to larger frontier models such as Kimi-K3 and GLM-5.3. Further experiments show improved performance on NatureBench Lite, agent scaffold optimization, and algorithm optimization, suggesting generalization of the learned optimization capability.
Limitations
- The abstract does not state explicit limitations or failure modes for MLExpert or the proposed training method.
- Generalization to other tasks is only suggested by the reported experiments, not definitively established.