RL Briefing

MLExpert: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Paper submitted October 15, 2026 · arXiv v1 · Read the preprint · Shangheng Du and colleagues

Problem

In a long machine-learning engineering run, an agent makes several decisions before an experiment returns feedback. One final result cannot easily identify which decisions helped. The authors seek training signals that arrive during a run, not only after its full sequence of experiments.

Method

The authors train a 35B-parameter mixture-of-experts agent with outcome-guided supervised fine-tuning and Windowed Interaction RL (WIRL). An execution window comprises the multi-turn decisions between successive experiment results. The window receives a shared outcome-based reward. Asynchronous collection and policy correction allow updates before the whole run ends. This is a description of the authors’ method, not a recipe for assigning individual decisions separate rewards.

How an execution window receives feedback

The drawing below is an illustrative sequence, not a reproduction of the authors’ experiments. A window groups decisions made before the next experiment result; that result supplies shared feedback for the window.

Four stages of Windowed Interaction RL A prior experiment result, two decisions within a window, the next experiment result and shared reward, then an update to the policy before the entire run ends. Previousresult Execution windowDecision → decision(illustrative) Next result→ shared reward Collect &correct policy The full experiment run can continue after this update.

Paused · stage 1 of 4: prior experiment result.

Reading order: previous result → decisions in an execution window → next result and shared reward → asynchronous collection and policy correction. The diagram remains understandable without JavaScript.

Findings

The authors report that MLExpert raises the base model’s average medal rate on MLE-Bench Lite from 37.9% to 72.7%. They also report improvements on NatureBench Lite, agent-scaffold optimization, and algorithm optimization. These are reported benchmark results, not a guarantee on arbitrary engineering projects.

Limitations

Our assessment: The abstract does not establish how much each training component contributes independently, nor whether the benchmark gains transfer to other engineering settings. Shared feedback across a window does not by itself identify which individual decision mattered most. Consult the full paper for ablations and evaluation conditions.

Practical significance

Grouping decisions by experiment-result boundaries offers a practical unit for credit assignment when tool-using agents run long experiments. Its utility outside the authors’ tested tasks remains an open question.