AgentSSL: Can MLE Agents Leverage Unlabeled Data?

The 30-second take

• AgentSSL treats semi-supervised learning as a search over programs when labels are scarce and unlabeled data are abundant.
• Under weak feedback and matched compute, agent-written code beats the strongest human baselines in 8 of 12 settings by up to 21.8 percent.
• Machine-learning teams with piles of unlabeled data should budget for program selection: the agent's own score can drop performance by as much as 47 percent.

What the paper actually did

New domains stall on two bottlenecks at once: getting labels, and hand-building the training pipeline. Recent machine-learning-engineering agents automate pipeline design, but they have been studied almost only in fully supervised settings. AgentSSL frames semi-supervised learning as a program search and asks how agents navigate it when supervision is limited. Even under weak feedback and matched compute, agents synthesize code that combines and adapts established semi-supervised techniques. That code exceeds the strongest human-engineered baselines in 8 of 12 settings by up to 21.8 percent. Program selection is the bottleneck. Performance drops by as much as 47 percent when the choice follows the agent's own feedback metric. To study that gap the authors introduce ProgramSelectBench, with 16,977 programs in total from agents and from human engineers, and they evaluate unsupervised and active selection methods. They also run AgentSSL on real-world ecology detection tasks and discuss what does and does not transfer past the controlled benchmarks.

What makes this disruptive

The scarce resource in many new domains is labels, not raw examples. Agents that only know the fully supervised recipe never touch that abundance. AgentSSL shows they can stitch existing semi-supervised methods into programs that sometimes beat careful human pipelines, at matched compute. The same experiments show the stitch is not the whole problem: a bad selection rule throws away most of the gain.

Why it matters (outside the lab)

A lab with a small labeled set and a large unlabeled set can ask an agent to search training programs instead of only tuning one hand-written pipeline. The 8-of-12 result is the reason to try. The 47 percent drop is the reason not to ship whichever program the agent rated highest. ProgramSelectBench, with 16,977 programs, is the object they built so that selection methods can be compared on purpose.

Limitations & open questions

The gains are in 8 of 12 settings, by up to 21.8 percent, not in every setting and not by a single fixed margin. The agents combine established techniques; the abstract does not present a new semi-supervised theory. Ecology detection is a transfer test, and the abstract does not report those accuracies. It also does not say which selection method on ProgramSelectBench closes the 47 percent gap.