LIBERO-PeRM: Benchmarking Personalized Robotic Manipulation

The 30-second take

• What: LIBERO-PeRM is a controlled benchmark for personalized robotic manipulation with five suites (312 tasks) of language-expressible preferences that do not change the task goal; current VLA policies struggle zero-shot, SFT makes preferences learnable, and mixed training trades off retention, prompting, and motion-preference strength.
• Abundance angle: today, a robot that finishes the chore your way is a scarce custom install. Preference benchmarks are a step toward cheaper default household robots if following taste becomes as routine as grasping (mid-horizon: reliability, safety, and unit economics decide defaults).
• Who should care: Household-robot and VLA teams, HRI researchers, and anyone who has watched a “general” policy succeed at the goal while violating how a person wanted it done.

What the paper actually did

Personalization is framed as what general-purpose robots need before they are practical at home. The authors argue personalized robotic manipulation (PeRM) lacked a controlled benchmark that separates task completion from preference satisfaction. LIBERO-PeRM studies five questions: zero-shot preference following; learning a single preference from demonstrations; sample efficiency as demos grow; generalization across layouts and tasks; and composing preferences plus behavior under conflicts.

They propose a taxonomy of observable preferences in simulation — motion, skill, and user-profile choices — restricted to language-expressible preferences that can be instantiated in LIBERO without changing the task goal, scored automatically, and reused across tasks. A procedural pipeline builds paired demonstrations with and without preferences. Five task suites (312 tasks) ship with high-quality demonstrations, metadata, and preference-satisfaction metrics.

Experiments: current VLA policies struggle with zero-shot personalization, while supervised fine-tuning makes preferences learnable. Mixed training retains no-preference behavior better than sequential adaptation and improves contextual-prompt performance versus preference-only training, but can weaken motion-preference learning. Preference generalization, decomposition, and conflict handling remain difficult.

What makes this disruptive

The scarce capability is measuring “did it do it my way?” as cleanly as “did it pick up the mug?” Without that split, personalization papers can hide inside task success. A 312-task, auto-evaluated, goal-preserving preference layer on LIBERO is the kind of boring infrastructure that moves a field.

The empirical snapshot is blunt: VLAs do not zero-shot your taste; SFT can teach it; mixing data helps you not forget the default; motion preferences and conflicts are still hard. That is a research agenda, not a vibe.

Preferences here are language-expressible and simulation-observable — a feature for control, a limit for messy human taste.

Why it matters (outside the lab)

Abundance lens: a robot that shares a kitchen must match scarce personal norms (where the mugs go, how fast to move) or it stays a demo. If preference following becomes a default capability, more household labor can become cheaper shared automation — after safety and real homes cooperate.

Near-term: train and score on LIBERO-PeRM instead of claiming personalization from a handful of demos. Medium-term: real-home transfer, conflict handling, and unit economics decide defaults. No butler date.

Horizon is mid: reliability and taste generalization, not a product year.

Limitations & open questions

Simulation on LIBERO-derived suites; preferences must be language-expressible and goal-preserving. “Current VLA policies” are not named in the abstract. Mixed-training trade-offs (motion preferences vs retention) are qualitative here.

Automatic metrics cannot capture unspoken household norms. Preprint ≠ home deployment. Abundance is not automatic: a benchmark does not personalize real robots on a schedule.