Coding Agents with an Obstacle-Aware Harness for Safe Robot Manipulation

The 30-second take

• What: The authors show coding agents chase the goal and collide despite prompts and traces that mention the obstacle, then add SafeHarness: waypoint routes over bounding boxes plus contact poses that avoid the obstacle.
• Abundance angle: today, safe physical labor from general coding agents is scarce — models write controllers but treat “don’t hit that” as optional. A reusable safety harness is a step toward cheaper shared robot capacity that is allowed near people and parts (mid-horizon: reliability and safety decide defaults).
• Who should care: Robot-foundation and language-agent labs, industrial safety engineers piloting LLM-written controllers, and anyone who assumed a “don’t touch” sentence in the prompt was enough.

What the paper actually did

Coding agents write a robot controller as a program so a language model can operate robots without robot-specific training. The authors ask whether that paradigm is safe. Each evaluation task pairs a manipulation goal with an obstacle the robot must not touch.

The agent usually completes the goal but collides. It treats task success as the only objective. Traces show it reasons about the obstacle, and the prompt already forbids contact, so the authors rule out perception and instruction. The failure is planning: the constraint never becomes a priority. Splitting behavior into a route phase and a contact-rich moment, they say the model has no idea of a clearing route, does not replan when a route is infeasible, and at contact does not treat execution as bound by the same constraint.

SafeHarness adds two obstacle-aware harnesses. Route planning grounds objects as bounding boxes, proposes waypoint routes, plans ahead, verifies, replans if needed, then executes. Contact execution chooses a contact pose that itself avoids the obstacle. They report 71.9% task success and 87.5% collision avoidance, 6.5 and 27.0 points above the previous SOTA, and 2.3× / 1.5× the same agent without harnesses.

What makes this disruptive

The scarce capability is general-purpose robot programming that respects hard geometric constraints without a custom policy. If prompt-only coding agents systematically de-prioritize “don’t collide,” shipping them onto real arms is a safety problem, not a UX nit.

SafeHarness is a structural fix: make routes and contacts first-class, verifiable objects (boxes, waypoints, contact poses) so safety is not a leftover sentence in the prompt. The large collision-avoidance jump versus the same agent unharnessed is the punchline.

This pressures the “just use an LLM coder” story. It does not yet prove industrial safety cases.

Why it matters (outside the lab)

Abundance lens: robot labor becomes a default only if it is allowed in cluttered spaces. Unsafe coding agents keep useful physical automation elite and fenced-off. A harness that turns obstacles into planned geometry is a step toward more ordinary, shareable robot capacity.

Near-term, treat this as an evaluation and scaffolding paper: do not deploy prompt-only agents around breakable objects. Medium-term, standards, hardware interlocks, and broader task suites decide whether harnessed agents become default.

No date when warehouses run on GPT-written skills. Safety and economics still dominate.

Limitations & open questions

Numbers are the authors' task suite versus a previous SOTA they define; we have not reproduced them. Bounding-box routes can be conservative or wrong if perception of boxes is bad — the abstract focuses on planning once objects are grounded. Contact-rich physics beyond “choose a contact that misses the obstacle” is not detailed.

Prompt-and-trace analysis that “neither perception nor instruction is at fault” is their interpretation of those traces. Preprint ≠ certified safety system. Abundance is not automatic: fewer collisions in a benchmark do not make unsupervised robot labor a default.