BiREPO: Learning Goal-Conditioned Bimanual Nonprehensile
Pose Reconfiguration Primitives on Diverse Objects

BiREPO pushes, rotates and flips large or irregular objects into a goal pose with bimanual setup, learning from wrench-guided simulation rollouts and transferring to the real robot zero-shot.

Abstract

Bimanual nonprehensile manipulation enables robots to move large or irregular objects that are difficult to grasp, but reliable control remains difficult: success depends jointly on coordinated contact, object dynamics, and kinematic feasibility. Learning these behaviors from teleoperated demonstrations can be costly across diverse objects and goals. We present Bimanual REconfiguration of Poses of Objects (BiREPO), a framework for learning 3D bimanual nonprehensile primitives for pushing, rotating, and flipping objects without human teleoperation. We begin with geometry-based scripted primitives designed for regular shapes, then extend them to irregular (nonconvex) objects through a wrench-guided contact projection that maps nominal contact points onto arbitrary surfaces while satisfying nonprehensile objectives. We construct a data pipeline that validates candidate executions in simulation, and uses successful trajectories to train one goal-conditioned diffusion policy per primitive, conditioned on the current and goal object geometry and the requested pose change. In real-world experiments, BiREPO achieves 65%, 63%, and 70% success on pushing, rotating, and flipping, respectively. Furthermore, we demonstrate that the learned primitives can be chained together to achieve multi-step object pose reconfiguration.

Method Overview

BiREPO method overview.

A: Wrench-guided contact projection initializes bounding box contacts, samples nearby mesh contacts, and selects them using primitive-specific wrench objectives.
B: Contacts are executed in simulation with collision-free motions and primitive-specific open-loop trajectories; successful rollouts are retained as demonstrations.
C: A goal-conditioned diffusion policy uses current and goal point clouds, robot states, and the goal vector to generate bimanual action chunks.

Wrench-Guided Contact Projection

The scripted primitives are defined on an object's bounding box, so running them on an irregular object means projecting the nominal contacts onto its actual surface. We compare three ways of doing that in simulation, on the same objects and the same goals: sampling contacts at random on the mesh, projecting each nominal contact along the surface normal, and our wrench-guided projection, which samples candidates near the nominal contact and keeps the pair whose combined wrench best serves the primitive's objective. Every object–goal pair is executed once per method, so the rates below compare like with like.

Additional metrics

Demonstration generation, as reported in the paper: how many rollouts each projection needs to produce one usable demonstration, and how many objects it leaves with too few demonstrations to train on. Lower is better for both.

PrimitiveRandomNormalWrench-guided (ours)
Attempts per successful demonstration (APS) ↓
Pushing5.44.12.1
Rotating5.94.81.9
Flipping3.21.61.6
Objects with fewer than five demonstrations / % ↓
Pushing40309
Rotating19244
Flipping1666

50 demonstration attempts per object and projection method, over the same 100 objects.

Explore the contacts in 3D

Six objects per primitive. The three views share one camera, so a contact hidden in one is hidden in all — drag to look around it.

Where the projection still fails

Four wrench-guided failures, and what the baselines did on the same episode. Coloured arrows show each contact's estimated moving direction; the grey arrow is the commanded goal. These are examples of a wider pattern: across all 170 residual push failures, 84% end with an orientation error, most failed rotations stop short of the commanded angle, and most failed flips never tip rather than toppling.

Simulation Experiments

We train one goal-conditioned diffusion policy per primitive on the demonstrations the projection generates, and evaluate on in-distribution objects with unseen goals over three seeds. Alongside ablations of the goal vector and the cross-attention, we compare against DP3, adapted to our goal-conditioned setting by concatenating the current and goal point clouds with an indicator, but without cross-attention or an explicit goal vector. BiREPO reaches the highest average success rate, 74.9% against 38.7% for DP3, with the largest margins on pushing and rotating.

Success rate by primitive for BiREPO, its ablations and DP3.
Success rate per primitive, mean ± SD over three seeds. Pushing and rotating count as a success within 30 mm and 10° of the goal; flipping uses its own orientation and settling criteria.

Qualitative Comparison

Each pair is one evaluation episode rolled out by both policies, with the outcome measured against the goal pose. Pick a primitive, then step through the objects.

Real-World Experiments

We deploy the simulation-trained policies zero-shot on a real bimanual platform: two arms with closed parallel-jaw grippers and two RealSense D415 cameras on unseen objects. The success criteria match simulation: within 30 mm and 10° of the goal for pushing and rotating, and within 10° of the desired 90° flip, settled on the target face after the arms withdraw. Since ground-truth poses are unavailable on hardware, the final pose error comes from ICP registration of the observed point cloud to the goal.

Detailed results
Primitive RegulareasyRegularhard IrregulareasyIrregularhard All
Pushing20 / 2516 / 2516 / 2513 / 2565 / 100
Rotating17 / 2516 / 2516 / 2514 / 2563 / 100
Flipping22 / 2514 / 2518 / 2516 / 2570 / 100

Hard trials request pushes beyond 10 cm, rotations beyond 45°, or flips whose support edge is oblique to the world axes; easy trials are the complementary cases.

Rollouts by object

Pick an object and a primitive to see all three trials we ran on it. Clips play at 2× speed.

Multi-Step Primitive Chaining

Every primitive begins and ends at the same robot home pose, so one can run straight into the next with no hand-off between them. The same three policies then compose into multi-step reconfigurations: two flips in a row to walk an object onto a new face, or a rotation followed by a push.

Robustness to Disturbance

When a person moves the object mid-rollout the next plan starts from where the object actually is and the arms keep working toward the commanded goal.

Conclusion

Takeaways

  • Nonprehensile manipulation offers a practical alternative for objects that cannot be grasped. Medium and large objects frequently exceed the width or payload of a parallel-jaw gripper, and pushing, rotating, and flipping with two arms reconfigures them without requiring a stable grasp.
  • Wrench-guided projection grounds demonstration generation in physics. Selecting contacts by the wrench they exert, rather than by surface geometry alone, produces rollouts whose interactions remain consistent with the objective of each primitive, and the policies trained on them transfer to the real platform without real world data collection or fine-tuning.
  • Structured goal conditioning keeps the policy aware of the target pose. Encoding the goal point cloud together with the requested pose change vector, rather than leaving the goal implicit in the observation, supplies the target at every replanning step, which we expect contributes to the corrective behavior observed under disturbance.

Limitations

  • Chaining and failure recovery remain bounded by the primitive definitions. Sequences are restricted to the primitives we define, and the policies lack an explicit recovery mechanism after large execution deviations. Extending the primitives BiREPO learns toward free SE(3) pose changes would broaden both composition and recovery.
  • Tabletop kinematics constrain the reachable workspace. Several failures in our experiments occur near the boundary of the region jointly reachable by both arms. Mounting the arms on a mobile base would enlarge the workspace and address this failure mode.