Residual Compliant Policy: Learning Compliant Policies from Few Demonstrations

Chunyu Jiang*,1,2, Yichen Cai*,2, Shuai Li2, Wenbin Sun3, João Carvalho2,4,
Kostas Daniilidis1, Oleg Arenz2, Cristiana de Farias2, Jan Peters2,4,5,6
1GRASP Lab, University of Pennsylvania, PA 19104, USA  ·  2Intelligent Autonomous Systems Lab, TU Darmstadt, Germany
3Karlsruhe Institute of Technology, Germany · 4German Research Center for AI (DFKI), SAIROL, Germany · 5Hessian.AI, Germany · 6Robotics Institute Germany (RIG)
*Equal contribution.
Paper Code (Coming Soon)

Results

Results video static/videos/results.mp4

Abstract

Contact-rich manipulation requires a robot to be precise in free space to establish the intended contacts, and compliant once in contact to avoid excessive forces. Imitation learning has proven effective for simpler manipulation tasks, but the compliance underlying a demonstration is not directly observable and is therefore difficult to infer from demonstration data alone. We present a framework that makes compliance explicit at every stage. During data collection, the operator teleoperates the real robot while a synchronized digital twin visualizes the error between the commanded reference and the achieved pose, making the current compliance visible, and selects between two discrete compliance modes. To generalize across object poses, we combine Laplacian editing, which retargets trajectories efficiently but is not physics-consistent during contact, with simulation-based trajectory optimization, which enforces physical plausibility but offers no manual guidance. Finally, a residual reinforcement learning policy refines the interaction forces by adjusting the reference pose of the underlying admittance controller. We evaluate on three flipping and pushing tasks on a Franka FR3 based on a total of 7 demonstrations across tasks. Our method is successful on all 40 out-of-distribution trials, compared to 33/40 and 24/40 for two baselines, with our base policy alone reaching 35/40.

Method

Proposed compliance policy learning pipeline
Method figure static/img/method.png
Proposed compliance policy learning pipeline. (a) The operator uses the haptic teleoperation device with our digital twin to collect Nh human demonstrations. (b) The collected human demonstrations are augmented via free-space motion Laplacian editing and in-contact trajectory optimization given randomly sampled positions and poses of the object in simulation. (c) A consistency flow matching-based policy is trained on the augmented dataset. (d) A residual RL policy is trained upon the base policy in simulation. (e) The system takes in the robot’s proprioception, stiffness of the admittance controller, external force feedback, and the object pose estimated by the digital twin to predict expected stiffness level and reference poses for implementing the task.