Dynamic On-Palm Manipulation via Controlled Sliding

William Yang1, Michael Posa1
1University of Pennsylvania
Robotics: Science and Systems (RSS), 2024
arXiv Video Code 
All videos shown in realtime

Abstract

Non-prehensile manipulation enables fast interactions with objects by circumventing the need to grasp and ungrasp as well as handling objects that cannot be grasped through force closure. Current approaches to non-prehensile manipulation focus on static contacts, avoiding the underactuation that comes with sliding. However, the ability to control sliding contact, essentially removing the no-slip constraint, opens up new possibilities in dynamic manipulation. In this paper, we explore a challenging dynamic non-prehensile manipulation task that requires the consideration of the full spectrum of hybrid contact modes. We leverage recent methods in contact-implicit MPC to handle the multi-modal planning aspect of the task. We demonstrate, with careful consideration of integration between the simple model used for MPC and the low-level tracking controller, how contact-implicit MPC can be adapted to dynamic tasks. Surprisingly, despite the known inaccuracies of frictional rigid contact models, our method is able to react to these inaccuracies while still quickly performing the task. Moreover, we do not use common aids such as reference trajectories or motion primitives, highlighting the generality of our approach. To the best of our knowledge, this is the first application of contact-implicit MPC to a dynamic manipulation task in three dimensions.

Task Models

We abstract the system into two models. The LCS model captures the contact forces \(\lambda\) between the end effector, tray, and supports. In the LCS model, the robot arm is abstracted away and replaced with direct inputs to the end effector \(u_{lcs}\). We use a robot-only model to track the end effector position \(q_{ee}(t)\) and force \(u_{lcs}(t)\) trajectories commanded from the MPC, so \(\lambda_{ee} = u_{lcs}\).

Interpolate start reference image.

Contact-Implicit Model Predictive Control

The difficulty of the task is primarily due to the poor predictive performance of dynamic frictional contact models. We utilize feedback on the object state to account for these modeling inaccuracies. Specifically we utilize contact-implicit MPC, which automatically plans for the states, inputs, and contact forces without pre-specified contact modes. Previous solutions would utilize motion primitives (to convert it to a planning problem) or reference trajectories (to convert it to a feedback problem), but our framework does not require either. Additionally, consideration of frictional contact modes in 3D significantly increases the number of contact modes that need to be considered, which makes this one of the most complex tasks successfully reasoned by contact-implicit MPC.

Execution on the physical robot. Tray has a mass of 1kg

The plan visualization for the same experiment shows the planned states, inputs, and contact forces.


Interactive Plan Visualization

Feel free to pan/pause/toggle visual elements. Meshcat animation recording made using Drake.

Animation can be reset by clicking the following ("Open Controls" -> "Animations" -> "reset" -> "play").


Rotation using a Wall

To highlight the generality of the chosen LCS model of our approach, we accomplish an entirely different task by modifying only the controller gains and environment specification. The task is specified only as a final position, where the final target state of the tray is rotated about the z-axis from its initial state.

Negative Rotation around Z-Axis

Positive Rotation around Z-Axis


Unknown Mass/Inertia

We place various objects on top of the tray. We do not inform the MPC of the objects, showcasing moderate robustness to inaccurate mass/inertia. In order to not occlude fiducials used to track the position of the tray, we place the objects closer to the edges of the tray which has the additional effect of shifting the center of mass.

Sugar Box (515g, ~50% tray mass)

Mug (319g, ~30% tray mass)


Advantages of a Model-Based Controller

Although we showcase moderate robustness to incorrect object mass/inertia, our model-based framework also allows us to easily accomodate changes to the object model. For the task below, we successfully manipulate a stack of two trays by giving the MPC an updated model.

Two stacked trays (MPC model has updated mass/inertia/geometry)


Ablation Study (Force Tracking)

Tracking the end effector force commanded by the MPC improves the speed and reliability of the task.

Individual Ablation Study Videos

Tracking end effector force trajectory (8/10 success)

NOT tracking end effector force trajectory (3/10 success)


Reliability Testing

We evaluate the reliability of our approach and its robustness to initialization by repeatedly executing the task without manual resets. Our framework successfully completes 6 cycles before deviating too far.

Video at 4x realtime

Acknowledgements

We would like to thank Alp Aydinoglu and Wei-Cheng Huang for helpful explanations about C3. We thank Bibit Bianchini and Mengti Sun for their tutorial on TagSlam. We thank Brian Acosta, Leon Kim, and Jessica Yin for their help with hardware experiments.

This material is based upon work supported by the National Science Foundation Graduate Research Fellowship Program under Grant No. DGE-1845298.

BibTeX


      @article{yang2024dynamic,
        title={Dynamic On-Palm Manipulation via Controlled Sliding}, 
        author={William Yang and Michael Posa},
        journal={arXiv preprint arXiv:2405.08731},
        year={2024},
      }