Interactive Navigation with Adaptive Non-prehensile Mobile Manipulation

Cunxi Dai^*, Xiaohan Liu^*, Koushil Sreenath, Zhongyu Li, Ralph Hollis

Carnegie Mellon University
University of California, Berkeley
^*Equal Contribution

Abstract

This paper presents a framework for interactive navigation using adaptive non-prehensile mobile manipulation. A key challenge is handling objects with unknown dynamics, which are difficult to infer from visual observation. To address this, we propose a method for learning adaptive dynamics via a learned SE(2) dynamics representation of common movable indoor objects. This adaptive model is integrated into Model Predictive Path Integral (MPPI) control to guide the robot’s interactions. The learned dynamics also inform decision-making for tasks like navigating around non-manipulable objects. Our approach is validated in both simulation and real-world scenarios, demonstrating its ability to accurately represent object dynamics and effectively manipulate various objects. We further highlight its success in Navigation Among Movable Objects (NAMO) task by deploying the proposed framework on a dynamically balancing mobile robot, Shmoobot.

Interactive Navigation with Adaptive Pushing: Time-lapse of interactive navigation with adaptive pushing on Shmoobot, demonstrating the robot’s ability to either push or plan a collision-free path (CF-Path) to navigate around an obstructing object based on its manipulability. The figure showcases two common object types: a light, manipulable box and a non-manipulable wheeled cart (e.g., a wheelchair with locked wheels).

Overall framework: In the offline phase, we first create simulation environments in Bullet [5] with three types of objects with randomized physical properties and train the basis function ψ and robot model R. The learned ψ are then used as parts of the adaptive dynamics, as described in Section.III. In step 2, we utilize the adaptive dynamics to formulate the controller for interactive navigation. Given a pre-built map, the controller switches between adaptive pushing and collision-free navigation based on whether the object is manipulable or is blocking the way. The details of the interactive navigation framework are described in Sec. IV. Finally, the controller sends control actions to the low-level controller of Shmoobot as described in Sec. VI. On the bottom-right corner, we show the schematic of Shmoobot and the necessary frames and notations.

Adaptive Dynamics Accuracy: We compared the accuracy of the adaptive dynamics model O prediction before and after learning on 8 objects whose dynamics were not seen during training. (m is the object’s mass, μ is the planar friction, σ is the wheel’s rotational friction and the red block on the data icon indicates a locked wheel). The left side is a visualization of the model roll-out of the different horizons, the interaction force is visualized in blue. We further compared with other models discussed in Sec. V of which we listed the prediction MSE at the bottom.

Adaptive Pushing Experiments

BibTeX

BibTex not yet available