We present a factor graph-based method to enforce temporal consistency in single-view RGB object pose estimation for robot control. By incorporating motion models, estimating measurement uncertainty, and performing online optimization, our approach improves pose stability and robustness. We validate its effectiveness on benchmarks and in real-time feedback control with a camera-mounted robot arm.

Jan 1, 2025

We introduce a video-guided planner that extends RRT using grasp and release states from instructional videos to solve complex, sequential tasks. Our method generalizes beyond the video scene and is validated on the real robot across three different tasks.

May 29, 2023

We propose a method for learning tool manipulation from unconstrained videos, removing the need for expert demonstrations. Our approach aligns the simulated environment with the video scene and uses tool-centric imitation learning to derive robot control policies. We validate the method on tools like a spade, scythe, and hammer, and demonstrate real-world success with a spade on a Franka Emika Panda robot.

Jan 1, 2022