Using A Surgical Physics Simulator As The Predictive Model In Model Predictive Path Integral Control For Soft-Tissue Manipulation

Tissue manipulation is a fundamental and frequently performed sub-task in robotic surgery. Automating tissue manipulation can significantly reduce the surgeon's cognitive and physical workload. Yet the tissue’s deformable nature and its hard-to-model interactions with rigid tools make its dynamics difficult to characterize thus the automation of this tasks still remains a challenge. This work presents a simulator-in-the-loop, sampling-based MPC framework that uses CRESSim, a heterogeneous, GPU-accelerated surgical VR environment (built in Unity with NVIDIA PhysX 5) as a black-box predictive model inside a Model Predictive Path Integral (MPPI) controller. At each control step, the controller relies only on fast forward rollouts of the simulator to compute the control actions without requiring analytical dynamics or Jacobians. We evaluate the approach in 2D and 3D tissue manipulation studies and achieve sub-2 mm target-point positioning accuracy, demonstrating the feasibility of using a high-fidelity surgical simulator “as the model” for closed-loop autonomous tissue manipulation.