Sensory substitution for haptic feedback

I used the master-slave system to conduct experiments by which the performance of human subjects in terms of task success rate, task completion times, and energy levels applied to tissue was compared for situations in which haptic feedback was substituted by the corresponding graphical stimuli. The motivation for presenting haptic information via graphical stimuli is the fact that, in today’s surgical systems, there is only a flow of camera vision data from the patient side to the surgeon side and incorporating full haptic interaction would require fundamental system re-designs and therefore remains a long-term goal. However, in the short term, it may be cost-effective and advantageous to provide substitute modes of sensory feedback to the surgeon, e.g., as graphical representation of instrument/tissue contact information.

Master–slave setup for performing telemanipulated soft-tissue lump localization (left) and stiffness discrimination (right).

For localization of a lump that is embedded in soft tissue (above left), performance comparisons were made for situations in which haptic feedback is substituted by a graphical display of haptic information. It was observed that the localization accuracy is comparable for both feedback modalities, which means that in cases where a haptic user interface is not available, "graphical force feedback" can adequately and cost-effectively substitute for haptic feedback. However, this comes at the expense of longer task completion times for graphical feedback. I also compared users’ performance under visual, haptic, graphical, and graphical plus haptic feedback modalities for a soft-tissue palpation task (above right). The goal was to study how the graphical or haptic cues can effectively replace a corrupted visual cue. It was found that graphical cueing and haptic cueing respectively lead to a higher and an equal rate of success in discriminating between two tissue samples with different stiffnesses compared to the visual mode, whereas the visual mode incurs the highest risk of tissue damage due to excessive tissue deformation.

Back to Research