Full-Body Haptic and Spatial Audio Cueing for Augmented Pilot Perception

This paper illustrates the development, implementation, and testing of full-body haptic and spatial audio cueing algorithms for augmented pilot perception, where haptic cueing is based on localized electrical muscle stimulation. Cueing algorithms are developed for roll-axis compensatory tracking tasks where the pilot acts on the displayed error between a desired input and the comparable vehicle output motion to produce a control action. The error is displayed to the pilot using multiple cueing modalities: visual, haptic, audio, and combinations of these. For the visual and combined visual haptic/audio modalities, visual cues are also considered in degraded visual environments (DVEs). Full-body haptic and spatial audio algorithms that are based on a proportional-derivative compensation strategy on the tracking error are found to provide satisfactory pilot–vehicle system (PVS) performance for the task in consideration in absence of visual cueing and to improve PVS performance in DVE when used in combination with visual feedback. These results are consistent with previous studies on the use of secondary perceptual cues for augmentation of human perception. The combination of these results indicate that the use of secondary sensory cues such as full-body haptics and spatial audio to augment the pilot perception can lead to improved and partially restored PVS performance when primary sensory cues like vision are impaired or denied.