Augmented Multisensory Interface Design: Performance Enhancement Capabilities and Training Potential.

This dissertation focuses on the design and study of augmented reality (AR) devices for the purpose of enhanced human performance. To do this the Augmented Multisensory Interface Design (AMID) method was developed to direct the design of AR devices by incorporating elements of perception, cognitive engineering, task analysis, system engineering, design process and experimental design principles. To test the effectiveness of the AMID methodology, the real world task of tele-operated landmine detection and removal was studied using both knowledge elicitation and archival methods. Following this it was possible to create a high fidelity test environment. This effort included the development and fabrication of tele-operated landmine detection robots, an indoor landmine field, designed using images and data from actual demined areas, and six AR devices. The landmine detection robot UTORR (Unmanned Tele-Operated Reconnaissance Robot) was equipped with ultrasonic and metal detection sensors, as well as two visual feedback cameras. The AMID method was used to identify two unimodal and one multimodal performance-enhancing AR interface solution (superior solution groups), along with an equal number of performance-inhibiting interface solutions (flawed solution groups). The experiment included 49 participants divided across seven conditions, representing six AR interfaces and one control group. All participants were given training in experimental tasks related to tele-operated landmine detection and disposal techniques. They were then tested in two sessions. In session one, participants used their assigned AR device, while in session two, one week later, the task was performed without the AR devices. Data from the first visit was used to determine the direct performance effect of augmented devices, whereas data from the second visit revealed the training potential of these conditions. The results have shown that AMID can be used to successfully identify AR design solutions for complex real world tasks. Findings have revealed that the enhancement potential for augmented devices is a function of the information type being enhanced as well as cognitive and physiological impact of the technology delivering that information. The results also indicate that these devices have potential value as training technologies. In the second session, where AR devices were not used, the performance of all three superior interfaces remained higher than that of the control or flawed interfaces. Closer analysis revealed that the users in these groups continued to use techniques learned as a result of the AR devices they had used previously. The results indicate that the AMID method can significantly improve human performance and, in some cases reduce task time while still improving target detection.