Improving Safety for Industrial Graphical User Interfaces of Augmented Reality Deployed on Wearable Devices
Researchers at Texas A&M are working with ABS to improve the understanding of augmented reality (AR) wearable devices. The goal is to provide guidance on the hardware requirements to be fit-for-purpose in marine environments – elements such as environmental noise, illumination, bandwidth, internet connectivity, storage capacity, set-up requirements, and streaming capabilities. The focus is on the impact AR wearable devices have on hazard perception capabilities of the wearer.
A laboratory was created in Texas A&M’s Laboratory for Ocean Innovation. Participants maneuvered through a mock-up of a ship, complete with tripping hazards, head knockers, etc. The navigation task was monitored with motion tracking while the participants engaged in the environment and with the virtual elements from the AR devices.
The Texas A&M team is led by Drs. Edgar J. Rojas Muñoz and Freddie D. Witherden and their students Lawrence Soberanis, Kyper Bezanson, and Graciela Fidalgo. ABS’s Vicente Hernandez, Technology Coordinator, is engaged and working closely with the team.
Augmented Reality wearable technology has the potential to increase the effectiveness of marine surveys. The use of such technology could enable real-time remote observations and feedback by subject matter experts. However, it has been found that augmented reality headsets, particularly Head-Mounted Displays, can have a negative effect on the safety and situational awareness of workers wearing the devices. As such, this project aims to sustain and reinforce workers’ safety and situational awareness as they perform their routine tasks using XR technologies in industrial settings. The objective is to develop extended reality (XR) enhancements that assist the workers in their tasks without introducing additional risk factors. Specifically, this report covers the initial steps toward the development of three safety-enhancing features:
1. Environment & Context-Aware Adaptive User Interfaces
2. Real-Time Estimation of Safety Hazards in an Industrial Setting
3. Real-Time Estimation of Unsafe User Behavior
Ensuring the safety of vessels and offshore structures requires regular surveys or inspections. These assessments evaluate and maintain the structural integrity, functionality, and overall safety of these entities. Surveying tasks encompass inspections for safety code compliance (e.g., identifying marked obstacles), validation of proper material storage, usage of appropriate tie-downs for cargo and tooling, and routine maintenance inspections involving check valves and pipes. Engaging in these tasks exposes surveyors to an array of environmental hazards, including obstacles, trip hazards, low ceilings, high ambient temperatures, humidity, and external forces such as high winds and pitching and plunging surfaces. Consequently, the safe execution of these tasks demands a significant level of situational awareness from the surveyors.
Given the intricacies of these tasks, AR technology is gaining prominence for potentially alleviating the cognitive load on surveyors and enabling certain aspects of the surveying process to be conducted remotely. In this envisioned scenario, the AR device would transmit a real-time video feed to a central command station, where a third party in a controlled office environment can offer assistance during the inspection process. For contextual reference, as of early 2023, the market for AR smart glasses alone is estimated at $11.7 billion, projected to reach $74.4 billion by 2030, with nearly one-third of revenue anticipated from the North American market. Furthermore, a significant portion of this growth is predicted to originate from the business sector.
Despite the potential of wearable technology to enhance the effectiveness of marine surveys within the uniquely challenging marine environment, many devices currently available on the market are not only inappropriate but may also potentially compromise the wearer’s safety. Consequently, it is imperative to comprehensively quantify these impacts, facilitating the establishment of guidelines and best practices.
To address this need, this project develops frameworks and algorithms that combine Computer Vision, Artificial Intelligence, and Computer Graphics to maximize the efficiency of XR technologies in industrial settings. The objective is to develop XR enhancements that can sustain and reinforce workers’ safety and situational awareness as they perform their routine tasks. These enhancements will mitigate the expected additional risks that arise from introducing XR technologies into the workspace, as showcased in Figure 1. Specifically, this report covers the progress made by three different overlapping aims within the project:
1. Environment & Context-Aware Adaptive User Interfaces
2. Real-Time Estimation of Safety Hazards in an Industrial Setting
3. Real-Time Estimation of Unsafe User Behavior