Japan’s Nexco-East is using mixed reality for tunnel emergency facilities training.
Japan’s expressway network has seen consistent expansion since the 1960s, resulting in an extensive road network exceeding 10,000 kilometers. However, a substantial portion of this network, constituting over half of the expressways, has been in operation for more than three decades. This prolonged usage has given rise to issues related to aging infrastructure, encompassing road surfaces, bridges, and tunnels, which have emerged as significant societal concerns. Additionally, Japan is confronted with the challenge of a declining population of expressway maintenance and management engineers.
In response to these formidable challenges, Nexco-East Engineering, the entity responsible for overseeing approximately 4,000 kilometers of Japan’s expressways, has introduced a training application harnessing Mixed Reality (MR) and digital twin technologies. This application is deployed within their training hub, known as the “Technical Training Center, (TTC)” located in Takasaki, Gunma. The primary aim of this application is to offer an efficient educational and training tool that equips engineers with the essential knowledge and skills required for their roles.
This issue was especially apparent in the education about the use of electronic control and communication equipment (ETC). According to Mr. Yoshitaka Nakamura, the manager of the facilities construction work division of Nexco-East Engineering, the company’s instructors used various materials such as chalk talk and PowerPoint slides to explain the operation of the equipment.
“However, it requires considerable time to gain understanding of associations among the training materials. Also, radio communication and control signal flows are impossible to see, so we had to rely on trainees’ imaginations. This resulted in gaps in understanding.”
To address this issue, the company started focusing on digital twin technologies and mixed reality. A project was then launched to develop a training material that combines a 3D model of an ETC equipment with various features such as radio communication and infrared sensors. This method would allow students to gain a better understanding of the equipment’s operation.
Over a decade ago, Nexco East Group recognized the potential issues arising from the aging expressway infrastructure and a shortage of maintenance and management personnel. To tackle these challenges, they initiated the Smart Maintenance Highway project, with the goal of integrating existing maintenance and management technologies with Information and Communication Technology (ICT). This project has led to the adoption of digital tools such as tablet-based tasks, Microsoft PowerBI for data analysis, and other ICT-driven innovations to enhance field operations.
Tetsuo Hideshima, Managing Director and General Planning Manager of Nexco-East Engineering, underscored the significance of comprehensive inspections and diagnostics for ongoing expressway maintenance and management. He stressed the necessity of streamlining and labor-saving measures through ICT to guarantee the safety of expressway users.
The Technical Training Center, established in 2012, plays a pivotal role in enhancing the skills of expressway inspection engineers. It provides a practical training environment equipped with real equipment, including power receiving and distribution equipment and the Electronic Toll Collection (ETC) system, identical to those used on expressways.
One notable challenge encountered by TTC was the varying levels of comprehension among trainees, particularly in the education on ETC equipment. The complexity of ETC systems made it challenging to standardize training materials and ensure uniform understanding among trainees. To address this issue, Nexco-East Engineering turned to MR and digital twin technologies.
DataMesh, recognized for its expertise in digital twins and extended reality (XR), collaborated with Nexco-East Engineering to develop an “MR for ETC equipment training” program. This innovative approach combined a 3D model of ETC equipment with infrared sensors, radio communication, and data communication routes. The objective was to provide trainees with a more intuitive grasp of ETC systems.
This development process prioritized iterative feedback from TTC instructors and project leaders, embracing an agile development approach. The outcome was a training tool that utilized MR to overlay 3D animations on actual ETC equipment, augmenting trainees’ comprehension of these intricate systems.
To ensure a consistent MR experience, DataMesh leveraged its proprietary digital twin platform, FactVerse, to standardize SLAM (Simultaneous Localization and Mapping) accuracy across various devices. This approach guaranteed that trainees could immerse themselves in the MR environment without alignment or freezing issues.
The MR training not only enhanced comprehension but also facilitated more intuitive interactions with the equipment. Trainees could adjust their viewpoint and interact with the virtual 3D models using devices like HoloLens 2. Instructors also found the new training method to be more effective.
The successful integration of MR technology for ETC equipment training prompted Nexco-East Engineering to develop “MR for tunnel emergency facilities training.” This fresh MR application addressed the challenge of visualizing the internal structures of tunnel emergency equipment, which had previously proven difficult to explain.
The MR for tunnel emergency facilities training afforded trainees an intricate view of the internal water flow and operational status of automatic water sprinkler systems in tunnels. This virtual training facilitated a comprehensive comprehension of the equipment, including the ability to adjust transparency and observe water flow.
The MR system was also designed with a “pseudo reproduction feature” to instruct trainees on how to identify malfunctions and perform necessary repairs, suggesting its potential for future remote maintenance applications.
Furthermore, the MR training tools were adaptable to remote training, allowing a larger number of trainees to participate. Microsoft’s HoloLens 2 proved particularly advantageous for indoor training sessions, offering a hands-free and interactive experience tailored to individual learning requirements.
Looking forward, Nexco-East Engineering intends to broaden its utilization of ICTs beyond hands-on training at the TTC. They plan to incorporate online training, on-the-job training, safety education systems, and remote maintenance solutions. The company acknowledges the importance of adapting to the growing complexity of equipment and facilities, ensuring that engineers are well-prepared to tackle challenges, even in the absence of experienced mentors.
Nexco-East Engineering recognizes the value of collaborations with ICT professionals like DataMesh and Microsoft Japan, enabling them to stay at the forefront of technological advancements and continue their mission of maintaining and enhancing the safety of Japan’s expressways. In doing so, they are working towards a future where ICT solutions play a pivotal role in addressing infrastructure aging and labor shortages, ultimately contributing to safer and more efficient expressway management.