By Bobby Carlton
Even with the news of Microsoft shutting down all mixed reality projects, employees at Toyota sees positive results using the Microsoft HoloLens 2 for training and collaboration.
Even though written instructions may be precise and thorough, sometimes a professional can’t master a new skill just by looking at them. This is where XR technology can help and why Toyota is using it with their workforce.
When Toyota started experimenting with Microsoft’s HoloLens, it discovered that the technology could be incredibly useful. In a case study released by both Toyota and Microsoft, David Kleiner, who heads Toyota’s research lab in North America, describes a particular use case that shows how the HoloLens can be used to improve the efficiency of his team.
In a logistics center in New Jersey, a Toyota employee was having a hard time finding the right way to install a door edge guard on a Toyota vehicle. To solve the problem, the employee and his colleague connected with a fellow Toyota colleague in California using the HoloLens 2 AR headset. They were able to see through the other person’s eyes and learn how to install the door guard.
The instructions were captured using the HoloLens 2 and were then used in Microsoft’s Dynamics 365 guides system, which makes it easier for workers to access the same help.
The case study shows how the technology helped teams improve their efficiency and reach their continuous improvement goals. The study also explained how workers at various logistics centers in the US used the HoloLens 2 for training and collaboration.
Toyota’s priority is making sure their teams are safe and have the tools needed to be successful, but in a way that is efficient, and according to Kleiner the company’s ability to solve problems and train people quickly is very important in order to bring new products to market faster. To overcome the limitations of time and location, Toyota is currently using mixed reality and the metaverse to enable their employees to share knowledge and improve their performance.
Toyota’s decision to use the Microsoft HoloLens 2 was also due to the company’s approach toward the development of the next wave of the “industrial metaverse”. Unlike other companies, Microsoft doesn’t just provide the right hardware to its customers. Instead, it offers a complete package that includes everything a company needs to get started with the technology.
Through its various offerings, including the Microsoft Azure and Dynamics 365 platforms, companies can create digital twins of their factories or supply chains. They can also use the cloud to simulate their operations and improve their environmental impact. With the help of Microsoft’s technology, Toyota can now benefit from the company’s extensive ecosystem to enable frontline workers to improve their performance.
Toyota’s goal is to make the HoloLens a “screen” for its frontline workers, enabling them to access all of their company’s tools without having to carry a laptop. With the help of the cloud, businesses like Toyota can now provide their employees with a variety of tools and experiences designed to improve their productivity.
“By using mixed reality applications on HoloLens 2, we have a 3D expression of the technical information, which means we free up both our hands. We’re far more efficient and look cool while we’re working,” said Hiroshi Sakai, Specialist and Lead of Mixed Reality Service Information at Toyota Motor Corporation
One of the tools that can be used by Microsoft to improve the efficiency of its frontline workers is Remote Assist, which is a remote assistance tool that can be used for monitoring and controlling the door edge.
This tool can be used by frontline workers to bring coworkers into their physical space, enabling them to receive calls and annotations from anywhere. They can also see what the other team members are seeing in real-time through the 3D space. Toyota has been working with Microsoft on the integration of the Remote Assist app and the Microsoft HoloLens since the latter was released in 2016.
Through the partnership, Toyota was able to create an enhanced version of the “Guides” experience for the HoloLens. It allows workers to start a session and get immediate access to a trainer, as well as additional support if they need it.
Before, technicians attached QR codes to the hood of vehicles to guide users through holographic instructions using the HoloLens and Guides. Unfortunately, these holograms often moved when the user moved around a car. To address this issue, Toyota and Microsoft collaborated to develop an “object understanding” solution, which allowed technicians to lock a hologram on a vehicle. This ensures that staff members have a consistent experience.
Toyota has continued to find the value in the “Guides” as a training tool. According to Kleiner, the company’s trainers can now use the tool to allow their students to work independently, which means they can learn at the same time. The training times for various individual users have been reduced by up to 50%.
Toyota’s innovations have helped the company move its efforts with the development of the HoloLens 2 platform into the enterprise. Kleiner noted that when the technology was first introduced to the company’s IT department, it was very easy to implement because it was just another Windows machine.
According to Kleiner, Toyota’s team is expecting the “Guides” to become a key component of the company’s workforce in the years to come. He also noted that the company will continue providing employees with the necessary tools and resources to improve their productivity.
Toyota has now been able to deploy a device that is easy to maintain and can be used by every employee. This allows the company’s workers to participate in various business conversations.
Since this case study was published, it was announced that Microsoft was drastically reducing all teams working on AR and VR and ending all of their mixed reality projects.
One of the teams that was let go was the one that created the mixed reality tool kit known as the MSRTK. This was a cross-platform system that was designed to create spatial anchors in virtual space. It was able to work with various VR headsets, such as those made by Meta and Microsoft’s own Hololens 2.
It’s unsure how this will impact Toyota’s endeavors with the HoloLens 2. But with the success Toyota had using XR technology as an enterprise solution, the company can easily shift and turn to headsets such as the Meta Quest Pro, the HTC Vive XR Elite, or a Pico 4 headset, which all provide full color passthrough, giving the user access to MR experiences.
There is also the rumored Apple mixed reality headset which would come packed with technology needed for Toyota for them to continue their work using XR.
One thing to keep in mind is that Microsoft isn’t totally out of the game when it comes to XR tools. The company will be launching their Microsoft Mesh application sometime in 2023.
