By Bobby Carlton
Through the M Mixed Reality initiative, BMW sets their sites on how XR technology will play a role with in-car entertainment and the rise in passenger economy.
BMW has created a way for drivers to actually be behind the steering wheel of a moving vehicle while wearing a VR headset designed to enhance your driving experience, and it puts the automaker on a new path as they dive into how XR technology, and the rise in passenger experiences and self-driving vehicles are becoming a reality.
The German automaker recently unveiled a new way for people to experience their M2 vehicle through a VR headset that has you actually driving the car. The M2 Project, which is part of the automaker’s “M Mixed Reality” initiative, allows people drive a car that includes your foot on the gas pedal, braking, steering, turn signals, even the radio! Except instead of seeing the real-world, you’re driving through a futuristic city. Check out the video below!
It sounds totally sketchy to be behind the wheel of any car while wearing something over your face that cuts out your real-world environment, but this wasn’t designed for normal streets; BMW designed their VR experience to be used on the company’s test track. The VR software can adapt and re-create the virtual course of locations around the world.
With that said, you could absolutely see something like this for your passengers!
Thanks to computer vision and simultaneous localization and mapping (SLAM), the car has some safety triggers built in to make sure you don’t get into an accident. As an added layer of safety, a BMW employee also rides in the passenger seat to watch the road and press an extra brake pedal in front of them.
One company that sees the potential of VR and passenger economy is Holoride. They first announced their work back in 2019 showing how passengers could access their VR experiences as part of in-car entertainment. Since then, they have improved the experience using HTC’s Flow VR headset and just recently announced the launch of their in-vehicle VR entertainment system in Germany. Select Audi vehicles can purchase the Holoride Pioneers’ Pack, which includes everything you need to transform your car into an “always-in-motion virtual space” where you can play games, browse the web, and more.
The inspiration for the BMW project came from the company’s digital city, which was known as M Town. Alex Kuttner, the engineer who developed the VR experience, said that the BMW fans wanted to visit the city if it was a real place.
“M Town is a mindset,” Kuttner said in an official BMW press release. “It’s a town where everything is possible, and that was the moment I realized we aren’t only here for selling products. We’re here for selling emotions and experiences. These two things combined in mixed reality are only the start of something really great in the future.”
Almost two years ago, the company started working on the mixed-reality project, which was initially supposed to be used for the M5 model. However, instead, it was designed for the M2. According to the company’s executives, the project could also be used to help drivers in racing competitions and training courses.
Frank van Meel, the company’s CEO, said that the goal of the project was to give employees a chance to explore new ideas without having to think about the business case for each new innovation.
“I think the interesting thing is now we have an answer, and the question is, what is the question to this answer?” van Meel said. “There are so many ideas. We haven’t found the final answers, but we’re working on all of these kinds of ideas.”
Although the experience isn’t yet available for the general public, the company invited several prominent gamers and content creators to participate in the development of the virtual reality experience. One of these was Cailee, a popular Twitch streamer and member of the G2 esports team. She said that she had previously tried playing video games in VR, but this was the first time that she actually used it. She believes that other games could also benefit from the technology.
“It’s just the most insane experience I’ve ever, ever had,” she said. “I play Rocket League, I’ve sunk so many hours into it and everything, but I really cannot describe the experience that I had in Munich.”
BMW hosted a demo of the mixed-reality experience in Lisbon. For those who were able to try it, the results were impressive, with everyone stating how the virtual course matched the actual driving experience, from speeding up, slowing down, and turns.
The first lap of the course featured a variety of obstacles that people had to avoid. To help “gamify” the experience, drivers had the task of collecting coins along the way. On the second and third laps, a timed element demanded that drivers accelerate, which provided them with the real feeling of racing. The suspension of reality allowed drivers to feel more comfortable with the way they drove.
According to David Hartono, the creative tech director of Monogrid, BMW’s interest in gaming is evidenced by the company’s decision to turn the vehicle’s internal display into a controllers. Last month, BMW partnered with AirConsole to allow players to play games using the company’s in-car display. He noted that the company’s use of VR technology could help reinforce its image as an innovative and pioneering company.
Sean MacPhedran, a senior director at SCS, a digital agency based in California, praised the BMW’s mixed-reality experience, saying that it was a step up from the traditional methods of driving luxury cars. It also highlighted the company’s capabilities in a more consumer-friendly manner.
“With BMW doing so much work with mixed reality and industrial 4.0, it’s hard to telegraph that to a consumer,” MacPhedran said. “A consumer doesn’t care about all the stuff you’re doing to make a car that much better. It kind of reminds me of how they show the car in the wind tunnel, but now they’re doing this to show how advanced a car is.”
Several car brands have started experimenting with VR and AR in the past couple of years. In 2017, Lucid, a luxury electric vehicle startup, opened its New York City showroom to allow people to explore its virtual models. In 2022, Porsche and Audi announced that they would be partnering with the startup Holoride to develop in-car VR systems that would be used to give passengers an incredible immersive experience.
While Nissan turned to AR to show potential car buyers how safe their cars are and used VR as a fun tongue-in-cheek way to start a conversation about recruiting mechanics.
Along with the auto industry, we are also seeing aviation look at VR technology to change the way their customers experience air travel.
According to Mike Ramsey, an analyst at Gartner, car companies are constantly looking for ways to keep up with the technological advancements that are happening in the industry. However, he noted that BMW was one of the first companies to invest in both virtual and augmented reality systems. Despite the company’s early involvement in the technology, he believes that the company’s use of VR is more about brand building and setting their targets on the rise of XR and passenger experiences.
“It’s one of those things that every single car company is investing in but nobody has figured out what the business value is,” he said. “Augmented reality, virtual reality, all of these technologies. For a company that has a performance-oriented orientation, they’re going to look at that as a way to expand their brand beyond the physical to wherever you travel, so to speak.”
According to Heiko Wenczel, the director of the UE business at Epic Games, the BMW experience was built using real-time sensors and interactions with the car’s surroundings. He said that being able to test and experience something in virtual reality is very beneficial for developing real-world products.
“You can translate that into any part of the manufacturing and automotive world,” Wenczel said. “Like when you design you get real-time feedback automatically, like what that is, and the human scale of designing cars and like understanding what mobility will be in the future needs that kind of interaction in real-time.”
Instead of rushing to develop something, companies often have to find a reason for their actions and how they can make money from it. According to van Meel, it’s important to start with a low-budget project to avoid investing too much money. Although BMW wouldn’t provide the exact amount of money that it spent on the project, van Meel noted that the company’s budget was relatively lean.
“If you take a step back and you say, well, it’s not finished yet, but I can see a lot of creativity and a lot of potential that is still a little bit unclear,” said van Meel. “You just should let it happen if it’s not insanely expensive, of course, because then you need to make decisions right away.”
With the more automakers looking at XR technology as part of the in-ride experience and focusing on passenger economy, and companies like Einride with their driverless technology, the automotive industry is entering a brand new phase.
