The use of augmented reality could have far-reaching implications for navigation systems, with multiple automakers already deploying the technology to enhance the way vehicles display information about driving conditions and the surrounding environment.
As advanced driver-assistance systems and autonomous driving continue to evolve, augmented reality technologies could also be an enhancement, providing a more immersive experience for passengers and drivers as their gaze can finally leave the road.
Basemark, a software company that specializes in automotive software, has started implementing AR applications for BMW Group, including an AR over-video application available in some of the automaker’s models such as the full-electric BMW iX crossover.
These apps use vehicle sensor data and computer vision functionality to put AR-enhanced information on the vehicle’s central information display when navigation is enabled.
The front-view camera sends live footage to a curved touch screen, where interactive arrows appear to show the driver exactly where they need to go.
Upcoming turning maneuvers and lane recommendations are shown to the driver, while the map remains visible thanks to a split-screen functionality.
In the technology’s application on German roads, “there is a functionality which helps drivers find parking spots and to understand the prices of the parking as well,” Basemark CEO Tero Sarkkinen told Automotive News Europe. “This is just the starting point, and we will have new features and functionality for BMW coming later.”
He added that AR helps the driver perceive the environment better, through highlighting other vehicles, pedestrians, distances and other factors.
With the next generation of AR use cases, Sarkkinen said, multiple information overlays and more sophisticated cameras will be able to provide heat-detection maps, which could help identify a large animal hidden in bushes alongside the road.
“You would then see a warning appear on the windscreen that would alert you to that presence, advising the driver to slow down,” Sarkkinen said.
Mercedes-Benz has been integrating AR technology into its navigation systems since the introduction of the A-Class in 2018.
Pedro Ribeiro Monteiro, director of infotainment systems at Mercedes’ R&D department, said the technology helps drivers find their way in complex traffic situations.
“MBUX Augmented Reality for navigation connects the virtual world with the real one,” Monteiro said.
“The technology shows graphical navigation and traffic information on live images.”
A camera in the windshield films the area in front of the vehicle, and that recording runs in the media display. Highlighted additional information, such as house numbers or street names, further eases the orientation.
“The system also supports you at traffic lights,” he added. “If you are in the first position, it will display the current traffic light situation.”
Monteiro said the company has been continually improving its navigation systems, pointing to the AR head-up display in the S-Class and EQS upper-premium models as the latest evolution. The systems in those vehicles integrate driving functions with AR visualizations.
“This head-up display complements your field of vision with virtual cues without distracting you,” he said.
“For example, large directional arrows hover over the intersection where you need to turn. You know immediately what to do.”
Monteiro said the goal is to use AR technology wherever it is helpful for the customer, leading to more comfort by reducing the level of stress while driving.
“When it comes to AR and heads-up displays, both driver distraction and safety are important aspects when designing the user experience. We consider them crucial for us when developing our AR systems,” he said.
“We look at AR as an important technology for intuitive navigation, and we plan to introduce new features and improvements in the future.”
If done properly, said Grant Courville, vice president of products and strategy for automotive and mission critical IoT at BlackBerry, AR technologies can boost the driver’s situational awareness.
“When I say ‘properly,’ that means making sure they are accurate and making sure they are not distracting,” he said. “Those are easy things to say but difficult things to do. It’s about presenting only the information that’s absolutely required and removing anything that is distracting.”
From his perspective, chances for innovation can occur through refinements in adding situational awareness to the environment the driver is in at a specific point in time and doing so in a way that’s not distracting.
“Anybody could put information up on a screen,” he said. “How do I put that information up so that it’s actually beneficial and not detrimental to the driver? If you’re in heavy downtown traffic, maybe there’s a certain amount of information that’s displayed which would be different than you’re on a highway moving at higher speeds.”
Courville added that as autonomous driving starts to enter the market and demands on driver attentiveness begin to relax, new opportunities for AR in navigation could emerge.
Foreign language challenges, for example, when driving in another country are an issue.
“Maybe Germany,” Courville offered as an example, “and there is signage on various storefronts that is in German. I don’t speak German, so the vehicle could offer, on the fly, translation on the windshield shown in English. These are things that could be possible down the road.
“Now you have flipped from safety of information to more personalization,” Courville said. “That’s where monetization of AR can come in as well.”