Cerence reimagines voice assistance with Co-Pilot

The mobility company Cerence showcased new technology at CES this month that reimagines the idea of an in-car voice assistant.

The technology, Cerence Co-Pilot, uses artificial intelligence and continuous learning to provide more intuitive in-vehicle assistance, with the ability to even anticipate a driver’s needs before being prompted for assistance.

Cerence Chief Technology Officer Prateek Kathpal said the Co-Pilot technology works to connect various facets of a driver’s life to the car, creating a much more personal in-vehicle assistance experience.

“The cars are now more software driven, but then they also have information within the car — which is sensor data, your personal data, your life that you’re bringing into the car from the phone, environmental data and also the road-condition data,” Kathpal told Automotive News via Zoom. “A true co-pilot needs to be aware of all that, but also guide you, inform you and help you along the way.”

The Co-Pilot information is conveyed to drivers on screen in the vehicle’s infotainment center, along with audible alerts and potential mobile alerts as well.

Cerence Co-Pilot, a 2022 CES Innovation Award recipient, is not only capable of alerting drivers of expected and unexpected vehicle maintenance issues; it is also able to deliver real-time information such as upcoming severe weather conditions, then offer to put the car in the appropriate driving mode. The AI-powered technology can also suggest ordering and paying for a driver’s morning coffee when they are a mile from their favorite coffee shop.

One of the key components of Co-Pilot is the Cerence Connected Vehicle Digital Twin platform, which includes a virtual replica of the car based in the cloud, featuring all of the vehicle’s sensor information.

“Whether that’s your tire pressure, or whether that’s your open sensor in the car, or whether that’s your maintenance errors that you’re getting in the car — all that information is available as a virtual replica of that car in the cloud,” Kathpal explained.

“That digital-twin data could be used to make intelligent decisions in the cloud and push them back to the car. But that digital twin can also be used to create more contextual applications in the cloud, based on the real-time car information data which the digital twin is pulling from the car real time, but much, much more intelligently.”

The Vietnamese electric vehicle startup VinFast is the first automaker to adopt Cerence’s digital-twin solution.

“The incredible capabilities of Cerence Connected Vehicle Digital Twin are key in our efforts to creating the world’s first truly connected car that will revolutionize the driving,” Hong Sang Bae, VinFast chief technology officer, said in a press release.

Cerence’s digital-twin capabilities enable VinFast to provide its customers with several unique tech features, including safety notifications when a driver misses a stop sign, calendar reminders about upcoming meetings or an intrusion alert when someone breaks into the vehicle.

Kathpal says future applications of Co-Pilot, with its digital-twin capabilities, could include providing intuitive assistance beyond in-vehicle experiences, such as when a package must be delivered to a person’s vehicle safely and securely.
“The digital twin knows where your car is parked,” Kathpal said. “It knows the car is locked, unlocked, the status, the different sensor information.” He said the technology could create a temporary digital “key” that would allow the delivery person to unlock the trunk.