New generation of AI Assistants

Washington DC (SPX) Jan 26, 2023 - The Perceptually-enabled Task Guidance (PTG) program is a cutting-edge initiative that aims to develop virtual task guidance assistants for military personnel. These assistants will be able to work with various sensor platforms to help military personnel perform complex physical tasks and expand their skillsets.

Unlike the AI assistants that are currently available, PTG technology will be able to see and hear what the user sees and hears. This will be achieved by integrating with a microphone, a head-mounted camera, and displays like augmented reality (AR) headsets. The technology will then deliver accurate instructions to the user, making the task at hand more manageable.

Recently, PTG performers have demonstrated early successes of their prototypes by using the task of cooking recipes as a proxy for more unfamiliar and complex tasks such as battlefield medical procedures, military equipment sustainment, and co-piloting aircraft. This is just a small sample of the many potential applications of PTG technology, which could greatly benefit military personnel in a wide range of scenarios.

"Today, the commercial sector is pursuing new and useful ways to present data to the user, but it doesn't go far enough," said Draper. "The gamechanger with PTG would be having perceptually-driven AI interfaces that can make sense of the real world, react to whatever the user is doing and provide advice. I'm really impressed at how quickly performing teams are making progress toward the goals."

For more information about PTG, you can also listen to Voices from DARPA podcast episode 41: The AI Tutor. In the episode, Draper expands on the motivations behind PTG in addition to other programs in his portfolio.

Video: A New Generation of AI Assistants