In a burning building there’s a difference between what someone can sense in the surroundings and all the environmental data around him. Researchers at NASA’s Jet Propulsion Laboratory (JPL) and the Department of Homeland Security (DHS) are working to fill that gap with an artificial intelligence system that can collect relevant information from the environment and relay it back to firefighters in real time.
AUDREY — or, the Assistant for Understanding Data through Reasoning, Extraction, and sYnthesis — is integrated with the Internet of Things, which lets the system connect to wearable sensors and head-mounted displays on each firefighter, communicating data about temperatures, hazardous gases, and even GPS locations from one team member to another.
“When first responders are connected to all these sensors, the AUDREY agent becomes their guardian angel,” Edward Chow, manager of JPL’s Civil Program Office and program manager for AUDREY, said in a press release. “Because of all this data the sensor sees, firefighters won’t run into the next room where the floor will collapse.”
As guardian angle, AUDREY can give guidance during training and real-world missions.
By tracking the movement of each team member, the system alerts individuals with relevant data about their immediate surroundings and communicates this information to the members in a given vicinity. Since AUDREY is designed to know the role of each team member, it can offer targeted information to the relevant firefighter, rather than overloading each team member with data.
The cloud-based machine-learning system can also learn to make predictions about missions, including what tactics should be used, and can offer recommendations for how the team can work better together.
These combined features make AUDREY unlike many other AI agents, according to Chow. “Most AI projects are rule-based — if this, then that,” he said. “But what if you’re only getting part of the information? We use complex reasoning to simulate how humans think. That allows us to provide more useful info to firefighters than a traditional AI system.”
AUDREY was developed as a part of the DHS’s Next Generation First Responder Program, which aims to innovate and invent sensory and communication technologies to keep firefighters, police, and paramedics safer in the field. The system was tested in a virtual demonstration in June and the researchers hope to test it in the field within a year.
