Reinforcement Learning and data drive NASA air taxi project

Before urban commuters take to the skies, a multi-institution team with Vanderbilt engineers and funded by NASA will develop and test the foundations of safety management for commercial, self-piloted air taxis.

Such aircraft must communicate with each other. They must respond to hazards, from weather to equipment malfunction to “uncooperative” other aircraft to prevent collisions and crashes. And all this must unfold in real time, in defined corridors separate from existing air traffic routes but without continuous air control support on the ground.

Reinforcement learning, a form of machine learning, is central to the $2.5 million project.

“This is a very exciting task,” said Cornelius Vanderbilt Professor of Engineering Gautam Biswas, who heads the Vanderbilt effort. “A machine learning algorithm is not like a person–it can do only what it has been trained for. It can analyze a situation better than a human, but it doesn’t have the intuition to deal with unusual situations.”

These vehicles, called eVOTLs, for electric vertical takeoff and landing, resemble a cross between a helicopter and a small airplane. They don’t need long runways of passenger jets or even smaller planes, but their lighter, pilotless profile makes them more susceptible to certain types of risk.

The project tackles three types of hazards: adverse convective weather, winds and fog; corridor incursion by non-cooperative aircraft; and vehicle and component level degradation and faults. Vanderbilt engineers, as experts in tracking performance of components and monitoring degradation, are focusing on the latter.

A key innovation they bring to this work is the idea of using Reinforcement Learning algorithms for online fault tolerant control. “What happens when a fault occurs? How can aircraft keep flying by adjusting its controller? Should it continue or alter its route, or is the situation so bad the vehicle must find a place to land immediately?” Biswas said.

“All of this decision making has to happen on board,” he said.

The approach also incorporates condition-based maintenance to support safe flights and will require a cloud infrastructure for prognostics, risk evaluation, and hazard response functions.

Automated aircraft control is not new. Autonomous systems handle more and more functions in commercial and military flights each year with software trained by system models. The difference with the eVTOL safety management project is that it will be data-driven.

NASA has been testing such aircraft at its research center in Hampton, Virginia. The agency is exploring how different aircraft technologies and configuration will perform in an urban environment, and researchers will use this data. With the information trove, the team will tackle not only what and how a “machine” learns, but also how to make the leap into “learning as you go.”

“All of this will be done in a data-driven manner,” Biswas said. “That is the interesting thing and the challenge–how rich is your data? If you don’t consider the limitations of your data, you will fail.”