$5 million NSF Convergence project revolutionizes neurodiverse employment through AI

An AI-based job interview tool tracks a user’s gaze and other metrics during a meeting with a virtual interviewer. Charts of attention and stress levels (below, right) during a simulated interview give users feedback about their performance.

REHABILITATION ENGINEERING
NSF grant aligns with school’s Inclusion EngineeringSM focus

The interview tool tracks attention and stress levels and adjusts to a user’s comfort level.

The National Science Foundation has awarded a highly competitive $5 million grant to Vanderbilt University that greatly expands a School of Engineering-led project for creating novel AI technology and tools and platforms that train and support individuals with Autism Spectrum Disorder in the workplace.

The significant federal investment follows a successful $1 million, nine-month pilot grant to the same team that forged partnerships with employers and other stakeholders and produced viable prototypes through immersive, human-centric design. The multi-university team includes Yale University, Cornell University, Georgia Institute of Technology and Vanderbilt University Medical Center as academic partners.

One in 54 people in the United States has ASD;

Each year 70,000 young adults with ASD leave high school and face grim employment prospects;

More than 8 in 10 adults with ASD are either unemployed or underemployed, a significantly higher rate than adults with other developmental disabilities;

The estimated lifetime cost of supporting an individual with ASD and limited employment prospects $3.2 million.

Approximately $50,000 per person per year could be contributed back into society when individuals with ASD—are employed.

The grant, made through NSF’s Convergence Accelerator program, advances the School of Engineering’s focus on Inclusion Engineering,SM which uses the disciplines within engineering to broaden meaningful participation for people who have been marginalized.

Through a new framework called Inclusion AI, the project supports “meaningful, individualized workforce engagement of an existing, underutilized neurodiverse U.S. talent pool, in addition to supporting a more inclusive national employment landscape,” said David K. Wilson Professor of Engineering Nilanjan Sarkar, who leads the effort.

“This is a critical but overlooked public health and economic challenge: how to meaningfully include individuals with ASD—a large, chronically unemployed and underemployed population,” said Sarkar, professor of mechanical engineering and department chair.

“We want to harness the power of AI, stakeholder engagement and convergent research to include neurodiverse individuals in the 21st century workforce,” Sarkar said. “We feel that there is a big opportunity to turn great societal cost into great societal value.”

The NSF in recent years has increased its efforts to have experts from diverse fields converge to tackle a profound challenge through its Convergence Accelerator program. By definition, NSF Convergence grants support fundamental research leading to rapid advances that can deliver significant societal impact.

For this project, organizational, clinical and implementation experts are integrated with engineering teams to pave the way for real-world impact. The multi-university, multi-disciplinary team already has commitments from major employers to license some of the technology and tools developed.

One of the AI-based tools allows two people to collaborate while virtually assembling a table.

Researchers will address three themes: 1) individualized assessment of unique abilities and appropriate job-matching; 2) tailored understanding and ongoing support related to social communication and interaction challenges; and 3) tools to support job candidates, employees and employers.

In all, the project includes further development, refinement and testing of five separate technologies prototyped in Phase 1. They are:

  • An assessment system that integrates a wearable eye tracker, scene cameras and computer vision algorithms to produce a detailed record of a person’s performance in visuospatial cognitive tasks;
  • A virtual reality-based job interview simulator that senses a user’s anxiety and attention through wearable computing and provides feedback and coaching;
  • A collaborative virtual reality platform to assess and help team-building skills through
    peer-based and intelligent agent-based interaction (graphic, above right);
  • A social robot for use in home environments to improve resilience and tolerance with job-related interruption; and
  • A computer vision-based tool to assess non-verbal communication in real-world settings.

Frist Center for Autism and Innovation Director Keivan Stassun is among other Vanderbilt experts involved and will be pivotal in coordinating with industry and advocacy stakeholders. Other key Vanderbilt collaborators are Maithilee Kunda (computer science), and Joshua Wade (mechanical engineering) and Tim Vogus (Owen Graduate School of Management).

Already, notable private-sector companies that employ people with ASD have committed to using at least one of the technologies developed under this program: Auticon, The Precisionists, Ernst & Young and SAP among them.

Two other companies, Floreo and Tipping Point Media, will make their existing VR modules available for adaptation to the program. Microsoft, which has a long-standing interest in hiring people with ASD, is involved as well and provided seed funding and access to cloud services for technology integration.

The five technologies can be used separately or as an integrated system, and the work has broader potential beyond ASD to expand employment access. In the U.S. alone, an estimated 50 million people have ASD, attention-deficit/ hyper-activity disorder, learning disability or other neurodiverse conditions.

“A more diverse workforce benefits employers as well as individuals with neurodiverse conditions,” Sarkar said. “These tools and technology will have an even larger impact in helping marginalized populations with meaningful employment.”

Dan Burger, BS ’12, MS ’13, demonstrates an early version of the AI interview tool during the grand opening of the Frist Center for Autism and Innovation in July 2019. (Vanderbilt photo/John Russell)