Mirror, mirror on the wall, show me … my collar bone.
Visitors to Nashville’s Adventure Science Center can get a new perspective of themselves with an augmented reality (AR) anatomy that creates the illusion of an X-ray view of the body’s muscles and bones.
Developed by a team of Vanderbilt Engineering undergraduates—with help from graduate students, several Nashville-area high school students and teachers—the exhibit was installed three years ago and has been used by hundreds of thousands of visitors.
Earlier this year, the Vanderbilt project was submitted to The White House Office of Science & Technology Policy (OSTP) Open Science Recognition Challenge in the Open Science to Advance Education category. Built on open-source computer code, the team has introduced 3D musculoskeletal models and expanded the range of educational content.
Exciting AR experiences already exist in combination with generative artificial intelligence (AI) and mixed reality (MR) intelligent overlays. Microsoft’s HoloLens has demonstrated the power of intelligent overlays in industries like healthcare and engineering, providing contextual information and virtual assistance. Numerous retailers experimenting with ‘try on’ smart mirrors in stores. New headsets like Apple’s Vision Pro and Meta’s Quest 3 expand inspiring possibilities for immersive experiences.
In education, AR holds immense potential for creating interactive content and dramatically transforming learning for students. Many medical students today are learning anatomy using a mix of lecture-based, lab-based, and VR-based material. From kindergarten through high school, AR visual technology has improved student engagement and perception, an extensive report by Fingent, a technology development company, noted recently.
“The thing I’ve always loved about the mirror is that it’s a project by students, for students. Learning anatomy can be difficult and dry, but turn it into a game? Make it personal by showing your own anatomy? And suddenly it’s an engaging, almost organic exploration of our own bodies,” said Cailey Kerley, PhD’22. Kerley mentored the undergraduate students on the team and helped set up the exhibit.
“We just got the TV mounted, the cameras setup, and the software running so everything was finally in place after many months of work. A little boy ran by, saw the mirror and stopped in his tracks. As he stared at the skeleton projected over his body in the mirror, he turned to his mom and said, ‘Are those MY bones? How did they do that?’ It was absolutely the best reaction we could have asked for,” Kerley said.
Over a period of five years, an interdisciplinary team of scientists and students gathered to create the project. In addition to Bennett Landman, chair of the Department of Electrical and Computer Engineering, and the project team lead who pioneered the AR tool development, computer engineering assistant professor Yuankai Huo contributed to user experience optimization. Morgan Rehnberg, vice president of Exhibits and Experiences at the Nashville Adventure Science Center arranged the exhibit. Fourteen engineering undergraduates, three Nashville high school teachers and five high school students also contributed to the project.
“The AR Mirror is a tremendously popular exhibit at the science center. Visitors love to see the mirror respond as they stand, walk, or dance in front of it. Every day, I see families learning together as they explore the body systems it reveals. We’ve had a great experience collaborating with Professor Landman and his team and look forward to continuing to work together in the future,” Rehnberg said.
The team developed the AR app using an Orbbec Astra Pro body tracking sensor that uses lasers to track a person’s position and then uses code to overlay muscles and a skeleton in 2D on a video that’s displayed like a mirror. The exploration mode allows users to identify bones and muscles by selecting them with their right hand. The game modes transform learning into an engaging quiz to reinforce the knowledge gained in the exploration mode.
“Using exploration and quiz game modes, the Vanderbilt AR application bridges the gap between theoretical knowledge and practical application. It makes anatomy education more accessible and captivating,” Landman said.
Adventure Science Center visitors can scan a QR code in the middle of the exhibit to learn more about the code used to create the exhibit, how to buy an Orbbec camera, about $160, and how to create their own AR mirror. The make-it-yourself project—Augmented-Reality-Programming-Challenge-Walkthrough —is offered on GitHub, an open-source project repository.
Contact brenda.ellis@vanderbilt.edu