Eye-gaze tracking system could dramatically improve kidney stone surgery training

When kidney stone surgery is performed using an endoscope, about a fourth of those patients will require a repeat surgery within 20 months due to residual stone fragments being left behind. These remaining pieces can lead to obstruction, pain, kidney injury and recurrent infections.

Jie Ying Wu

Gaining and assessing surgical experience in endoscopic kidney stone surgery is particularly challenging given the limited field of view in the devices, said Jie Ying Wu, assistant professor of computer science at Vanderbilt University, whose research focuses on using machine learning techniques to transform surgical robots from a teleoperated tool to an intelligent assistant.

Wu models the interactions between surgeons and tissue to better understand operational environments. One of her research applications is skill assessment for kidney stone surgery and 3D reconstruction from endoscope images.

“Successful endoscopic stone surgery requires a surgeon to visualize the entire renal collecting system and locate all kidney stones during treatment,” Wu said. “Complete kidney navigation relies on clinical experience that associates with the risk of postoperative residual stone fragments.”

Wu is creating an eye-gaze sharing system to complement the current standard of care in which trainees get only verbal feedback. The system allows trainees to see the expert’s gaze. The expert can use their gaze in real time to point to objects on the screen or where the trainee should go next.

Gaze sharing has been shown to improve skill acquisition and retention in robot-assisted and laparoscopic surgical settings but has not been explored in the environment of ureteroscopy, said Wu, an affiliate of the Vanderbilt Institute for Surgery and Engineering. VISE is an interdisciplinary, trans-institutional entity designed to bring engineers and physicians together to impact health care.

Left, physicians wear Microsoft HoloLens 2s to both track gaze and project the augmented reality markers. Right, a possible interface where the expert’s and trainee’s gaze are denoted by gray and red triangles, respectively, in the endoscope video.

“As our training system is solely composed of software running on augmented reality devices, all existing endoscopic and laparoscopic surgical procedures could, in principle, immediately benefit from the results of this project,” Wu said. “In this way, we believe the success of our project will improve training procedures and mitigate repeat complications or surgeries, benefiting patients, surgeons and society.”

Nicholas Kavoussi is Wu’s collaborator on the project. He is a minimally invasive urology surgeon at the Vanderbilt University Medical Center. Kavoussi also is a member of VISE and actively involved in multiple research projects furthering the integration of technology and surgery. “I really couldn’t do it without him,” said Wu, who also holds appointments in Biomedical Engineering, Electrical and Computer Engineering, and Mechanical Engineering.

She is one of four Vanderbilt researchers who received a Vanderbilt Scaling Success Grant for “Development and Validation of Gaze-Based Training for Endoscopic Kidney Stone Surgery.” Scaling Success Grants aim to turn ambitious, early stage research proposals into tangible achievements and secure significant expanded external funding.

All Scaling Success applicants receive detailed feedback from reviewers, enhancing their chances for future success. “Having other perspectives will help us strengthen external proposals,” Wu said.

Wu has applied for external funding to evaluate whether the gaze-sharing platform improves complete kidney visualization.

Scaling Success is administered by Research Development and Support in the Office of the Vice Provost for Research and Innovation. The fund focuses on projects that have already shown potential and have received interest from sponsors. For more information on Scaling Success and other internal grant programs, visit the Research Development and Support internal funding webpage.

Contact: brenda.ellis@vanderbilt.edu