A multidisciplinary team of clinicians, scientists, and engineers at Mount Sinai has developed a deep learning pose-recognition algorithm that analyzes video feeds of infants in the neonatal intensive care unit (NICU) to track their movements and assess key neurological indicators. The findings, suggest that this AI-based tool could offer a minimally invasive, scalable solution for continuous neurologic monitoring, providing critical real-time insights into infant health.
Each year, over 300,000 newborns are admitted to NICUs across the United States. Infant alertness is a highly sensitive component of neurological assessment, indicating the overall health of the central nervous system. However, unlike the continuous monitoring of heart and lung function through cardiorespiratory telemetry, comprehensive neurotelemetry has remained out of reach for most NICUs, despite advances in EEG and the development of specialized neuro-NICUs. Neurological assessments typically rely on periodic physical exams, which can be imprecise and may miss subclinical changes.
The Mount Sinai team hypothesized that computer vision techniques could be used to monitor infant movement and predict neurological changes. “Pose AI,” a machine learning method capable of tracking anatomical landmarks from video data, has already transformed fields like athletics and robotics. The team adapted this technology to neonatal care, training their algorithm on over 16,938,000 seconds of video footage from 115 infants at The Mount Sinai Hospital who were undergoing continuous video EEG monitoring. The study demonstrated that Pose AI could reliably track infant movement and identify anatomical landmarks from video data, effectively predicting sedation and cerebral dysfunction with high accuracy.
“Although many NICUs are equipped with video cameras, until now, deep learning has not been applied for continuous monitoring,” said Dr. Felix Richter, senior author and Instructor of Newborn Medicine in the Department of Pediatrics at Mount Sinai. “Our study shows that an AI algorithm applied to video feeds can detect neurological changes early, potentially enabling quicker interventions and improved outcomes.”
As reported by medicalxpress, the team was impressed by the robustness of Pose AI, which performed well under various lighting conditions, including day and night settings and when infants were undergoing phototherapy. The Pose AI movement index also correlated with both gestational and postnatal age.
Dr. Richter emphasized that this tool is designed to complement, not replace, the essential assessments performed by physicians and nurses. “We see this as an augmentation of current practices, providing continuous monitoring that clinicians can leverage to make informed decisions,” he said. The goal is to develop a system akin to a neuro-telemetry strip, similar to current heart rate or respiratory monitors, which could alert staff to changes in sedation or cerebral status. Clinicians would be able to review both video footage and AI-driven insights, creating an intuitive tool for bedside care.
The study’s authors acknowledged limitations, noting that the AI models were trained using data from a single institution. As such, further evaluation on video data from different facilities and camera types is needed. The team plans to test the technology in other NICUs and conduct clinical trials to assess its impact on patient care. They are also exploring its application for other neurological conditions and its potential use with adult patients.
“At Mount Sinai, we are dedicated to leveraging new AI advancements to enhance patient care,” said Dr. Girish N. Nadkarni, System Chief of Data Driven and Digital Medicine and a co-author of the study. “AI is already improving clinical outcomes across our health system—reducing hospital readmissions, aiding in cancer diagnostics, and delivering real-time care based on data from wearables. We are thrilled to bring this safe, non-invasive AI tool into the NICU to benefit our most vulnerable patients.”