While immune checkpoint inhibitors (ICIs) offer lifesaving treatment for cancer patients, they can also lead to immune-related adverse events (irAEs), side effects that can affect nearly every organ in the body with varying levels of severity. However, understanding the frequency and impact of irAEs in real-world data has been challenging, as combining cases across institutions has proven difficult. Manual approaches to investigate irAEs are inefficient, prompting researchers at Mass General Brigham to utilize a prebuilt large language model (LLM) to identify irAE instances in hospital settings more effectively.
As reported by massgeneralbrigham.org, the LLM was able to detect the most common severe irAEs, such as ICI-induced colitis, hepatitis, pneumonitis, and myocarditis—some of which can be fatal. When compared to International Classification of Disease (ICD) codes, which are used retrospectively to identify irAEs, the LLM demonstrated superior accuracy. These findings were published in *The Journal of Clinical Oncology*.
“The LLM not only outperformed ICD codes in detecting irAEs, but it also identified additional cases that manual adjudication missed, with excellent specificity and sensitivity, completing each chart in just 9.53 seconds,” said Dr. Kerry Reynolds, Director of the Severe Immunotherapy Complications Program at Mass General Cancer Center and lead author of the study. “As a free, open-source model, this LLM pipeline offers other institutions the ability to quickly create similar databases, potentially fostering unprecedented collaboration.”
The study analyzed a decade of data from a meticulously curated dataset of patients who were hospitalized after receiving ICI therapy. “The model consistently achieved sensitivities and specificities above 90% across the four irAEs, which is remarkable,” Reynolds stated. “Historically, irAE research has been concentrated in large academic centers, limiting the involvement of smaller community hospitals. This study has the potential to change that. The LLM is lightweight, requiring minimal computational resources, and can be run on a local machine. We’re excited to share it with the broader community.”