Originally published by our sister publication Specialty Pharmacy Continuum

By Gina Shaw

Among the top 10 patient safety concerns facing the healthcare industry in 2025, as ranked by ECRI and the Institute for Safe Medication Practices in a new report, are two directly related to technologies that frequently involve hospital pharmacies: insufficient governance of artificial intelligence and cybersecurity breaches.

Although AI is being incorporated into an ever-growing array of healthcare applications—including many that are commonly used in pharmacy, such as clinical decision support tools—few healthcare organizations have policies governing the use of AI, noted the report, issued March 10. For example, a 2023 survey of 31 hospital executives conducted by the Center for Connected Medicine found that only 16% reported that their organization had a systemwide governance policy for AI usage and data access. “Medical errors generated by AI could compromise patient safety and lead to misdiagnoses and inappropriate treatment decisions, which can cause injury or death,” the report warned.

“AI has many potential benefits, but it requires governance and performance and safety metrics that are monitored over time,” said Francisco Rodriguez-Campos, MSc, PhD, the principal project officer of device safety at ECRI. The new report recommends that institutions form multidisciplinary committees to evaluate new technologies that incorporate AI and determine risks, with representation from leadership, clinical services, human factors engineering, clinical engineering, patient safety and risk management.

Institutions should also implement reporting systems for tracking AI-related medical incidents, errors and adverse events, the report stressed. At present, it’s unclear whether there are any such mechanisms at a federal level. 

“The FDA’s MAUDE [Manufacturer and User Facility Device Experience] database tracks errors related to medical devices, but you don’t find AI-related medical incidents in that database,” Dr. Rodriguez-Campos said. “That doesn’t necessarily mean that such incidents or errors have not happened; it’s more that people may not really associate the AI with the incident yet. For example, let’s say you have an incident involving a PET scan: Does the error or problem have to do with the scanner or with the AI that is running on the scanner?”

ECRI has been asking institutions to report suspected AI-related incidents to its reporting network; so far, the handful of events that have been reported were not related to the AI, but more data are needed. In a webinar ECRI held for its members on managing risks associated with AI-enabled health technologies, 7% of attendees polled reported they were aware of instances at their facility in which an AI technology could have contributed to an event that did (or had potential to) adversely affect patient care. “We definitely need a more robust reporting system regarding errors and incidents involving AI, including those related to privacy, accuracy, misdiagnosis and potential bias,” Dr. Rodriguez-Campos said.

Cybersecurity is another leading patient safety concern, the ECRI report found. In a 2022 survey on healthcare cybersecurity sponsored by Proofpoint, 89% of respondents reported that their organizations had experienced cyberattacks in the past year: 72% believe that they are vulnerable to a ransomware attack, and 53% said their organizations lack in-house cybersecurity expertise.

“Cyber threats can take many forms and can enter the organization in a number of ways, including social engineering attacks; ransomware; loss or theft of equipment or data; insider, accidental, or intentional data loss; and attacks against network-connected medical devices,” the report explained.  Among the multiple detrimental effects of such incidents on patients: “Access to prescription medications can be compromised, leading to missed doses that may contribute to poor outcomes or out-of- pocket expenses in order to continue therapeutic treatment.”

ECRI urged health systems to take an enterprise risk management approach to help the organization achieve a comprehensive understanding of cybersecurity risks, using guidance from the National Institute of Standards and Technology, The Joint Commission, and the Department of Health and Human Services for small healthcare organizations and medium/large organizations.

Dr. Rodriguez-Campos reported no relevant financial disclosures.