FDA faces research hurdles in regulating AI for imaging
In a keynote presentation that shared the US Food and Drug Administration (FDA) perspective on AI / Machine Learning (ML) as medical devices, Ravi Samala, PhD, highlighted four driving forces that have led to unique regulatory challenges. for the agency:
- Emerging applications – algorithms to identify new model observations in human physiology; monitoring response to treatment; and more
- New specialty areas for AI
- The unique nature of medical data – characterized by low prevalence of disease, lack or difficulty in obtaining truthful data; large variations in imaging technologies from manufacturer to manufacturer; and variations based on gender, age, ethnicity, imaging modality and disease categories
- Advances in algorithms – new techniques such as generative adversarial networks (GANs) for data generation; reinforcement learning; continuous learning; transformers, etc …
âBased on these observations, our team identified five AI / ML gaps in regulatory scientific research that could help the agency fulfill its mission of giving patients in the United States the first access to software in the United States. as medical devices, âsaid Samala, a staff member of the FDA’s Division of Imaging, Diagnostics and Software Reliability (DIDSR) at the Center for Devices and Radiological Health (CDRH) of the FDA. Office of scientific and technical laboratories of the agency.
These research gaps concern:
- Data size
- New and multimodal data
- Computer Assisted Sorting Applications
- Quantitative imaging
- Adaptive algorithms
Data size issues include issues related to the scarcity of medical imaging data; data augmentation techniques; transfer learning; and GANs, Samala said.
“It is absolutely necessary to fundamentally understand the limitations of smaller data sets, as well as to develop techniques to maximize information and improve AI / ML training,” he said at the meeting, hosted by the Society for Imaging Informatics in Medicine (SIIM).
The FDA is working to address this issue with efforts like its Virtual Imaging Clinical Trials for Regulatory Evaluation (VICTRE) project, for example, according to Samala.
New evaluation paradigms
The second regulatory research gap is the development of new types of AI software that use multiple types of data sources, including radiology, physiology, pathology, patient demographics, and other patient record data. electronic health, according to Samala. These new types of algorithms often require new paradigms of evaluation – including clinical and non-clinical testing – as well as the measurement of safety and efficacy when used in new types of applications.
“Although most devices currently on the market are diagnostic in nature, we are seeing more and more prognostic devices – trying to predict response to treatment, assessment of risk in therapy – which requires different metrics of assessment as well as different standards, “he said.
The FDA is working to identify assessment approaches that can assess the performance of these types of devices, he said.
Computer Assisted Triage
Computer-aided triage has been one of the fastest growing areas of AI, with more than 30 AI software applications granted authorization in the past two years, Samala said. There is, however, a critical need for estimating time savings in the clinical environment for these types of algorithms.
Samala highlighted the new research that was accepted for presentation at the next RSNA 2021 meeting. The study will report on a method for evaluating clinical efficacy based on query theory for rapid device analysis. computer aided triage and notification.
Another gap in regulatory science is quantitative imaging and radiomics. There is a need for well-characterized quantitative characteristics and a clear reference standard, Samala said.
The FDA is also working on a number of projects in this area, including an effort involving the use of radiomic features to assess bone fragility.
Another active research topic concerns adaptive algorithms, i.e. continuous learning algorithms that modify their behavior through a defined learning process. There is a need for a validation testing framework and a need to account for uncertainty in the referenced standard, according to Samala.
âDeveloping a clear scientific framework to develop this type of regulatory flexibility is a major challenge,â he said.
The FDA is working on a project that uses online benchmarking for continuous learning systems, for example
âWhat we’re trying to see is if there are any methods that can be used to calibrate these lifelong learning systems so that they canâ¦ maintain a good risk / benefit profile,â he said. .
Several emerging applications and new areas for AI continue to emerge, which has, in turn, led to new classes of AI software categories such as computer-aided triage and acquisition and computer-aided optimization, Samala said. The number of AI applications involving risk assessment and patient prognosis is also expected to increase.
“And these new classes of devices require new methods of evaluating performance,” he concluded. “This creates new challenges as well as opportunities in the field of research.”
The FDA has several initiatives underway to address these AI challenges, but also continues to rely heavily on research and contributions from communities such as SIIM to assist the agency with its regulatory activities, Samala said.
Copyright Â© 2021 AuntMinnie.com