Speakers: Christine Moundas, Dr. Bernardo Bizzo
On this episode of Ropes & Gray's podcast series, Decoding Digital Health, Christine Moundas, a health care partner and co-chair of the firm's digital health initiative, is joined by Dr. Bernardo Bizzo, a senior director at Mass General Brigham AI and assistant professor of radiology in the Department of Radiology at Mass General Hospital. Dr. Bizzo discusses his work in the field of digital health and AI, including the development and clearance of AI products by the FDA. They also explore the promising aspects of AI in health care and the challenges involved in developing and deploying AI tools.
Transcript:
Christine Moundas: Welcome to Decoding Digital Health, a Ropes & Gray podcast series focused on legal business and regulatory issues impacting the digital health space. My name is Christine Moundas—I'm a partner at Ropes & Gray. I sit in our health care group and participate in our data practice. I also co-chair our digital health initiative. On our episode today, we'd like to welcome Dr. Bernardo Bizzo—he's a senior director at Mass General Brigham AI. He's also assistant professor of radiology in the Department of Radiology at Mass General Hospital. Dr. Bizzo is focused on clinical AI activities and leads collaborations with industry to develop, validate, and implement AI and machine-learning-based software as a medical device and other clinical AI applications. Over the last decade, he's led the co-development and successful FDA clearance of multiple clinical AI products. Dr. Bizzo, thank you so much for joining us today.
Dr. Bernardo Bizzo: Thank you, Christine. It's a pleasure to be here today.
Christine Moundas: Can you tell our audience a bit about the work you've done in the field of digital health and AI?
Dr. Bernardo Bizzo: I am a trained diagnostic radiologist with a computer science background, and joined Mass General Brigham back in 2015, when the hype of AI and deep learning was just beginning. And that's when our Center for Clinical Data Science was created from the Mass General and the Brigham radiology departments, which evolved into the Mass General Brigham AI business, focused on bridging the gap between academic research and AI product development, validation, and clinical implementation. This means supporting end-to-end what we call the "AI product journey," so we can develop AI clinical applications, especially regulated medical devices, so they can be used clinically in our hospitals, and also commercialized throughout the country. That's not common for academic medical centers that are typically focused on research activities only. We made that possible for Mass General Brigham to act as a device manufacturer of record because we invested a lot of time and resources building a very robust quality management system that is ISO 9001 and ISO 13485 certified so we can follow the industry and the FDA requirements.
During this time, we co-developed with industry around 30 or so AI products, especially imaging-based applications, like stroke and fracture detectors, also AI tools for hospital operations, like missed care opportunities, cardiology with wave form data, women's health, and so on. As some of our first products were fully developed, the industry partners that we built them with came back and asked for our help with their next steps for these products. So, about four years ago, we created our digital clinical research organization, an AI CRO, to take fully developed AI base products that we built either at Mass General Brigham or from companies anywhere in the world who want to get in the U.S. market and assist with the go-to-market strategy, including regulatory clearance/clinical impact assessment. In the last years, we implemented over 30 AI solutions in different capacities across the MGB hospitals, and designed and conducted studies to support the FDA submission of over 50 AI findings, with almost 20 FDA clearances from over a dozen different companies from the United States, Europe, Asia, and Australia.
Christine Moundas: That's amazing—that's a lot of great work. You referenced throughout your work here to get devices through the FDA clearance process. Can you talk a bit about how the FDA regulates this space, so that our audience understands the FDA regulatory angle?
Dr. Bernardo Bizzo: Per the current FDA framework, AI products that are locked in substantial updates or continuous learning would require re-submission and re-approval, except for changes within the same intended use of the device through the predetermined change control plan (or "PCCP"). The FDA has defined several categories of AI software that analyzes medical images, such as computer-aided triage (or "CADt"), CADe and CADx. When the FDA evaluates a product for clearance, a benefit-risk analysis is performed to assess if probable health benefits outweighed any probable injury from the use of the device, and this analysis then determines the class of device, and the type of special controls required. Most clinical AI applications, as we know, are defined as class II devices that are typically cleared under the 510(k) pathway, meaning that a device that performs a similar task, even if on a different body part, or imaging modality that was approved before (what we call a "de novo," which means the "first of its kind").
There are two main types of special controls based on this benefit-risk analysis. First is a standalone performance assessment that is required for all imaging AI applications and measures the model accuracy comparing ground truth from experts with the AI results. The second type of special control is a clinical performance assessment, also known as rigorous study (or an "MRMC"), which is a requirement for CADe and CADx devices (computer-aided detection and computer-aided diagnosis devices). And this rigorous study is aimed at assessing the impact of the AI on physicians who would be the end users of the AI tool, to assess how much faster, more accurate, or more consistent those physicians are when they're using the AI tools. The FDA expects a superior performance of the physicians when using these tools, compared to when they are not using the tools.
Christine Moundas: That was a great overview. It's a very complex space, but obviously you've been very deep in the weeds for years now. A lot of people aren't as familiar with the FDA framework, so it's really important that when potential digital health and AI tools are being considered, that there is a legal and regulatory analysis to make sure there's an understanding of what the FDA treatment of the software as a medical device might be, and what the clearance expectations would be. Overall, I'd say this field is evolving very rapidly, and over the past eight to 10 years, we've really seen a proliferation of these types of technologies being developed. Can you talk about some of the most promising aspects of AI in health care?
Dr. Bernardo Bizzo: There's a lot of promise about AI enhancing the accuracy and efficiency of physicians in diagnosis, prognosis, and treatment decisions, and studies have shown that AI algorithms can analyze medical images with a level of precision that rivals or even surpasses expert human radiologists. This means that abnormalities such as suspicious lung nodules or brain bleeds can be detected earlier and with greater reliability. Early detection often leads to earlier intervention and treatment, ultimately improving the patient prognosis and survival rates. So, there's a lot of promise that this technology will really help improve patient care, especially with the increasing demands on our health care professionals.
Christine Moundas: Given what you've seen thus far, what would you say are the most challenging aspects of developing and deploying AI in health care?
Dr. Bernardo Bizzo: One of the biggest challenges we are seeing with AI across the board is the return on investment. It takes a lot of time and resources to develop these AI tools with a high performance, following the required standards. Then, similarly, a lot of effort is needed to get them approved by the FDA. So, it takes a lot of time and costs a lot of money to get these tools to actually be ready for clinical use. And then, AI developers, they need to recoup their investment with AI license fees, that are typically of high cost, so providers, hospitals, and health systems can use these AI tools. Implementing these tools in health care systems' complex clinical workflows is typically very challenging, and providers end up bearing a lot of the cost for deployment, including IT, monitoring, support, and the AI license fees. So, all that needs to happen to get these tools in the workflow and expect them to perform at a level that will increase physicians' efficiency, improve patient downstream impact, and other health economic levers that can justify all the investment in the technology. We see many AI tools just not performing so well with the real-world data. Another issue is that most AI tools do not have reimbursement codes associated—only a handful of them have those types of reimbursement codes. So, they really need to add a lot of value to justify the investment by the providers and health care systems.
Christine Moundas: Are there any particularly promising approaches that you're seeing emerge in this field?
Dr. Bernardo Bizzo: Yes, for sure. As of late, in the last five years, we have seen a lot of promise with the foundational models, or the large language models, that use the transformers architecture, including generative AI tools that had a big hype in the last couple of years with ChatGPT and other similar tools. These transformers, at a very high level, they can take any kind of signal, whether it is a diagnostic signal from radiology data, from pathology data, words, audio, and it can bring that through an encoder, and now you have features in a features space. For example, you take a sentence in English, and then you decode that same set of features through a different decoder, and you could get that same sentence out from this model, for example, in French. But you can encode anything, and this has been shown to work very efficiently. What's interesting is whether you bring genomics, medical images, digital pathology, text, like patient notes, you encode them into the exact same feature state, and you can output another modality if you want. You can input an image and ask a question, and get an answer. You can input a medical image with pathology and genetics, and get an answer. That's really groundbreaking to have this cross-feature space. These transformers, they have a feature called "self-attention," which means they don't need supervised learning, like the deep learning, neural networks that have been the status quo for imaging-based AI products on the market for the last decade. So, they can do unsupervised learning, and learn from anything that can be ever recorded digitally. We have these massive data sets, and because of that, the networks are massive as well, and that's how we get this emergence.
What we mostly have to date are just foundational models that were trained on all data that's available on the internet, which are not meant and approved for medical use, but we know they can do a reasonable job for some specific medical tasks. Now, what we're seeing is that foundational models are starting to be trained on health care data, and we at Mass General Brigham AI are doing that with a few strategic partners, so we can have data provenance. There's high-quality multimodal health care data used to train these foundational models, and then you can fine tune them for specific use cases, so they can perform tasks across modalities or body parts as comprehensive AI, instead of the very narrow AI applications we have seen to date. For example, just detecting lung nodules on chests or detecting those brain bleeds on CAT scans—instead, these large language models trained on health care data and fine-tuned for radiology can, for example, draft an entire radiology report. But the main challenge is that if these tools are impacting clinical care, for example, providing support for diagnosis, prognosis, or treatment decisions per the current regulatory framework, they are considered by the FDA as a medical device, and they need to be regulated, meaning tested against expert physicians performing the same task, to prove they perform at a level that is safe for people to use. And until now, there are no FDA-approved generative AI tools. It is my opinion that an expansion of the FDA's regulatory authority beyond traditional medical devices, to include AI applications outside of its current scope should be established, especially in this new era of foundational models and generative AI. That will hopefully bring more high-quality AI solutions to the clinical workflows, so we can better help our health care professionals and patients.
Christine Moundas: That all sounds amazing—this is really an incredible emerging field. The past few years have shown that there are really some applications that could potentially transform how patient care is delivered, so continuing to see how the technology develops and how the use cases develop is a very exciting aspect of this field.
Dr. Bernardo Bizzo: Yes, for sure—it's a very exciting field. It's a great field to be in and see the evolution of technology. We always think about how we can apply this technology to help our communities and help evolve how we improve quality care throughout the world.
Christine Moundas: Thank you so much, Dr. Bizzo, for this insightful discussion. And for our listeners, we appreciate you tuning in to our Decoding Digital Health podcast series. For more information about our digital health practice and other topics of interest in this space, please feel free to visit ropesgray.com/digitalhealth. You can also sign up for our mailing list as well as get invitations to our digital health-focused events. Finally, you can subscribe to this series wherever you listen to podcasts, including Apple and Spotify. Thanks again for listening.
The content of this article is intended to provide a general guide to the subject matter. Specialist advice should be sought about your specific circumstances.