Leveraging AI for Safer and More Efficient Healthcare Delivery

Ashley Beecy MD, FACC at UXDX USA. Video: https://youtu.be/1XJo1oKp0NE

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Building products in healthcare

[00:00:10] It's such a pleasure to be here. What an exciting conference. Love the energy. It just shows what a great time it is to be building products across industries. What I want to talk about is how we build products in the healthcare space.

[00:00:24] In healthcare, we use cutting-edge technology: the way that we communicate to other providers, the way that we communicate to each other, the way that we communicate and give data back to our patients, and even the way our patients communicate with us as physicians. I'm going to go through some of the work that we're doing at New York Presbyterian to develop and innovate in the cardiovascular space. I gave some examples of legacy technology, but we really do cutting-edge work in healthcare.

[00:01:01] You can see from this graph the trajectory of the publications that have been submitted to journals in the last few years related to machine learning. It's exponential, and this only goes to 2021. If you were to take this to 2024, you'd see there's over 40,000 publications. Now when we look at how many AI-based models have been submitted to and approved by the FDA, that's maybe just about a thousand. What's the difference between what's being generated in ideas and research projects in healthcare, and what's being productized and delivered into healthcare systems? That's going to be the focus of what I'm going to talk about today.

[00:01:46] Some of these challenges, you'll notice, aren't necessarily specific to healthcare, but I'm going to try to focus on what we face on a regular basis. First, identifying the problem. How many people here build healthcare-related products? It's hard with the lights here. A handful. How many people building healthcare-related products have somebody on a clinical team, or with a clinical role, developing the product alongside you? Okay, it's actually pretty one to one. You're thinking about it in the right direction. I think that's an important point to note: having those clinical team members as part of your development team can really help you identify what the needs are.

[00:02:30] AI deployment: we're going to go over some of the challenges in bringing the results and the output of this technology into a clinical workflow. Solution performance: I'm going to talk a little bit about the difference between model performance, how we assess that accuracy, and how we really add value when we're integrating these products into clinical care. And lastly, something specific to healthcare: regulation and trust. It's great to build a good product, but if people don't adopt it and use it, then we're not going to achieve what we're looking for.

Asking the right question

[00:02:58] Asking the right question: what's relevant? Some of the things that you want to address in the healthcare system are the organizational priorities and the needs. Right now, we know that healthcare providers have an incredible administrative burden. They spend over two hours in the evening doing what we call pajama time, and that's working in the electronic health record to do charting. That's a big focus of some of the products in the space, so that's one of the priorities. Reducing the burden of chronic disease is another big priority for healthcare systems. We spend a lot of money on caring for these patients, and we really want to prevent the advancement of disease and take the best care we can of them.

[00:03:41] Understanding the current workflow. If we're building an AI-based technology, are we actually improving what we have to date? What is the standard of care? Really understand what the standard of care is and what we're improving on. Make sure the solutions we're focusing on are practical: the more practical, the faster we can evaluate it, and the faster we can understand whether it's going to work or not.

[00:04:02] Understanding the model integration and the affected roles. I say affected roles because there are often two sets of stakeholders that you're balancing here. There are the stakeholders that see the need for the product; maybe it's a quality solution or a care solution. And then there are the stakeholders that are actually going to use it and adopt it. Make sure you understand who those two stakeholder groups are and that they're incorporated in your design process.

[00:04:26] Understanding if there's a cheaper alternative to AI. We use a lot of rule-based criteria, weighted risk scores and informatics solutions that are quite effective in the healthcare space, and it's important to keep these in mind. Then making sure you have the data and labels. We'll talk a little bit about how we are overcoming the challenges of healthcare-related data in the cardiovascular space at NYP. And lastly, understanding the key components of success. When you're building the product, understand again who's going to use it, and make sure that the effector arm is going to be part of the way that you're designing the product to achieve the intended outcomes.

The enterprise heart failure program

[00:05:03] I'm going to focus on the cardiovascular space. This is a big priority for our healthcare system. Over 8 million patients will have heart failure by 2030. This is a big problem. One in five patients will be at risk for developing heart failure after they're over the age of 40. And we spend a lot of money as a healthcare system taking care of these patients. We developed an enterprise heart failure program. It spans across our ten-hospital system and it's run by Dr. Nir Uriel. What we're trying to do is really improve the care for this patient population.

[00:05:39] What we said was: we have affiliates at Cornell Tech. They are doing cutting-edge work. Let's make an investment, work with them, and build development teams and products that will improve the care for these patients. We invested about $10 million to build a collaborative engagement with them, and I'm going to tell you some of the work that we're doing in this space.

[00:05:59] It's a big team. We came together as a group at a virtual meeting, and the first meeting, I'm not lying, was a little crickets. We had our clinical team, who really didn't understand the concept of machine learning, and we had our technical team, a lot of whom had not worked in the electronic health records, so they weren't even familiar with some of the nuances that you have to consider when working with medical data. We spent about three months, which was a long time, iterating on how we come up with the right questions and what the problems are that we need to solve where this technology can really make an impact.

[00:06:36] We started here. We looked at the left and said: in healthcare we can prevent, we can diagnose, we can do prognostication or risk stratification, we can treat and monitor. Where the technology is, I think we're really in the stages of diagnosis, prognosis and risk stratification. When we think about problems, I often think about what's feasible and what's lower risk, and these areas are the more feasible and lower-risk categories at this point. We said these are the areas where we think we can answer questions that will make a difference in patient care: heart failure, some infiltrative cardiomyopathies, and valvular disease.

[00:07:14] One of the things that we wanted to focus on was how we leverage the data that's in our healthcare system to do what we call opportunistic screening. What's opportunistic screening? That's looking at all of that data and identifying disease that may not be recognized right now. That's a little bit different from traditional screening, where you may be over a certain age and you get a colonoscopy or a mammogram. This is saying: you come in with a cut on your knee and you happen to get an electrocardiogram, but what data can I get from that electrocardiogram to maybe tell you that you may be at risk for cardiac amyloidosis, or at risk for structural heart disease? That's an area where we've said there's a lot of value to impact one of our strategic priorities, which is to reduce the burden of chronic disease.

Where the work really is: data curation and deployment

[00:08:01] I want to take another poll here, because I'm curious to see this crowd's opinion. When we moved on from coming up with our ideas to model development, we looked at the development life cycle. As you know, we have data curation, model training and model deployment. How many people think data curation is the most resource intensive? Very few people. What about model training? Okay, no. What about model deployment? Highest hands for model deployment. I think I'm getting you to that direction: this is quite challenging in the healthcare space.

[00:08:38] This is how I would lay it out. This is not to say that model training isn't resource intensive; it requires a certain level of subject matter expertise. It's just that I think we often underestimate the work to get the data AI-ready and to find the actual application in the workflow.

[00:08:55] This is a picture of Legos. This is what our data often looks like when it comes out from our electronic health record and hits our data lake. We've done a lot of work across the organization on data strategy and on leveraging a data platform to improve our data readiness and maturity. In the cardiovascular space in particular, we have done hardware upgrades so that we can take data from electrocardiograms, for example, and bring it right into our data platform so that we can use it for AI model building. We've enhanced the way we archive data, and we leverage our national registry data and even our research repositories.

[00:09:35] This is an example with electrocardiograms. We have over 10 million ECGs, but we had to upgrade the hardware to be able to get the electrocardiogram data into our environment in the cloud so that we could build models from it. That's a lot of work that has gone on for the last couple of years just to build the foundation for us to develop models. And this is an example of EHR-based data. When we're developing models from EHR data, there are a lot of considerations. I can't even just take a medication name. There are different formulations: it could be oral, it could be IV. There are different brand names and generic names. There are different doses. There's a lot to consider with EHR-based data.

The models: opportunistic screening and remote management

[00:10:19] These are the types of models that we're building. I mentioned opportunistic screening, and I'm going to give a couple of examples, but it follows a theme: how do we take data that's available in our healthcare system, from patients who come in through routine care and interaction, and leverage it to build models that identify disease earlier so that we can make an impact? We are looking at echocardiograms to detect peak VO2, so we can understand whether a patient has advanced severe heart failure. We're looking at chest CTs to see if we can identify patients that have a reduced ejection fraction. Then we have remote management, which I'm going to go into a little bit more.

[00:10:58] And even remission HF. What that is: when we take echocardiograms and we find that a patient has heart failure, can we take echocardiogram pairs and identify who's going to have recovery of that ejection fraction? Because we can't do that now. If you come to see me as a cardiologist, I want to be able to tell you your chances of recovery, because that's one of the first questions you're going to ask me. And then ECG dynamics: how can we take an electrocardiogram and predict things like the data that we would get from an invasive right heart cath? This is cutting edge. It could save patients procedures.

[00:11:33] We talk about equity and fairness with AI models, but think about this. If I can use an echocardiogram, which is a lower-cost, more readily available study, and I can predict what the outcome is for a cardiopulmonary exercise test, now I can make that available in places where cardiopulmonary exercise tests don't exist. Now I'm using technology to actually improve equity.

[00:12:00] I want to go into the first two examples in a little bit more depth. An echocardiogram is an ultrasound of your heart, and we're looking for structural heart disease. Do you have problems with your valve? Do you have problems with your heart function? A cardiopulmonary exercise test is a specialized test, which we have over there on the left. We understand what your oxygen consumption is to tell if you have advanced heart failure. Do you need a heart transplant? Do you need a mechanical support device? Oftentimes we catch these patients too late, so they're not a candidate for certain procedures that could be life-altering.

[00:12:33] So we're going to build a model to detect whether a patient has abnormal peak VO2 based on a readily available test. Here's what we did. We took our structured data features plus our DICOM files, which are the echo image files. We built a model to detect whether a patient has abnormal peak VO2 or not, and we found that we can detect with an AUC of about 0.8 whether patients have an abnormal or normal peak VO2. It's been very successful. This paper was submitted to a journal just this past week, so we're excited about that project.

[00:13:08] The other one I'm going to tell you about really briefly, so that I can put this in context when I talk about the rest of the life cycle, is our CT-EF. You come in, you get a chest CT, maybe you have pneumonia or another problem, and we detect whether you have a reduced ejection fraction so that we can get you to see advanced heart failure early. There are over 40,000 CTs done yearly, so this is a lot of data and a lot of opportunity to identify patients. We built a model using data from one of our medical schools and medical centers at Columbia, tested the model at Cornell, and again found that the performance is around 0.8. These are examples of what we're building that we think can have impact.

[00:13:51] The last example I'm going to give is the remote management program. We've been running it for about five years. We have over 1,500 patients who have worked with our physician assistants on a regular basis. They've had calls, they've had medications adjusted, they have wearable devices. There's a lot of in-depth data, so we can build a reinforcement learning model using this data to determine what the next steps in that patient's care will be. With our limited resources and the physician assistants we have now, we can scale the program and offer it to more people. That is our goal in building a model in this space.

Integrating model output into the clinical workflow

[00:14:21] I think that we can use technology to empower our providers, and it can be effective, but in order to make an impact we need to deliver it effectively into the healthcare system. This is outlining the MLOps life cycle. We're evaluating model performance, which I talked about. I talked about the data infrastructure, and I want to mention model integration.

[00:14:53] This is a UX conference, so I'm going to tell you: when we build these models, I have a data output. I have a prediction of whether the patient has an abnormal or normal result. When we think about integrating this type of model output into the healthcare system, there are a couple of options. I can put it in the electronic health record, I can keep it as a standalone system, or I can integrate it into some other information system, maybe my echocardiogram system or another system providers use. We tend to want to integrate it into the electronic health record. Why? Because every click that I have to do as a doctor outside the electronic health record makes me want to pull my hair out. This is not unique to healthcare. I'm sure in every industry, if you're requiring your user to leave the system to go to another system, there's a higher likelihood of failure and less adoption.

[00:15:47] When we think about integrating these data results into the electronic health record, we have a few options. We use Epic, which is one of the largest electronic health records. We can notify the provider with a pop-up that says, "Hey, we notice that the patient you're seeing right now has an abnormal risk score for heart failure based on their chest CT. We think you should do something about it." That has its pros and cons. A lot of people click dismiss, you'll see that. But it is one option that exists.

[00:16:15] I will say something about BPAs before I go on. We actually had so many of these that we had to go back and remove them. We had to do an analysis of how many BPAs there were, what they were and what they referenced, to cut them out of the system, because they were becoming ineffective. I just want to point that out when we're thinking about the ways to integrate this information that are going to be effective. You can put it at the top of the electronic health record. There's a monitor watch [?] where you can show graphs and display the risk score with other related information. You can have lists, you can have reports, you can have dashboards. And with each of these you can have varying levels of transparency on which features are weighted higher in the risk score, so that the providers know.

[00:17:03] I will say a lot of people emphasize explainability in healthcare, but as a doctor, I don't know how all the medications I use work. So I'm going to give a little pushback on that. You'll get different opinions on how valuable it is. I think what's most valuable is that when you're building products, you have the evidence generated to support that they're effective. Because what do I do with medications? Even if I don't know the mechanism of action, I want to know what clinical trial there was, which patients were included in the clinical trial, whether it was effective or not, and what the limitations are and the scenarios in which I should not use this medication. I think the same concept should apply to AI in healthcare.

[00:17:48] We talked about the options for integrating it into the electronic health record, but we have to get it there. How are we getting it there? We're building out the architecture within our cloud, and we're building out APIs to integrate into the electronic health record. This is what we've done so far for the models that we have deployed, but we are working with a third-party vendor to be able to do this at scale. I didn't include the vendor name here, but if you have questions about it, I'm happy to answer them offline.

Monitoring and solution performance

[00:18:15] What are we monitoring? Some of the same things we monitor for AI-based products in other industries. We look at the inputs, the outputs, the data drift. We look at regular application metrics like runtimes, security and version control. And then we look at solution performance. What's really unique to healthcare is some of that solution performance, the feedback mechanisms, and whether we're adhering to regulation and compliance.

[00:18:42] I mention solution performance again because when we look at our model and show that it's effective, that's very different from when it's actually running in the healthcare system. We need to know from our providers whether this is working or not. We need feedback mechanisms. Right now, the way we deploy in our healthcare system is very linear. It's like the software development life cycle, and we do periodic updates after we understand, over a certain cadence, what the performance is and whether we're achieving it. But we need to move towards more of an adaptive system. We need to be collecting these metrics on a regular basis. We need to be doing regular audits and fine-tuning in real time on continuous data. We're not there yet. This is maturing. I think we'll get there, and I think by building the MLOps systems to support this type of work we will, but this is certainly in progress.

[00:19:32] I mentioned feedback mechanisms. This is the same as adverse event reporting. I think for each product, if we're asking doctors to use this in their decision making and they find that it's wrong, we should have a way to understand whether it's wrong, so that we can look into what the risk is in patient care, listen to them, and enhance our products over time.

Governance and trust

[00:19:58] A while ago I asked DALL-E to create a picture of the medical director of AI operations giving a presentation, and I did it again the other day on GPT-4o. Clearly there's a change in the pictures over time; it's gotten much better. But I've asked this question no less than 40 times, and there has still never been a woman in the picture. Maybe the first time I didn't recognize it, maybe the second time, but at 40 I was like, "Wow, this is really against probability." I think that highlights the fact that we need to do this carefully if we're going to introduce products into the healthcare system.

[00:20:35] How are we making sure that we're meeting quality standards? By creating an AI governance that's focused on three main areas: risk to patient privacy, risk to quality and patient care, and regulatory risk. We have a multidisciplinary group that's come together, made up of doctors, informaticians, data scientists, legal, regulatory. I'm probably forgetting somebody, but you understand the concept: it's a lot of people who are really thinking about this and building up the architecture, the processes and the resources to support doing this over the long term. Because there are certain things that we haven't even addressed, like when we have models running at the same time, how will one model impact another model? These are really novel concepts that I think will continue to evolve.

[00:21:24] We don't do this alone. There are a lot of national organizations thinking about best practices in developing solutions in the healthcare space. I recommend you check them out, because there's a lot of good material there that can help you as you're thinking about this.

Summary

[00:21:42] In summary: spend a lot of time on the idea, and include clinical partners. That's really important with what we're doing. Make sure that the value proposition is there. The concept of fail fast is not unique to healthcare, but it certainly applies. I've heard people use the term pilotitis, that it exists, so try to move from pilot to scaling in a somewhat rapid fashion. Understand that performance is not just the performance on retrospective data; there are a lot of considerations when you're integrating it into care.

[00:22:19] We talked about high-feasibility, low-risk products. I gave some more cutting-edge examples related to opportunistic screening, but a lot of the products that we're focusing on now are reducing the administrative burden on providers: taking away that workload, coming up with ambient scribes. When we think about opportunistic screening, I'm taking data and introducing it back into the healthcare system, so I'm actually increasing the burden on the provider. There are different models to reduce that burden, such as having centralized or decentralized care teams focus on acting on the data. But really think about that whole life cycle and what the integration will look like. I think you'll be more effective when building products in the space.

[00:23:00] And then building consensus among the stakeholders. People ask me a lot whether doctors will want to adopt the technology, whether they're concerned, what their concerns are. People are excited. They just want to know that it's safe. They want to know that it's validated, that it's tested, that it's going to be effective, and they want to be included in the process. I think the takeaway here is that building that trust through evidence generation, getting these products into the hands of the doctors, and working with them to build them will certainly get us there. That's all I have, so I would love to open it up for questions.

Q&A

[00:23:49] Host: Thank you so much, Ashley. That was wonderful. We've got some great questions here. Phil wants to know if overdiagnosis is a potential concern when applying AI in healthcare.

[00:24:00] Ashley: That's a really good question. When we think about screening, there's been a lot of work by specialty societies to figure out at what age you should have a mammogram and at what age you should have a colonoscopy, so that we make sure the benefits at a population health level are greater than the risks, and patients aren't adversely affected by false positives and increased downstream testing. I think this comes back to evidence generation in clinical trials.

[00:24:32] I'll give an example. The Society of Gastroenterology, I think it was just recently: there's been a lot of data to support AI-enhanced polyp reviews for colonoscopy. A colonoscopy is where you look for colon cancer; it's the endoscopy that goes from below, not above, and one of the things they detect is polyps. You can enhance your detection of polyps with AI-based solutions evaluating that video. But they found that the number needed to treat was too high, and that the false positive rate really didn't warrant that additional screening, so they didn't include it in the recent guidelines.

[00:25:14] Another thing I'll mention that's related: we look at our model performance retrospectively, and in that we have to understand what the test characteristics are. What is the prevalence of disease? What's the positive predictive value? How many false positives will we have? When we build opportunistic screening models, we really try to optimize for specificity to reduce that false positive rate, because we want to add value. We want patients to get confirmatory downstream testing that they otherwise wouldn't, while trying to limit over-testing. But it's a great question, and something we need more data on.

[00:25:58] Host: We've got a question here. I don't know if you've read Women in Data [?], which is a very depressing book, but I know they only started testing medication on women in the 90s or something very late like that. The question is about the history of healthcare research being focused predominantly on men, and how, if at all, that is impacting AI advice, with leaders anchoring towards that history of healthcare.

[00:26:24] Ashley: I think one of the points this brings up is the data we use to train our models. If we're training on data or doing our research predominantly on men, then it's not going to be generalizable to our female population. But the same holds true for race and ethnicity, for age, for patients with disability. What we do, as part of both our AI governance and our model development process, is make sure that the data we're building the model on is going to accurately reflect the population to which the model is going to be applied. It's actually a question we ask, as part of AI governance, for products that are developed commercially. If you developed the product commercially in the middle of the country in a small rural town, I want to know that, because my population in a major metropolitan area may not be reflective, and your model may not work as effectively on our patient population here. It's something we have to be very cognizant of.

[00:27:24] The other thing I'll mention related to that question is that AI has a lot of utility not just in diagnosis. There are a lot of products being built for better recruitment for clinical trials, to engage patients and find patients that would benefit from being participants in clinical trials, so that we can improve the equity and representation of patients in these trials. I think it's making sure that our products apply to all of our patients, and then using the technology to include more patients in the research itself.

[00:27:57] Host: Yeah, very important. What about personally identifying information? Any challenges you come up against, legally or ethically, when utilizing that?

[00:28:09] Ashley: We take data privacy incredibly seriously, but that's not specific to AI. That's through anything we're building, any product, any way we're using patient data. When we do research, for example, we go through our IRB review board, and we get consent where it's appropriate through their governance. There's a whole slew of governance processes to make sure that we're using the data appropriately and that we have consent. The other thing is transparency. We prioritize transparency so that when we're building models and introducing them into patient care, you know when we're using AI. We have deployed, through our electronic health record, an automated in-basket response system, and at the bottom it tells you that the draft was partially created by an AI solution, so that you know.

[00:29:04] Host: It's very important, for sure, making that part transparent.

[00:29:07] Ashley: Exactly. Absolutely.

[00:29:09] Host: I have a question for you. You mentioned the importance of partnering with clinical folks, and you showed that image with all of the circles of the various teams. How do you help them understand, and vice versa, the technology piece, the product piece, the design and development piece that's going on behind the scenes of the tools that you want them to be using and acting on?

[00:29:34] Ashley: That's a great question. I want to say it was easy, but it wasn't. Even with the data science 101, it was hard, so it was an evolving process. What we did was take those 30 or 40 people that you saw on the screen and build pods, and it was a very effective team structure. We had a technology faculty member represented. We had a postdoc or master's student on the technology side. We had a clinical lead, and usually someone a little bit more junior on the clinical team too, and then an informaticist, who helped do the translation, and a project manager. By breaking it down into those smaller groups and pods for each of the projects, we were able to dissect the question, understand what data was available, and really iterate. I'll emphasize the iterative process that it took, and it was quite successful. Now what's even more is bringing the work to the next level, which is clinical trials. What we're building now is going into production, and we continue those conversations in that journey of product development.

[00:30:41] Host: Can you give us a peek into that world? I think so many of us are curious about healthcare but don't necessarily see what happens behind the curtain.

[00:30:49] Ashley: Clinical trial design is something we have obviously done for a long time in the healthcare system. What we're trying to do in AI is make sure that we understand the test characteristics, understand what that workflow and that effector arm is, and make sure that the evidence we are generating will help us make a determination, as a healthcare system, of which products we want to prioritize and which products we want to take down the FDA pathway and get regulatory approval for. And making sure that we do it right, we do it effectively, and we do it with our partners, who have that long-term history of research experience.

[00:31:32] Host: Absolutely. There's a question here at the bottom I'd love to see more of; there's an ellipsis. We already asked the questions above, and I just want to make sure I'm asking it fully on behalf of someone. Here we go. We've got some more that have been uploaded as well. As hospital systems are building these models, are they made available for other providers? Is this proprietary? Are you distributing across your network?

[00:31:55] Ashley: I love that question. That comes up a lot. One of the things that we do when we publish: the two examples that I went into in a little more depth, we submitted to journals as open source. The code will be open source for people to look at and understand how it works on their data. As far as disseminating this work, there are a lot of challenges. We usually do external validations to understand that generalizability I was talking about: if the model performs well in our health system, does it perform well in another? Not to get into too much technical detail, but with Docker containers we share models and not data, and then another healthcare institution can apply it and understand on retrospective data whether it was generalizable.

[00:32:48] Then to disseminate it more on a productized level, I think we're going to partner with industry. That's one of the avenues, or creating companies within healthcare systems themselves is another opportunity. What I think we'll see, and this is my own personal opinion, is that this type of work becomes almost like a marketplace approach. There'll be companies that create platforms, and models that we're building as a healthcare system will be able to be deployed and disseminated through them. It would be hard for me to go into another healthcare system and build out this whole architecture that I was talking about for successful deployment, and the surveillance and monitoring tools to understand it's effective, and it wouldn't make sense to do it for one model that I'm building. If I have a lot of research teams developing these models at scale, making those models available on a platform through some kind of licensing agreement may be the direction that we see things go.

[00:33:50] Host: Absolutely. That's our time for questions. Thank you so much, Ashley. This was phenomenal.

[00:33:55] Ashley: Thank you all.