AI in Healthcare UX: From Burden to Breakthrough

Anya Gerasimchuk at UXDX USA. Video: https://youtu.be/JRXxyW-YSac

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Ontada and the oncology ecosystem

[00:00:08] I'm very happy to be here and to speak to you about AI in healthcare UX. This is not the hype AI conversation; it's about the real people: real clinicians, nurses, patients, doctors. In this particular space, I represent oncology and multispecialty. How do I... oh, there we go, wonderful. In particular, I represent Ontada. These are our interconnected businesses within O&M, or oncology and multispecialty. What they do is provide an interconnected portfolio of solutions in order to serve the patient. Of course, the patient in oncology is our main persona. All these sub-businesses do various things to serve the patient, whether it is finding clinical trials, or developing drugs, as we recently heard from Paul on the life sciences side, or finding practices close to home.

[00:01:13] Ontada in particular focuses on technology. We are powering this whole ecosystem with our technology. We have one of the largest EHRs specializing in oncology, and we also have interconnected solutions which serve the EHR and work together with it. We're also collecting data from the patient journeys, which is of course de-personalized [?], and then fed to real-world research so that we develop better drugs, better treatments, better clinical trials and so forth. As shown on the slide, we have about 2,800 providers throughout the United States using our solutions, and they in turn serve about 1.6 million patients. So it's a pretty big reach within the oncology space. But probably our biggest benefit is the data that we collect from the patient journeys.

[00:02:14] Representing design within the space, I need to say that the space is extremely emotional. Unfortunately, many of us have in one way or another experienced cancer. The space can be very emotional, but it is also very complex from an operational perspective. There are a lot of handshakes, a lot of conversations, a lot of decision points happening behind the scenes. In design, we're not designing just for one person; we're designing for an ecosystem of workflows. And failure for us is not just a usability failure; it's a failure on the human level. There are a lot of risk factors involved in our design principles.

A complex patient journey

[00:03:00] Let's look at the patient journey, since this is our main focus: the patient and the patient's outcomes. On the surface, the patient journey seems relatively straightforward. You have initial symptoms. You have the referral to the oncology practice. You have an initial diagnosis. You are sent to more advanced testing labs, maybe precision medicine tests. You come back, and maybe you're matched against clinical trials. You go through treatment. Hopefully the patient is in remission. And then after all these steps, the data is collected and fed to life sciences companies in order to make better drugs and better decisions for patient outcomes.

[00:03:53] So again, it's a very emotional journey and very operationally complex, with a lot of decisions and a lot of handshakes. We know that one patient is served by roughly 40 people, and amongst those 40 people we have about 20 different specializations. Specializations can range from the administration side, like scheduling appointments and finding practices, to more clinical roles: doctors, nurses, infusion nurses, lab technicians and so forth. You can imagine how complex the workflows can be. There are about 30-plus workflows, which can be very complicated. To give you an example: insurance authorizations for drugs and for regimens, scheduling multiple series of appointments and so forth.

[00:04:51] From a UX standpoint, this creates big friction, because you have to design for this ecosystem, but you know that there is a huge pain point: you have to administer this ecosystem somehow. To administer this number of people, and this is just an example of one patient, you can imagine how many intricacies you need to introduce into your system.

EHRs and the administrative burden

[00:05:17] From that perspective, EHRs. Who knows what an EHR is? Lift up your hand. All right, quite a good number of people, because we all interface with some sort of EHR on a regular basis when we go to physicians. EHR stands for electronic health records. They are extremely useful. They've been around for many, many years. They help digitize workflows. They are extremely crucial for patient safety programs. They hold documentation for the patient. They help practices make better decisions on how to serve their patients. They also provide insights to physicians on what treatment to choose. So there is a lot which goes into the EHR.

[00:06:08] But EHRs, on the other hand, introduce a new problem. I stated the issue of administrative burden, and that's what EHRs and digital applications in healthcare actually create. They create administrative burden by putting a lot of manual tasks on the physicians and the people who are using these applications. For approximately one hour of face-to-face time with a patient, the physician goes back and spends about two hours documenting the findings in the EHR. Sometimes they document these findings on the patient in their pajamas, hence the term pajama time. What that means is that physicians don't have time to do it in the office, so they go back and, instead of spending time with their families, they document the findings on the patient after hours at home.

[00:07:11] Physicians, or practices, using EHRs have about 50% burnout rates compared to those who are not using digital tools, which is kind of strange nowadays, right? But those exist, and they're probably not within the practice space. The cost of that is on the patient, because the patient now has less eye-to-eye contact with the physician. They have less human connection with the physician, and the patient sees less presence from the clinician, from the doctor. And this is a must-have in oncology care, because the care is so complicated, and a lot of it is based on observations.

The opportunity: ambient AI

[00:07:58] From the product perspective, this is the opportunity that we identified within our space. We need to remove the burden on the physician of being a data clerk. We need to put the physician back in the office, back face-to-face with the patient. We still need to preserve documentation. Documentation in the oncology space is a must-have and extremely important, because the nature of the disease is very long. You can imagine that a doctor needs to know what happened to you in year one and have that context in the subsequent years. It spans the entire life cycle of the disease, in order to make better decisions. So all of that we need to improve. This is our opportunity.

[00:08:52] And guess what? This is the hype talk about AI, after all. We understood that with documentation, AI can actually help us. But there is a specific type of AI. Very often we hear that AI needs a certain interaction with a computer from the user. But our problem is unique, because we need to remove that. We don't want the physician to be a data clerk, typing something on the computer and giving commands. So we realized that the way to start our journey to improving the administrative burden, to removing burnout from physicians, is actually an ambient AI solution. Ambient AI is the kind which works in the background, in the real moment. Whenever an interaction is happening, the AI listens, it transcribes and then it summarizes. But more than that, it can then lead to certain other actions, and this also becomes very important to us, which I will describe in a couple of seconds.

[00:10:02] Just to mention, in a couple of slides, the different types of ambient AI. One of them is screen-based. This is where the intelligence tracks user actions within the application, and then, based on those actions, it can pre-fill various text fields or suggest the next steps. Another type is multimodal. Oops, I went too fast. How do I go back? The front button? Never mind, it's okay. Oh, there we go, thank you. Multimodal is when some sort of sensor is connected to the intelligence, and after that it goes back to the UI, and you can fulfill the next steps with it. Sensor-based, or environmental-based, works the same way as the previous one, but it's connected to IoT devices or environmental sensors. It is used in healthcare to a lesser extent because of regulatory issues, of course.

[00:11:04] The final one is workflow-aware ambient AI. This is more future-looking AI, meaning that it's still evolving; not by us, but by those who are developing the intelligence. It is future-looking because it can preemptively prompt the next steps or give you insights on what to do next. We at Ontada and McKesson are also looking to explore that solution, and we are very close to getting there. But of course, combining the AI types is what gives you a very powerful solution, and I'll show a couple of examples there.

Clinical noting and the AI scribe

[00:11:46] Let's now go to the concrete examples and use cases of what we started to explore. In Sam's talk and in yesterday's workshops, we spoke about needing to concentrate on small chunks to deliver. This is the so-called MVP, or scope. Our scope became a special type of documentation which is the backbone of oncology care, and that is called clinical noting. Is anyone familiar with the term clinical noting? Okay, it sounds like I can go a little bit into describing what that is. It is the backbone of the documentation. Imagine that you are in the exam room, the doctor comes in, you start the conversation, and you can see the doctor typing something into the computer. Or you can have another person enter along with the doctor, not a nurse, who is called a scribe, and the scribe is actually typing what he or she hears in the room. So they transcribe. Currently we have a lot of tools that do this. The scribe or the doctor is actually taking notes on your conversation.

[00:13:09] Clinical notes are extremely complex, because they combine not only the narration of the doctor or the nurse or any other type of clinician; they also need to combine it with medical jargon and various terminology, because all of that information needs to go to the other teams so they know what to do with the patient as next steps. The typical workflow of taking clinical notes is described here, in an extremely summarized way. As I've described in the first steps, the patient and doctor interact, the visit finishes, and then at some point in time the physician finally goes to the EHR system, to the application, and opens up the correct template for the note. There might be different types of clinicians adding the note: it can be a surgeon, it can be a nurse, it can be someone from social work, for example. Based on the particular usage, the templates might be slightly different. We also have to structure the note in such a way that various types of information go into the correct sections of the note, so that the system then knows how to work the note further and how to trigger the next workflows from the note.

[00:14:37] Then, of course, come other complicated steps. Physicians need to be paid, so they have to find billing codes for everything which they do. For that, they might remember what they need to add, or they go to other parts of the system to find it. Finally they find it, and they input it. The same happens with other clinical information about the patient. They need to find lab results, various medical history, perhaps allergies the patient had, find them in the system and input them into the correct macro, because we use macros in order to separate that type of information into chunks within the note. The other teams also need to know about that, so they look at the note and take notes on their part in order to understand what to do next, whether to schedule appointments or order new medications for the patient.

[00:15:38] Now let's go back and think about how ambient AI can help here. Imagine that you don't have this person called a scribe, or you don't have the physician sitting here doing this: this is you, this is the EHR. Imagine instead that it is an AI scribe. The AI scribe listens in real time, transcribes, summarizes and, based on how it was programmed, inputs the information in the correct order into the correct template. It finds the billing codes and inputs those. It talks to other teams. And then finally a human in the loop reviews everything which was generated, and the provider signs the note, which becomes a living organism that informs the next steps for the patient. So with the AI scribe we are now simplifying this workflow. We are removing some steps, and we are automating them.

[00:16:50] An interesting point is that when the patient and the physician start talking to each other, of course the patient has to give consent so that the physician can use the ambient AI solution. Ambient AI currently usually sits on some sort of mobile device and listens to the conversation. The difference between this and any other ambient model that we all currently use is that the LLM is different as well. By LLM I mean large language model. We are using ones which are trained in healthcare. But even that is complicated, because healthcare-based LLMs are very generic, and we have to work with oncology terminology, which is extremely specialized. So we are definitely facing some difficulties.

[00:17:44] However, it is already working for us. This solution is in practices, and we already have some wonderful sentiment from many doctors and many providers. It is starting to save them time: roughly two hours from their workday. It saves them two hours, in other words, roughly. Pajama time is partially, almost, eliminated, which is exactly what we tried to achieve. And I need to note that installing the solution at the practices is also very, very simple, which is a beauty, an extra add-on for us.

Outcomes first, trust and data

[00:18:32] Let's now step back and look at this whole solution, not just the ambient AI. It is extremely important to always focus on the outcomes. What outcome do you want to achieve? For us, it is bringing back the personal time in the exam room between the patient and the provider by removing the screen barrier. We realized that we can enable this by providing a particular experience to the provider, and the experience is no clicks, no finding information within the system. This is possible via ambient AI; this is our solution. So we concentrated on the outcome first, and then we found the solution that might work for it.

[00:19:27] There are two aspects: trust and data. Healthcare is extremely risky. We even have a patient safety committee, and we have to work with them to analyze our designs and various other decision points. It's a new thing for us, but it does exist. So how do you trust whatever the AI introduced? We have several checkpoints, even from the design perspective. Our designers came up with a solution where, when the physician reviews the output, they can control what they see, and they can even edit it and monitor it, remove or add, which does add clicks, but it's still a preventative hook in the workflow. The other one is data, which is extremely difficult for us, but it's one of the huge things that McKesson and Ontada in general [?] have, because we have a rich amount of data. But data can come in different shapes and forms; it can be structured and unstructured. Some of the difficulties are that data comes from different endpoints, and interoperability between applications in healthcare in general is not very good. Differences in workflows is another one. Everyone works their scribe [?] differently.

[00:20:50] To wrap up, additional use cases we can probably cover during the Q&A session, or reach out to me and we can discuss those as well. But I do want to talk about McKesson's Aim 28 [?] and mention that the organization in general is trying to achieve, within three years, a certain level of maturity with AI. So everyone at McKesson, one way or another, participates in the maturity with AI. I do want to point to this last bucket here, which is AI-first SDLC. We heard a lot about how we can improve the workflows between design and engineering and product, so we can streamline, develop better-quality outcomes with rapid prototyping, validate them faster and deliver quality production code. This is the engineer of the future, or it can be a design engineer professional as well. But to wrap this up, I do want to mention that we, as professionals in this industry, need to understand why we're doing something. We have shiny tools, and very interesting outputs that we can achieve with these tools, but we need to focus on what exactly we're trying to achieve and how to use them to benefit the people for whom we're building them. So, thank you very much.

Q&A

[00:22:26] Host: Good stuff. Thank you. We sure have a lot of questions rolling in here; let's just give it a moment. Okay. How do you mitigate the inherent bias involved in LLMs in healthcare that may negatively affect communities of color, the LGBTQ community, etc.?

[00:22:46] Anya: Honestly, we have not encountered that. I don't think there is bias in the LLMs, because they are also developed by different people from different communities. There is a lot of multiculturalism, and there are different communities of people who develop those, so I have not seen any. So I don't think there will be bias from that perspective. I myself am a person with an accent, so I might be biased myself in thinking that, but I don't think so.

[00:23:17] Host: How is the data collected in ambient AI workflows managed in a way that complies with HIPAA, particularly personally identifiable data? And who has access to it?

[00:23:27] Anya: That's a great question. This is what our product, engineering and partially UX teams are struggling with at the moment: who owns the data. We work with vendors providing the solutions to us, and the question is exactly that. How do we keep it, who owns it, how does the vendor, if it is the vendor, preserve the data, or should it be kept with us? There might be some technical elements which might solve this, in terms of where we introduce the AI and whether it's actually within our system or outside.

[00:24:08] Host: Okay, so a lot here on security and privacy, I'm sure. What is your team's practice on security and privacy of the recording data? Is it erased after an allotted time? How do you account for communicating your practices to the patient?

[00:24:23] Anya: Yeah, absolutely. We are not recording any interaction where real data is present, where HIPAA, for example, applies. Any time we have an interview at a practice and are looking at the real interfaces, we shut down the video recording. So we try to take notes and do whatever is possible from that perspective. That's a good question.

[00:24:51] Host: You have an upvoting battle going on here, so we're jumping around. Okay: you have talked about the physician response. How have patients responded?

[00:25:04] Anya: That's an interesting question. I guess you mean in terms of being recorded and using the ambient AI scribe. So far we've been collecting consent from the patients, which means that they are probably totally fine with the solution.

[00:25:27] Host: Have you experienced any initial friction points when getting physicians to adopt ambient AI? And if yes, how did you resolve that?

[00:25:35] Anya: Yeah, absolutely. It's a question of trust for many physicians. It's a matter of their practice of taking those notes. Some of them don't want to use anything or anyone; they want to take their notes themselves. That's the biggest friction point. And for some physicians it's a matter of word of mouth: "Hey, I've used this cool feature, it's AI. Do you want to use it?" That's how they find out. So it's also about knowing about the features. Even though our enterprise educates people on what they have, it's not always visible and apparent to physicians during practice.

[00:26:12] Host: Have you had physicians who are just scared off by AI in general?

[00:26:17] Anya: Yeah, of course.

[00:26:17] Host: Okay, of course. How does the accuracy compare between the human scribe and AI?

[00:26:26] Anya: It might also be a matter of the technical jargon the AI's LLM is trained on. It is definitely more accurate than a human. During one of the talks here we also heard that AI is pretty accurate at capturing information. So it is usually more accurate than human transcription.

[00:26:44] Host: And how do you work with a vendor when you have a discovery finding from your doctors?

[00:26:51] Anya: We come up with an ideal scenario and workflows first, and then see whether the vendor matches or doesn't. If we have to adjust, we work with the vendor to match the workflow we are trying to achieve. And if they have a better solution for us, because they are experienced in the matter, we work with them and use whatever they can offer.

[00:27:18] Host: Jumping around. Okay: how do you overcome patient skepticism when this AI solution is presented to them?

[00:27:26] Anya: As far as patient responses to the AI go, and this goes for the previous question as well, we do not have a lot of insights. We need to do a bit of research, since the solution is new. We also have restrictions with practices: they allow us to interact with the patients, but not directly. So we have not yet collected a lot of data on this one, but that's a great one to follow up.

[00:27:52] Host: I think this next one's interesting. While AI may eliminate the need for a human scribe, save the hospital money and potentially eliminate jobs, does it add more work time for the HCP examiner to double-check that the AI is accurate?

[00:28:05] Anya: Oh, absolutely. Well, maybe not more time. Maybe not more time than they would spend if they created the clinical note manually. But as I mentioned, yes, it does introduce extra clicks and extra cognitive load to review what has been generated. But so far, according to what we see in the industry, it is a smaller step compared to the overall solution that we provide to them.

[00:28:35] Host: How accurate is the AI at understanding and correctly transcribing people with heavy accents, or just in general? Do you record accuracy rates?

[00:28:45] Anya: I cannot recollect the accuracy rates. I do remember the questions which we put to the vendors we worked with before, and so far the results were very positive. So the accuracy rate is pretty high.

[00:29:03] Host: Is there an opportunity to also provide real-time interaction with the AI, rather than just talking and transcribing? Can you allow the doctor to queue up other actions, or even ask questions?

[00:29:15] Anya: Yes, that's one as well. This is our first initial scope. We are looking at further downstream workflows that come from the note. One of them is the billing codes which I mentioned, but another one which is even more interesting is triggering orders for labs and for medications. Imagine that the provider says, "I think that you experienced this symptom, so maybe you need to try this particular medication." The system automatically grabs the suggestion and pre-orders something, and the physician, of course, has to go in, review what might have been ordered, and allow the system to execute the order.

[00:30:00] Host: You mentioned that you do not think there is bias in the AI model because it is built by a diverse group of people. Does the diversity of these groups reflect the diversity of patients and physicians?

[00:30:13] Anya: A lot of our practices are throughout the United States, and one of the biggest bases of practices that have started using the solution is in Texas. Texas has a huge Mexican population, a Spanish-speaking population, and so far they've been pioneers in using it, and they do not have any complaints about that. So it is already being used with a diverse population, from the culture perspective and from the language perspective.

[00:30:44] Host: If the data entry was affecting patient outcomes, why wasn't the data entry structure modified? And do you need all that data to begin with? Did you eliminate anything on that side?

[00:30:58] Anya: If the data entry was affecting patient outcomes, why wasn't the data entry structure modified? It was modified. As I've mentioned, within the note there are different subsections, and each subsection corresponds to data which needs to be used by different teams. So it has been modified, and we actually matched our AI solution to what had been done before. I hope that answers the question.

[00:31:26] Host: How deeply does design work with legal partners?

[00:31:31] Anya: Oh, big time. In order to use AI, first we have to go through legal. But then, when we are introducing design features, we always have a legal partner in the room while we are discovering what it needs to be. And this is not only with AI; it's throughout all our application builds. We have to involve our legal partners. So that's a great question, but it's even more intense with AI here.

[00:31:56] Host: That's interesting. How do you go about user research in the discovery phase for the product?

[00:32:03] Anya: That's my favorite one. As I mentioned in one of my last slides, user research is here to stay. AI might speed up research and maybe improve its quality, but I think that in the age of AI, user research is here to stay, because that's how we can figure out the outcomes that we want to build. So user researchers, you're safe. You'll be here for a long, long time.