Transforming Enterprise AI with Scalable LLM Deployments

Mahmoud Fahmy at UXDX EMEA. Video: https://youtu.be/Jq-GAqGpUew

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and an opening question

[00:00:08] Mahmoud: Thanks, guys. Thank you. Hi, everyone. My name is Mahmoud Fahmy. I'm a lead AI engineer at Mastercard. Today I'm part of a team that is working on putting the model into production. So today I will give you a talk about what we are doing day by day, what our task is, and the challenges that we are facing.

[00:00:33] Before I start talking about AI and LLMs and generative AI, I'd like to give you a question. This is an opening question. In our daily use of GPTs, are we using single LLMs, or is there an agent behind it? This is a question I don't need any answer to right now, even after the talk. You have to think about it when you are using them day by day. Yeah, this is our agenda. Let's go into the talk.

Why generative AI matters for enterprise

[00:01:01] Artificial intelligence, machine learning and deep learning models have been doing a really good job in the last 20 years. For 20 years we've been talking about how we can diagnose images, how we can work with classification, how we do fraud detection, a lot of things. But unfortunately this is not working well with enterprise. Enterprise has a different way. Enterprise is dealing with a lot of unstructured data. There is so much unstructured data: it's estimated that 80 to 90% of the data in enterprise is unstructured. This creates a huge dilemma for our enterprise, because we have to deal with labeling a lot of data, and then we have to make it work with deep learning or machine learning. Now, generative AI gave us a really good way: you can put your data, unstructured data, into your model or your generator. Then from there, it will take care of your data and generate answers.

[00:02:08] Yesterday I was looking into UXDX. There was someone talking about how a lot of people say that AI will take our place in the future. Unfortunately, the problem that we are facing today is not about creating a future. AI poses a lot of risks today that we have to deal with. So stop talking about AI or generative AI doomsday. Talk about the risks that AI poses today. Machine learning or LLM generators create a lot of data compliance risks, from GDPR, a lot of things, in deploying your models. This is a huge problem. I recommend you read this article. It talks about stopping the talk about AI doomsday and thinking about the risks AI poses today. It's from Nature.

[00:03:07] Yes, there's a lot of misconception that generative AI was created by OpenAI. Unfortunately not. OpenAI got a really good innovation, but model generation, or the language model, has been there since 1991. That's the oldest one you can find, and it generated a lot of text based on old text. But what OpenAI did with ChatGPT and all of this stuff is create a user interface. Before ChatGPT, we could not talk with our models. Now you can just put your data in and start talking with your model. That's it, like you're talking to each other.

[00:03:57] And yes, Mastercard is not far away from AI. Mastercard has been working with AI for a long time, making a lot of things related to AI, and using generative AI we achieved a 20% accuracy improvement in our fraud systems. I'd like to recommend you read this CNBC article. It talks about how we have achieved 300% as well in current use cases.

What it takes to put a model into production

[00:04:28] If I'd like to create a model, or put my model into production using AI, there are a lot of things that you have to think about. First of all, you have to think about your environment. Second, you have to think about your model foundation. Then the software layer, and at the end, your infrastructure. In this graph I just put the challenge and depth that you will face with each and every concern or consideration when you are putting your model into production.

[00:05:04] First of all, the foundational model. Thanks to open source, there are a lot of generative models you can get from Hugging Face, you can get from everywhere. Then you can create an environment to do your experimentation, which is easy as well. Every enterprise right now has its own environment, and they can create their LLMs. Then, if you have a budget, you can buy GPUs and create your infrastructure, putting a lot of effort into this infrastructure. But the problem is how you can manage this infrastructure with your environment. So the software layer is the thing. There are a lot of challenges: how can I communicate one tool with another tool? How can I generate a model? How can I get my model to be in production, user friendly, and scaling on demand?

[00:06:01] This is a picture from a NeurIPS paper. It's from Google folks. They're talking about how only 5% of your machine learning application is just the model. The problem is not there. There are a lot of concerns: your data governance, your infrastructure, your consideration about where you get your data from, and a lot of things you have to think about. But still, the machine learning model is the core of your machine learning application.

Closed-book challenges and retrieval-augmented generation

[00:06:37] So I'll talk about the challenges that you can face when you are just experimenting with your generative AI or your LLMs. We have two approaches. The closed-book approach, or it's called the parametric approach, is where you just get your model from anywhere and start with prompt and context, or you can build it from scratch, or you can fine-tune it. But this has a lot of problems. The problem that we have is hallucination, which I think a lot of you face when you are using generative AI. Hallucination is when your model is confident in an answer that is not really the right answer.

[00:07:22] Then attribution. Today we are working with a lot of risk systems, and we have to think about how we can make the generative AI give its reasoning for generating its response. So attribution is very risky. The third thing is staleness. Because of the evolution of AI and LLMs, there are a lot of models, so maybe your model will be outdated tomorrow, or even in the next hour.

[00:07:54] And then we have revision. This one is about when you have a contract with a team that gives you your data, and they can opt out and say, hey, we'd like to delete this data. So you have to retrain your model again, and imagine this is multiple teams that have to do that. Then customization. There's a lot of domain-specific data, so I have to create a model for each domain. Imagine that you have 30 domain-specific applications or use cases. You have to create a model for each one.

[00:08:30] The solution is that you can couple a memory to this model. So how can I couple this model? You can add a memory outside your model, which is called RAG right now. There are a lot of things, but RAG is the most popular right now for everyone. With RAG, you can put your domain-specific data in, and this domain-specific data will create your context. So now you might think that hallucination will be gone, like this paper called "Retrieval Augmentation Reduces Hallucination in Conversation".

[00:09:14] But that's not the whole story when we add a memory to our generative AI model, or our model at all. It raises a lot of questions that you have to think about when you are creating your model or deploying your model. These questions aren't even half of the questions you have to think about. Should I add this data in this way? Should I make chunks? Should my phrase be very long or very small, whatever, based on your model as well? So there are a lot of challenges.

Challenges of deploying LLMs at Mastercard

[00:09:52] Now, this is our paper we published last year. It talks about the depths that we face when we are deploying LLMs at Mastercard. The LLM is just 3% of our work: getting it fine-tuned, putting in RAG, doing all of that stuff. Then we ask ourselves how we can create our compliance data, how we can make a segregation. Because this is a transaction, you have to make sure that only these people can access this data, because generative AI can share data without any compliance on it.

[00:10:36] So the key factors in deploying your model are how you think about utilizing your GPU, then efficiency with your scalable data. You have users, and you have to make sure that your user demand is scaled and working well. Then, thanks to open source, you have a big community that helps you focus on your area of expertise. Then, when you are designing the software layer that we're talking about, you have to think about how you can make it very, very good. It should be good at integration. It should be good at adding new models, because of staleness as well.

[00:11:19] Scaling and deployment considerations in fintech. The considerations that you have to think about are: are we going to the cloud, or will we work on premise? Second, the AI-driven risks. This is one of the biggest risks that AI poses right now, because imagine that you have your data, and it's very risky data, and you can access it using chat, which should not be possible. Then data governance, as I said. Then model optimization. Once we put the model into production, our work is not done. We have to keep it scaling and keep it working with performance, because the users will increase, and it increases like rush hours.

[00:12:05] Then seamless AI integration. You can imagine that I'd like to integrate with a legacy system. Can I do that? That's your software layer again. And this is a roadmap for how you can take your model from vision alignment to the changes that happen in your data each time.

The rise of AI agents

[00:12:29] The rise of AI agents. I'd like to ask you a question, because a lot of you have heard about AI agents and how AI agents work. An AI agent is not like single LLMs. An AI agent is different. This is the future of AI, the AI agent. But there are some core things you have to think about if you're trying to create your agent in an enterprise, or even in a startup. Don't build agents for everything. It's not a trendy thing that I have to build for everything. Keep it simple. Then think like your agent. This is a cool thing that you have to think about.

[00:13:19] This is an image. First of all, we have the single LLM, then we have workflows, then we have the agent, and the future will be physical AI. The single LLM is just the way we used to use it in 2022: you have a message, you put it to your model, and the model will generate a response. Workflow: the scientist creates multiple LLMs and sends the same message to all of them, then aggregates all of these messages. But the agent works in a different way. The agent tries to think like it is a human. It should take your message, then think about it and make a feedback and action. So the agent will create a more reasoned response.

[00:14:12] Then physical AI. Physical AI is about having an agent that can think and create things in a niche area. Right now, if you ask your LLM about something in a niche area, it will never get an answer. It will try to overfit to the things that are there. But physical AI should think about the niche area as something we didn't think about, and try to do it.

Should I build an agent?

[00:14:40] Should I build an agent? This comes down to three questions. First of all, is the task complex? If yes, then build an agent. If no, use a workflow. You don't have to care about all the headaches of building multiple agents, multiple LLMs together and communicating together. If yes, you have another question: are all the parts of the task doable? If yes, keep it as an agent. If no, you have to think about your scope. You have to reduce your scope. Then if it's yes again, and we are with an agent, you have to think about the error. Is the error a huge error? Will the error create a risk? If yes, put a human in the loop, so a human is still there if we are using an agent. If no, you can make it an agent. It's like sending an email: there's no risk in sending an email.

[00:15:42] And agents are just a model using tools in a loop. An agent sounds like a very different thing that we didn't see before, but it's just an LLM model that is communicating with an environment. It has a lot of tools, a lot of LLMs, a lot of databases, APIs to external memory, whatever. From there it takes the message, tries to reason about this message, and sends it back to you.

[00:16:12] Now I'd like to give you an example of how to think like your agent. The agent is something different; it's just trying to augment our intelligence. So think like your agent. This is a use case where I'd like to search for Mastercard global. I'm a computer, and I have a tool to type, move, click. First of all, I will move the mouse, click on Chrome, and close my eyes for three to six seconds. Then Chrome is there, and Google is there. I'll use another tool: move, click, type "Mastercard global". Then, hooray, I got Mastercard global. So try to think like your agent. This is the way the agent works.

Reflection, tool use, planning and multi-agent collaboration

[00:16:58] Now, the next evolution in enterprise AI. First of all, you have the static LLM, then the LLM with RAG, or external memory. Then we now have an agent. This agent is something that has reinforcement learning in the loop, so it thinks to create reasoning. Then the future will be multi-agent. Multi-agents will communicate together to create innovation for us. Again, think about a niche area that we didn't touch, and they will create something new we can use. So how does the agent do that? There are four things that you can think about: reflection, planning, multi-agent, and tool use.

[00:17:49] Reflection. Imagine there are two agents, or two LLM models. The first one will create a task, say, create a website. The second one should evaluate this task, and they will go in a loop until they get the task done perfectly. And this is an example I just told you.

[00:18:08] Then tool use. What is tool use? You have an agent. Imagine this transaction tool, or transaction agent, is just multiple LLMs together, and all of them are communicating together. There's a scientist there, and multiple LLMs, and they're creating stuff, but they need more information. So imagine that you are this agent. You are a transaction AI. What you will do is go to a computer and start searching: I'd like to get this profile, I'd like to get this data from this database, I'd like to understand what is going on with this research. Then all of this stuff will be reasoned in your head to create something new. And reasoning, by the way, should be reasoning all day.

[00:19:02] Then planning. Again, you are an agent, and you have to think about how you can do a task. Imagine that you have a task: you should do market research, you should go into hiring some people, then you create your startup. So what will you do? First of all, let's see if my brand is good. Second one, and so on. It's cutting the task into small tasks, and from these small tasks it will execute all the tasks as well. Again, the reasoning stuff should be reasoning.

[00:19:34] Then multi-agent collaboration. This should be the great thing that we should think about. Multi-agent collaboration: imagine that all of them are agents. Imagine that there is a room, and this room has four or five scientists, and there's someone in the middle who is a coordinator. He is a scientist as well, and every one of them is an expert in something. All of them should communicate together. The first one has expertise in mathematics, the second one is a statistician, the third one is someone from AI, and someone else is from physics. All of them communicate together to get new stuff or to do a task. So this is how the multi-agent way should work.

MCP and A2A protocols

[00:20:27] So how does this happen? How does the communication happen in tool use, how is your agent communicating with each of the tools, and how are the agents communicating together? Thanks to Anthropic and Google, they created two protocols, actually: Model Context Protocol, MCP, and the agent-to-agent protocol, A2A.

[00:20:54] How is MCP working? Before MCP, we had a lot of problems with how we can build an API, a RESTful API, that can communicate with multiple external tools. I'd like to get the data from a database. I'd like to get something from Medium, and so on and so on. With a REST API, you have to create something for each tool you would like to use. But MCP helps us have a unified way: if you'd like to communicate with LLMs, please use MCP. It will be easy and faster and straightforward. And yes, this is an example I just gave you.

[00:21:39] The A2A protocol. Some people think that A2A and MCP contradict each other. They don't. They complement each other, because A2A also creates a unified way that helps an agent communicate with another agent. Imagine that you have a room of scientists, and there's another room of scientists, and both of them are communicating using A2A. So this will help with multi-agent collaboration.

[00:22:17] And this is an image. This image shows you agent A and agent B. Agent A would like to get data from a database and from external tools. What it will do is use the MCP protocol to communicate with the external tools, as will agent B. But how they communicate together is using A2A. So they complement each other. Now you can build a multi-agent system that can use external tools.

[00:22:58] And sorry, this is from a reviewer when we published our paper. He asked, after all the challenges, after all the risks AI poses today: will LLMs and agents help us in the future, or will we help them? This is a question. So don't worry about the future. Think about AI, how you can create innovation, how you can build more things that help in this innovation and help us create things in a niche area. Thanks very much. And this is my box [?], and this is my QR code for LinkedIn. You can talk to me if you have any questions, if you would like. And yeah, thanks.

Q&A

[00:23:46] Host: Amazing. I loved it. As a very non-technical member of the audience, the way you put all of that together in a very simplified way meant even I feel I understood it. Really, really lovely work from you. One of the interesting points for me at the end there was when you were talking about this shift to multi-agent collaboration, all coming from different perspectives to come out with a really useful thing. It's a really interesting mirroring of how we've spent the last 10, 15, 20 years talking about cross-functional, cross-collaborative teams. In our AI, we're almost creating the same kind of team dynamics that we know help us build products as well. There's maybe something there, a mirroring of the way we've built organizations and the way we're building AI. I will stop rambling.

[00:24:36] We've got some great questions that have come in. Actually, I was interested in this as well. You had in your boxes, as you were going through, something called UX negotiation. As a UX researcher, I'm impossible to negotiate with, so I would love to know what UX negotiation is in the AI space.

[00:24:58] Mahmoud: Which one?

[00:24:59] Host: It's the second one down. It's just dropped there. What do you mean by UX negotiation?

[00:25:02] Mahmoud: A2A. Yeah. It's negotiation. It's not UX negotiation. It's a negotiation between multiple agents. Imagine that, again, you are working with your teammate: "Hey, it's not working this way," and he will negotiate with you about how we can create it. So this is the way. The agent is like you. This is why I said, make it simple: think like your agent. You are an agent, you're creating something, and you have to think about how you can communicate with your teammate and how they will create something. It's not perfect, for sure, but there's an evaluation. You have to think about it like someone is talking to you and you reply back, and this goes on. And how do they do that? We are using our mouths to speak; they are using the A2A protocol to communicate. So this is the way. It's not like UX.

[00:26:05] Host: Not like UX, but similar to that mirroring. As UX, we talk endlessly about putting ourselves in the shoes of our users.

[00:26:12] Mahmoud: Yeah, same thing.

[00:26:14] Host: You're putting yourself in the shoes of the agent: what is the process and the task they go through. I love it. Brilliant. Brilliant question up next. Again, one of the things I really loved was your checklist of whether we should build an agent for this, because there's an awful lot of excitement, that pendulum swing with something new: we deploy a million of them, it's great. The question has now disappeared, but it was basically asking how you focus a team, or a business, that gets excited about deploying lots of agents, and try and rein them in. How did you manage that at Mastercard?

[00:26:50] Mahmoud: Mastercard has a budget, so we can just ship all of them. We don't care. We have the money. It's fine.

[00:26:58] Host: Okay, that's it.

[00:27:00] Mahmoud: Yeah. So Mastercard has a budget. Yes, we can build an agent for sure, but for what? This is something we have to consider. If our task is very complex, and again it poses a lot of risks, we cannot go with an agent, because we need a human in the loop, which is a lot of headache. So we go with workflows, which have multiple LLMs. Even more, the problem with agents is that you have multiple agents, multiple LLMs communicating with each other, and you have outside databases that should communicate with them as well. This is a lot of headache and software infrastructure. There are a lot of things you have to think about, not only the budget. You have to think about how you can keep it in production. This is a huge problem. I gave you the challenges and depths that we published. There are a lot of things you have to think about with one LLM. So imagine an agent.

[00:27:58] Host: Yeah, absolutely. A difficult question coming out of the top here. I think we're all aware of the carbon footprint that generative AI is using. What are the carbon costs of using multiple AI agents? Do you measure that at all? Is it something that you have to consider as lead AI engineer, or is that something that someone else is worrying about?

[00:28:24] Mahmoud: Unfortunately, yeah. I think there are a lot of things they talk about at Nvidia and Intel as well, because we are using the infrastructure as an AI team. I know there are a lot of risks about the carbon, but yeah, this is a problem they are thinking about. I think the CEO of Nvidia, Jensen, was talking about this at the last GTC. He was talking about how to reduce the carbon used by GPUs.

[00:29:00] Host: It's probably one of those things, right? It comes in quickly, and maybe in a particularly inefficient way, and as human beings...

[00:29:06] Mahmoud: Yeah, this is one of the risks that AI poses right now. I think this is one of them.

[00:29:12] Host: It's got a very big environmental footprint. But as with lots of things in human innovation, I think if the benefit is seen, there will be a push to...

[00:29:20] Mahmoud: Yes. They are thinking about it. And for sure, if we can use CPUs, it will be better. So I think there are multiple things that they are thinking about and we could do.

[00:29:30] Host: But I guess it comes back to that initial quote you put up from Nature, right? Let's stop worrying about tomorrow's doomsday, because there are risks today, and actually the environmental thing is there. Cool. We have three minutes left for this very existential question: mate, why are we trying so hard to create replicas of ourselves? What's going on? Why are we creating AI? Why are we not just doing it ourselves?

[00:29:55] Mahmoud: Yes. As I said, generative AI is augmenting our intelligence.

[00:30:01] Host: Nice.

[00:30:03] Mahmoud: From day one, using deep learning, we were trying to augment our intelligence. We were trying to create a neural network. This neural network works like our brains. We cannot understand our brain, and we cannot understand how neural networks work, for sure. I'm not joking. We cannot understand how neural networks work. We just bring some algorithms to understand or evaluate them. But it is about making more automation. We have to automate our day-to-day. We have to reduce human labor to give humans the chance to think about something new, innovation. Humans are about innovation. So if we can create something new, why should we waste our time on operational things? So AI will not take our place. It just helps us. It's a shift.

[00:30:56] Host: It's an augmentation.

[00:30:58] Mahmoud: Yeah, a shift. That's it. So you just create an operational fix.

[00:31:03] Host: Fantastic. I guess another way of looking at it is that shift to a search engine from an encyclopedia. You don't have to open a physical book up. Those sorts of paradigm shifts.

[00:31:14] Mahmoud: Yes. And you shift from using papers; now you are using tools on websites, like Docs or whatever. So this is the way. This is it.

[00:31:23] Host: Great. Oh no, it disappeared. I'm sorry, we're not going for that one then. We have answered the top question, I believe, when we were talking about trying to limit the amount of LLMs we were using. You've mentioned data privacy, of course. In the banking space, you mentioned, huge risk mitigation must go into this. So what are the issues around data privacy with having an autonomous agent doing its own data mining? How are you making sure it's not going rogue and breaking the law?

[00:31:56] Mahmoud: This is one of the questions that I put on the slide. There are multiple questions. You have to measure the error. If there is a huge error, you put a human in the loop. If not, no. So yes, you just put a human in the loop in your agent, and for some tasks the human should evaluate that output.

[00:32:15] Host: Fair enough. So always keep a bit of human in the loop to evaluate the output, for risk mitigation at this stage of things. Fantastic. We only have 12 seconds left, so if you can answer "is this the real life, is this just fantasy" in 7 seconds, then you're a better man than I am. I think that's all we have time for. Ladies and gentlemen, thanks so much. Thank you, mate. Brilliant stuff. Love it.