How Intercom built ‘Fin’, a GPT-4 powered chatbot

Fergal Reid at UXDX EMEA. Video: https://youtu.be/eu95OO6dhuQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Intercom, and the bot we built

[00:00:00] Hi, I'm Fergal. I work for a company called Intercom. Thanks so much everyone for coming to the talk. Hopefully you get something useful out of it today. I'm going to tell you a little bit about how we at Intercom built Fin, which is a large language model, GPT-4 powered product that we shipped generally available there in June. I'm going to try and keep a good bit of time for Q&A at the end if I can, because there's so much going on in AI at the moment. The world is moving so fast and it's really hard to know what people are interested in and where to meet the audience.

[00:00:40] Can I just ask you, to try and calibrate, can I get a sense — can you raise your hand if your primary function is design or UX? Okay, so a very, very UX, very design crowd. That's really good for me to know, great. And can I ask you one more thing: could you raise your hand if you've heard of Intercom before? Okay, so we're fairly widely known.

[00:01:03] I have a few slides at the start explaining Intercom. I'll go through those pretty fast. Intercom started out to be a customer communications platform, one place you could go and handle all your customer communications, and more recently we're really focusing on customer support and customer service. How can we get people to have great customer service on the internet? Intercom is perhaps closely associated with this messenger, this thing that lives in the bottom right hand corner of a screen where you can go as an end user and talk to a customer service team or a product team or whoever is responding to it. Intercom has done a lot of work to make this messenger customizable and it's fairly widely deployed.

[00:01:45] But we spend a lot of time thinking about the experience of the support agent, the support representative that answers questions all day every day. We spend maybe about half our time in terms of AI and machine learning thinking about that, and then the other half of our time thinking about bots in the messenger. That's a picture there of the bot experience that you might have gotten in the Intercom messenger over the last few years. What I want to talk about today is the story of how we took our previous generation bots, which did use machine learning but weren't as sophisticated and weren't as advanced as this new bot we've built, Fin, is.

[00:02:26] So I want to talk a little bit about Fin. Fin is this bot that we built over the last — really from about February through June, and we're still working on it. Why am I talking about Fin specifically? What's different about it? This uses modern AI, uses GPT. Someone can come along and they can ask a question here, like the classic question we always use for testing is "how do I delete a tag?" Fin would just go and look at the knowledge base and answer the question from the knowledge base.

[00:03:06] What's really impressive or interesting about this, as opposed to what we had before, is if you see this follow-on question here, "how do I add one instead?" It's smart enough to know that "one" there refers to one tag. And then in the Intercom knowledge base there are many different places you can add a tag, so it goes and looks across all the different help desk articles and synthesizes a new answer here. This is an example that I just made two days ago when I was looking for an example for this. It just kind of works.

Three levels of getting to grips with GPTs

[00:03:44] So how did we do this? What changed in the world that made us want to build a product with this sort of next generation AI, and to do it so quickly? Really it's GPTs and large language models, like ChatGPT. We really feel this is an internet-sized technology change. This is a huge leap forward in AI. It's going to change all our products at some stage, and this tech has also changed how we build and how we use AI. So I want to talk to you about that today.

[00:04:18] I want to start off by talking a little bit about the technology, because this is a technology-driven change that hopefully I'll be able to introduce at a high level in terms of how we see it. And then I want to talk a little bit about the process that we use to actually build with this technology, and then hopefully open it up to questions and discover I have explained everything badly, but we'll do our best.

[00:04:39] For us there's three levels of getting to grips with and dealing with this new technology, which I'm just going to refer to as GPTs — you could call it large language models either. Level one is like, oh my God, these things are incredible. I'll give you an example of this. If I go to GPT-4 and I just say, hey, what's my biography, it knows this really quite detailed specific information about me. But then you get to level two, where you realize that a lot of the information that it knows about me is actually wrong. It's made up. It's plausible but incorrect. You get to level two of your understanding, which is: GPTs make things up and they aren't trustworthy.

[00:05:18] My thesis for this talk, and the reason why I think this is an interesting thing to think about, is that there is, I would argue, a level three, which is: actually they can still be incredible when they're used right, but you've got to look at them in the right way. You've got to see them as engineering components or design components. See them as building blocks. We've all looked at maybe ChatGPT, and ChatGPT is a product. It uses the technology but it's got certain design trade-offs. For Fin we're trying to build a product that has different design trade-offs. To do that we need to separate out some aspects of this technology that I think have been accidentally bundled together.

Why token prediction is not a useful way to think about them

[00:06:01] What do I mean by that? I want to start off with: how should we think about GPTs? What does this even mean? These are big machine learning systems, and what they're setting out to do is something called token prediction. They're just setting out to predict the next word or the next part of a word, what should come next. So you say to ChatGPT "the quick brown fox" and it says "jumps over the lazy dog," because it's been trained on a lot of text on the internet that has the sentence "the quick brown fox jumps over the lazy dog." You give it the start of a mathematical sequence of numbers, the Fibonacci sequence, and it just predicts what's coming next. This kind of simple stuff.

[00:06:43] But where it gets interesting is you say something like this: the smallest household item too big to fit in a suitcase. That's an unusual query. There may not be too many lists on the internet of smallest household items that are too big to fit in a suitcase. But it says, oh, it's a microwave. However, this could also vary depending on individual preferences and the size of the suitcase, and so on. So there's something going on here that's not that easy to predict. There's some form of reasoning here over and above just token prediction.

[00:07:15] You see a machine learning person and they'll tell you all this stuff is a sequence model, uses attention. I say, hey, that's not really a useful way to think about these things. That's like thinking about what's a human — well, a human is genes and evolution, and you wait a long time and you get a human. This is true, but it's not useful. So in the same way that's not a useful way to think about humans and your users and what they do, thinking about token prediction is not a useful way to think about these systems. You should distrust anything you read that says, oh, it can't do something because it's just trained to predict the next word. These are new systems. They're different, and we're still learning exactly what they can and can't do.

A reasoning engine attached to a database

[00:08:02] I think a better way of thinking about these systems is that it's like a big database, and it's a reasoning engine attached to the database. Really the reasoning engine is key. What's new and different about this AI stuff is that it can do a bit of reasoning for you, whereas in the past if you wanted a piece of software that would do reasoning it was really hard or almost impossible. We focus a lot on the database that comes with ChatGPT if we use ChatGPT, and often the database is wrong. It's a liability. It's this compressed representation of things. So for Fin, and I think for a lot of products that you want to build, you want to use the reasoning part of this AI tech but you want to try and get away from thinking about what facts it knows or doesn't know.

[00:08:54] Again, the reasoning capability is quite sophisticated. A toy example here: a mouse wants to steal a piece of cheese, there's a cat in the room, there's a bed in the room, what should the mouse do? You go to ChatGPT or GPT-4 and it makes up a pretty good plan. If you haven't worked in machine learning that may or may not be that impressive. This is really impressive. Machine learning systems — people struggled for decades to get them to do anything vaguely like this and they just didn't, and now suddenly it almost works out of the box across many different domains. You say the cat's deaf, and it'll give you a new plan. What about the bell? Ignore the bell, the reason being the cat is deaf, the bell sound won't alert it. A lot of general knowledge about the world. Even if this tech was to stay static where it is, I think we'll be figuring out how to use this reasoning and combine it with our software products for years and years to come, decades to come.

[00:09:54] I'm spending some time on this because I think it's important to understand the limitations and the strengths of this new technology that we can design with and build with. A lot of talk was focusing on the fact that it hallucinates. But if you ask a human to answer a question — you get a history expert and you ask them a deep question about history, well, they may misremember something too. They may recall something badly or make something up that's plausible but wrong. If you give them a history book and then ask them a question and they can go look for it in the history book, they give you something more reasonable. That's how you want to use these, and that's how we use this, that's how we built this product, Fin. There's limits to what you can put into them, but overall they're pretty good when you work well with them. We're still learning how to do this.

Interpolation, extrapolation and retrieval augmented generation

[00:10:49] Another model — this may be a little technical — another model that we use to think about what tasks these systems are good at versus bad at is: are they interpolative or extrapolative? You can think about this in a mathematical way. Interpolation is when you're asking a model to do something that's similar to situations it's seen before, and extrapolative is where you're asking it to think about something very outside its experience. They're really good at interpolative tasks. You should favor tasks where you can give it examples or training data, or ask it to do something that it's probably seen on the internet, perform a task given a context.

[00:11:33] Can you raise your hand if you've heard of retrieval augmented generation? Okay, that's much fewer people. That's really the game you're in when you're trying to build good products from these things. You're trying to say to the system at runtime, hey system, here's a bunch of context, maybe context that I've pulled from my database, now figure something out. Again, you don't want to be building systems that say something like "who was the president of America in 1900?" Instead you want to be building systems that say "hey GPT, given this article about the history of US presidents, who was the president of America in 1900?"

Intercom's journey after ChatGPT launched

[00:12:21] Okay, so that's background technology. Hopefully that's useful. I want to tell you a little about Intercom's journey using this technology to build Fin. I run a machine learning team in Intercom. We sit alongside a large number of product teams. We are technical machine learning product development specialists. Our world and the company's world changed on the 30th of November when ChatGPT came out. A lot of people in the company, in Intercom, were up late on Slack playing with ChatGPT going, oh my goodness, this has really changed customer support. That change hasn't happened yet, but it's going to.

[00:13:04] We've been spending years building bots. The bots are pretty good when you ask them specific questions that someone has manually set up an answer to, but this thing can just dialogue with you. What do we do? The first thing we did was we set out to build extremely fast, simple features — an extremely fast product development cycle to just build some very simple features to get this power of ChatGPT into our customers' hands. We focused on the inbox. We built features in the Intercom inbox for our support reps to help them summarize a conversation.

[00:13:45] Summarizing a conversation is an interpolative task. You've got this big set of customer support conversation, you're trying to make it smaller, you're trying to summarize it, you're trying to take data that's there and press it down. So we had a good enough mental model of the system that we said, hey, summarization will probably work pretty well, edit tone of voice will probably work pretty well. We built these simple exploratory features quite quickly, to do things like have a support rep be able to edit the tone of voice of something before it sends to an end user, or to write a shorthand and then try and expand that shorthand out. We built this quite quickly.

[00:14:29] We started on the 5th of December. The 20th of December we had prototypes used by the Intercom customer support team over the holiday break. The 13th of January we launched the customer beta with fewer than 10 customers. The 31st of January we did a launch with testimonials from those customers based on several weeks of actual real use. And by March we had these inbox features live with thousands of customers.

[00:14:56] In parallel with that, we realized we were like, oh, this is going to hallucinate too much. I don't know if we can build something to help end users directly, an end user question answering bot, with this. We were unsure about that. But by the end of January, by February, we had one thread of work which was, oh, actually we've got increasing conviction that we can build something that will actually answer end user questions. We got early access to OpenAI's GPT-4 around then, and in our early prototypes we were like, wow, GPT-4 can be told to hallucinate, told to make things up, even less than the previous generation of models. And so we were like, wow, we think we can build something here that's actually a really next generation bot that doesn't just augment human support reps but actually directly answers end user questions.

Design goals for Fin

[00:15:52] So we came up with some design goals. What did we want this next generation bot to do? We wanted it to converse naturally. We wanted it to be able to converse in that amazing natural way that ChatGPT does. But unlike ChatGPT, we wanted to constrain it to answer questions about just your business. We wanted to reduce hallucinations a lot. We wanted to have minimal configuration.

[00:16:17] Intercom had built customer support bots for years, but our customers really struggled to activate. They really struggled even though we thought it was worth their while and we spent a long time sweating the design of it. People just don't have time. That initial usage friction of, you have to set the bot up, if you wanted it to answer 100 different questions you've got to do a little bit of work, you've got to do maybe 10 minutes of work to teach it each one of those questions — people just didn't do that enough. That activation friction was just a killer. I actually think one of the most exciting things about AI is its ability to reduce first use and activation friction. It's huge. It won't be perfect out of the box, but it'll get you to the 80/20 part of value extremely fast. I think that's very exciting.

How we work with this technology

[00:17:02] Okay, so how do we build? That's what we set out to build with Fin. How do we actually work with this? Intercom has all these general product principles that run right across R&D: start with the problem, think big, start small, ship fast, deliver outcomes. It really sweats starting with the customer problem. But sometimes with AI you've got to work backwards from the technology instead, which is anathema to a lot of what we've learned as good product practice over the last decade, where we've had maybe somewhat static technology since mobile.

[00:17:39] I actually think a skunkworks style approach works well with this. My team often starts with technology and reduces technology risk initially, and then partners with product teams in the wider organization to try and say, okay, now that we've reduced the technology risk, what can we build in terms of product? It works well for us. We have a senior team of ML folk. We tend to own this technical core with engineers and scientists that help establish a wide design envelope, that then we can work with designers and product experts: now that we've defined the outsides of that envelope, what's the best product to build within that envelope? And they have more expertise on that.

[00:18:25] I think the ML team needs to have quantitative skills. You need to be looking at data. These systems in production do surprising things you didn't see in training, because every customer's data is different. You need to be able to monitor and to quantitatively use data science or analytics or whatever you want to call it to monitor these systems in production. And I'm a strong believer that if you have an ML team they need the ability to ship code. They need to be a skunkworks team maybe, but they need the ability to ship code to the product autonomously.

[00:19:01] We really believe in fast customer contact. I almost think that the biggest predictor of success for an ML product is how fast it ships to customers, how fast it ships anything to customers, even something with no machine learning. I think we're going to see a lot of people where, in the past, if you wanted to build an AI product or a machine learning product it was really expensive, and now you can use OpenAI's APIs to build a version one way quicker. Maybe it won't be cost effective in production, or maybe it'll be rough around the edges, but where it used to take machine learning experts ages to build a V1, now you can maybe do it quite quickly, in a matter of weeks. I think that's going to be very exciting over time.

[00:19:50] Test your real data as soon as possible. These systems will do surprising things when you put real data into them. You want to front-load that. A huge part of design for this is: how do you design a system that is graceful under failure conditions? It's not about what does a perfect design look like, it's what do I have to do to compromise the design to make a system that will tolerate failure in the AI component or the ML component?

[00:20:20] An example of this, one of my favorite examples, is something we see every day, which is the predictive text keyboard on your mobile phone. It's doing prediction. There is a little bit of AI or machine learning in there, but it's been beautifully designed so that when it gives you a suggestion that's irrelevant, which it does a lot, you can ignore it, but if it gives you a suggestion that's relevant and useful your eye is drawn to it. So it's quite a nice synthesis between design, machine learning and the human attention system there.

Shipping Fin, and what comes next

[00:20:52] Like I mentioned, how we build AI, how we build machine learning, has changed. People say, oh, large language models aren't all of AI, there's all this other AI stuff we've been doing for years. But to a first approximation they kind of are. They're completely changing AI, completely changing machine learning.

[00:21:13] So we shipped Fin. We started building it at the end of January, and then we shipped a demo. We really wanted to ship an online demo for people to actually interact with, to build credibility. We shipped that around the 14th of March, and then after that we moved into full production. Intercom probably had 100 people working on this, and shipped a full featured product generally available at the start of June.

[00:21:43] The current state: there's always this diffusion of innovations adoption curve. We're probably in early adopter still, or maybe starting to enter early majority. I would describe it as good adoption. We're working on a V2, and I can't share numbers, but tens of millions of dollars of ARR rather than thousands. We're working on future versions of Fin.

[00:22:03] Customer support I think is going to be hugely changed by AI. If I have to guess what's next, there's going to continue to be breakneck progress in the underlying technology. It's going to change what we can do. I think you're going to get smaller machine learning models that you can run in your own cloud system, or your developers can run. I think they're a little bit overhyped today, but I think they're exciting, and I think it's going to come more and more. We're all going to have to learn how we productize these systems. They're not going to go away. There's going to be continuing progress as well with large models. I think you're going to see a GPT-5 or competitors at some point, and they're going to continue getting pretty amazing. Okay, thank you very much. I think we've got maybe about seven minutes for questions.

Q&A

[00:22:53] Host: Thanks, thanks very much. I'll go over to the slides. I suppose the most important question is, where did you come up with, how did you get the name Fin?

[00:23:01] Fergal: A team that I was not on came up with the name Fin, and I think it's a reference to a lot of things. It's a short name, it's phonetic, it's easy to spell. It's a friendly name. Maybe it's a bit of a fact that it's an Irish name, and the salmon of knowledge probably came up somewhere. So yeah, probably that works.

[00:23:23] Host: And I just had one other question as well. How do you measure the success and the effectiveness of Fin in addressing those support questions as they are now?

[00:23:33] Fergal: Yeah, great question, how do you measure success. We are in the happy position that we've had bots live in production for years and we have well-established success metric loops. So when Fin gives an answer there's these little buttons that pop up at the end of the answer where you can say "that helped," in which case your conversation will end, or you can say "wait for the team," in which case another button dismisses the bot and says, look, you need more help from a human. What we do is we gather this telemetry from across our customer base and we use that to make sure that, hey, this is actually moving the needle, this is actually working.

[00:24:11] Fergal: One thing we did that I think is pretty dramatic is we bill on a resolution. We bill you a dollar when it has successfully closed a conversation, when the user hasn't escalated to get more help from a team. So we deliberately set ourselves up an incentive that's hopefully customer aligned and end user aligned too.

[00:24:33] Host: Did anything unusual ever come from that? Is there anything that stood out from that?

[00:24:37] Fergal: A lot of unusual stuff happens. There's a lot of ways of breaking your experience, of accidentally building deflection, something that gets in people's way, and so we've all sorts of guard rails and checks on that. So yeah, we see unusual things there frequently.

[00:24:54] Host: Great, so we'll get to the audience questions, I should say. Did any of your customers have any concerns on OpenAI having access to customer data, and do you have to train models with customer data?

[00:25:08] Fergal: So two questions. I'm going to take the second one first. As we use Fin today, we use the static OpenAI models, and so we're taking your data — when an end user asks a question, we take that, we use an old school neural network to search your knowledge base to shortlist relevant pieces of content, then we send them with the question to GPT-4 to try and get it to answer. So we're not training models with customer data, certainly not at the moment, and we'd have to look at our terms and conditions to do that.

[00:25:47] Fergal: Did any of our customers have concerns on OpenAI having access to customer data? Yeah, they do, because you have to send the customer question in some sort of summarized form to OpenAI to make the system work, and right now that's a trade-off that you have to be willing to accept. OpenAI give us pretty good terms. They don't retain the data, they promise they don't use the data for training. We've an enterprise agreement with them, quite favorable terms, so I don't think I would be worried about it. If you need your data to not leave the EU then you can't use it at the moment, because we only have it using OpenAI US servers. However, we'll probably move on from that soon, it's becoming available globally.

[00:26:26] Host: Okay, and then can you tell us how was the usability testing process used to validate Fin?

[00:26:29] Fergal: There's a lot in there. One thing we do is gather telemetry and metrics at scale, so that's a quantitative process. Other than that, we have a research team, a pretty well-developed research team in Intercom, that actually sat down with customers and asked customers what they thought of Fin working on different conversations. And we also did usability testing with end users. So there's a lot in there, and there are big issues of flywheel that works there, identifying problems which we then try and fix either at the product end or at the ML end. It's not perfect but it's pretty good.

[00:27:09] Host: Okay, and bad Fin: what was the biggest complaint customers have with Fin?

[00:27:14] Fergal: Good question. There's a few different ones. A lot of customers initially balked at the price. The dollar, we initially maybe charged $2 or something — that's mostly driven by computation. Large language models are very expensive to run at the moment if you want to bill per resolution. That's mostly gone away at the dollar price point, and people realize what's driving it. A lot of customers also are unhappy with what we call answer quality, because it's imperfect. Overall customers love it, but there's definitely a strand of feedback where people's expectations are, hey, this should be perfect. It's still not perfect. It's a lot closer to perfect than it was two years ago, but it's still imperfect.

[00:27:58] Fergal: So if you are in a regulated domain it may not be suitable for you to deploy this technology at the moment. If you're an entertainment or B2B SaaS or something like that, it probably is. So customers will have to make that decision. Can you tolerate occasional error, or is your tolerance for error zero? That's something people have to weigh.

[00:28:18] Host: And that one leads into the other one here. Can you tell us from your user research how comfortable and trusting are your customers using AI?

[00:28:26] Fergal: Look, it varies, but people are getting increasingly comfortable with this. And also we've done a lot of work with Fin to put guard rails around it and reduce that discomfort. This is something we're all learning about, but I think it's still going.

[00:28:42] Host: And can we be sure it can reason? These models assume a computational model of cognition, which is only one perspective. Should we care if GPT can reason, or if it can simply appear to be reasoning?

[00:28:55] Fergal: Yeah, it's interesting. I guess it's sort of a philosophical question there. If you think that — I mean, it's not what the question's saying, but if you were to say, oh, only entities with souls can reason, if you take a position like that then you'd be like, well, it can't reason. There is a computational model of cognition here, where it's like, oh yeah, something that does enough compute can reason. You get into a definition of reasoning. No, it's very hard to be sure of anything with these models. I would say I have a position, which is that what they're doing is indistinguishable from reasoning to me, and so I would say at a functional level I'd say it can reason. But I think the jury is still out in terms of the philosophical or psychological aspects of what's going on here.

[00:29:41] Host: Okay, and how do customers react when they know they're talking to an AI bot? Similar to chatbots?

[00:29:47] Fergal: Yeah, good question. Overall similar to chatbots, but I think end user expectations are changing. I think ChatGPT is changing expectations. Two years ago, as soon as people realized it was a bot, a subset of users, but a large subset of users, fell into a search oriented paradigm. They started to really start writing search queries, like you use Google, small short keyword-based queries. People don't do that with Fin so much, maybe because it starts dialoguing with them in a way that seems more natural and so that changes user expectation, or maybe it's because ChatGPT has changed user expectations generally. I don't think we know, but I definitely do think that we are seeing a difference in terms of the affordances and we're seeing a difference in terms of what end users expect, driven by ChatGPT and driven by the deployment of bots like Fin.

[00:30:46] Host: Okay, and then someone was asking, do you do any localization market testing or research on the name Fin?

[00:30:54] Fergal: That is outside my department, and we've a lot of great people who work in product marketing. How much market testing and research they did on the name Fin, I don't know. I'm sure they did some. On the other hand, we were moving very quickly here, and time to market was critical. We thought that, hey, we really wanted to get this stuff live quickly, learn fast. We thought AI was going to disrupt the space, we wanted to be ahead of that. And so right across the entire product development cycle we made intuitive calls, and so I don't know exactly how tested that would be. I'm sure they did some, I don't actually know the details on that.

[00:31:37] Host: Well, it's a cool name in my mind anyway. So what's your best prediction where LLMs will be in two years' time?

[00:31:41] Fergal: I don't know. I think things are going to get pretty crazy, frankly. There are different theses. I think that as people continue to dump huge amounts of computation at these they will continue to change qualitatively and quantitatively. There is an opposing thesis, which is, hey, the current architectures and current data sets maybe are reaching some sort of plateau and maybe progress will slow down. I definitely know that the amount of money flowing into this space is insane. The amount of GPUs being used for training, everything we're seeing is strategic investment everywhere is just absolutely huge.

[00:32:28] Fergal: So I don't think we know. My best prediction in two years is it's starting to get difficult to tell apart from human level cognition across many, many, many tasks. That would be my best point prediction, but nobody knows. Will they continue to advance, and how far will they continue to advance with scale, we don't know. I'm definitely a maximalist. I definitely think they can go absolutely crazy places with continued compute.

[00:32:56] Host: A little excited or a little scared, or a little both?

[00:32:58] Fergal: Both, yeah, both. I'm very excited for AI as it approaches human level intelligence from a product development perspective. As a human, if it starts to exceed that, if we don't hit some scaling thing, I think we need to talk about regulation and all those things, and people are starting to talk about those things. I welcome our overlords.

[00:33:09] Host: Can I point Fin to a selected database to make it safer to use in healthcare?

[00:33:24] Fergal: Fin only works with the data that's in your database, so you can definitely point it to a database. To make it safe to use in healthcare, that gets blurry. I would say you need to do your own testing and evaluation there. I can't guarantee that it'll never misinterpret something. Now, look, if you have a human support rep I also can't guarantee they'll never misinterpret something that's in your knowledge base. Humans are still a little bit better at knowing what they don't know than these systems are.

[00:33:58] Fergal: I wouldn't deploy it in a healthcare setting for anything life critical or anything like this. I wouldn't deploy any current generation AI. But I live in a country that has a first world kind of healthcare system. All these things are trade-offs and they depend on your domain. If you were a customer in a less developed country and there was no access to healthcare professionals, I don't know. Talking about generative AI generally, we'd recommend not deploying Fin for healthcare settings at the moment, generally.

[00:34:32] Host: Okay, and can you tell us from your user research how comfortable and trusting your customers are using AI?

[00:34:38] Fergal: Again, it varies. There's a spectrum out there. Some people should be legitimately slower adopters of this because maybe they're in healthcare, maybe they're in some extremely critical finance application. Other customers should probably adopt much more optimistically. It depends on your current customer support team, if you're outsourcing your customer support at the moment. So it depends on many variables, and I wish I could give a single answer, but this is new technology, it's a new product. I definitely think Fin is human competitive in many, many domains.

[00:35:19] Host: Okay, and has Fin reduced numbers to real agents?

[00:35:21] Fergal: Fin has absolutely reduced the volume of support queries that reach real agents, human agents. Resolution Bot, our previous product, also did this, but you had to configure it. Fin will frequently get good performance. Maybe it'll knock 25% off your inbound customer support volume literally within a couple of days of setting it up, where it's legitimately answering questions that previously humans had to answer, and successfully answering them. It depends on the business, it depends on how much is in your help center, how much is in your knowledge base. But yeah, it's definitely working, probably better than we expected, better than I would have expected.

[00:36:07] Host: Yeah, okay, great. Listen, thanks to Fin and thanks to yourself, Fergal. These people need coffee, it's 11:00, so please, a warm round of applause for Fergal. Thanks so much, everyone.