AI and Emotional Design: Creating Empathetic User Experiences

Bhrunali Gokhe at UXDX Community: Empathy-Driven Design: AI, Emotional Intelligence, and the PM-UX Partnership
. Video: https://youtu.be/PaS6lNEuXB4

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction and the three categories of AI

[00:00:07] Hello Ro, thank you for the small introduction. Very good evening everyone. Myself, Bhrunali Gokhe, and I work with Accenture Design Studio. So today basically we are going to explore the fascinating intersection of artificial intelligence and emotional design. We are also going to see how this pairing can help to understand the user experiences. I'll be showcasing real world examples in order to understand the ideas in a much better way. So let's dive in.

[00:00:55] So what is the role of AI in emotional design? To understand that, it's very, very important to understand how AI basically functions. AI is being categorized into three different categories, that is artificial narrow intelligence, generative AI, and artificial general intelligence. I would like to quote a few examples for you guys to understand what these categories are basically talking about.

[00:01:23] So artificial narrow intelligence: you must be aware of products like smart speakers, the platforms like web searches, the AI inbuilt car dashboards and so on. That's where artificial narrow intelligence comes into picture.

[00:01:45] Generative AI, which is the next one. Everyone in our day-to-day activities must be using any of the platforms which are being generated using generative AI. A few of them I would like to name: Claude AI, Google Gemini, Copilot, and a few of the designing softwares are Midjourney and DALL·E.

[00:02:12] And the third one, that is artificial general intelligence. It's nothing but an AI which can do anything a human can do. So this is the basic differentiation, or the way AI basically performs.

The five roles of AI in emotional design

[00:02:31] Now moving on to the role of AI in terms of emotional design, there are five stages which everyone should understand to understand the role of AI. For every point I'll be sharing an example in order to convey the concept in a much better way.

[00:02:51] Talking about creating an emotionally intelligent system: when we say emotionally intelligent system, it means empathizing with the end users, a platform which can empathize with the user. There is a very good application named Affectiva, which actually gains insights from the user's facial expression. So for example, if the user is using any product, then by looking at the expression, or capturing the facial expression, the platform can actually gather the emotional insights. So that's what creating an emotionally intelligent system refers to.

[00:03:40] The second one is bridging the human-machine gap. Again, the relevant example to this particular section would be Tesla's in-car AI integration, which actually recognizes the driver's fatigue and pushes an alert message which ensures safety. It also makes decisions based on the inputs which are being provided by the users, such as traffic conditions, the conditions of the road, and so forth.

[00:04:21] The third point, which says application across industry. So here I would like to mention one of my latest examples where I have worked on. I was creating a DAM platform for one of my clients. So basically DAM is nothing but a platform, a repository for storing their assets, brand assets. There we have integrated AI searches to explore more of the metadata. Secondly, we integrated AI with the design softwares so that the designers can edit those assets then and there, instead of having a third party software installation. Even if it's not there, the functioning won't stop.

[00:05:17] The fourth one, that is emotion recognition and real-time feedback. For this again there is a very good example. There is a platform named Zendesk. So this platform is nothing but a customer support platform, and what it does is it actually detects the frustration. So if the platform detects any negative sentiment, the query will be escalated to a human agent. So that's how the emotions are recognized and the right solution or feedback is provided.

[00:05:57] And the last point, what it says, is enhancing personalization. So we all know that in this era of AI, every tool which is there in the market is providing a personalization or a customization in some way or the other.

The three principles of emotional design

[00:06:15] So moving on, we must understand the principles. When we talk about emotional design, I think almost everyone, or especially the designers, must be aware about all these concepts, but not particularly in technical terms. So let me take you to this.

[00:06:39] Three principles of emotional design are visceral, behavioral and reflective. When we say visceral, it's nothing but the aesthetical appeal of the product. The second, behavioral: how that product is being used, what is the user experience of the product which is being provided to the end user. And the third, that is the deeper understanding of the product and what value it has been adding while using that particular product. Let's see this with an example.

[00:07:15] When I think of Apple as a brand, the first thought which comes to my mind is the sleek design which it provides and the minimal look and feel. Talking about its behavior, yes, it gives a fantastic user experience, also it provides best safety measures. And similarly for reflective, everyone must have been using the smart watches. So in spite of just providing the time, it also gives you an additional value added, for example giving you health or fitness updates, maybe adding a value to your current lifestyle. So that's how these principles are measured and kept in mind.

Techniques for measuring emotion

[00:08:08] So as we will be designing the experiences, or a platform or a product, using emotional intelligence, it is also required to understand how the users are going to react to that particular product or platform, and those need to be measured using a few techniques. Here are a few of them.

[00:08:35] First is emotion mapping, user surveys, AI-powered tools. So I'll support all the three techniques with an example for better understanding. Emotion mapping is nothing but visualizing the emotion in the form of a journey, like how the user reacts. So for example, let's take an application, a login screen: if it doesn't have a Register Now button, just imagine the frustration a user must be having. And similarly, there is a very good example for emotion mapping where a well-known brand like Coca-Cola has used emotion AI for their latest ad campaign to connect with the end users.

[00:09:30] The second technique, it's user surveys. So what user surveys do are not different from the Google surveys what we perform during our research phase.

[00:09:40] And the third is AI-powered platforms. When we say AI-powered platform, so as I already quoted one of the very good examples, that is Affectiva, which takes the insights using the facial expressions. Another example I would like to quote here is KLM Airlines, which is again a customer support platform which actually tracks the queries using the tone of voice and the text to understand the user behavior. So these are a few techniques which we should consider.

Designing for emotional intelligence: the Calm app

[00:10:23] Talking about designing for emotional intelligence, there are four areas which we should consider, and we'll understand these four stages with a very good example. So there is an application named the Calm app, which is basically used for meditation and sleep support.

[00:10:49] So let's understand this with a scenario. For example, we are using a Calm app where the user puts an input prompt, maybe, which says, let's say, get asleep faster, or relieve stress. So based on this input the application will collect data and throw a reply based on the current emotion. It will provide a mood slider to the end user.

[00:11:23] Now moving to the emotional analysis. The user input which has been taken by the application, it will understand how the emotions of the user are being gathered. So it will perform an algorithm where it will see how frequently the user uses these phrases. For example, is it every so-and-so period the user actually uses this phrase, for example relieve stress or get sleep faster? So based on that it will create a pattern.

[00:12:02] And based on that pattern it will throw an adaptive response. When we say adaptive response, it means that the user is facing an emotional sentiment for which the user might need an extra emotional boost-up or support. So the application will throw a push notification, a motivational push notification, which might say that you need to take extra care of yourself, it's high time to get down to bed. So these are a few examples of how adaptive responses actually react.

[00:12:46] And moving towards the last one, that is user feedback. It is nothing but, for example, if the user has completed one of the meditation sessions within the application, the application will throw a popup message saying, can you rate this application, or do you have any comment, how can we improve the user experience, in order to tailor the content in a much better way. So these are a few practices which we actually should consider while creating any product or an application based on these ideas.

[00:13:31] There are a few case studies, out of which Calm we have just discussed. There is Pi[?] and Duolingo. I would like you guys to go and check these case studies, which are particularly based on AI and emotional intelligence. We don't have that much time that I could explore this in detail, but yeah, a very good recommendation, I would say.

Challenges and considerations

[00:14:00] So talking about the challenges and considerations, when we talk about AI and emotional intelligence, privacy concerns definitely are a very crucial part of it, where the consent with the end users and how the data is responsibly used needs to be conveyed.

[00:14:21] When we say ethical considerations, we must follow the ethical guidelines in order to be safeguarded with the recent deepfake crimes which are going on in the current scenario.

[00:14:41] To tackle biases and fairness, I would say the AI model needs to be frequently monitored or audited, and also AI can be trained with a vast data set, with multiple diverse demographics, cultures and contexts, where we can actually reduce the biasness and move towards the fairness of providing the solution.

[00:15:15] And the last, that is user transparency. So when we see user transparency, here the end user must know how AI is being used to influence their emotional experiences, in order to have that trust while using any of the applications.

[00:15:38] So here I would like to end my session and open the forum for Q&A. I would like to have open thoughts, not just the questions: any suggestions, any thoughts on the topic. I know it was a small presentation due to time constraints. There is a lot we could actually explore, but yeah, I would like to hand it over to the team.

Q&A

[00:16:07] Host: Excellent, thank you very much, Bhrunali. So yes, as Bhrunali mentioned, write in your questions on whichever platform you're watching this on, whether it's YouTube, LinkedIn, our platform, and I'll gather all those questions through. But I'm going to just get the ball rolling, because it was a great example of the adaptive responses with Calm. I'm just thinking practically, how would a designer or researcher or developer, how would you prototype that out, or how would you design something that's adaptive so it can go in many different ways?

[00:16:45] Bhrunali: So when we say adaptive, it's basically more towards the responsive side of it. So for example, nowadays it's very, very in trend and very important to have a design which is very responsive across all the platforms, right? And for that we do have plugins within Figma, we have multiple tutorials which we need to explore, because Figma has recently come up with very latest AI integrations which can help us create prototypes, high fidelity designs, in a much more responsive way.

[00:17:25] Host: So yeah, I guess my question was less about adaptive, like I guess that's the screen sizes and being able to kind of work on a mobile as well as a desktop. But where I'm going with my mind is that when I'm designing something I think of a nice linear flow, and it's nice and neat and it's very easy because it's kind of step one goes to step two goes to step three. And I guess, unless I misunderstood the point about the adaptive, is it kind of realizes, okay, you don't need to go through steps two and three, you can jump to step four, because it understands more about you, rather than being a dumb system that always makes people go through all the steps? Is that correct in my understanding, or did I misinterpret it?

[00:18:17] Bhrunali: So when we say adaptive, we go through a particular process, right, as you mentioned. So we just cannot jump to the final stage, and we know that designing, it's all about process, right? So we do the iteration maybe from different stages, but we cannot jump to the final stage. So for creating any of the applications or products we have to go through multiple stages. When we say stages, there are five different stages which we actually follow while creating any design or a solution, right? So those five steps are definitely mandatory to be followed.

[00:19:01] Host: Okay, so it's actually following the exact same process internally when you're building it as you would a normal pre-AI application.

[00:19:11] Bhrunali: Right, right.

[00:19:11] Host: And the other question I had, I know that you were kind of tight for time, but you mentioned Duolingo, and I've heard a lot about the potential for AI particularly in education, because you have that one-on-one tutor which is kind of infinitely better, or well, two orders of magnitude better, than a classroom environment. So do you have any examples of what Duolingo are doing in the space, because I've heard a lot about it but I've yet to see a really good product coming out in the space?

[00:19:49] Bhrunali: Yes, so AI integration, when we say in the educational sector, it has taken a vast turn and has done a lot of it, given to the educational sector. There was one of the case studies where I was researching. I will give my live example. I was a person during my schooling where I never used to like to study history as a subject. But nowadays there are platforms where history has been made so interesting using the 3D animations which we have, the 3D integrations which we have within AI, also using storytelling, which is one of the very good aspects when we talk about presentation.

[00:20:37] Bhrunali: So instead of having just boring textbooks, today they actually have a 3D animation to be explained how history happened, how the events happened in the past. As we all know that visuals actually give a human a visualization, an image of what has happened, which actually brings curiosity within the students to understand and bring that curiosity to learn more. So that's how AI has been contributing and adding a value within the educational sector.

[00:21:20] Host: Great, yeah, and I guess that's that interactive bit as well, because each student goes at a slightly different pace, right? And we have a question coming in from Matthew. Thanks for the insightful presentation. I'm curious if you have any specific examples of how one can address or avoid inherited bias in AI.

[00:21:42] Bhrunali: Can you please repeat?

[00:21:46] Host: So Matthew's asking if you have any examples. You talked about the risk of bias, so he's asking if you have any examples of how one can address it or avoid bias, so what can they do to proactively make sure that doesn't happen.

[00:22:00] Bhrunali: When we say bias, it's not just with the AI which is happening nowadays. Even we as humans have that bias factor, right? For example, if you like any particular application, and if you are frequently using it, you will have that bias intent towards using that again and again, even though the competitor of that particular platform might have come up with some new add-on feature or so.

[00:22:36] Bhrunali: So in that case, we as designers should avoid having biases, reason being, if a designer is on the neutral phase, one can actually explore many opportunities which are being provided to the competitive products, with whom we are working on. So to have that open-mindedness we should have that neutral space and explore the various opportunities, being a competitor, or the new trends which are there in the market. So that's how you grow.

[00:23:16] Bhrunali: And yeah, so I'm talking about the bias in terms of AI, so how AI basically has that bias effect. I would say it's because of the machine, the particular algorithm which is set. And to tackle those biases, I've already mentioned that we can have multiple diverse data sets where we can train an AI, or audit an AI model, by providing multiple diverse demographics, cultures and new contexts, so that even the AI should not have that bias in providing any end result to the users.

[00:23:55] Host: It's a great point about making sure this diversity input. There's also a fantastic interview, I can't remember the name of the person in OpenAI, but they talk about red teaming, where they actively go in and try to find all of the biases and all of the breakage points in their models. So yeah, I'd recommend looking up red teaming. That's a great input.

[00:24:24] Host: We actually have another question, but unfortunately we're just out of time. So maybe what we'll do is we'll post that question on our Slack channel. I'll invite you, Bhrunali, to come and answer it there as well. And there's a comment about sharing the video link, so if you just want to post that on our private chat here.

[00:24:44] Bhrunali: Of course, of course, I'll just do that right away.

[00:24:46] Host: Brilliant. Well, thank you very much, Bhrunali.

[00:24:50] Bhrunali: Thank you so much for having me, and thank you so much audience for being there till the end. Thank you very much.