Beyond the hype: turning Net Promoter Score data into actionable insights

Carolina Carbone at UXDX Community: Evaluating Performance The Right Way. Video: https://youtu.be/Ok0yCQTdtDI

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Context: who Mibanco's clients are

[00:00:07] Hi Rory, how are you? Hi everyone. I'm Carolina, I am head of customer experience at Mibanco Peru, and today I'm going to share with you a case study of how at Mibanco we turn our net promoter score information into actionable insights.

[00:00:23] So, a bit of context first. Mibanco is the number one microfinance institution in Latin America. We're number two worldwide. We're part of Credicorp, one of the largest Latin American financial holdings. And for us, everything's about transforming lives and writing progress stories with our clients, and how we do that is by giving them access to the financial service industry.

[00:00:56] Many of our clients, we have almost 2 million clients, many of them do not save through the formal banking system. Our clients value in-person, close, reliable advisory. Many of them are not digital, so the way in which we connect with them still relies a lot on face-to-face interactions, even though our digital penetration is growing a lot. And for you to have a bit more context, the loan size, or the average ticket of our loans with our clients, is a little less than $3,000. So many of our clients are informal, and for them we as a microfinance institution allow them to access the financial service for the first time.

[00:01:50] So that's a bit of context for you to understand how we have different challenges at the moment when we want to do product or service discovery. We need to connect with these clients and understand who they are. Following that, I think there's something that is key when you're doing discovery, getting to know your clients, be it if you're a UXer or you're somebody who's working in customer experience: to understand how you as a business reach every single individual that you offer a product to.

The hybrid relationship model

[00:02:29] So we focus on building this customer relationship model, customer relationship through a hybrid model. What does that actually mean? Traditionally microfinance is an industry that is highly reliant on face-to-face interactions. When I'm talking about face to face, it's either a loan officer or business advisor visiting their client portfolio, or a client interacting through bank branches. Most of the loan evaluation happens in the field, where advisors go and visit each and every single one of their clients to understand the cash flow of their business.

[00:03:09] So there's a high dependency on people, on physical spaces, and there's a high level of operating interaction with the branch. This is something that for our clients in a way was not necessarily convenient. Our clients are working all the time, they're entrepreneurs, so what they are looking for is a fast, secure, sustainable credit process for them to have a loan, and they also seek convenience.

[00:03:39] So one of the things that we've changed in Mibanco is we have a hybrid model, and this allows us to be more convenient for our clients. We not only rely on our business advisors and our bank branches, but many of the processes are now digital. Loan officers can work from home and connect remotely with their clients. Our loan officers have different digital tools that allow them to evaluate and have information from the clients without being face to face with them, and that saves time so that they can connect with clients in different ways.

[00:04:15] And there's something that's also very important, that we assess risk centrally. What that means, it might seem like a lot of financial jargon, but what that means is that it's faster for a business advisor to give a loan if the evaluation has already been done centrally, so that's something that our clients value a lot.

[00:04:36] So I've given you a bit of context on how microfinance works, because I know that for many of us it's very different to the traditional bank branch that we might see in the high street of the city that we live in, and because it is really relevant when we are understanding the client, to get to know what that go-to-market strategy is, in order for us to see different areas of improvement for CX.

NPS is just an average

[00:05:03] So in Mibanco we work with the net promoter score. I know there's a lot of methodological debate on whether that should be the indicator to measure CX or not. In the case of this presentation it really applies to any CX indicator that you're looking for, and we have chosen NPS score and that works very well for us.

[00:05:26] One of the things that I think is a lesson that we took when we started working with the NPS score, and that was really recently, it's been like three years, is that it's just an average. So if you're looking at numbers, it might be 14 or 66, but if you really don't segment that number you really are not discovering much. So many presentations might say, well, the number is X, or the number went from A to B, or we have more or less promoters, or less detractors, more detractors. That's the sort of analysis that we wanted to avoid.

[00:06:03] In order to do that, what we did was first of all try to go beyond the average and identify different kinds of segmentation that connected numbers and surveys with who our client was and at what stage of our business they were. It might sound very anecdotal, but one of the first segmentations that we did, and sometimes it's natural when you're analyzing data, was, well, let's see how our NPS score varies according to client gender.

[00:06:37] Today more than 50% of our clients are actually women entrepreneurs, and when we took the data we saw that there was a plus 10 point difference between the NPS score of clients that were women versus men. And when we had that discussion, jokingly somebody said, well, what we need to do is to have more female clients. So that was clearly a joke, but it shows how these kinds of simple cuts are not necessarily going to lead you to a discovery that leads to product design and delivery.

[00:07:15] So we tried to understand why that happened, and the answer really lay in understanding how the NPS score really changes according to the client profile that we have and the levels of recommendation that our client has. Because recommendation is for us key, because we are heavily reliant on our clients recommending to their peers that they can get a loan at ours and get into the financial service through word of mouth. That's really relevant for us.

[00:07:50] And we asked ourselves a lot of questions. So the first one was, why does this happen? What are the underlying pain points, what incentivizes[?] these clients? And what we found out was that we had a higher proportion of female clients that belong to the clients that recommend us the most. Today, the clients that recommend us the most are the clients that have smaller loan tickets, clients that we are their first encounter with the financial industry, the first bank that they've banked with, clients that have smaller businesses.

[00:08:29] So if you see all our clients, there is a higher proportion of women that have smaller businesses. And we understood that this is not a biased gender conversation, but it's something related to the profile that they have, that is in higher proportion female when it comes to smaller businesses, that on average recommend the most.

Analyzing unstructured feedback, and the pandemic's shadow

[00:08:53] There's another thing that for us is very relevant when we do segmentations. The first one, as I mentioned right now, and then we'll go into a bit more detail, is the business indicators behind the profile of each of our clients. And the second is to analyze unstructured information.

[00:09:10] When we do our net promoter score survey to our clients, we usually ask the question and then ask why, and that provides us with a lot of text data, where it's really rich because you can understand different ways in which clients communicate. And for us, one of the things that took a really, really big toll on our connection with clients, as happened not only in microfinance but in many industries worldwide, was the pandemic. It was something that the analysis of this unstructured data allowed us to see that this was not a one-year issue, but it was something that still can be seen in many of the comments, both of our clients that are promoters and the detractors.

[00:09:57] And for us in Peru, we had one of the highest death tolls related to the pandemic. We had one of the strictest social distancing policies. It took like six months where we couldn't really leave home. So that took a toll on the face-to-face connection between our loan officers and the clients. Analyzing this information really helped us understand that for our clients proximity is key. Even though we're working in a hybrid model, what they want is definitely convenience, but also to have us really close by through different channels. Analyzing this data allowed us to see how the pandemic and that distance was so heavy, and it took so many months and actually years in the impact that it took in our levels of recommendation.

[00:10:57] So these two things are key at a high level. The first one is, okay, go beyond the average. Check the relationship between the NPS scores and the business indicators behind that show you who your client is, where they are in their stage of the customer journey. And the second is to understand what they say, what they feel and what's a concern for them, by analyzing unstructured data.

Two insights that set the CX strategy

[00:11:25] So I'll continue now telling you a bit about a case study where we managed to take customer insights, design and pilot an initiative and then scale it, deliver it and measure impact, by using information related to customer experience, specifically using our NPS score.

[00:11:44] So we had two key insights that set our CX strategy. The first one was that our NPS scores, or level of recommendation, really went down as months passed from the moment when we gave our client a loan. Actually the specific number is that after six months the recommendation levels from our clients halve.

[00:12:14] So that showed us that we were heavily reliant on giving our clients that loan. It was a magic moment, because we were giving them something that was important for them to grow their business. But we didn't really want that necessarily to happen. We didn't really want to only rely on that moment of giving a financial product. We wanted to be with the client, or close to the client, and have a higher level of recommendation after that, in every single interaction that comes afterwards, like paying your loan or being advised, asking for help, etc. So that was the first insight that we got by cutting that NPS survey and joining it with business indicators, in this case the moment, or the month, that they had received a loan.

[00:13:01] The second insight that we had was that contact with the business advisor has a very, very strong correlation with the NPS scores. So a client that knows their business advisor, or loan officer as we call them, or has had a very recent contact when we survey them, usually has a higher NPS score. So that showed us something that we learned, or that was emphasized by the COVID context. Even though we have a hybrid model, we have a lot of channels in order to be convenient, we need to be close by to them, and that is shown by data as well.

[00:13:46] So those are two insights that we took when we both crossed NPS scores with business information, and by correlating to questions in a survey that showed us how contact with the officer is definitely a lever for recommendation levels.

[00:14:09] And what we did as well was we put every single insight together in a mixer and understood that, even though COVID social distancing was over a long time ago, the clients still remembered that that distance really took a toll. Second, as months pass from loan disbursement, clients' NPS scores go down. So that distance is also not only caused by COVID, that's already something in the past, but it's something that when months pass from that relationship or that loan, clients feel that they no longer recommend us because they don't feel that we're close to them. And thirdly, the clients who have closer contact with that loan officer recommend us the most.

[00:14:52] I think that made us feel that even though CX is at the heart of microfinance, it's not necessarily an explicit component of a monthly advisor routine. It's something that we identified that needed to be part of their day-to-day interactions, beyond providing resources to our clients.

CX visits: the initiative

[00:15:10] So what we tried to do is, how can we be close to the client beyond that moment when we give them their loan? What we identified and designed was an initiative that we launched back in 2021 that is called CX visits. We ask every single business advisor or loan officer to visit 10 clients a month, focused on building a relationship. They can definitely have conversations on their loans or any other business offers that they can have. But what we do centrally is, just as happens in the commercial world where many of our advisors receive sales leads, we gave them CX leads.

[00:16:01] Those leads are prioritized according to the moment a client is in their journey. So we usually focus on those clients that have already received their loan and are in that moment between the two disbursements. We integrated this in a visit choreography so that we could gather feedback as well from the client in this visit. It's not always visiting for visiting's sake, it's to ask them for feedback and ask them if everything is okay, if they need something else beyond a commercial offer or a commercial need.

[00:16:39] The leads help us a lot. They're prioritized according to the different profiles of clients, their different stages, and it's something that changes a lot according to the priorities that we have and the information that we see in our NPS score survey. And we integrate it as well not only in their day-to-day routine but also in their digital education. So they have the client information online so that they can see it when they're in the field or in the branch or at home. Many of those visits can be done remotely as well if that's convenient for the client.

[00:17:19] So today, after almost two years, we do more than 40,000 visits a month. That's a lot of screen time with our clients beyond that moment when we give them their loans. And these are a couple of pictures of how these visits are done. Our clients are entrepreneurs, many of them work in their own business, many of them work at home as well, so we do these visits in the place where our client feels more comfortable.

Measuring that it actually happens

[00:17:53] And for us what was key was, we launched this initiative, how do we measure impact? One of the key challenges we had was to show different stakeholders how this was actually being done, to ensure that it was happening. How do we ensure that these visits are being done and those 40,000 visits are real, and that the client knows that they're happening?

[00:18:16] So what we decided to do was to ask for client feedback. What we did was ask two questions, in order to have two leading indicators that would show us that this was generating impact and that it was being adopted by both advisors and clients. We asked our clients, after the visit that was registered by the loan officer: did your loan officer visit you during the last week? And we also asked them to rate the interaction. So we could see and compare different profiles of clients, what their evaluation was, or if there were any differences between the evaluation according to different types of loan officers.

[00:18:56] And we not only asked for this information, but we very proactively communicated it to the leaders of these loan officer teams, so they can understand if maybe a group was not necessarily visiting or had other priorities that month. We connected it as well with geolocation, so that we could know and ensure that our loan officers were visiting, and give them as well the tools so that they could know where the client was.

What we learned

[00:19:30] So this helped us a lot, and what we managed to learn from this was, I think first of all, that intermediate time feedback is key to ensure adoption. You're not going to see an impact in the NPS score right away. Operational controls are important to ensure consistency of the visits. We don't necessarily want a one shot. We have almost two million clients and we want to be close to them every single month.

[00:19:55] Client feedback tends to be unstructured and hard to measure with visit surveys. We have had several iterations on the survey that we have with advisors, and face-to-face surveys are sometimes very difficult to control, they are very unstructured, so that's been a challenge as well.

[00:20:17] We need a lot of patience and time. Our NPS survey is relational, so we ask a survey to represent the group of our whole entire stock of clients. So you're not necessarily going to see an impact month to month or quarter to quarter. You need to wait for some time, because it's a habit. It's not only one visit, it's a visit every single month for many of the clients.

[00:20:44] Integration with methodology: it has to be part of the routine. Just as collecting is part of a loan officer's routine, or selling is part of a routine, CX is also a habit. The lead generation logic is key, and loan officers need to trust the process and understand why they're visiting a client. It's not just visiting for visiting's sake.

[00:21:05] Loan officer feedback is key. There are some clients that actually don't want to be visited, and we've been very respectful of that in our model as well. Sometimes they are very busy, or sometimes they might be very risky and don't necessarily have a connection with us that is that strong. So we have that feedback as well, and we don't visit a client that does not want to be visited.

[00:21:30] On-site and remote visits work very well. During the beginning of the year we had a lot of social protests in the southern part of the country, and that was unfortunately something that impacted our teams and our clients, and there was a lot of distancing and protests, so for safety a lot of people stayed at home. So having remote visits helped us to adapt to external situations that are beyond our control. We don't want to generate distance even in situations that are complicated. We want to be even closer with our clients. And leadership involvement is also key for adoption and habit formation. This is not a one shot, it's something that is methodological. It needs to be in everyone's DNA in a way.

Impact

[00:22:24] So, impact. We had a very strong reduction in client complaints related to contact with the loan officer, and that we saw over the last year in our NPS survey. The NPS score for clients that have been visited is more than 10 points above our clients that have not been visited, and we've had significant results when we've seen the stock of clients over time.

[00:22:50] And we've also seen something, and this goes back to the initial insight, which was, we want our clients to recommend us a lot, not only at the moment when they've received their loan or the moment close to that disbursement, but we want them to recommend us throughout their journey with us, beyond the financial sale. So we've also seen an increase for clients with loan disbursements more than six months beyond the moment they received their resources or their loans from their advisors.

[00:23:23] So these impacts are real, but as I mentioned before, and I'll close with this as well, it really sometimes needs time. So these intermediate indicators, like confirming visits and seeing the number of registered visits that clients confirm, help you to see the evolution and adoption.

Key takeaways

[00:23:45] So, key takeaways. I can't really stress this enough: it really takes time when you're working with a go-to-market model that is hybrid and combines the digital world, different channels, and face-to-face interaction. It really takes time and sometimes patience to see metric impact when designing and delivering new services in this particular environment. It's not necessarily like an app with high transaction levels where you can tweak something and then you'll see the impact right away. This is different. We're talking about building relationships over time with a team of more than 4,000 business advisors nationwide.

[00:24:32] Intermediate CX levels are key to know how well a CX project is performing. You're not necessarily going to have an NPS impact right away. You need to understand what other indicators show you that you're moving the needle.

[00:24:45] NPS scores are just an average that needs to be segmented with client and business data. Understanding how your business works is key for product discovery, because that will allow you to get to know your client better and get to know more about the areas of improvement.

[00:25:04] Text analytics is really helpful to turn hypotheses into concrete client feedback. I showed you how we track the sentiment related to COVID in both promoters and detractors, and that's been very helpful to help us see how that's been reducing over time, but it's still there and we need to take that into consideration when we're designing.

[00:25:25] Data is key to set a CX strategy that helps prioritize initiatives. One of the hardest tasks when you are working in CX is that you really want to fix everything, but that's not necessarily possible in the short term, so prioritization is absolutely key, to see where you're going to get more impact and help more clients.

[00:25:48] And lastly, including other questions in NPS surveys allows us to understand client behavior. It's been really helpful for us to not only have a survey that includes the NPS core score, but other six levers that are key for us, like for example the last contact that a client has had with their business advisor. So that's all for us today. Hopefully we have some questions and we can see them right now.

Q&A: what else the visits surface

[00:26:18] Rory: Excellent, thank you very much Carolina. I really enjoyed that. That was a great example, with the data to back it up, of how you can tackle net promoter score, see the trends and figure out how to solve it. And one thing that really came up to me as I was going through that is, that's a wealth of information now that you're getting with those visits. So other than just the NPS score that you're seeing, are you seeing information flowing throughout the organization, because each account manager is meeting 10 customers and they're really understanding what their challenges are, and trying to come up with ways that you can improve not just via the visits but also improve your product?

[00:27:02] Carolina: Sure, sure. And it's definitely been a challenge to centralize that information from every single visit that happens. We have a registration form in the business advisor app that helps to see what the things they have. And we are trying to tweak it constantly, because sometimes our clients are busy and they say, no, everything's okay, thank you. So we've tweaked it over time as well, so that we can learn what are the things that they want to know.

[00:27:33] Carolina: It helps us as well to understand, or have feedback, not only on service but also on commercial improvements. There might be a client that has absolutely no problem related with the service that they have with the business advisor, but they might have a complaint on something else. So that really helps us understand and have that feedback. But it's something that always shows you different things according to different contexts, and also according to the different geographical regions of our country. So we try always to understand if clients that belong to a particular region of the country are feeling something different than others, and gather that and then take that insight and say, well, maybe we need to tweak something over time.

[00:28:33] Carolina: So yeah, definitely it's something that gives us more information. And for those who are more on the methodological side, one of our aims is to exploit that more, because it's also a cheaper way to get client feedback than surveys. So that's something that makes our voice of client methodology stronger but also more efficient, so we can have more information in a more efficient way.

Q&A: did it result in more business?

[00:29:03] Rory: And just a devil's advocate question now. Did this result in more business? Because I could see somebody arguing, well, our NPS is high at the point of making the sale, so what does it matter if it dips off?

[00:29:20] Carolina: Sure. So that's usually the first question that gets asked: what's the business impact? We did the analysis, especially at the beginning, because we needed to convince a lot of people inside of the organization that this was something that was generating not only CX impact but also other impacts over time. And we did see that we had a higher percentage of clients that got a second loan afterwards, after the contact, controlling by type of client, right? Because you have different risk profiles, so you might not really give somebody a loan at that particular time because they're not necessarily going to be able to pay it afterwards. So we're very careful on keeping our clients in the financial industry.

[00:30:11] Carolina: So we did see that, in a sort of, you're getting a visit, you're being closer to the client, you're increasing the NPS score, and you're also ensuring that that client sticks with you as a business advisor and then asks you for another loan over time, when it's applicable for them. So that's been helpful.

[00:30:36] Carolina: And we've also taken advantage of visits to, as I mentioned, we have screen time, right? So it's also a way in which we can communicate things to clients. Many of our clients, even though digital is growing really, really strong, many of them are not digital with banking yet. So it's not like in other industries where you're just doing a lot of Google ads and Facebook ads, that will not reach necessarily many of our clients, especially those who are in rural areas where connectivity is not necessarily that strong.

[00:31:09] Carolina: So having that network in that space, beyond commercial, beyond collecting, really helps: if there is something that we want to communicate, that's the space. So we're leveraging on getting not only value as we're getting more business, but also value in terms of other services that we might want to, or things that we want to communicate in the relationship over time. For example, if we want to educate our clients on a topic, that might be the half hour to do that. Being very careful on times, and on what the client also expects of you.

[00:31:55] Rory: Brilliant. No, that's a fantastic outcome, as well as a fantastic generation of information. That actually brings us to time, so thank you once again for sharing.

[00:32:06] Carolina: Thanks to you for having us here.