Change Management to Enable Company-wide Experimentation

Ruben de Boer at UXDX Community: UX Research - Facts & Feelings. Video: https://youtu.be/f0gLGH5ysPQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Change management to enable anything

[00:00:00] Thank you, and thank you for having me. It's the second time I get to present for the UXDX community. The first time was four years ago in Amsterdam at a live event, so very happy to be back and see that this amazing community is still thriving, because I'm absolutely enthusiastic about the mission and the vision of this community.

[00:00:22] The title for my talk is change management to enable company-wide experimentation. When I was preparing the talk, with the community in mind, I realized I could have also called it change management to enable outcome driven working, or change management to enable continuous discovery, or change management to enable anything to create the best possible digital products.

[00:00:51] Change management, we can use it to motivate colleagues to do something and to help create better products. In this presentation I'll refer to experimentation and A/B testing, because that is what I do and that's what I love, but know you can apply the theories in this talk for similar goals, including your personal growth, but we'll get to that. Whatever your goal is when trying to change the culture, mindset or way of working of your team or your organization, you must understand that a large proportion of your work should be dedicated to change management. But change is hard, change can be difficult. So the next 25 minutes you'll learn something about change management, change management tactics, and things you should do and should not do to accomplish your goals.

[00:01:42] Let's first briefly introduce myself. This is basically me on one slide. My name is Ruben and I've been in the business of experimentation and optimization for over 14 years. Today I'm the lead CRO manager and consultant at Online Dialogue, which is an experimentation-only agency based in the Netherlands, and I'm the owner of the company Conversion Ideas. With this company my goal is to help people learn and excel in conversion rate optimization and experimentation for affordable prices with online courses, and now have over 12,000 students worldwide.

[00:02:19] I'm also an international keynote speaker, I post a lot on LinkedIn, I appear on regular podcasts, and I love optimization, not just for my clients, not just the business, but also my personal life. For instance, I love sports, I do sports every single day. I also love the city Amsterdam in the Netherlands, that's where I live, and I love Amsterdam for everything it has to offer, but it did not seem very appropriate to show a picture of me with a beer at a festival in Amsterdam, therefore for the third picture I chose a picture from vacation while enjoying nature and hiking with my girlfriend Zoe and my dog Happy. So that's me in a nutshell.

The Sunday morning run: system one and system two

[00:03:00] So, change management. Let's start with a story. Imagine you want to make a lifestyle change. This summer you're really going to be in great shape with a great body and, of course, a nice six-pack. Now it's Sunday morning at 7:00 am and an alarm clock wakes you up. You decided to set an alarm clock the night before to get out of bed early on a Sunday morning and go for a run before having breakfast with your family.

[00:03:33] And now that alarm clock rings, something interesting happens. One part of you knows you have to get up and take that run to get in shape. This is the same part of you that set the alarm clock the night before. But as you know, there's another part of you. It wakes up in the darkness of the early morning in the weekend, still feeling sleepy, with a warm blanket over your body, your head resting on the comfortable pillow, and this part of you wants absolutely nothing to do with waking up. All it wants to do is to turn off that alarm clock, turn around and get some more sleep. And so the struggle begins.

[00:04:20] Now, you might recognize the situation. Maybe this is a struggle you experience every single day, when your alarm clock goes off and we have to go to work. Or you might recognize this from a similar goal, for instance when you're trying to lose weight, save more money or drink less alcohol. So what is happening here?

[00:04:41] Well, most of us, or some of us, will know these two parts of us as system one and system two, as described in the book Thinking, Fast and Slow. So let's check it out. System two is a metaphor for our conscious, rational brain. This part of our brain can reason, it can think in the past, present and future. It knows what is good for us and it knows we have to make an investment now for a benefit in the future. This part of the brain knows we have to get up for that run so that later this year, this summer, we are in great shape. This is also the part of us that set the alarm clock the night before.

Educating system two

[00:05:28] So for behavioral change to succeed, system two has to know how to change. It has to be smart enough. And to make system two smart enough for change, to have a different way of working, we must educate our colleagues and show them the importance of experimentation and validation and make them understand it.

[00:05:52] So you could show proof that experimentation works. When you start in an organization you can begin with low hanging fruit and get some quick winners and show the proof that it works. You can also give presentations and workshops about whatever goal you're trying to achieve: continuous discovery, experimentation, everything. You can share the results and insights from experiments. And you can test the ideas of your colleagues, and especially contradicting ideas I find most interesting, because if ideas are worth a discussion in the team and especially with higher management, it is definitely worth testing. And whatever the outcome of the experiment, some people will be surprised and everyone will understand the importance of experimentation.

[00:06:46] Now unfortunately, just making system two more intelligent does not result in lasting change. Otherwise it would be easy and I would not be here giving you this presentation. And this is obvious, right? We know this, we cannot read a book and change our entire life, or change our way of thinking or a way of working. But even though it's obvious, I see it still happen a lot in organizations, possibly with devastating results.

[00:07:15] So this is what happens. You have a specialist, let's say a CRO or experimentation specialist, and this specialist is very enthusiastic about his way of working, and he's trying to convince his colleagues to work the same way. But by doing so he's a bit demanding, he slightly forces his way of working onto his colleagues and onto his manager. And this is accompanied by convincing others using mere facts, and facts only target system two. So the specialist explains how, with this new way of working, we get better results for the organization and our customers, and we have to work outcome driven, and our competitors do it, and Booking and Facebook do it.

[00:08:03] But the colleagues, they're not enthusiastic. The colleagues think the specialist is pushy and demanding and not showing any interest in their way of working. So as a result they start ignoring or even resisting the desired behavior, they start resisting experimentation. Now this again makes the specialist think that the colleagues are stubborn, that they don't see the benefits and that they do not want to change. And this makes him more demanding and pushy, and before he knows it he finds himself with his colleagues in a very negative cycle, in a very negative circle, and it's really hard to break this.

Motivating system one

[00:08:50] So to change behavior and way of working we need a lot more than educating system two. So let's check out system one. System one is a metaphor for our subconscious brain, our primal brain. This is the strongest part of our brain. These are our reflexes and instincts. This part of the brain, it cannot reason, it cannot think in the past, present and future. And a rule of thumb is that 95% of our decisions are made by system one. So this is the part of us that loves staying in bed on the Sunday morning. It doesn't care at all about a well-trained body somewhere in the future. All it cares about is the here and now.

[00:09:36] So for behavioral change to succeed, system one needs to be motivated. So make experimentation as easy and as small as possible. System one does not like big changes. So don't bother your colleagues with statistics, crafting correct hypotheses, advanced interview techniques, etc, etc. It's something we're very passionate about, but for colleagues it is something difficult and they are not so enthusiastic about such things.

[00:10:17] You also want to give sufficient autonomy to your colleagues and your team. System one wants to stay in control. And you want to make experimentation accessible and make it easy for colleagues to hand in their test ideas. System one does not like to put in the extra effort. And this is experimentation, of course, but again the same accounts for other desired behaviors. If you want more discovery by the team, make it easy and start small.

[00:10:48] Well, unfortunately, motivating system one is difficult. Change often fails because system one is not motivated enough and not working together with system two. So to create lasting change, system two should be smart enough, we should educate system two. System one should be motivated and work for us, and this one is especially important, system one to work for us, because it's the strongest part of our brain. And system one and system two should work together. This creates lasting change. And to do so we can tweak the environment.

Tweaking the physical environment

[00:11:39] First up is the physical environment. Working in online experimentation, we continuously tweak the physical environment of our website visitors. With A/B tests, smoke tests or ad testing, we try to change their behavior by tweaking something in their physical environment. So for successful behavioral change we can also tweak the physical environment.

[00:12:07] Let's check out our example once more, for the run on a Sunday morning. Students in the United States invented this alarm clock and named it Clocky. Clocky, like any alarm clock, when it goes off it makes a lot of noise. However, as you can see in the picture, Clocky is a clock on wheels. When it goes off it makes the horrible noise, but also its wheels start spinning, and it rides off your bedside table, racing through your room and still making that noise.

[00:12:39] Now this morning system two wants to get out of bed to go for that run, and system one now also wants to get out of bed because it wants to stop that horrible noise coming from Clocky. So you get out of bed and on your hands and knees you try to catch that alarm clock, and by the time you catch it and turn it off you're out of bed, awake and ready to go for that run.

[00:13:04] So if you want to change the culture and way of working of your team or organization, you must tweak the physical environment. Create Clockys in the office environment. You can send update emails with insights from your experiments. You can apply gamification, like a "which test won" competition. You can create accessible data dashboards and perhaps place monitors with data in the office. You can share a form through which colleagues can easily hand in their test ideas. You could also add a big feedback button on your website and digital product targeted to the IP address of your office. You can have a Slack channel to share successes and failures. You could have a fail board so that the actual acceptance of failure becomes better.

[00:13:55] Or you can have a fail trophy, and this one's quite nice. I read this one in an article, that Amazon has a fail trophy. Whoever breaks the website badly gets the fail trophy for a week, and this person gets to place the trophy on their desk. And this trophy is considered a badge of honor. This means this person did something so bold that it messed up the website, and this person will get praised for it throughout the week and receive high fives even from higher management. I thought this was an excellent example of a tweak in the physical environment to accept failure and take risks.

Tweaking the social environment, and being a likable authority

[00:14:35] Well, besides the physical environment we can of course also tweak the social environment. Working on websites and digital products we know this all too well: ratings, reviews, testimonials, success stories, we find social proof everywhere. And of course we can use it for cultural change, behavioral change, as well.

[00:14:55] When thinking about the Sunday morning, coming up with an example is quite easy, right? You can meet with your best friends at 7:30 am in a nearby park to go for a run together. Now when your alarm clock rings, system one does not want to let that friend down, and gets out of bed as well.

[00:15:16] So for behavioral change in your organization, tweak the social environment. Involve your colleagues, create a community of enthusiasts and promote experimentation. Host brainstorm sessions with colleagues. Celebrate success and failure together. And have colleagues watch usability tests together in a room. I always love this one. Usability tests get a lot of valuable insights, but also by getting people together in a room and seeing someone actually use their product, that's when they start understanding that optimization, validation, experimentation is very important.

[00:15:52] What is also very important is you. You are probably the most crucial person in the social environment when you want to change the behavior and way of working of your colleagues. As we have seen, you can facilitate or completely block organizational change with your beliefs and behavior.

[00:16:15] Most of you will be familiar with this book, Influence by Cialdini. In his book he describes the six principles of influence. These are reciprocity, commitment and consistency, social proof, authority, liking and scarcity. Two of these apply to you as a person in the social environment. If you want behavioral change to succeed, you have to be a likable authority.

[00:16:44] So be passionate but humble. Be approachable. Show interest in others. And never tell people they are wrong. So build trust, and never tell them they are wrong. System one hates it when someone tells it it has done something wrong, so always respect the opinion of your colleagues. And most importantly, help your colleagues achieve their goals with experimentation instead of only trying to achieve yours.

Rules, norms and accepting failure

[00:17:19] Two more tweaks to go, almost there. Let's continue with rules and norms. Rules and norms can make certain behavior mandatory. If we check back on our run on a Sunday morning, it would be a bit weird if your partner or government set the rule that you have to wake up on Sunday at 7am and go for a run. That would be absurd, unless your partner is very upset with your big belly and set you that rule to go for that run, but that's out of scope, I guess, for this presentation.

[00:17:50] But if you think about it, what made you run as a kid at school during physical exercise? It was your teacher, and you complied. And when we think back about the COVID period, the COVID situation, we have shown very different behavior due to governmental regulations. And the same applies for the workplace: arrive on time at work, clean up your desk at the end of the day, come prepared to meetings, run an A/B experiment for every change on the website, or interview one customer every single week.

[00:18:27] So norms and especially rules generally come from higher management, therefore it's essential to involve them as soon as possible. You need their support when you want to change the way of working of your team. So you can inform them with results, insights, or share business cases with them. You can involve them, you can test their ideas, invite them to a brainstorm session or a usability test. And once you have a higher manager on board, you can craft a plan together to get experimentation higher on the agenda in the organization. If you are a higher manager, you can also set the rule that ties experimentation to your personal assessment.

[00:19:12] Looking at norms, we already discussed a little bit, accepting failure is a very important norm. Failure should be seen as part of the learning process. When management accepts this, it shows the organization that failure is not something to be scared of. Instead, it is necessary to achieve goals.

[00:19:35] And we know failure is important, right? We read about it a lot, we hear about it a lot, I'm telling you now during this presentation. But how do we get there? And the answer is simple, we use this model. We already learned about Amazon with the fail trophy. There's also a nice example for a tweak in the social environment at Online Dialogue. Every last Friday of the month is so-called Oops Friday[?]. From 4pm to 5pm on this Friday everyone stops working and shares their biggest oops[?] of that month, and the rest of my colleagues respond with applause, laughter and empathy, and together we learn. We learn from our failures. And this makes it a very safe environment to fail and thus take productive or educated risks.

Rewards and incentives

[00:20:31] The last tweak, rewards and incentives. Rewards can make the new behavior stick. It's pairing something we want with something we have to do, and rewards can come from within, from our physical environment or our social environment.

[00:20:48] If we look at our example for the Sunday morning for one last time: when we go for a run our body produces endorphins and these make you feel great. But also what could make you feel great is if you put a check mark on your goal list after the run. And perhaps after the run, when you are in the shower, your partner makes you your favorite healthy breakfast as a reward. And all these rewards can help you go running again next week.

[00:21:21] So if you want to change the way of working, have the right rewards and incentives in place and make it fun. So assign nice rewards to a "which test won" competition or an idea competition. Celebrate failure and success together with free food and free drinks. Show developers the positive impact they made instead of simply what they delivered. And nominations for external awards can also motivate the team, it shows they're on the right track.

[00:21:54] For higher management, tie experimentation to a salary increase, for instance. And for higher management, ensure you have consistent results. When results contradict, we will not start experimenting more. For instance, when the organization states you have to experiment but failure is not desired and is tied to your personal assessment, the rewards contradict and this leads to less experimentation.

There is no one size fits all: research and experiment on the change itself

[00:22:22] So as you've seen in this presentation, we already have quite a lot of knowledge about this model. Some of you will know about system one and system two. We tweak the physical environment of our website visitors or our digital product visitors every single day. We know about influence, about social proof, and we know we have to be a likable authority. We also know how to do research, set up a process and experiment. And we have to do that in this case as well.

[00:22:52] For instance, occasionally I get the question by someone of how they can get the board of directors more involved in experimentation. Now my answer is always the same: I don't know, I have never met your board of directors. You have to experiment. Do your research and experiment. For some boards showing business cases works absolutely perfect, for others, involve them, yet others, test their ideas.

[00:23:20] The same for colleagues, right? Sometimes they like a "which test won" competition, sometimes they want to be involved in a brainstorm session, yet others simply want update emails. Dashboarding early in the experimentation maturity works wonders for some companies; for other companies colleagues look at the data, draw wrong conclusions and act upon those conclusions. Some colleagues will like beer as a reward, others want wine, and yet others want a cake.

[00:23:47] There is no one size fits all. There's no five, six or seven steps to get to an experimentation culture and change the way of working. We have to do research and experiment. We have to do the same as we do for our products: interview colleagues, survey them, see how you can best help them, and experiment to see what works. So do your research, get many ideas to motivate colleagues through this model, set up a process, experiment and see what sticks and doesn't stick.

[00:24:24] So I hope you get started as soon as possible. Sit with your team or experimentation enthusiasts, brainstorm ideas based on this model, plan and start experimenting. We plan our experiments already, so let's also start planning our change efforts and our personal growth efforts.

Using the same model for personal growth

[00:24:43] Because we saw the example for running on a Sunday morning, but we can use this model for our personal growth much more. If you get frustrated with colleagues because they don't adopt your way of working, learn mindfulness and use the same principles. If you want to become a better listener, use this model. If you want to become a better interviewer, use this model.

[00:25:08] For instance, let's say you want to get better at mindfulness. So system two: talk to people, read a book, download the app, learn how to apply mindfulness. System one: make the change small, don't start with 30 minutes meditation, start with three minutes a day and just watch your breathing. Physical environment: a meditation calendar on a prominent spot in your house, an app on the first screen of your mobile phone, a big Buddha statue in the middle of your living room, whatever reminds you of meditation. Social tweak: have your partner join you, or if it's at work have your colleague join you. Rewards: when you meditate you get to make yourself your extra delicious favorite cappuccino. And finally rules: if you don't meditate in the morning you have to make breakfast for the whole family, or if you're not meditating at work you have to clean the office desks of all your colleagues at the end of the day.

[00:26:09] So I use this model myself for everything I want to optimize in my life, basically: my work, my sports, food, sleep, mindfulness, my own business, everything. I really believe that change is possible and you can accomplish your goals and dreams. So I hope with this presentation you get started with change management for your team and your personal growth and devote a lot more time to it, and I also hope that with this knowledge your Sunday morning now looks like this. Good luck, have fun, and please share your successes and of course your failures with me, I'd love to hear from you. I share a lot of information about this topic on LinkedIn, so please feel free to connect there. And for now, thank you for listening, have a wonderful day and keep experimenting.

Q&A

[00:27:04] Host: Excellent, thank you very much Ruben. Nice, inspiring personal anecdote at the end.

[00:27:10] Ruben: Yes.

[00:27:12] Host: Great. So if you have any questions for Ruben, please feel free to enter them into the chat, whichever platform you're watching this on. But I want to get started, because I was listening there and it felt like you were speaking to me with that cycle of pushing too hard and then getting people's backs up. That's something I have definitely been guilty of in the past. But when I was looking at all of your answers, it seemed like you needed to come from a position of authority. You needed the authority to change the environment, to change the incentives. So what happens if you're a person on the team and you don't necessarily have that authority? Now, I know you mentioned board of directors, but what about even just getting to your manager, or trying to get somebody like that involved in that cycle?

[00:28:02] Ruben: Yeah, very good, of course. And you need to match the goals of your organization and your team, because otherwise you're going to do something different than others want. But to the question, what I've tried a lot and have succeeded with is to use this model. Make colleagues enthusiastic about experimentation. This creates noise. And then at some point some higher manager, who probably has some interest somewhere about experimentation and outcome driven working, contacts me, or through someone I know who knows that manager I get to have a meeting with that person, and from there the ball starts rolling. So it's best of course to have a manager that is a fan of outcome driven working and experimentation, but you can make people enthusiastic, create noise in the organization, and then sit with a higher manager to get it higher on the agenda.

[00:28:59] Host: Excellent. Well, thank you very much. I think that's actually brought us to time, so thank you once again, Ruben.

[00:29:07] Ruben: Very welcome, thank you for having me.