Why UX Research Doesn't Need To Break The Bank

Shilpi Sinha at UXDX Community: Nordics. Video: https://youtu.be/PNYcnWb8U7g

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

The AR game at IKEA and why it was complex

[00:00:00] Hi everyone, I'm Shilpi Sinha and I'm a UX researcher at Pandora Group, and today I'm going to talk about why UX research doesn't need to break the bank.

[00:00:11] I have been a UX researcher at the Danish jewelry company Pandora for the past six months, where I have joined them on their digital transformation and omnichannel journey. Before Pandora I worked at the Swedish furniture retailer IKEA, where for the last year and a half, in the middle of their digital transformation, I was the acting global head of UX/UI, with the responsibility to build up an in-house UX/UI team from the ground. I also did some hands-on work on some products as a UX researcher, and one of these was an augmented reality game.

[00:00:50] So IKEA wanted to explore the opportunities of creating an AR game that would be played in store, and this was a rather complex product. First we have this immersive technology where there's still a lot to learn in terms of best practices. We're not only designing for a user-friendly mobile experience, we also have to keep in mind the technical requirements needed for the app, because augmented reality apps tend to eat a lot of battery.

[00:01:21] And then of course we wanted some substance in the form of gameplay and storyline that at the same time was aligned with our company values. So we couldn't just build a fun game, it had to be relevant in the context of IKEA.

[00:01:41] When designing for AR you need to consider the physical environment, which in this case would be an IKEA store, so there would be a lot of rooms, objects, as well as other people, and we needed to make sure we wouldn't cause problems that would disrupt the customer's shopping experience.

[00:01:59] And last but not least, all of this was being created to be put in the hands of children aged seven to 12. One big challenge here is obviously the age range, because the development of a seven-year-old is very different from a 12-year-old. In addition to designing for children, we're also designing for the parents and need to consider their needs and concerns. And of course there are a lot of legalities, not just for the product itself but for the research, which makes it a lot more time consuming and costly when doing research with kids.

Going back to basics: generative research

[00:02:38] So with all of these challenges and unknowns, the only way to start going forward was by going back to basics and start with generative research, also called discovery or exploratory research, as early as possible.

[00:02:56] This included doing a qualitative survey with parents in our own network that fit the target group, so it didn't require a large sample size, nor were we expecting a statistically significant result. We just wanted to get a high level understanding of parents' experience of shopping with children before diving in deeper. One thing we learned from the surveys that surprised me, who doesn't have any kids, was that parents weren't especially concerned about their children getting lost or breaking things or getting injured in the store, which I had originally thought.

[00:03:35] We also did some ethnographic field studies and some guerrilla research, which I will go into now. So we went off to the IKEA store where we did two things. Some guerrilla interviewing of store co-workers, which meant we didn't need to spend any time or money on participant recruitment or incentives. The co-workers shared struggles around having to monitor kids from going into restricted areas or climbing onto the shelves in the warehouse. So while kids running around wasn't much of a concern for the parents, it definitely was for the store co-workers.

[00:04:16] We then did a field study where we moved from area to area and made observations over time of the activity in that area. We wanted to learn what different family constellations there were, the dynamics and interactions in those, and about the kids' behaviors throughout the store journey. What we observed was a very dynamic store journey for the kids, where they were clearly more engaged in shopping and browsing with their parents in some areas of the store, while not so engaged in other areas.

[00:04:47] We also learned from the survey with the parents that while shopping with kids could be stressful and require more planning, parents did enjoy getting to know more about their children and their preferences. So we already knew that we didn't want the game to be too immersive for safety reasons, but now we had learned about how this could also affect the shopping experience for the entire family as well.

[00:05:16] So as we started to have some design pillars in place for the game, we needed to start testing, and as I mentioned before, recruiting kids can be a bit time consuming, with both legal and logistical considerations. So we did some guerrilla testing where the team members who had children tested with them at home, and here we also observed that kids had a tendency to run while playing.

[00:05:44] So with all the attitudinal and behavioral data we had collected so far, we concluded that we wanted the game to not work in certain rooms including restricted areas, and that it would sense if you were running while playing and it would stop. We also wanted it to be easy to pause and resume the gameplay without losing progress, so the kids wouldn't be too distracted from the shopping experience.

Comparative analysis

[00:06:13] And on the topic of going back to basics, I want to talk about doing comparative analysis. It's a little bit like comparing apples to oranges. If you sell apples, a competitive analysis would be looking at other companies that sell the same products, while a comparative analysis looks beyond a specific product or company.

[00:06:38] Here I wanted to use an example from something I worked on at Pandora, which was a gift finder quiz. So we did the usual competitive analysis where we looked at other jewelry and fashion retailers, but we also did research with a product agnostic approach, to learn about gift finding in general, to see if we could find common patterns, overarching themes and underlying needs. We wanted to learn about the users' mental models and what conventions were familiar to them in other contexts that we could incorporate in our own product.

Building the research capability rather than being it

[00:07:18] Another thing that I've learned is really important is that I, as a researcher, don't see myself as the research capability, but rather as just a person to start building the research capabilities in the organization.

[00:07:39] So I've been working with UX in companies where the design maturity has been quite low to begin with, and not a lot of resources have been set aside to do UX research. If we go back to the AR game at IKEA, we were a small, so-called two-pizza team. This two-pizza team rule I believe was coined by Jeff Bezos of Amazon, who said that a team should be small enough so they can be fed with two large pizzas.

[00:08:12] Seeing that this product was much more complex than if we were just building a website, for instance, a lot of these pizza slices had to be reserved for the technical development of the product, and not a lot was left for UX. So in order to make ends meet and conduct quality research, I needed to maximize insights and minimize having to negotiate and justify conducting research, which isn't uncommon, especially as it is difficult in general to measure the ROI of generative research.

[00:08:51] So right from the start, at our kickoff meeting, I had prepared a presentation about UX in the context of an augmented reality game. I needed my team to be my partners in the research, and for that to be possible I gave them an introduction into some of the areas we would need to do research in, and why. This was to give them a foundational understanding of important topics, such as the physical and cognitive development of children, so that we could have a more collaborative approach to the research, and when insights and recommendations would be presented it would hopefully make more sense to them.

[00:09:35] If we jump over to Pandora again, where I had joined a small digital experience design team as the first UX researcher, here my goal again was not to be the research capability but to build the capabilities within the team to plan, conduct and interpret research. I work with some really skilled UX designers who all have some experience doing research, but UX design is a very broad subject and everyone has different experiences and sets of skills at different levels.

[00:10:10] So one thing I'm doing to make sure I don't go through subjects people already know is a self-assessment exercise, where we put up a bunch of words and topics and do dot voting to identify areas my team members could use more deep diving sessions into. And then I run sessions based on those gaps, so that the entire team is being upskilled when it comes to research.

[00:10:38] And all this talk about building the research capabilities by including and involving the team, it's not just to alleviate my workload. It's because when people feel included in the creation of something, they will also feel ownership and responsibility for that, which in the long run helps get stakeholder buy-in and build the maturity in the organization. I personally prefer thinking in terms of partnerships and co-creation than stakeholder management, which I feel more sounds like a chore.

Making the most of the research you already have

[00:11:14] And finally, maybe a bit of an obvious point, but working on reinventing the wheel is not completely uncommon and it's such a waste of resources. So in making sure to get the most of what you have, at IKEA one thing we did was to create a research channel. For this we used Slack, but the tool doesn't really matter here. We had a channel called Experience Design Research with day-to-day communication, sharing, asking questions.

[00:11:48] There's so much data and insights floating around in people's inboxes, private folders and in their heads, so this was a way to connect people across the company to each other much faster, faster than having to hunt someone down by going from person to person. And the channel was for everyone who worked with or benefited from data and insights about the customer.

[00:12:16] So the channel could be used to see if there's anyone who has experience in something, not necessarily from the company but from previous jobs, or to announce what you're doing and create transparency of the work, and anticipation as well of there soon being some insights to be shared. So it helps connect people to people, to insights, easier. It could also be that someone would ask a question that you yourself asked a while back and got help with, so that you can now help others with that question as well. This alleviates the pressure of people being the single point of contact for certain types of questions, and it helps resurface existing research.

[00:13:02] And one way to make more out of existing research is to increase the visibility and accessibility of it, and not have it buried in tools and dashboards that require special accesses and a PhD to decipher. Here we had monitors placed in the middle of the office with customer insights data and the live social media feed, where we could see in real time what people were saying about IKEA. So this gave everyone the opportunity to get a snapshot of the customer experience and brand perception of the company, regardless of what team you worked in. And similarly, I have seen other companies put up posters of their personas or journey maps around the office, and I think that's a simple but great way to make the user experience and needs more visible and tangible for all co-workers.

Building a network outside your industry

[00:13:57] And finally, one of my favorite things to do is to extend to people outside my company and create an external network for sharing knowledge and insights. One example of doing this is when I worked with UX research for a recommendation engine at IKEA. I was doing some comparative analysis of other recommendation engines and found Netflix to have a very interesting one, so I reached out to them, and me and a few of the engineers on the team met with the team of Netflix to exchange learnings.

[00:14:33] Jakob Nielsen said that users spend most of their time on other sites, which I think is true for all: your customers spend most of their time with other companies. Which is why I find it so valuable to extend to organizations outside your own industry or sector who are working with similar challenges, features, or have the same target group as you, and then you can have a mutual exchange of insights that help you better understand your customers. And this relates back to what I was saying earlier about comparative analysis, and to look to build a greater understanding of underlying needs and motivations of people in order to build better products.

Summary

[00:15:19] So to summarize what I have found to be key components of gathering insights through research when resources are scarce. Doing generative research such as ethnographic field studies or comparative analysis in addition to competitive analysis, and doing it early as a means to mitigate risks. If you are juggling a lot of requirements and unknowns, not conducting research until after you've started building the product can prove to be a very expensive lesson.

[00:15:57] Now if you are someone who does research in your company, try to build the research capability of your team. Train them in assessing research needs as well as planning, executing and interpreting research. Have your colleagues be your partners in research. And if you are not conducting research today, find out who does and see how you can contribute to being more data and insights driven.

[00:16:23] And finally, create infrastructure that allows you to make the most of existing data and insights inside your company today, and build a network outside your industry or sector to share insights and ideas about your users as well as about research practices in general.

[00:16:43] So that was my two cents on how research can be done without breaking the bank. Feel free to reach out or connect with me on LinkedIn, and thank you for listening.