How to use Secondary Sources to Guide Product Decisions

Lade Tawak at UXDX Community: EMEA Online Series. Video: https://youtu.be/0pOwRHtEL0M

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

What secondary sources are and when to use them

[00:00:00] How to use secondary sources to guide product decisions. But first, what are secondary sources? When you do research, there's primary research and secondary research. Primary research is when you go directly to the source, the people you want to find out more about, the thing you want to learn about, and you do that research by yourself. You do the interviews yourself, you do the surveys yourself, all of that. And secondary sources are the opposite: you're relying on secondhand information. Basically, it is research culled from other people's research and fact-based findings.

[00:00:40] And when can you do secondary research? I think in the first talk we saw that one of the major problems is that there's no time for research, or there may be time, but the stakeholders are not really convinced that research is necessary or that it's a good thing to do. And so you could just jump straight into doing your wireframes and prototypes and development, or using your intuition and your assumptions about what people need and all of that. Or you can use what you have available.

[00:01:13] The goal of research, building on what I said last week, is really to eliminate risk as much as possible. So think of the biggest risks in your assumptions and try to find the answers, to be sure that you're on the right path. Secondary sources can help you in finding insights, but beyond that, they can also help you with exploratory research, or inspiration. If you're not sure what problems to tackle, or what research to do, or what features people are requesting, when you use secondary sources you can find direction for new projects, new challenges, new research.

[00:01:52] And the third thing that secondary sources are useful for is competitive research. Typically when you do competitive research, most people are doing feature comparison: they have this feature, the other company has that feature but we don't have it, or these people don't have that feature and we have it. But you can go beyond that, beyond feature comparisons, when you do your competitive research with secondary sources.

Social media and app store reviews

[00:02:16] So let's get into some examples of secondary sources. Think about the last time you had a bad experience with a product. Where did you share it, besides your group chats? Probably on social media. People share everything on social media. When you have a bad experience, when you have a good experience, the most likely place you will go to first, beyond your friends, is social media. Oh, I tried this new product, it was horrible, do not recommend. I watched this new movie, it was terrible, do not recommend. Watch it, don't watch it, all of that.

[00:02:46] So this is a very good source. You can simply do a keyword search. Think about your product, your service. Think about the keywords that are relevant to your product, your service or your company, and even for your competitors as well. This is where extensive competitive research can also come in. You'll see what people are enjoying about your product, what people are frustrated about, what they would like, what they think is missing.

[00:03:09] And now more people are aware of terms like UX, so you can even search for things like "the UX of" and insert your product name, or the service that you provide, or the name of your company. Those things give you insight about what people wish your product had, what they think is missing, what they hate about the experience, what they love about it. And it gives you insight on the problem areas of your product, or maybe feature requests that people have, like, oh, I wish this app could do XYZ. And then you know that XYZ is a feature that people want, and then you can do more research about that, and so on.

[00:03:46] Another way to do this is by looking at app store reviews. Honest question, think about it: when did you last actually look at app store reviews for your product? If you've not been doing that, you should. In app store reviews, sometimes people just put five stars and "great app," but sometimes people go into detail about what sucks, what they love, what they wish it had, and so on. So you find complaints, you find compliments, you find questions, you find suggestions.

[00:04:18] The good thing about questions is, if you have a feature already and people are commenting in your app store reviews saying, does this app allow you to do this, then you know that there's a discoverability problem, a findability problem, with that feature. And that tells you, okay, this is a problem that we need to solve, if people are not finding a feature that we already have that they would like to use.

[00:04:40] Another thing is you should check all the app stores your product is listed on. Don't just check one or the other, check all of them. And a good tip, especially for B2B products, where there's not really an app store that you download and install from, is to check other review websites. There are many websites that do review comparisons and give feedback about different apps. Basically they've done the competitive analysis for you. So you can go to those websites as well and see what people think about your product, especially versus your competitors' products.

Desk research

[00:05:16] A third secondary source is desk research. Desk research is literally you getting your Google game on and looking for research from other publications, other sources, on topics relevant to your product, your features, the customer needs that you're trying to solve for your target audience, your assumptions, all of these things. You have companies like Nielsen Norman, MeasuringU, Baymard Institute, where they publish research on user behavior, best practices, preferences, benchmarking reports.

[00:05:51] Another good source for desk research is tech companies, like Meta, Microsoft, Google, Spotify. They often post case studies and best practices based on the research they've done, and they often post on sites like Medium, or even on their own company website.

[00:06:05] And the research that you search for doesn't necessarily have to be focused on UX, or come from tech company websites or publications like Nielsen Norman and the ones I mentioned. For example, if you're working on something in VR, AR, voice, the automotive industry, there's a lot of academic research. Papers publish insights on best practices, on experiments, tests, things that can answer questions that you might have and may not have the time and resources to get into, especially in the social sciences, social psychology, human-computer interaction. You can search for academic papers. You can check sites like Google Scholar and ResearchGate and find valuable research there.

[00:06:52] And desk research can also be internal; it's not always external. If there are other teams within your company that are able to do research, you might want to see what research they have done in the past and what findings they have. And you can buy reports from external parties as well.

Looking internally: customer-facing teams and research panels

[00:07:07] And finally, the final secondary source that you can look to when you can't do primary research is to look internally, look inwards, look at your company. I already mentioned existing research that might have been done. But another way to look internally is to look at your customer-facing teams, your user-facing teams. Whether you're a B2B product, a B2C product, a B2B2C, whatever kind of product you offer, there are always people or teams within the organization that are always in contact with users.

[00:07:39] The first that really comes to mind is customer support. If you have a customer support team, they're definitely in contact with users. Marketing, sales, account managers, whatever you want to call them, there are people within your organization. It may even be the founder, if it's a small startup. Founders are usually in contact with users, especially in the B2B space. So talk to those people. Ask them questions, ask them about feedback, ask them about complaints, ask them about what people are saying about the product, and so on.

[00:08:08] A good way to also look internally is to create an internal research panel. A research panel is simply a group of people that you can reach out to constantly to get feedback and ask questions. If you do research, you should probably have a panel. It makes recruitment so easy. But an internal research panel is for if you're not able to go external, whether to your user base or using third-party sites like UserTesting, like I mentioned the first week. You can look at people on your team. You can have interviews with them, you can do usability tests, show them concepts, particularly when they are similar to your target audience.

[00:08:41] For example, if your startup is creating something for new parents, you can look at your team and see if there are any new parents on your team and ask them those questions. Or, with the previous example, maybe you're creating a tool for amateur artists to sell their artwork. I know he said professional, but suppose amateur artists selling their artwork, and you have people on your team who are amateur artists or beginner artists who want to sell artwork. You can also talk to them.

[00:09:12] And very important: if you can't find people who fit the specific attributes of your audience, it's fine, but try as much as possible to consider the behavioral things, the psychographic things, because those are the things that are really important. Behavior is the more important thing, more than demographic things like age and gender and all of that. What is most important is behavior.

Secondary sources don't replace primary research

[00:09:38] Cool. So now what next? You've gone to all the secondary sources, you've found so many useful and relevant things. But before we go into the what next, it's important to note that secondary sources do not replace actual user research, actually going out and doing your own primary research. What they are is a good starting point when you have limitations around what you can do. When you've done all of this secondary research and you have valuable insights, it helps you build a case for doing primary research.

[00:10:12] It also helps you if you're not sure where to focus. An example is when I worked on Chrome. We were looking at where I should focus and what targets, because Chrome has lots of different kinds of users. There are people from whatever behavioral or demographic definition you want to give; there's a wide range. So where should I start from? I started with doing exploratory research. I spoke to people that I knew personally, I checked the app store reviews, and that gave me a sense of where to narrow down my research. So that is what secondary research can also help you do: help you focus on where you need to look, or where you need to ask more questions and do further research.

Synthesize, share, prioritize, act

[00:11:00] So now, what next? With every kind of research you do, primary or secondary, you need to do analysis and synthesis. Very, very important. There's no point in just collecting all the data and then just leaving it there. What you need to do is categorize what you found. A way that I've done this in the past was to put it into feature requests, customer complaints, positive feedback, and whatever other category makes sense. Just create whatever categories make sense for your product or service offering, and use that to make sense of what you have gathered. You can do affinity maps, whatever makes sense for you to synthesize. Just do that.

[00:11:47] The second thing to do after synthesizing is to share. There's no benefit to doing all of this great work if you do not share. You need to discuss the findings with your team. You need to discuss what you have noticed, what you have seen, what stands out to you. You need to discuss all of those things with your team. Quantify whatever you can. For example, if you look through the past year of app store reviews, you can say something like, oh, 10% of app store reviews in the past year are people requesting a particular feature. Or if you're talking to customer support or looking at customer support tickets, you can say, oh, in the past three months, 70% of customer support tickets have been about a specific complaint on this screen or flow or feature. Or three in five requests are for Y feature, which our competitors have but we do not have.

[00:12:44] And sharing all of this with the relevant people, usually the product manager or whoever else is making the product decisions, leads to the next step, which is to prioritize. So you synthesize, you've shared, and I couldn't find an S word for prioritize, so: prioritize. You prioritize with those people. You need to think about these insights that you have. How relevant are they to the business goals? Is it low effort and high reward, or high effort and low reward? Is this already on the roadmap? If you see from your research that most of the feature requests are for a particular feature, but it's already on the roadmap, you need to ask, okay, do we need to move this up on the roadmap? Or do we need to add this to our roadmap? Is it relevant to us? Do we want to do what other people are doing? Is this a thing that matters?

[00:13:34] So you need to prioritize all of what you found with your team. You need to define how severe the problem is. Are you losing users? Are people saying things like, I'm [leaving][?] because I need this feature, this app doesn't have it, so I'm going to use this other one instead? Is that what you're seeing on social media or in the app store reviews or wherever? Are you seeing increased frustration because of the experience, and is that impacting things like customer satisfaction scores and all of that? Those are the things that help you prioritize.

[00:14:03] And then finally, once you have prioritized, you need to act. It's not enough to synthesize, share and prioritize if you do not act. Acting can be you following up with the product manager. If you as a designer have no additional thing that you need to do, it's following up with the PM, or whoever makes the decision on what's next. It can be you directly implementing wherever possible. For example, if you're seeing that people are constantly complaining about a particular flow or screen in the app or the website or whatever, you can start making design changes. You can create a new wireframe or a new prototype to test, to improve that flow. Acting can also mean doing more research to uncover how severe a problem this is, or how relevant a need this is, or how important a pain point this is to address for your business and to solve. And that's it.