Innovation Without Adoption Is Meaningless

Rhiannon White at UXDX EMEA. Video: https://youtu.be/QSc1pSuiwKY

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Clue, and how I got here

[00:00:00] Good morning everyone, thank you so much for having me here. I'm really excited to be here at UXDX. A couple of years ago colleagues of mine in my former role spoke here, and we actually used that content in our onboarding with our teams for literally years afterwards. And our current CEO at Clue, who used to be the CPO, Audrey Tsang, spoke here, and I watched that UXDX talk that she did here, and that was the beginning of my rather large girl crush on her. So for me to be here today with you all is really exciting, thank you.

[00:00:28] As I mentioned, I'm Rhiannon and I work at Clue. Some of you may have heard of Clue. I may be lucky and there may be Clue users in the audience. We're a period and cycle tracking app. Our founder, Ida Tin, coined the term femtech. We have over 10 million monthly active users across the world, in every country that you can imagine, including Antarctica. And at Clue we have a very profound belief in the importance of taking charge of your health, because I think we've all had that experience where we, or a loved one, are in pain or something goes wrong, and you're not able to live the life that you want to live because of poor health, because of something going wrong with your health, and how everything just falls away. Without health you really can't do anything else.

[00:01:09] And so for us, we have this really profound belief that we can support people in taking charge of their health through tracking, because we really believe that tracking equals data, and data equals insight, and insight equals action, and action is agency. And agency is the ability to take control of our own lives. And for those of us with the biology that is driven by the hormones that come out of ovaries, agency and control over our reproductive health has long been the thing that's been quite hard to get, bluntly.

[00:01:42] So as I mentioned, I'm the chief product officer at Clue, which I find just a constant delight. Every day I feel lucky when I wake up and get to go to this job. I'm originally from New Zealand, that's the accent that you can hear. I believe there's a very important rugby game coming up between our two countries this Saturday. I got a little bit of stick in the taxi on the way in from the airport because I didn't know that, and probably lost my right to my Kiwi passport, but anyway.

[00:02:06] Normally this is the slide where I show some logos and then briefly talk through why I'm talking to you about this. But because I'm from New Zealand, I thought I'd reinforce every stereotype you might have about Kiwis and New Zealand. This is actually where I grew up. Those are my three children on the beach, so I do have the right to show that photo. And that large one on the left is actually why I'm at Clue, which I'm happy to talk about at some point as well.

[00:02:32] I actually started my career in marketing at L'Oréal in London, not in New Zealand. And then I moved home to New Zealand and spent a few years in marketing roles there, including four very formative years doing political marketing, which is a trip. And then I finished my marketing career in London again at the BBC, where my last marketing role was as the director of marketing on BBC iPlayer.

[00:02:52] And I had been falling in love more and more with digital over that time, because the reason I fell in love with marketing in the first place was, if you look at marketing textbooks from the '80s, the very old school definition of marketing is building profitable customer relationships, and all three parts of that, profitable customer relationships, were the bits that I found really exciting. But I was increasingly finding that marketing wasn't doing that for me. Marketing was more of a push stuff out. And the more I got to work with product teams, and when I talk about product teams I really mean the product team, so engineering, design, and product management, the more I started to feel that that was where the customer relationship was being built. It was really in the product.

[00:03:35] And so I was fortunate enough to be mentored across into product by the person I'd worked with at the BBC, at Shazam, which is a music recognition app, I'm sure plenty of you have used it. And I spent three years in the Silicon Valley office there before going back home to New Zealand. In New Zealand I did a stint in a large bank, which I won't bore you with, and then I took on the chief product officer role at Vend, which was a Kiwi-based SaaS company. We built cloud-based point of sale systems for small and medium businesses. And I was part of the management team that led the exit for that business, and speaking of stereotypes, we were acquired by our largest competitor Lightspeed, who conformed to every stereotype you might have of Canadians. They were just the most lovely people, it was an amazing experience.

Why innovation without adoption is meaningless

[00:04:22] But the thing that really happened to me along this way is that as I went through this journey from being in marketing then to being in product, I kind of turned my back on marketing. I got very product purist. I got very fixated on shipping is all that matters, and it's really about shipping, and it's about the product, and I don't want to hear about anything else. I usually like to say to the teams that I love disconfirming data, because disconfirming data means that we've learned something genuinely new. But I had to have the disconfirming data shoved in my face a few too many times before I came back from my product purist stance, because I just kept seeing over and over that it didn't matter how amazing the product we shipped was. If nobody was using it, then it doesn't matter. You can have the greatest thing in the world, but if nobody knows about it and they're not using it, then actually you haven't done anything great.

[00:05:04] And so that's why this talk is called innovation without adoption is meaningless, because I had to learn over and over this point that the product isn't great unless it's being adopted. And loads of people here have talked about that experience already, of when you ship something and you don't see it being picked up, and how heartbreaking that is. And so I really have come to learn that that process is a process of teams working on that, and a lot of that is the product team, but you have to think about it, you have to build for it, it won't just happen on its own. And the reason for that is that frankly no one's waiting around, waiting to see what they shipped for me. People are trying to get on with their busy lives, and we're lucky to play a tiny part in supporting that, and so we have to go to them and explain how we're solving their problem.

The four jobs to be done at Clue

[00:05:50] But let's talk about innovation a little bit. It's so nice being at UXDX because I don't have to explain what jobs to be done are. Everybody here knows about them, and it seems like everybody's pretty passionate about that. But I'm a huge fan of the jobs to be done framework because it's all about people. We're trying to serve real people who have real problems in their lives, or are trying to make some progress in their lives. And if you really understand the job and the progress that someone's trying to make, then the technology kind of doesn't matter. The technology just enables you to get there. And so that way we can always be innovating.

[00:06:24] So here is what our jobs look like at Clue. We have four jobs that we need to do for our users, and the thing that makes these very exciting and really satisfying to work with is because they hold true across all the various life stages. If you think about the reproductive cycle for someone with a period, it starts very young, ends quite old, there's a whole lot that can happen in between there, but these jobs hold true across all of that. And they are linear: you have to meet each job before you have the right to do the next one for the user.

[00:06:56] The first job is trust. Now I think in the period tracking space a lot of people immediately go to thinking about data, which is true, people do think about data, but it's actually a secondary trust driver. Accuracy is the biggest trust driver. The second is what we call the orient job, and sometimes we refer to it internally as sensemaking. So this is where people are really asking, what is going on, is this normal? Is this normal, along with the last question, am I alone, are the two most profound and really emotional questions that people ask in this space. The third job is what we refer to as the empower job: what do I do now? And then the last one is this comfort job: am I alone?

[00:07:31] And this is so strong for people that even when you're doing research with people and you're just trying to hear about their experience, they'll ask you at the end of the interview, is this normal, am I normal? Because this feeling of being alone, the sense of, you'd think in today's day and age that actually people do have all the information they need, but they really don't. It's actually shocking how little information we have about our own biology, and consistently people feel alone in this.

Applying the jobs to product strategy

[00:07:55] And so this is how we apply it to our product strategy. On this point about accuracy, we focus first and foremost on accuracy, and accuracy, I mentioned, is really subjective. It's super interesting, because you would think it's objective, but it's not. It's really based on the user's lived experience. So someone with a highly irregular period will say, oh, the product's really accurate, it predicted it within a week, whereas somebody who has a highly regular period says that's shockingly inaccurate. So it's really about your own lived experience, and users always bring that lens to it. So we have to focus on accuracy. And of course the key thing about accuracy is, bad data in, bad data out, so we have to make the product easy to use so that people can put the data in and get the results out that they're looking for.

[00:08:41] The orient job, I mentioned, is what we call sensemaking internally. This manifests in terms of people saying things like, I felt like I was going crazy and then I opened the app and I saw it was just hormones. I had a great conversation with a user a couple of months ago who's in perimenopause, and she was talking about how she gets to a point where sometimes she just feels particularly exhausted, and she felt like she was exhausted all the time and that her life had completely changed. But through tracking she could see that actually she was really exhausted in the second two weeks of her cycle. So it wasn't that her life had completely crumbled and that she was so exhausted, actually it was cyclical, and she could see what that pattern was and understand it.

[00:09:21] Which leads to the third job, which is around helping them to choose what they do next. And that choose point is really important, because not everyone's going to do something. In fact, I had a conversation with a user in the US a couple of months ago who has this brilliant, she through tracking had realized that on cycle day 19 she always got into a massive fight with her husband. So now when she feels like they're really getting wound up and she thinks he's being particularly annoying, she checks the app, and if it's say cycle day 19 she walks away. So she's choosing not to do something there, which is great. I don't know what she does all the other days. But this is this point around enabling people to choose, because they have the insight, therefore they can take action, which gives them agency over their behavior.

[00:10:01] And this comfort job is really about connection. We are humans and we are a connection-seeking species, and the thing that people really want is to know, am I alone? And so we do this through content, and we often have an internal saying that content strategy is product strategy. We have two strands of content because this turns up in two different ways for users. It turns up in terms of, I want to know the facts, give me the facts, give me the scientifically validated article that has all the references that I can go and check myself if I want, and I know that you're giving it to me straight and it's not all the myths and taboos that are out there, like eat citrus and that'll reduce your cramps. That's not true, people.

[00:10:36] And then the second strand is, people really want to know this, we talk about it as the people like me. So you want the facts, that's really helpful for connecting to, okay, I can figure out what's going on here. But there's also an emotional side around reading people's stories and hearing from people like me who are going through the same thing. When people are struggling to conceive it is heartbreakingly difficult and lonely, and when they can see that they're not alone, and that actually huge, huge, huge numbers of people struggle to conceive, it really helps them.

What meaningful adoption looks like

[00:11:08] So what does adoption mean? Because it's not just people having a look. I unfortunately have shipped plenty of things where people just have a look and then they go away. Every product has its own adoption metrics, or it should, which I always think of as the behavioral or usage metrics that show you whether or not your product is really doing the job that your user hired it for. This looks like nothing to you, but 5T7/2, this is actually our key activation metric. If a user records five tracking points in their first seven days over two separate sessions, that's really important, then they are 70% more likely to become a permanent monthly active user. If they don't do that, the likelihood of becoming a permanent monthly active user is 25%. So it's a huge difference, if they hit this behavioral sequence.

[00:11:55] And I know monthly active users is pretty hotly debated as a metric for many products. I mentioned my previous product was a cloud-based point of sale system, who cares about MAU? But I think for a period tracker a monthly based metric is quite good, it's kind of useful. But this is an example of meaningful adoption.

[00:12:12] This is really interesting to us, because I mentioned that monthly active users is really important and that we really want people to become monthly active users. And so we've been doing a lot of research looking at people who left us. Often what happens is people drift away from the product, they don't kind of quit the product in one go, they tend to drift away when they do churn. And so we've been doing a lot of research trying to understand what happens, what drives that. And we had found that there are a lot of folks who have irregular periods, or who want to enter all the data in one go. They don't go in and track their period every day or track anything else every day. And bluntly the interface just wasn't easy for that: you had to track it, save, track it, save, track it, save. Doesn't sound like much, but when you think about people's busy days it was annoying. And so we had been working on solving for that, and I'll actually show you the impact of that a little bit later on.

[00:13:01] So just to be clear, we were very clearly and specifically thinking about people who were monthly active users and who were drifting away. So they had been, but now maybe they were once every two months, every three months. They weren't new users at all. We shipped it, and we found that users were using it on day zero, which was new and surprising. And then, I mentioned that this activation metric is really important and drives increased retention, and we saw people who hit this particular innovation in the product, this fast track, is what users called it, we didn't call it that but that's what they called it, had a 25% uplift in retention. I don't know if any of you have done much work in retention, I love it, but it's a grind, and anything that improves by 25% is crazy, especially when you weren't setting out to do that.

[00:13:44] And what was happening, this is where qual and data become so good together. So we obviously have a huge amount of data off the product and we could see what people were doing, but once we got to really speak with people and really dive deep into what was going on, we found that they were actually using it not at all for what we had envisaged. They were using it to basically import their data from previous or other products. Because we all like data interoperability, and bringing your data from one product to another is always the hardest thing. You'd think it'd be quite a simple data schema, a period is a period is a period. It's not, it's really difficult to match those data schemas, and so it's always been in our backlog as something we would like to do, but the effort is so huge. And users had found a way around it, which they will always do, to try and solve their problem. And so this was a really interesting piece of where innovation and adoption come together to be very meaningful.

[00:14:31] So that's what I meant around meaningful, and it has to be something that is meaningful to your business. And so for us that's monthly active users, and then the other part of course is we're a subscription based business, so then do they become a subscriber or not?

The multi-day tracker and rebuilding reminders

[00:14:45] And so here, I mentioned that we were working on this, we called it a multi-day tracker, users called it the fast tracker. We were working on this, and here we can see the impact on monthly active users. This is our monthly active users day by day as a graph. You can see there's a weekly pattern in it which is quite curious, Sunday, Monday, Tuesday are our highest days. And you can see on the left hand side of the graph, although I kind of cut off the edge, it had been declining, and it had been gently declining for a long time. And this was a real concern to me because as a consumer based app, that is the core of what we do, which is why we'd been doing this work, thinking about how we solve for this.

[00:15:26] And this was just such a really fascinating example for me of how these two things have to come together, because we had built a solution, we had tested it with users, we were pretty confident and happy that we had a good thing. We shipped it, and you can see the day that we ship it. Now this is on iOS and it's an app, so it goes out to 100% of new users immediately, but then has a phased rollout. And you can see as it phases out, you can see there's also a regular rise, but the regular low is not as low as the week before. Oh, it's starting to work, something's starting to work. And then we bring in telling people about it and communicating with them, and not just hoping that they find it. And we did that through a combination of CRM, this one was very strongly driven by CRM, but also in-app prompts of course. And then you can see the lift, and you can see how it just keeps rising. And I'm super happy, this graph is just gently rising, rising, rising all the time now.

[00:16:17] And this is another example. We had to rebuild our reminders service. Reminders are a pretty cool part of a product like ours. Why would you track your period if not to be reminded about when it's coming? Unfortunately the reminders had gotten very old and had a huge amount of problems in the code, so we had to rebuild the whole thing, much to our users' immense frustration and mine. But we got it done. And you can see we launch it, and for new users it goes in onboarding, classic, everybody's gone through onboarding with reminders, and you can see, yay, nice steady adoption, great.

[00:16:42] But then we launched an in-app prompt. So this is about thinking about adoption in terms of, how do we let people know that this functionality that they want, and that frankly they had been missing for a while, was there and was back? And so after someone had tracked, we popped up something saying, hey, do you want to be reminded about tracking? And you can see that hugely lifts it, because we took it to them. We didn't expect them to come and find that we'd shipped this thing, because they're really busy. We took it to them when they were trying to do the thing that they wanted to do with our product, and said, here's how we could make it easier. And then we introduced the CRM as well, and then you can see even more spikes happening. So this is bringing all of the components of the system together, and I think this is the key point here: bringing all the components of the system together is what helps us bring the innovation to the user, that's hopefully solving their problem and gives them a chance to do it a little bit quicker, a little bit faster, a little bit more easily.

[00:17:33] I used to think that really it was only the product that mattered and only shipping was what mattered, but I've had to learn over and over, through graphs like that in many places, that it's not. It is the system, and it's the coalescing around the aim, which is adoption. Not shipping, adoption is the aim that you get to.

Launching perimenopause

[00:17:53] An example of how we do it, again in real life, with a really big thing, was that we've just recently launched perimenopause as a functionality in the app. I don't know if many of you know anything about perimenopause. The odds are you don't, because we surveyed our users and 70% of people don't, and that seems to be the thing that comes up over and over again. This was something we were really excited to be doing because our users had been pulling us towards it for a long time. And actually we had many of the symptoms associated with perimenopause in the app already, so you could use them and you could track. But one, users didn't know that these symptoms were associated with perimenopause; two, they couldn't find them; and so three, they weren't using them. Dare I say it, innovation without adoption is meaningless. We had the stuff there, it wasn't working.

[00:18:41] So we started out super lean. We shipped just a tracking shortcut that brought together those symptoms under the heading perimenopause tracking, because this job for users is about comfort and about telling me what's going on. Just telling users, hey, this is a symptom that is associated with perimenopause, you're not going crazy, is incredibly helpful and people really value that. And so we did that because we want to ship something rather than nothing. A constant drum beat for us is time with value, not time to value. Users don't care if you did it in four weeks or three, they care about solving their problem now. So having time with value is what really matters.

[00:19:14] So then we started building out the more complete mode, and started to work together cross-functionally on how we could drive that adoption from the beginning. We started with our users and worked back from there. So we ran a survey in the app asking users about their levels of understanding about perimenopause and what they wanted to understand. We thought we'd get a few thousand answers, you know, who wants to fill out a survey, I mentioned how busy people are? Well, when it's about something that's going to happen to you but you don't know about, you do. It turns out we had over 26,000 responses in 24 hours, and we ended up taking it down because we just had so much data. So then we used that data and that insight to shape the content that we prepared, and the CRM, and the UA, and the PR, and the social. Everything together came from that first insight, and grounded in what users were telling us they don't know, so that we can know that we're solving that problem.

[00:20:04] And then when we launched, our support team is very small. It was really exciting hearing Fergal[?] talk from Intercom. I've worked on products with big support teams, and support is so critical. We have a tiny team for support at Clue, but they're amazing, and they went back to everyone who had ever contacted us about perimenopause and said, hey, can you give it a go, can you tell us about this? And we had a lot of feedback and interaction with them.

[00:20:25] And then we rolled out the various product pieces as they came. Again, not waiting for anything. So when it was ready for existing users on iOS and Android, roll it out. Without new user onboarding? Fine, roll out new user onboarding the week after. And the same with the comms: as it's ready, we roll it out. We develop in English, so we're always in English first, but we're in 20 languages. But we don't wait for the localization. We get it out and then we do the localization afterwards if we have to, because time with value is always more important than waiting until everything is all perfect.

[00:21:00] And so then we started to roll all that out, and as we saw that come out, we just see continual spikes. You can see it in the data, and it's very new, I didn't actually even have it to put in the slides, it all started to come through over the last couple of days. But you can see distinct spikes, the same spikes that you can see associated with all those adoption tactics, if you like, and those moments of working together, all combining. The product grew over 200% last month and it's on track to grow the same if not more this month. That's our fastest growing new product ever, because thinking holistically together from the beginning has meant that we're continually bringing it to users and letting them know about it, not just hoping that they might find it and that it might be relevant to them.

[00:21:38] We also have a very deep interviewing practice. This is very important. This is one of the few things that I am very religious about in product: speaking directly with users. And so we have early users involved, and we are all interviewing them weekly as well, in terms of what we're learning and how we keep then iterating on the product.

Anti-patterns: fixating on when, and waiting for a safe point

[00:21:59] I don't want to pretend that this is easy or that we've got it all worked out, because we really don't. I think there are two very common anti-patterns, or tensions, that I have observed both at Clue and in other places that I have worked.

[00:22:12] If we use product development as a broad term for all the functions involved in product development, and if we use comms as a broad term for all the functions involved in speaking directly with users, comms tend to get very locked on when, and really want a date. When are we going to do this, what's the launch date, why won't product development commit to a date, why does product development miss its date? This all probably sounds familiar to you. But this focus on when is actually really a holdover from CPG, which after all is dominated by physicality. You have physical products with physical distribution and physical media, so you do have to have this massive physical coordination to get something live. I remember when I started at L'Oréal, just the intensity of trying to line all that up for a new product launch is really insane. And with physicality it is a binary choice: you're either on the top of the shelf or you're not.

[00:23:09] But we work in digital products. And while I'm a huge, huge advocate of really considering the physicality that your user, their physical experience while they're interacting with your product, where are they, when are they, what are they doing, what are they feeling, what's going on, the fact of the matter is that we have digital products with digital distribution and digital media, which means that we can ship at any time of any day, and we can ship comms at any time of any day. We're not restricted in the same way around trying to work back from these dates.

[00:23:38] And so this means that we should start from the what and work forwards, because that's the thing that's going to matter the most. The when doesn't really matter. When a user comes across it, what matters is whether or not it's the right message at the right time. So we can make sure that we are working together on that and then planning forward, because with digital products there are so many different ways in, and paths that they might take to get to it, you can't really know. Whereas with physical products, again, there are only so many billboards that you can buy, and there are only so many routes into the store, and stores are very carefully planned so that you go certain routes. There is a physicality to it that you can think about and plan, but you can't do that in digital. You really have to think about the what, and that's how you can coalesce around there. And we have found that when teams can let go of the when, when they let go of the we need a date, and sometimes I've had to give them a fake date so they can let go of it, and say, hey y'all, we all know this is not a real date, just like, then when they can let go of that and focus on the what, suddenly it becomes really easy to work cross-functionally.

[00:24:46] The second anti-pattern I often see, and this is one that tends to apply more to the product development teams, is waiting for some kind of safe point. I think this is really interesting, because product development teams are really good at being clear around scoping down, and so we'll be quite tight on, you know, the comms functions might say we need all this functionality and the product teams will say no. But there's something about safety, and I think this is also a holdover from a different time as well.

[00:25:17] An example of this for us: as I mentioned, we've been rebuilding reminders. We rebuilt on iOS, showed you the data, then we had to rebuild them on Android. Android is, as you know, harder often in terms of development, and so the team that had worked on it was just really reluctant to start the push, because they had seen the power of working together on adoption and the volume that could come in. And I was getting a lot of, ah, so-and-so's away, maybe we should wait till they're back, or we should wait till we get more data on this, or we should wait to see if the product's more stable, there might be bugs in there. A lot of hesitation.

[00:25:49] And I think here again this desire for safety comes from a time when shipping was less frequent than it is now. This is one of the great benefits of us all being much more focused on DevOps. Once there was a time when you shipped software once a quarter if you were lucky, and even though many of the people I work with haven't lived that life, there is still kind of a collective memory. And the fact of the matter is, though, that we don't live in that locked-in world anymore. Even in our product, which is a mobile app, so we do have the gatekeepers of the stores, but we ship every week. And my previous product, which was primarily a web product, we shipped code to production every six minutes. And I'm sure you all do the same. Shipping is the heartbeat of a software product, and just like anything to do with your heart, the more you exercise it the healthier you are.

[00:26:35] And so again, with this anti-pattern of anxiety, I think it's really critical to focus on the why. Why are we doing this, what is the value to the user? And again with reminders, all the value lies, it's core to the product, so the longer we don't ship it, the longer we're not serving those users and their needs. We're just not doing the job that they hired us for. And so I think this foundational belief, it's really helpful to come back to this why. Why are we doing this? We build things because we believe it's going to create value, and so if we can anchor in that, that helps people get past the anxiety. Spoiler alert, we did ship it. There was a bug in it, but it was one of those needle in a haystack ones, you don't find it unless you have huge volume. So it was good, and now it's being fixed, the fix is shipping today.

Three things to leave you with

[00:27:23] So I think there's probably three things I'd like to leave you with. I think the first is, don't be like early product me. Shipping is not the end goal, adoption is the end goal. Solving your users' problems is your end goal, and you're only solving them if they are adopting what you're building. And to truly drive meaningful adoption you need to understand what that looks like for your product, what your adoption metrics are. Not just the standard product metrics, those are useful, but actually what really matters is your product and your users.

[00:27:53] And finally, a product is a system. It's not just shipping, it's a really broad system, and if you can understand how all the celestial bodies in your solar system for your product interact with each other and help or hinder each other, then you can really drive so much more, and you can drive a lot more progress against those goals that you're trying to move. Because shipping code is always the riskiest thing that we can do to solve a problem, because it's the most expensive thing we can do to solve a problem. Whereas if you can ship something and then work with all the other parts in your system, whatever those components may be, sales, support, CRM, PR, social, whatever it is, then you can continue to impact those metrics long after you've shipped. And you can also test and try things without shipping, which is always going to be something that we have to keep on our radar, and that's going to help us move more rapidly.

[00:28:43] So that's it from me, thank you so much for listening.

Q&A

[00:28:50] Host: So from the audience, the first question we have here is, how do you deal with inaccuracies in prediction, and how it can have a huge impact on a woman's life?

[00:29:01] Rhiannon: That's a good one. There are several ways that people use the predictions. Mostly people use them to plan, so everybody wants to know when it's going to happen so they can plan around that, so not necessarily a huge impact if you're out by a day or two there. Some people use them when they're trying to conceive, and accuracy around when ovulation is happening is really important, and there's a lot that apps can help with, but also the evidence is really clear that using biometric data is really helpful there, so people should be doing both. And then there are folks who, for whatever reason, other forms of birth control don't work and they might use this type of product. This is not our recommended purpose, and all the apps that have this are really clear around that. If people are using predictions to manage their contraceptive journeys, then it's really a good idea that they use biometric data as well, in conjunction with that.

[00:30:00] Host: Okay, and you mentioned content strategy is product strategy. What ways did you identify winning, most valuable content?

[00:30:08] Rhiannon: I'm going to sound like a broken record on all of this. It comes from our user base, you hear it over and over again. So a really classic example, perimenopause, I mentioned, is a huge area that we're working on at the moment, and the information gap there is even more shockingly bad. And there's a huge amount coming up from our users about ADHD in perimenopause. Here's a really interesting bit though. Where the research, it's politely referred to in health and women's health as the research gap or the diagnosis gap, it's more like a flaming crater. There's just not that much research on women's health. You're probably all familiar that for a long time women weren't even included in clinical trials, they were just treated as smaller men for drug interactions. Well, it turns out that for example with ADHD, the drugs that people can use to treat that interact differently with estrogen, and so when estrogen levels start to fluctuate and change a lot in perimenopause, then the interactions start to change. So our users are saying, give us the content on that, give us the information on that. We go to the science, because we're science-based, and there's not much science on it because no one's doing any research. So that's where we start, for example, looking for areas to do research.

[00:31:15] Host: Gotcha. Okay, you mentioned content strategy is product strategy, what ways did you identify winning and most valuable content? Oh sorry, that was what we just talked about, wasn't it? Sorry. How do you agree on UX and product metrics? Did you use a framework, and how often did you validate these metrics?

[00:31:31] Rhiannon: So in terms of how do we agree on them, we're pretty small and lean, so it's a constant process of insight and learning for us. I showed you the activation metric. When I joined we didn't have activation metrics, so we started working on uncovering and getting the data around this, and then we look at the data and we look at it together, and away we go. It's not very hard to agree, we're really small.

[00:31:57] Host: Okay. And what do you use to track your analytics?

[00:31:59] Rhiannon: Do you mean like the system? I don't know, very broad question. We have a huge amount of data off the product. We use Looker for everyone's day to day, and Redash for the data science people.

[00:32:13] Host: Okay, we'll go with the last question here. How do you innovate and stay ahead of the competition with companies like Apple entering?

[00:32:21] Rhiannon: This is a super interesting question. Yeah, and Apple is very fully in the space. I don't know, maybe I'll get in trouble saying this in a public forum, what's going on, Apple? They make such a great product generally, their products are great, they're funding women's health research, which is amazing. I don't know if anyone's tried to use their period tracking feature, it's like not living up to their normal standards. I don't get it and I don't understand it. And I speak to loads and loads of users who use Clue and Apple Health, or who use Flo and Apple Health. Basically many, many people just use both together, and they will use the period tracking app that's a specialist to do the specialist job for them, because there's something about Apple Health that's not quite meeting their needs. I don't know why. Yeah, maybe one day we could have a chat about it.

[00:33:07] Host: Cool. Okay, ladies and gentlemen, put your hands together for Rhiannon. Thank you everyone.