Building Features Without Breaking Speed

Catherine Cornell at UXDX Community: New York. Video: https://youtu.be/JAXVPDSRreo

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Stash and hackathon hot takes

[00:00:05] All right, good evening, ladies, gentlemen and those beyond the binary. Thank you so much for being here, and welcome again to Stash. My name is Catherine, I'm a director of product here at Stash, and I'm going to be talking to you about our moonshot hackathon project. First, just a little context on Stash. Stash is pioneering the future of investing by uniting banking, investing and education into a seamless experience. We're doing this by helping four million Americans continue to grow and manage the over a billion dollars in assets that they've deposited onto the platform, 28 [?] at a time. Stashers tend to be no-nonsense Americans striving to make ends meet, so they don't have a lot of cash lying around. The fact that they're investing with us and trusting us with their funds really says a lot. But we're not here to talk about Stash. We're here to talk about my hackathon hot takes.

[00:01:03] All right. How many of you currently work at a place that has hackathons on a regular cadence? Very few. Wow. Oh no, this talk is sunk. All right. How many of you are pro-hackathon? You think it makes the product better, it leads to innovation, it's good? All right. I appreciate that most of you are honest. Designers and UXR people, you think it's maybe not the most productive week, but it's fun team building. First I'm going to talk about my experience. I think hackathons are an accelerant for innovation, but a boost of fuel without a steering wheel can be a really bad thing. It's like in Mario Kart when you get that boost and you can't steer anymore. I'm going to be talking through how we built a really ambitious feature called Portfolio Builder at a hackathon, and how we had to navigate through the process of making it better and iterating on it.

[00:02:07] First, why hackathons? Many of you said you're not currently doing them, so let me tell you why you should do them. They foster collaboration: you get to make your own team and work with who you sit with at lunch. Skilling: at a previous hackathon I was a backend engineer, so I got to practice my real skills. It sparks innovation, sure. And then also employee retention. Whenever I see this on a Medium post or anywhere hackathons are being discussed, it makes me laugh, because what employee retention is essentially saying is that the engineers like doing what they want to do, and not what maybe the product people are telling them to do.

Four types of hackathon project

[00:02:50] Before we dig into how hackathons challenge this, I just want to talk to you a little bit about product development at Stash. This flowchart probably looks really familiar to all of you; you probably follow similar patterns at your own jobs. We start by empathizing: listening to our users, figuring out what their wants and needs are. Strategizing. Then we ideate. We move on to building, we measure, we learn, rinse and repeat. What hackathons do is intentionally subvert this. You start right at step four. Bypass go.

[00:03:26] There are a couple of different types of projects, and they're not all created equal. If you see any of yours on here, don't feel bad. The first is the personal crusade. This is a hackathon project where you're building exactly what you want to use. I would really like to have a feature where I can order ice cream through the Stash app and it gets delivered to my desk, so I'm going to build that at the hackathon. Our users don't want that, but I want it. It's my personal crusade. Next is the roadmap item. This usually ends up happening when the PM is the captain of the hackathon team. It's not a bad thing, but it's not the best use of this week for exploration.

[00:04:18] The crowd pleaser is actually very similar to the personal crusade, but the crowd pleaser is the personal crusade of maybe someone on the C-team, maybe one of the hackathon judges. And then the last type is the type that I really encourage you all to do when you have hackathons at your own company: the moonshot. A moonshot is a fantastic, almost overly ambitious, pie-in-the-sky, literal reach-for-the-stars type of project. I think hackathons can be enormously valuable in moving the needle on these types of initiatives.

The problem: first-time investors and the cannabis-Tesla effect

[00:04:57] Before I talk about what we built at our hackathon, I want to tell you about the problem, the customer need that our hackathon moonshot project was trying to solve. We heard on so many different channels, probably channels that you use as well, on our social, our contacts with customer service, our surveys, account closure feedback forms, that first-time investors lack the education and confidence to make their first purchase. They get all the way through our funnel, all the way through our onboarding, get dropped on our Invest home, see this beautiful design and say, "Bye, I'm overwhelmed." This is Sally Stasher, and she's out of here. It's too many choices. They come in and they see hundreds of options, and they don't know how to get started. So we wanted to give them the confidence to feel they could get started.

[00:05:49] The next problem we wanted to solve is that when users did get started, they usually didn't make a purchase that was the best choice for themselves. We found that 78% of Stashers, at the time we started the hack, had a portfolio volatility higher than their target range. Portfolio volatility is essentially just jargon to say their risk was too high. They were buying things that were too risky for their risk profile. I call this the cannabis-Tesla effect. I hated adding the GIF; I was like, this is a really serious thing, these lovely people are coming all the way from Ireland. But this is it. Our Stashers, our investors, are coming onto the platform, and what they're actually buying the most is cannabis ETFs, cannabis single stocks, or really sexy companies they know about, like Tesla, because this guy goes on podcasts and smokes weed. As a result, the risk in their portfolios was really high, again because this guy goes on podcasts and smokes weed, so the valuation jumps up and jumps down.

[00:07:01] This isn't just a judgment on Stashers buying what they want to buy. Having a portfolio volatility higher than your target range actually really impacts our metrics as well. Last August, when the S&P dropped by almost 3%, we saw that users that had a portfolio volatility higher than what we would want them to have were abandoning marquee features like recurring investments. They were selling off what was in their account, and worse yet, they were churning off the platform. So correcting this problem was also going to help protect us when we have market downturns, like this week.

The mission and what we built at the hackathon

[00:07:41] So we wanted to help people get started, and help them make a better choice when they start. Our mission: simplify the initial investment experience through personalizing a portfolio based on the Stasher's risk level, empowering the Stasher to add and remove investments from the portfolio, and explaining how and why the customizations impacted the overall portfolio. This is a lot of text; this is a lot of stuff to want to do. It was challenging not just from a technical perspective. It was also challenging from a UXR perspective and a design perspective. How do you even begin to do all of these things? We pretty much wanted to create a Reading Rainbow, Blue's Clues, Dora the Explorer robo-advisor. Crazy challenges. I told you: a moonshot, really ambitious. But we did it. We got the technical part down during the hackathon.

[00:08:42] This is what I call the design process scribble. It's by Damien Newman, and we use it a lot at Stash to help visualize why we do design sprints and take time to do UXR. What the hackathon did is it cannonballed us right over all of the scribbles, the uncertainty, the patterns and insights, the clarity and focus, which in my mind I think of as the rubber-stamping process. Even though we all work in agile environments where we're all empowered to make our own choices, I think we can all admit that we have some rubber-stamping processes in our companies. At the hackathon, we started here. We just started building, so we bypassed a lot of those challenges.

[00:09:30] This is what we built. Pretty simple. Again, the front end was really simple; we spent a lot of time working on the back end, which was by far the most complex. Three steps, single CTA. Put in an amount, not very smart. Here's what you get: a nice little visualization of what's in your portfolio, with a happy little guy in the middle telling you that you have a recommended balance. This is an example of someone that's gone in and added five dollars of cannabis. You can see this happy little guy is almost off the edge; he's very risky. If you tapped on him, you would get some user-friendly content on why it was risky and what you could do to have less risk.

Winning, and the first A/B test

[00:10:12] We won. Whoo. You need to clap for that. We determine our winners by letting our very engaged Facebook group vote on who they want. We took that as, well, that's the UXR that we need: people like it, let's go. We took a couple of weeks to get the data engineering correct and to get the model right, and then we launched it to production in an A/B test, obviously, because we want to make sure we're measuring and learning. Sidebar: we have a homegrown tool called Canary for our web testing, which I'm happy to talk about later, but since it's homegrown I figured it wouldn't be relevant to most of you.

[00:10:58] The results of our first A/B test. Number of invest buys: we found that new users were 24% more likely to make an investment if they were shown our hack, and cautious users were 20% more likely. Distinct ETF buys, which is a proxy for correcting the cannabis-Tesla effect: new users were 30% more likely to buy a diverse portfolio, and cautious users were 31% more likely. The main takeaway is that our hack, Portfolio Builder, encouraged Stashers to get started at a higher rate and to invest smarter. We won again. Another giant check.

Taking it to native, and losing

[00:11:40] All right, measure and learn, the next step. Moving on. We did some Mixpanel analysis. Again, I'm happy to talk about Mixpanel at the end and answer questions about how we use it; it's pretty standard, though. We saw that once users got to the portfolio screen, they were very likely to convert. On some days conversion was as high as 56%, which for us is stupid high. So we decided we needed to drive users to that portfolio visualization screen, with the happy little guy in the middle, as quickly as possible. The next thing we learned from Mixpanel is that only 27% of users who converted attempted to customize their portfolio, and we had spent quite a bit of our engineering time on that feature. We thought it was going to be really important.

[00:12:26] Time to build. We're going to take it to the native apps. Let's do it. This is what we made. We pared the funnel down to the screen, and we took out the search and customization feature. When you think about this, we got a shorter funnel, a simplified design, fewer choices, fewer jam jars on the grocery store shelf, for the design nerds. What do we think the outcome is going to be? Better? Wins, right? Of course we're going to keep winning. No. Our A/B tests on native weren't winners; they were losers, and we weren't sure why.

Going back to empathize: building trust with Lessons

[00:13:01] To find the answer, we actually had to go all the way back to the beginning of our design cycle and start to empathize, to think about why people wanted to use a tool like this in the first place. We talked to six users, current Stashers that had access to our hack. They all indicated that the tool made them feel either confident or very confident in their investments after using it. But they had a lot of questions. How does it reflect a moderate risk? What does that actually mean? How does Stash select how much money to put into each one? And one quote that really stood out to me, direct quote: "How do I know this is good for me to buy, and not just what's good for Stash for me to buy?" What this guy was asking is: how do I know I can trust you? You're telling me to buy something, I don't know what it means, I don't have the education. You're supposed to be teaching me what's good for me so that I can make an educated choice, and right now it feels like you're just shoving me toward the end of a funnel and wanting me to click purchase.

[00:14:06] This was our UXR takeaway: once people see the hack and try it out, they're highly engaged, but it might be challenging to communicate the value prop before they actually see it. We needed to build trust and credibility with the user before they got to the end of that funnel. So our very talented design team on the Advice squad went away and created this: Lessons. Lessons are an Instagram story-style walkthrough that teaches users the very basics of investing, and at the end you're driven right to Portfolio Builder to make your first investment. It helps build trust and credibility in Stash, so that when they get to the end, they feel like they know what they're doing, and they feel ready to commit to making a purchase and investing money that they really don't have a lot of.

[00:15:00] When we started the hack, we wanted to make people feel confident in investing. When we went through the iteration cycle, we were so focused on making the funnel shorter that we forgot we wanted to make them feel confident in the first place, and that's what it was doing. So Portfolio Builder paired with Lessons had a 16% lift in conversion. Great. And it built trust and confidence in Stash.

How to get a giant check at a hackathon

[00:15:31] A hackathon helped launch us over the initial hurdles of building an ambitious moonshot feature quickly. I really don't think I would have been able to build a semi-robo-advisor that tells you your volatility score and that you can customize as quickly during the normal design process as we did during the hackathon. However, I can also see in hindsight that not going back to the beginning of the design process after the hackathon cost us some wins when we launched on native. So what can I do for you? How to get a giant check at a hackathon: the next time you have a hackathon, or if you want to take this away to your company and say, "Let's have a hackathon, I want a giant check," shoot for the moon. Go for something crazy ambitious. Use that week of fun and exploration to actually explore what's out there, and not just focus on a roadmap item you want or a personal crusade item you want. Think about the entire design life cycle when you're in the hackathon. Don't just start here. Start here. Thank you.