Q&A With Patricia Trejo

Patricia Trejo at UXDX Community: LatAm. Video: https://youtu.be/wraVjYvYCCQ

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Q&A: who builds the flow metrics charts

[00:00:00] Host: Excellent. Welcome, Patricia. Again, another talk I really enjoyed there, and it was all around diving really deep into the data to uncover all of the insights that you can. I've got a few questions, but for the audience, please remember, as Jovi [?] said, if you have any questions, put them into the ask-the-speakers channel on Slack or in the box to the right of the screen now, and I will hopefully be able to ask those questions of Patricia. I'm going to start, though. That looked great, but one question I have is: it looks like a lot of work to generate those graphs and those charts. Is there a centralized team that does that for people, or is each team responsible for doing their own research and things like that?

[00:00:56] Patricia: Let me talk to you from my experience, because when I started with flow metrics, I started by myself, with the team I was working with. I started to connect through a very simple script to the API of the issue tracker, for example Jira, Trello and a lot of other ones. That's super easy. Usually issue trackers have their API exposed, and it's super easy to create a script. Sometimes you can also find some scripts on the web. You connect it to, say, a Google spreadsheet, and you start to retrieve information and then analyze it.

[00:01:49] The first times you do that, it's not that easy, because you have to start to understand what the data means and what you want to measure, or what you want to see or analyze. So at first it's sometimes not that easy, but once you start, it's like a whole world that opens in front of your eyes. And the data is always the same; you have more or less the same registers. It's as simple as that. So you can start doing it. In fact, with all my years working, when I started to work in a new team or with a new tool, for example with a new issue tracker, it was super, super easy.

[00:02:38] But for example, now I work at Cornershop by Uber, and now we have a development team that extracts and retrieves information from the issue tracker for all the development teams. We created a first MVP to connect to just a Google spreadsheet and start creating the charts. Then we did a second release, when we started to create some Data Studio reports connected to the Google spreadsheets, and now we are trying to release even more evolutions of this product, which we leave available for all teams.

[00:03:28] I have these conversations and try to instruct people, POs, engineering managers and team leaders, so they understand the data and start to analyze all their behaviors. Because each team has a super different context, a super different behavior, and you cannot have a one-size-fits-all chart or any improvement. It depends on the team. So we also have to do that work of going to the team leaders or to the teams and trying to explain to them what these metrics are. When they start using them, it's incredible how they start to see the value. But they have to start with that, and they have to start improving their development process, their daily process, to see: oh my God, yes, we made this improvement, and now my day-to-day is even easier, or even better.

Training teams to use the data

[00:04:40] Host: Excellent. I like that, because there are probably a lot of people who don't have a dev team, but you've said you've done this before yourself, and it's possible to do it. That's great. Can you share a bit more about the training? Because I've seen in some companies people are given lots of data, but they don't know what to do with it, and then it just gets left. So what kind of training do you give the teams?

[00:05:10] Patricia: At first, I start with super simple metrics, as I said in the talk. When you start with super simple metrics, to know, for example, how many cards or tasks you delivered in a week or in a month, or things like that, it's already a super easy way to see how the team is behaving. So you can see: okay, why are we sometimes delivering more than other times? Or why do we deliver more bugs some weeks? Or why, in these last three weeks, have we delivered so many bugs and not features that bring value to our product? What happened?

[00:06:07] That is a super, super simple thing that you can start with. Usually in your data you have timestamps, so for example you have the timestamp when a task gets completed. If you start to look at that data, how many cards the team delivered in that week or by that date, you are going to start to see throughput, that super basic flow metric, and you can start asking yourselves as a team about all the things that are related to that. Maybe at first you just see the whole bunch of different cards delivered in a week without considering the different kinds of cards, and then you can start splitting them out: these are bugs, these are features, these are technical tasks, this is technical debt, et cetera. And you start to see more different behaviors.

[00:07:22] For example, I always remember, years ago I was working on a team, and one day a stakeholder came to me and told me, "Patty, I'm very, very, very concerned." "Oh yeah? Why? Please tell me." "Because I see that the team is working the whole time just delivering technical debt and not features." "Really? I work here with the team the whole day. I haven't seen that." "Yeah, I have seen that. I'm super worried, because I want to have features delivered in production." "Okay, let's see."

[00:08:02] So I opened my laptop, took my flow metrics, all my charts, and showed him: "Look at this. We paid down just one card of technical debt three weeks ago, and now the team is working on one more. But see all the features we have delivered all these weeks, and all the technical tasks that we have done to support all the development." He just saw the chart and was like, "Okay, yeah, you're right. Sorry, I thought I was right." Cool. And a discussion that could take more than 30 minutes took about five minutes, no more than that.

What pairing brings

[00:08:54] Host: Brilliant. That's a great example, because people often do have those assumptions. One thing, just on the pairing: it was very interesting that you couldn't find any difference, because in every other study I've seen, particularly on the bug side of things, there are fewer bugs when pairing. What was the outcome for you when you didn't find a difference? Did developers start working separately then, or did they continue pairing?

[00:09:22] Patricia: Well, I think the major difference when developers are pairing is, how do I say, the innovation, the different ideas the developers bring to a solution. Sometimes, for a technical task that has a more straightforward solution, it's not as necessary; maybe yes for a more complex one. But sometimes, for features, for value that you bring directly into the product, you see amazing things when developers are pairing. Solutions could sometimes take longer, maybe yes, maybe not, but they usually finish with better quality, with better solutions for the users, with fewer bugs, with less tech debt, because you have two minds working on the same thing at the same moment.

[00:10:34] That's super, super cool to have. And as I said before, when you have complex technical tasks, it's also super cool to do pair programming. For me, maybe all teams would be doing pair programming all the time, but since sometimes that's not possible, maybe you can focus on those kinds of situations: when to use and when not to use pair programming.

[00:11:10] Host: Great. Well, thank you very much, Patricia. I really enjoyed the talk and the questions, and thank you for sharing.

[00:11:18] Patricia: Thank you. See you, people.

[00:11:21] Host: Now we're over to Daphne and apology [?].