Unpack The Problem - Defining Objectives

Rija Javed at UXDX Europe. Video: https://youtu.be/7WAXOYioUxI

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Being sold on tech debt

[00:00:06] Thank you. Hi, everyone, I'm Rija. As mentioned, I am somewhat new to the UK and the European tech scene over here. My prior experience has been in the Bay Area, and before that I actually grew up in Canada and did some internships here. So I thought something that would potentially be useful is some of the lessons that I've learned and that I've seen play out through my career.

[00:00:31] First up, tech debt. I assume this is a term that everybody over here is familiar with and hopefully has seen in some capacity. Unless you're working on a completely new piece of infrastructure and system, you probably have some of it within the system that you're currently working on. If you don't, then I probably won't believe you, but I'm totally open to learning that you have a somewhat old system and it doesn't have any tech debt associated with it.

[00:01:01] Very early on in my career, at one of my past companies, when I first started I got to work on a greenfield project. It was a new client-facing initiative that got a lot of visibility inside the company and had PR associated with it. It went well, it was super great, and I was now ready for my next challenge. I had a conversation with my manager, the VP of engineering: "Given the bread and butter of the company and what we deliver to the client, and we're building out this next new thing, that's where I'd like to focus." His response back to me was, "Okay, but we also have this tech debt that has been accrued over the past six to seven years, and to be honest this is the biggest business risk that we have at this point. Unless you're telling me that you really are focused and motivated and want to work on this next shiny new client-facing feature." I think especially when you're new and junior, that's where a lot of your focus is going to be.

[00:02:00] As an aside, I think what companies sometimes tend to do, because ultimately you are delivering the best service and the best product to your external clients, is that even internally a lot of the focus in terms of visibility, communication and potentially appreciation tends to be on those particular things. So you yourself as an engineer, regardless of which department and which area of the system you're working on, especially if you're somewhat young and inexperienced, may not necessarily be thinking so much about that other work being the best way for you to provide delivery.

[00:02:35] But as a manager he did a good job of, quote-unquote, inspiring or selling me on this being the best way for me to make an impact. I was totally on board with that. It was also the most challenging piece of infrastructure, so there's that part of your brain that's like, "Oh yes, I want to conquer the biggest challenge and make the best impact." Okay, cool, I'm going to go and do that, and work on fixing this tech debt and all this operability and on-call stuff. I got onboarded onto it and onto on-call.

Finding signal in the noise

[00:03:06] Then I soon realized the problem, and you may have encountered this if you've had to tackle it and figure out exactly how you're going to scale the system: initially there's a lot of noise associated with it, at least to me at that point, somewhat young and inexperienced. Are you familiar with the garbage island that has been created in the Pacific Ocean? That was the visual that came to my mind. There's this huge mess everywhere. I didn't even know what to focus on, and I was literally playing whack-a-mole all over the place.

[00:03:41] There's some noise that shouldn't actually be happening, like waking you up at 2:00 or 3:00 a.m., when ultimately it's not that severe an issue and you can deal with it later on, at 10:00 or 11:00 a.m. Or there are things that people have just become immune to because of the number of times they have been escalated, and you just accept that it's totally fine for this to come up every so often, and this is the somewhat manual, hacky fix you do, as opposed to doing the root cause analysis. Or maybe you have done the root cause analysis, but you just don't feel like you have the time to implement the root cause fix such that this and similar incidents don't occur again.

[00:04:24] My job was to navigate this whole system and organize it in terms of these one-off incidents that were happening: do root cause analysis, then scope analysis, then work out what exactly the business impact was going to be, and then come up with "This is the priority order that we're going to associate with this," and that overall larger plan. Okay, cool, this is something that I can now go ahead and execute on.

Using data to get staffing

[00:04:55] The next problem that came up for me: especially within a startup, or even within a big company, I don't think any company can ever say, "We've fully hired the talent we need, and we have all the staffing we need in whatever area." You're constantly looking for great talent, and you can never have enough staffing, because the amount of things you want to do is potentially to conquer the world and build these unicorns and rainbows, and there are client-facing features and there's all this fun behind-the-scenes stuff. When it came down to staffing, I couldn't do this on my own, and given the size we were at, how did I actually get more staffing onto this?

[00:05:35] At that point it was again a little bit of a fun conversation, because a lot of our staffing and our focus seemed to be on those client-facing initiatives as opposed to this area. While I had come up with this plan, I was going to need more engineers and more of a high-level company focus on this as well. So my manager, who had so amazingly sold me on this, now I had to sell him on the vision that we needed more staffing. His response back to me was, "Rija, I believe you." And he should totally believe me that this is an area that needs attention, because that's literally what he sold me on. "I believe you, but in order for me to convince the C-level staff as well as the rest of the company outside this area," so not just within engineering, but outside engineering, like product, data, design, and those cross-functional areas like CS, ops, marketing, research and various others depending on which industry you're in, "we as human beings are going to be extremely subjective. So we need the data to show these people, those metrics, KPIs, whatever you want to call them, to actually say, 'This is a priority, and this is the business impact it's actually going to have.'"

[00:06:53] So while I had come up with that overall larger roadmap, I then had to take that data and ultimately create those shiny dashboards to show to the various other people and get their focus. Now this is a little bit of an interesting one, because you do want a data-driven approach. Nevertheless, one of the things about communication, especially as you go through your career and become more and more senior, is that the level and type of communication you'll be doing with people is going to vary.

[00:07:26] The communication, data and types of metrics that are relevant to the people working on the system are going to be a lot more granular and a lot more specific. The stuff you're going to communicate to the front-end engineers, for them to learn about this area, or the client-facing engineers who aren't working on it, is going to be a bit higher level. To people outside engineering, it may be a bit higher level again. What I showed my VP of engineering and CTO, maybe they'll be looking at the more granular stuff, but at a high level their day-to-day consumption is going to be different. Same thing with the C-level staff, and ultimately if you're doing board presentations on it. So think about what exactly those are for the different business areas of the company. This was more on the platform infrastructure side: why should the business actually care about this area?

[00:08:20] Ultimately that was a bit of a reverse approach, because we should already have had those metrics before creating the plan. The cool part was that it became a bit of a model for the rest of the company, given the earlier stage we were at and that we were trying to revamp and reshape everything.

How a roadmap comes together

[00:08:39] It was also my first lesson in coming up with a roadmap and how exactly you do that. The company is always going to have some vision or goal that it's ultimately trying to accomplish, and it's necessary to have it. It may be that your vision is to provide the best invoice financing support to small and medium-sized businesses, but the reason, in terms of the mission, is that you actually want to help the little guy out there as opposed to the big corporations, a similar vision to what we have at MarketInvoice. Then there's going to be the strategy associated with it, which is based on some very high-level input combined with exec staff and other input, in terms of, at a high level, what we believe and how we're going to go through it. Then of course there are various cross-functional areas within the company and various stakeholders, potentially even external ones if you're working with third parties, and how that might dictate the constraints you're functioning under. And the biggest thing, of course, is the data associated with it.

[00:09:50] In some parts, especially the platform infrastructure area, we have a lot of those metrics, so data is going to be a much bigger thing. But when it comes down to the client-facing areas, there's going to be some sort of risk that you take. You will have done some market research. Let's say you know that you don't have any mobile app, and you notice that a lot of people have been signing in using their iOS phones, while you see little to no sign-ins from Android phones. Based on that data, you don't know for sure, but the data is pointing towards it, so you decide to build an iOS app for your product. Then, as you ship and see more traction or you don't, you can figure out whether you want to add more features onto it or not, and maybe over time you'll get better data on the Android side as well. All of that then comes together for you to create that roadmap.

[00:10:53] I was able to do all of that for that particular behind-the-scenes area. Later on, when we decided to create this bigger area and take on more client-facing work as the area itself expanded, it was a nice little juggle, especially in terms of stakeholder input, because there are going to be other areas that have an input into this, and you're ultimately providing them with those services.

Iterating and learning from post-mortems

[00:11:25] The other part is being iterative and continuously learning. You do that with the roadmap, as I said, with some of the client-facing initiatives where you took some level of risk and how exactly you wanted to balance that. As you're delivering the product and features, in terms of the different milestones associated with it, you take the learnings.

[00:11:46] The other place where it's extremely important, especially when you're on the more engineering and operability side of things, is post-mortems. Given the area we were dealing with and that tech debt, you come up with the roadmap and you've done all that analysis, but let's say an incident goes off, maybe because something at a third party has changed, or maybe you made a mistake in the data you were looking at and something was missed, and some production incident went off. Of course you do the immediate fix, you do the root cause analysis, and you try to make the fixes associated with it.

[00:12:27] But I think the overall process you want to follow is working out exactly how the problem occurred, doing the five whys approach until you arrive at "This is the root cause, and these are the fixes we want to make," assigning some sort of priority order to them, and actually putting them on the roadmap, as opposed to treating it as a one-off incident. Sometimes people get in the habit of saying, "We did this post-mortem, we went through the overall exercise, we've done the immediate fixes and identified the longer-term fixes." But if you're not actually able to go back to the roadmap and allocate them to it, then you've not made that learning part of the process.

[00:13:15] The other thing is, once you've come up with the roadmap and now you're working on a particular project, you have some sort of tech lead associated with it. There's going to be someone from the engineering side, and if it's a client-facing one, probably someone from product and design, and depending on which industry you're in there's probably going to be other cross-functional input as well. The data-driven approach you're trying to take is not just about the roadmap and the projects you'll be working on, with some understanding of the high-level timeline, but also about the milestones associated with the project. Come together as an overall group and look at the various features you're discussing and the business impact each is going to have.

[00:13:57] What can sometimes happen is, because everybody has different perspectives, you lose sight of the higher-level goal of what you're trying to accomplish and how it fits into the larger business. Maybe you're putting more weight on your own context as you come up with the different milestones and features. As anybody who's done some sort of leadership role in tech knows, people are always going to want unicorns and rainbows. Even when something is delivered, there's that interesting discussion, which also went back to the tech debt thing for me: "This is a bug." "No, actually this is an unsupported feature, based on what we came up with at the time." It's educating people, especially non-tech people who may not be familiar with it. Of course later on that becomes one of their favorite phrases when discussing with the engineering, product and design teams: "No, this is tech debt," or "This is on-call," or "Oh no, this is an unsupported feature."

[00:15:03] That continuous learning approach, as well as the data-driven approach, is what you want to be taking even while you're coming up with it. The other part is that when you launch something, as I said for some of the client-facing areas and maybe the infrastructure stuff, you will potentially have made some assumptions even with the data you have. So you want to give yourself some post-launch time on the roadmap to take in those learnings once you've launched: "These are the issues we're seeing, given the features we came up with, and this is the time we want to spend making those fixes." As opposed to, "Okay, I launched. What's the next feature for me to work on?" and then everything is dumped onto the on-call engineer, who potentially has no context on it. The flip side is that at some point you do need to do the handoff, so that the individuals associated with the project don't become single points of failure: "You worked on that particular integration with that third party, so I'm always going to call on Alice and Bob, and they'll always fix it," even six months down the line.

Aligning people's motivations with the company goal

[00:16:10] The other part that was a fun learning experience for me is people. Again, going back to that data-driven approach, coming up with the roadmap and figuring out the overall company focus. I think the job of any person in some sort of senior, leadership or manager role, whatever term you want to associate with it, with its various definitions, is that when you have some level of responsibility, and you become responsible for an area and/or the people associated with it, your job there is to make the company successful. But you're also beholden to the people associated with that area and that overall larger team, so to speak.

[00:17:03] One of the good quotes that I've read, in a book by Ray Dalio, which I focus on in my overall management and engineering philosophy, is that the job of a leader is to figure out an individual's motivations and then make sure they align with the high-level company goal and strategy. Some people might want to work on the most challenging things. Another person cares more about their overall team, the type of people they're working with. Another person may just want to work on the most shiny features. Another person just wants to work on the most impactful things, and somebody else maybe just wants whatever is going to get them to the next step in continuously growing and learning. So figure out exactly what those motivations are and then align them to that higher-level goal. I believe that's what one of my managers in the past was able to do so well, and how he inspired me to focus on that particular area, versus my very initial thinking that I wanted to work on the client-facing initiative.

[00:18:08] Going back to the data-driven approach: once I was able to come up with those metrics, for staffing, in terms of hiring people externally as well as some of the people we moved in internally, the process we followed was: what are the company priorities? This was definitely a company priority after we had justified it with the metrics we had. What are people's skill sets? There were quite a few people who had the skill set and could help. And then what's their interest level? That's where the motivation part comes in, figuring out exactly what that motivation is. It's extremely subjective, and it comes down to communication, because that's the most important aspect of any relationship, especially in terms of overall growth for an individual. The cool part was that, given those metrics, and being able to show them how best they could make an impact and how it aligned with what they were trying to achieve and their overall larger career goal, not just while at this particular company, I had a bit of an interesting problem one year down the line: most of the people actually wanted to work on this area, and where we were heading and the impact it was going to make, versus the client-facing initiatives, so to speak.

Onboarding during rapid growth and people first

[00:19:17] The other interesting part that came out of it, and this is something nobody teaches you in school, industry being the best learning experience you can ever have, is that we were hiring a lot of people and onboarding them: interns, people straight from college and university, as well as senior hires. When you're going through these rapid growth phases, you may not necessarily have full capacity at that particular time, and for some period of time onboarding them may actually be a negative for you in the short term. Earlier in my career that's certainly something I did not appreciate as much. While trying to deal with the system, onboard people and keep up with some of the other new client-facing initiatives and areas we were looking into, it was certainly hard to grasp or even understand. I think it's one of those things you have to go through yourself, rather than somebody telling you, "No, no, Rija, believe me, this is how it's going to turn out, and this is exactly how we should do it." It's interesting, as I'm now trying to onboard the younger members of my team onto the same thing, as we're hiring quite a few people at MarketInvoice.

[00:20:34] When I reflected back on what we had been able to accomplish in the one-to-two-year time frame, given what we were trying to accomplish with most of the team just hired within that time frame, what I appreciated most was this. Yes, a lot of what we did was very technically challenging. We had to deal with various constraints, both internal and a lot of external ones that were just outside our control, but we had to somehow fit everything together, and in one case with a compressed timeline that we just had to put up with. But in all of this, when I reflected back, I didn't for a single moment doubt that we would be able to accomplish any of it, purely because of the people I had within my team.

[00:21:21] The high-level lesson, and something I've taken away from whatever career I've had, is that people are the most important aspect of any entity, organization or project, and if you have the right team you can overcome any challenges associated with it. It is extremely important to have the right talent, to make sure you understand them and they understand you in terms of what the higher-level vision and goal is, how their motivations fit in, with as much of your input as you can provide, as opposed to purely being the company's megaphone. There's going to be that high-level vision, and it's not like you're gossiping behind its back, but your job, and the reason they're reporting into you, is that you're able to provide them with information they may not all be privy to, or some more color on it from your perspective.

[00:22:09] This is also something I'm focused on as I try to grow my team at MarketInvoice, from where they were when I started four to five months ago, to where we are now and where we plan to be in the next couple of years. It's the highest priority for us and we're keenly focused on it, because ultimately these are the people who are going to be delivering on that high-level vision and strategy for you. It's also one of the reasons why I think working within a startup, versus a very large corporation, can deliver a lot of results for you, because people can be motivated very early on and see the impact very directly. That's all I have. If you have any questions, happy to take them.