Decentralised Teams: Do they perform better?

Silvanus Lee, Swara Mehta, Swetha Shankar at UXDX APAC. Video: https://youtu.be/7XI-d8upcjA

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introductions: Twitter, Stripe and Red Hat

[00:00:00] Host: I'd like to start off, if that's okay, with getting a little bit of background. If you could give the audience a bit of background on your teams and how they're structured. Silvanus, if we could maybe start with you, if that's okay. Gentleman first, I guess, because we're a panel of females.

[00:00:18] Silvanus: Thank you, Catherine, for the intro. My name is Silvanus. I am based in Singapore; I've got my "I love Singapore" t-shirt on. I'm currently the director of engineering for the APAC engineering center at Twitter. We are about 100 engineers, data scientists and product folks, distributed across our tech hubs in Singapore and India, and a handful in Australia and Japan as well. Of course our teams work closely with our global engineering teams in the US, in the UK and elsewhere. Here in APAC the teams work on all aspects of Twitter's products, including consumer experience, machine learning, data platform, and even traffic and network engineering to keep the lights on.

[00:01:03] Silvanus: Apart from my day job at Twitter, I'm also the co-founder of an enterprise AI company called BasisAI, which I co-founded in Singapore to help companies accelerate the path to productionization for AI applications. Before moving to Singapore in 2018 to co-found this company, I was in the Bay Area for 12 years, and I was fortunate to be part of many high-growth companies, such as Dropbox and Uber. At Uber in particular, that was my first experience managing globally distributed teams, as the director of data science, and the teams were spread out across San Francisco, New York and Bangalore. So this topic is timely. I'm really passionate about building high-performing engineering teams at scale, and I believe that decentralization can be a compounding advantage for any high-performing engineering organization. So glad to be here.

[00:01:56] Host: Don't give away too much yet. Swara, how about you? What's it like at Stripe?

[00:01:59] Swara: Hi, everyone. I'm the data science manager at Stripe. I've been here for a little under a year. At Stripe we build programmable infrastructure to facilitate the movement of money, so it involves multiple solutions, including payments but not just payments. I've previously worked in early-stage startups as well as large multinationals, so just like Silvanus, I've been through different variations of geographically distributed companies. I'm currently located in Singapore, and I lead Stripe data science in Singapore.

[00:02:38] Host: Nice. And Swetha, can you talk a bit more about Red Hat?

[00:02:42] Swetha: Sure. First of all, I'm Swetha. I'm director of engineering and site leader at Red Hat in Bangalore, India. It's a pleasure to be with you here. A little bit more about myself: as you know, Red Hat is a truly open source organization. Not only do we have a global team spread across all parts of the world, and even remote people working together with us, but we are truly open source communities, working across the world with communities, with partners and with customers equally. Prior to Red Hat I worked for around 18 years with Cisco, so I've been across the journey from traditional software and hardware systems to completely cloud-based systems, understanding how we work in a global environment, working with global teams in different kinds of software industries, be it traditional or modern. It's a pleasure being with you, and I'm looking forward to the panel.

Why decentralized teams: talent, time zones and empathy

[00:03:52] Host: I'm actually going to stick with you, if that's okay. I'll dive into the first piece, which is why do decentralized teams? What's the value?

[00:04:08] Swetha: I would say we cannot contain talent within a single boundary of an organization, and a decentralized team really unlocks our ability to tap into the potential across the world. You have the best talent coming from various parts of the world, working together in a cohesive environment. Second, you have different perspectives and ideas coming through. Today we keep talking about having diverse teams in order to bring in various perspectives, and in a very similar way decentralized teams bring the various aspects together.

[00:04:58] Swetha: Talent is the main aspect when we look at decentralized teams. The second aspect we talk about is that in certain functions it's very important that we have round-the-clock support for our customers. How do we actually enable customers to have a fantastic experience through the systems? So I would say a lot of things, in terms of time zone experience and in terms of talent coming together, really work well with a decentralized system.

[00:05:35] Host: It's an interesting one. Silvanus, how about you?

[00:05:42] Silvanus: I think Swetha covered a lot of the main points. Talent obviously is number one. There's this realization now that great talent is not constrained to one single geography; in fact it's globally distributed. For us, today is actually a proud moment, because at Twitter we just announced the official opening of our Canada international hub, and this complements our international sites in Singapore, Bangalore and London. This is a company where we went on record to say that our ambition is to increase our impact by enabling anyone anywhere to work for the company. To do that, we must be able to tap into global talent regardless of location, and it's also a chance to build more diverse pools, which is an area of great priority for us.

[00:06:29] Silvanus: The second benefit, as Swetha mentioned, is follow-the-sun operations. At a company like Twitter, we are always keeping the lights on; the service is available 24/7. For teams such as SRE or traffic engineering, the fact that we're distributed in Europe, in APAC and in the US helps us ensure there's always time zone coverage, and this has actually turned out to be a major strategic advantage, as we're able to scale to meet the huge spikes in demand.

[00:07:03] Silvanus: The third one, which I think Swara will also be able to touch on more, because I think Stripe is really the leader in a lot of this, is a sense that if you have teams in every region where you have customers, it gives you a lot more empathy. You're able to have boots on the ground and build your company, products and services to better reflect your audience, by leveraging the local context and cultural insights. For us at Twitter, 80% of our users are outside the US, so it's imperative for us to build teams that are able to serve these markets. In fact at Twitter we're looking to better serve emerging markets like India, like Nigeria, places in sub-Saharan Africa, and we're thinking about how we want to build our engineering and product teams there.

[00:07:53] Host: It's a really interesting one. We had Jesse Link [?] talking about the London office and the piece on Twitter globally, what it's like. It is fascinating the way Twitter are moving towards decentralized teams globally. Swara, as Silvanus mentioned, how is it for you from a local point of view? Why would you promote the benefit of decentralized teams?

[00:08:21] Swara: In addition to everything we've spoken about, one other benefit in my mind is the locally relevant products you can build with much greater efficiency if you have decentralized teams. Tech companies today, whenever they're looking to launch in a new market or grow in a certain market, need to get the right product-market fit and offer solutions that are relevant to the local population. Sometimes customizing existing solutions is not enough. Sometimes it requires building entirely new solutions from scratch, and that's where decentralized teams add incredible value.

[00:09:00] Swara: For example, at Stripe we've always been strongly positioned in the credit card space, but credit card penetration in APAC is extremely low. If we were to really do well in APAC, we would have to build products and payment methods, like supporting e-wallets, digital payments, direct debits, all of those things that are more relevant to the APAC payment ecosystem, rather than just customizing the existing credit card applications we have already built globally. So I think building locally relevant products is a great way for decentralized teams to get started. In addition, I think companies like Stripe can also empower local APAC businesses to grow globally, which means we build hyperlocal solutions but we enable their growth globally as well.

The challenges: knowledge sharing, time zones and tooling

[00:10:02] Host: You're muted. Apologies. And what are the challenges at Stripe?

[00:10:07] Swara: The challenges are a few, but the first one that comes to my mind is the cross-pollination of knowledge. We have to make sure that if each decentralized team is working in a relatively autonomous manner, we are still cross-pollinating knowledge across different teams and different areas, so that as an organization we're still sharing ideas and learning. That same thing benefits individuals as well, because it makes sure individuals get enough visibility and acknowledgement of their work outside their immediate regional team.

[00:10:43] Host: Interesting. Swetha, would you have anything to add on the risks from your perspective?

[00:10:52] Swetha: I would say that just as time zones are a fantastic advantage, they're equally a risk or a challenge to face in decentralized teams. Imagine someone in New Zealand working with someone in the US, probably in a central time zone. It's really difficult to match the communication and keep in sync, so there are a lot of challenges in getting over the different time zones, as much as it's an advantage. The other aspect, I wouldn't call it a challenge, but something we need to iron out, is how we want the tools and processes set up in decentralized teams to enable this. Sometimes just ironing things out between all the global teams is equally a challenge, just to get this decentralization working together.

[00:11:48] Host: It is really interesting. I'll just take a quick second to say to people: if anyone has any questions for the team here, please do bring them in, because I know there are probably a lot of people who work with decentralized teams. I think most engineering companies and designers are working globally, so if you have any thoughts, ask the speakers and we can go from there.

Structuring teams across sites

[00:12:13] Host: I'll move on to teams and structure, because there is the age-old question of culture: can you get people on board globally if they are decentralized? Silvanus, I'll ask you: how does the structure of your team work?

[00:12:37] Silvanus: That's a great question. It's not all roses and butterflies; there's a lot to overcome. The one thing I'm always mindful of is that we have to be really intentional as a culture in investing in a decentralized model. It can't just be an afterthought. You can't just say, "Let's hire up a team working on XYZ in this location." We have to be thoughtful, and that's what we have done when we think about setting up these various sites. There are various models that can work. There are sister teams in different locations working on a common service area, where you define the asynchronous handoffs and how you're going to ship code effectively in that model. I've seen some companies adopt a center of excellence model as well, so one site is known, for example, for its prowess in mobile engineering. In the case of Twitter, we've tried to cluster the teams in Asia Pacific to work on various aspects of machine learning, data platform and data engineering, in part because of the high-grade talent and specialization we're able to find.

[00:13:36] Silvanus: So the first thing is being very intentional about how we set up these teams, and then, as Swetha talked about, having the tools and processes to overcome the challenges you will face in terms of time zones and asynchronous handoffs. Going back to the team at Twitter, we started with that nexus of machine learning and data platform, but as we scaled we were able to bring in more teams once we reached critical scale. Now we have probably over 100 engineers in the region, so we're able to invest in more adjacencies. For example, we spun up a traffic and network engineering team, and now we have teams that cover a large part of the product. We even have teams working on the new moonshots, as Twitter is thinking about reinventing its products. It's been heartening and encouraging to see the trajectory we've taken with the Asia Pacific site.

[00:14:35] Host: And Swara, what about you, from a data perspective? What is it like with your team?

[00:14:44] Swara: One other thing I've seen work particularly well in terms of team structure is making sure that different roles are represented within the decentralized team. We have enough engineering, data science, product and design folks, different roles, within the same working group, and I think that cross-functional working group really leads to autonomy. I've seen that if only a few roles are represented, there can be times when you're still blocked on another decentralized team or waiting on someone else. That works particularly well at Stripe. We have a lot of cross-functional teams with representation from various different roles, and that's when we all work on a problem together and can develop code and ship it sooner.

Convincing management to set up a site

[00:15:30] Host: Swetha, I'll ask you: how do you convince management to set up a team in a decentralized location? Has that been difficult for you?

[00:15:41] Swetha: Actually, it's not been so difficult. One of the interesting things is that when we have open source communities across the world, you recognize that the talent pool is something you need to tap into. So convincing management that the talent pool is available throughout the world is a very easy job from that standpoint. But once you've convinced them there's a talent pool, how do you go forward in establishing it, actually locating these decentralized offices and making them work? That's a much bigger challenge, I would say.

[00:16:26] Swetha: The second aspect: do you have to convince management on the cost effectiveness of decentralized teams, or on their performance or effectiveness? That comes secondary, because once you look at the talent pool that's there and that you want to tap into, it's a much easier job to overcome the rest of the challenges. So I would say the talent pool is the convincing factor for most teams setting up decentralized teams across the different global centers. That's the crux, the core of what matters.

[00:17:05] Host: Silvanus, how is it for you? How do you convince management to set up decentralized teams?

[00:17:12] Silvanus: One of the things I was fortunate to experience at Twitter, where I've been for the last year, and I was explicitly attracted to this role because of it, is the commitment to decentralization. Even in the early innings of the pandemic, Twitter was very quick to double down on its commitment, and it's not just globally distributed teams, it's also the commitment to remote work. We said that anyone should have the option of never having to set foot in the office, and this is something we take quite seriously. With this backdrop, the culture is set. There's a recognition that talent is globally distributed, and to attract the best people we need to scale our processes, our systems and our culture to fast forward to a future where it's really decentralized.

[00:18:12] Silvanus: Again, there are challenges to overcome, and we talked about a couple of them. One is ensuring that teams can ship effectively and with high velocity, even in a decentralized setup. Another is ensuring that your tools are able to scale. Hopefully we'll get more chance to dive into technologies we've used to reduce friction in the collaboration of these teams. But it's also things like the tech stack. If you have a very monolithic architecture, it makes it really difficult for different engineering teams to ship effectively, because they're always blocked on each other's work. So how do you refactor your tech stack to enable a world where different teams are unblocked and able to collaborate effectively? That could be an upfront engineering investment you have to make. Going back to your question, at Twitter we're fortunate to be able to say that this is something we want to solve for, and we're prepared to make the investments in technology, processes and systems to get there.

[00:19:11] Silvanus: From there on, my job is really to work with engineering leadership to understand which teams would be ripe for bringing to the various sites, and how we can situate these teams for success. We talked about clustering people with similar skills together when it makes sense. This is also great for the site, because first, it helps build a reputation for what that site is known for, which is great for internal positioning as well as storytelling, and second, it provides more opportunities for skills transfer and mobility between teams for the engineers who are going to be hired into your site. To wrap up, I would say it's important that the company recognizes this is a commitment they want to make, and then has site leads in place, actually hiring people who can help navigate the different complexities: what kinds of teams to set up, how we make sure we have a strong hiring bar, how we make sure these teams have a strong culture that maps to the global engineering culture, and then making sure we scale those teams according to what we think makes sense.

Working with a monolith

[00:20:26] Host: It's really interesting, and, Swetha, I saw you were nodding there. Do you think you can do decentralized teams with a monolithic stack?

[00:20:36] Swetha: That's something I really faced at Cisco. We were in digital video software, and it was a completely monolithic stack, not in the modern software sense, but completely hardware and software related, and one single codebase. What becomes very interesting is that while we'd like to put in the modern practices and refactor the code, like Silvanus said, there will be a period of time when you have to deal with the monolith. How do you have the decentralized team deal with this monolith of code? Here I would say there is no silver bullet.

[00:21:15] Swetha: But first of all, even in a monolith you do have modular functions, so it becomes very important for the decentralized team to pick up one of the functional areas within the monolith and go forward. The second is that the contracts, the negotiations of how the software will work between the parts, are very, very important. It's pretty much like a waterfall model: you have to communicate upfront, "This is how my software will start working when I put in this piece of code." So there's a lot of negotiation and contracting before the teams actually go away, do their bits and put it together.

[00:22:04] Swetha: Finally, when the entire piece of software comes together, integration in a monolith is a little more tedious, to say the least, because you're not working in a system that is running all the time and that the decentralized teams can visualize. So sometimes monoliths do have the overhead of having the upfront contracts in place, bringing things together and working through the testing systems and the integrations and all of that. It's possible, but I would suggest to teams who are in monoliths to do continuous refactoring into modern software practices.

Setting goals and roadmaps, and avoiding conflict

[00:22:45] Host: It's a valid point, and it leads on to the next piece, around decision making. Swara, I'll ask you, if that's okay: how are goals and the roadmap set out for your team?

[00:23:02] Swara: Decision making is definitely something that needs to be done with deliberation. One model I've seen work particularly well is when the strategy principles of the entire organization are laid out in advance. In my mind these are the fundamental vision of where the company wants to go, and that should be done at the organizational level. After that, how to achieve this vision is where decentralized teams can decide their own strategy, their own goals and roadmaps. I've seen that it works particularly well when the "where" is aligned across the teams. How to get there is definitely something we should give the teams autonomy on.

[00:23:52] Swara: One thing within that which works well is that once the distributed team has decided its strategy and roadmap, let's say for the next six months, each sub-team or each cross-functional working group can then break that down into individual projects or items, and they should have the autonomy to pivot or change some of those implementation details, as long as the overall vision is still aligned. That basically ensures that we have alignment, but enough flexibility to make changes on the ground as needed.

[00:24:30] Host: Silvanus, when you're talking about decision making, how do you ensure within the Twitter team that you avoid conflict?

[00:24:41] Silvanus: I think Swara mentioned a lot of great points that we definitely try to adopt as well. What I'd add is: have really clear drivers and decision makers, make sure it's well defined, and write everything down, because in an asynchronous environment you've got to make sure you have artifacts and documents that clearly index how decisions are being made and what action items need to be taken. We've also got into the habit of recording our meetings, because we cannot assume that everyone's able to attend the meeting synchronously. It's cultural. The first few times it might feel awkward: why is every single standup or team sync being recorded? But after that it becomes second nature. "I know that 80% of the team is here, but 20% of people might want to refer back to what we discussed."

[00:25:37] Silvanus: So we do that, and obviously overcommunicating, making sure we have really clear systems for project tracking and accountability. And of course, if there's a possibility of leveraging time zone overlaps for synchronous collaboration, we try to do that. I'm in Asia Pacific, in Singapore, and I try to start my day at 7:00 a.m., so it gives me at least a three-hour overlap with my West Coast peers. Similarly, the folks in India might start their day later, but it gives them some overlap with the West Coast, and of course overlap with London later in the afternoon. The combination of synchronous and asynchronous collaboration really helps.

[00:26:20] Silvanus: The other thing I would say is that because everyone has gone remote, I actually think it has leveled the playing field in many ways. Now it's not as if 90% of the team is building in one location and we are the odd one out. Everyone is remote, wherever in the world they might be, so everyone's a bit more intentional and a bit more aware, and I think that's really helped our cause.

[00:26:44] Host: That's really... Swetha, sorry, you were going to add something?

[00:26:49] Swetha: Yes. When the decision making is within the organization and within the decentralized teams, that's one way to look at it. But when you're working with open source communities, and your product is truly distributed between open source, the external communities and the internal organization, decision making can be a little trickier, and it becomes very much an act of influencing as well. You not only need to communicate and overcommunicate, like Silvanus said. It's also about ensuring that others understand the intention of why you're doing a certain thing. So a lot of influencing, and trying to align the different people and stakeholders, becomes equally important when you're doing the decision making, I would say.

Career progression away from headquarters

[00:27:42] Host: Absolutely. I want to move to the final piece. I might have touched on the pandemic if we had time, but we only have a few minutes left, so finally I'll go through career succession, if that's okay. I think it's really important. Your head offices are effectively in the US, so how do you get ahead in your role? Swara, I'll ask you first, if that's all right.

[00:28:21] Swara: We've tackled this problem in multiple ways. We definitely ensure we have lots of mobility options for people. There are career progressions within your current role, in the trajectory going forward. We also have career progressions into a different role; for example, if you just want to try out something new, that's always an option at Stripe. And within your role, it's always possible to partner with a different business team, for example. So we ensure we have as many mobility options as possible across roles.

[00:28:57] Swara: In addition to that, one thing we typically try to do is have a career plan for each person, and we ensure that the products being built out of, let's say, EMEA or APAC or any of our other decentralized teams have enough critical mass, enough actual product to ship, so that they're not just making incremental value additions to what headquarters has already built. I think that has typically worked as a good motivating factor for everybody who's looking to build a career out of an office that's not the headquarters.

[00:29:38] Swara: The last thing I would say, and I think Silvanus pointed this out earlier, is that the fact that everybody is remote has leveled the playing field, which means the only thing distinguishing people in a decentralized team outside headquarters from people at headquarters is the time zone. As long as the communication piece is there, we do allow people outside headquarters to be tech leads or project leads of projects that are not headquartered in their region as well. So that's a career option for some of our folks.

[00:30:13] Host: Swetha, how about you? What is your opinion on that? How do you get ahead?

[00:30:21] Swetha: I would say much the same as what Swara said. In today's environment there's a level playing field, and that level playing field actually gives us the opportunity to rotate roles within Red Hat, taking on something new or taking on stretch goals, which allow you to explore the gamut of management options or technical options. This allows you to explore your growth path on both the technical ladder and the management ladder, and we have done that on both: how you can rotate roles on the management side and on the technical side.

[00:31:10] Swetha: The second, I would say, is that there are regional roles as well as global roles, and identifying both allows a person to progress in their career and make a jump. And like Swara said, we have mobility options: if a person decides, "I can play this role much better in an EMEA location," we always enable that. So I would say it's a combination of career setup, rotation of roles and progression.

[00:31:47] Host: Great. And Silvanus, did you want to add something?

[00:31:52] Silvanus: In addition to all the great points, I would add that, having built my career at headquarters in the US, and more recently having taken on a role as a site lead for a decentralized outfit of a US company, I would say something that hopefully is not too controversial: with the right company or approach, being part of a decentralized team can actually be a career accelerator. Particularly with the culture we're trying to build at Twitter, we've seen engineers and data scientists branch out from locations in the States, where maybe they're American or they've worked in the US, and come to Singapore with a new team to try different things.

[00:32:39] Silvanus: I think the pitch is that in a decentralized site you almost have to optimize for being a bit more scrappy. You have to hustle a bit harder, but that translates to a lot more learning opportunities and growth. The reason you have to be a bit more scrappy, almost entrepreneurial or resourceful, is that sometimes things are less structured, and you've got to make the case for why this team has to exist or what you want to do with it. But again, I say with the right company and culture, because at companies that give you a lot of autonomy and leeway to experiment and try new things, if you come in with the right attitude you can take on a lot more scope than you might otherwise be able to.

[00:33:20] Silvanus: The other thing about being part of a decentralized team is that you get to pick up really strong communication and stakeholder management skills, because you have to learn how to influence, pitch your impact and scale your impact, and I think this is really good for opening up growth and learning opportunities down the line. I don't know if this will become commonplace, hopefully, but I actually think the job at this site has been the most rewarding, exciting and challenging one yet.

[00:33:52] Host: Very interesting. I'm going to have to leave it at that. Thank you all so much.