CI/CD for Product Designers: Continuous Research/Continuous Development and a Path Forward

Matthew Conti at UXDX Community: Continuous Delivery. Video: https://youtu.be/zxfe2LLxUoY

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

From webmasters to automated CI/CD

[00:00:00] Hi everyone, my name is Matthew Conti. I have been an independent contractor and consultant for over a decade, originally starting in aeronautical design, then international development, the military, and much more. Over the years CI/CD has become more prominent as the industry has evolved and become more mature.

[00:00:28] Walking down history lane, what CI/CD originally was, back when webmasters were more prominent, was a continuous improvement, continuous development process, where webmasters were trained every six months to a year. They needed to update your systems for security reasons and make sure everything was relevant and up to date. Over the years it's now become fully automated. Usually I like to approach things as a level one, level two, level three CI/CD approach, but for product design I definitely want to get into what continuous research and continuous development are for a product designer.

Research operations: user, operations and market research

[00:01:09] Research operations, research ops: what I have found to be a nice Goldilocks zone for research, primary and secondary, is breaking it down into three different buckets. Each one can be primary, secondary or tertiary. Typically users are the primary effect towards the implementation, utility and usability of findings and the implementation of solutions. So user research, which is more common these days: journey mapping, getting into personas, doing surveys, doing empirical or ethnographic research. It's a very common environment to gain IAO variables, or interests, attitudes and opinions.

[00:02:04] Then operations research is getting into the operational considerations. I break that down into coulds, shoulds, wants and needs. Based on user input, secondarily I will look, from a policy perspective, a legal perspective, a doctrine perspective, at whether this is something from a user perspective we can actually implement. What could actually hurt our users, even though it might make it more effective or useful for them? Are there any bylaws or any legalese that we need to be considerate of?

[00:02:42] And then the market research elements: market trends and considerations of how even the updates to the system, application, software, anything in between, are impacting the market, or how the market is influencing us within our solution. So it's not really one size fits all of user input. I think it's a triangulation of wants and needs, consequences and risks.

[00:03:14] From a research operations perspective, especially when we know society constantly evolves asymmetrically or asynchronously to how we implement, from a human-centric perspective we are not going at the same pace as society is. So there are always trade-offs and considerations, primary, secondary or third-ranking consequences, that take place based on implementation, and what that looks like within the roadmap and how you plan for it. But the great part of implementing research operations and continuous research into the process is really taking into account that, no matter what, the ground that we are on is always shifting, evolving, adapting and adopting to us, or vice versa. If you think of ontological design, we shape the world and then the world shapes us back. As we influence the world, the world influences us in return. It's like M.C. Escher's hand, for any artists out there: it's the hand that's drawing the hand, which is then drawing the hand.

Design operations: single, multi and full factor ideation

[00:04:22] Then, getting into the design operations of things, I like to break it down into three buckets as well. Single factor ideation or development is a one-to-one relationship, when you're thinking of a single slice, segmented designs or implementation within the process, where it's not really touching on or impacting many different pieces of the puzzle. I genuinely think of that as single factor design or ideation, development thinking that you're taking into account.

[00:04:54] The next is multi-factor ideation, a one-to-many relationship, meaning that it isn't involving all things, but it's involving things that are in the closest center of gravity towards that design process, to troubleshoot, to problem solve, to think through, to chew on, to validate, dealing with the divergence and convergence of effects.

[00:05:25] And then full factor ideation: one-to-all relationships. How does this, from a 360 view, impact everything around us? Usually, when you get into a wider scale of relationships, you're dealing with specificity and sensitivity, usually because it's either very revolutionary, it's innovative, it's disruptive, it requires a lot of change. So think of single all the way to full factor: as you go from single to multi to full it gets more complex, but it can also become more risky, and you have to take in the challenges, the compliance and the consequences of those things. And again, it's not one size fits all. When you're designing something you don't do it in broad strokes. Even when you're implementing a vision, the vision can also be distilled down into vertical and horizontal design prototyping.

Design thinking grounded in design theory

[00:06:23] Design thinking as a whole: I came from a background of design theory. You can't design think if you don't know what design is. So my foundations are in design theory, not design thinking, because that helps me understand how you think about design. So instead of just bucketing it into one design thinking umbrella: as I'm sure many people have gone through the design iterations, chewing on ideas, going back and forth with developers and things like that, there's a push and pull, almost like concentric circles and an overlapping Venn diagram of different types of how you think within that design process. Whether it's design critique, design criticism, pluralistic walkthroughs, design workshops or design studio, it's not predicated on just one element over any other; they're all important.

[00:07:23] Some might consider visual design a subcategory of communication design, but it's all based on the organizational structure and what makes the most sense for your workflow and process, which is all valuable and very useful. But each one has its own pockets of very robust and nuanced considerations, and even philosophical nuances.

[00:07:45] From that design thinking practice, as we evolve and as applications have matured, or we put out a release: six months later, three months later, "OK, now that we've gotten to this point, where are we now? How do we think about users now? Did we meet our mark, given certain limitations because of our infrastructure? Let's get user research going again, or operational research, or market research, and think about how we can further synthesize this for the sake of our users, putting them first." So these different pockets of design thinking are ubiquitous within the whole continuous development process, and even within a CI/CD practice.

Systems thinking

[00:08:34] The next one is systems thinking. I usually don't apply just design thinking. I think in systems theory; again, I come from a systems theory perspective. What is a system, or systems, in order for me to think about it systematically? You have information systems, decision support systems, branding systems, sometimes design systems, management systems thinking, workflow, product service systems. Some of these may overlap, but it's to show you that it's not one size fits all, and it can be very nuanced. If you use specific types of component libraries, or ones that are not fully refined, while some components are more detailed and more thorough: are they one-offs, are they repeated patterns? How are you taking those user interface conventions and supporting them systematically? And how does product versus service correlate, and how those workflows all come to be, is very important.

[00:09:37] If you're applying Conway's law, we create a system or an application that reflects the organizational psychology of our day-to-day. And think of models: you're getting into human-model interaction, or model-based engineering, or model-based decision-making, kind of what was shown with the user journey flows. Those are mental models for people to act upon systematically. But it's not necessarily predicated on a blanket system. It's pertaining to specific nuances of how that user journey is to operate systematically, from a holistic perspective, divergently or convergently for that matter.

[00:10:25] This is to give you a more broken-down distillation of that. Don't limit yourself to what you've already done, and think about scale and scope. How can you build out more, but also how can you build up pockets of unique and novel ideas as you're building the further foundation and activity-centered considerations through the process?

Lateral thinking and futures thinking

[00:10:56] The holistic immersion elements between the research ops and design ops practice: we don't bring this into the full gamut of what divergent and convergent thinking really pertains to, whether using single diamond or double diamond design thinking. When you're bringing in things, whether it's research based, design based, development based or management based, when you start blending that stuff together, pushing and pulling, having the conversations, having the tough messages relayed: is it being communicated effectively? Are we getting past the finish line? Do we have all the needs that are there for us to execute in the way we'd like to? When you bring that stuff together, you're getting into lateral thinking and futures thinking.

[00:11:54] Lateral thinking means you're getting creative. You're getting creative about how you're implementing things, and in what kind of way. What makes sense, what doesn't? What is the intention, what is the purpose? What deadlines do you have to meet? How does the user feel now that that's what we've implemented? How can we creatively still solve their problem, even though we've already delivered it? Now we have a release in three to six months; what can we do now?

[00:12:21] And along the same line is futures thinking. "OK, this is where we are right now. We've implemented everything from a user perspective to a T. Operationally and market-related, it's going to help us better position ourselves competitively, but also meet user interests. What can we do now, with users involved, stakeholders involved, and people involved, and deliver something into the future beyond where we are now?" So it's getting more imaginative, thinking of blue sky, green pastures. How can we now take all of these dynamics in the process?

Experimentation and degrees of reliability

[00:13:01] Some of that gets into experimentation. We get into DOE, design of experiments. And some of that deals with discriminant reliability, where you don't have all the data points you need, where you might think of it as envisioning what it may look like even though you don't have the necessary data points to make it valid. You have inter-rater reliability, where you have some exposure to what could be reliable, but you don't have complete validation on all considerations. Let's say you're 50/50 there, or 60/40, or 70/30, or 20/80; you have some semblance of validation and reliability to that.

[00:13:50] And then you have complete internal reliability, even within futures thinking, where we know we can take this down the road. We know the users love it, because even though this might be a new implementation, we've been following the empirical research or ethnographic research for the past eight months and know that this could be exactly what we need. Or even getting into A/B testing, gathering secondary research, getting into pair testing, getting into split testing, and doing all these different ways to measure effectiveness qualitatively and quantitatively.

The product designer's job once pipelines are automated

[00:14:43] As CI/CD has come to full fruition and it's more automated now, even though CI/CD has been practiced for several years, now that we have these automated systems, the next step for product designers is: we've got to make sure that we're continuously developing our designs to evolve with our user ecosystems and our communities, because they're moving at a different pace than we are. They have different affordances, they have different sentiments towards our application six months down the road, a year down the road, two years down the road. They say you can easily underestimate what you can do in a year, but completely overestimate what you can do in 10.

[00:15:39] So as you are implementing and automating and building up all of these things from an engineering perspective, a development perspective, the product designer's responsibility now is to evolve that product. Sometimes it's leaning things out if they're too costly, and making sure that we're downscaling in appropriate areas because they're not as effective. You don't just lean out within process; you can also lean out within your application, product or service as well. Those are the necessary trade-offs towards making sure that you always meet operational, user and business interests. But more importantly, you're staying modern, you're staying with the zeitgeist of the industry.

Benefits of continuous research and development

[00:16:38] So the benefits overall, as I was discussing a little bit, are having a continuous research and development process, using research ops and design ops as the segue towards the necessary conversations between product management, design and development, because it's never over. The CI/CD pipelines now afford you the opportunity, more so than ever before, to consistently look into user interests primarily, but also think about the operational benefit of making sure you're always meeting market and user needs.

[00:17:18] Then there's the acculturation and advancement of a more human-centered design. There are certain designs out there, certain websites, certain applications that are five to eight years old that are just not doing their job as much as they used to, and yet they're not really trying to change. They're not trying to reinvent the wheel, they're not trying to address more common user interests or demands from what the market says. It can't be human-centered design if it's not human-centric anymore. Because the dynamics of the landscape, of the community, of society, of technology, of the ecosystems around us, are all moving at a different pace, you can easily fall from something that was once human-centric into something that no longer makes sense. That's what CI/CD has afforded us as product designers: always looking into how we can do this better, and where we can think abundantly and with a growth mentality within our services currently.

[00:18:24] But there's also the further proliferation of human-centered computing. This is more common now than it was in the past, but getting into HCC means taking into account how developer experience is just as important as the user experience and design. Human-centered computing is bringing in information systems management, business science, industrial organizational psychology, along with computation, and building out the model-based decision-making that's necessary for human-model interaction. And model essentially means, overall, let's say you're using agile, DevOps, let's say you're applying Six Sigma or some other practice. Human-model interaction is users, and society as a whole, interacting with you as an organizational structure, to say the least. But it's also so deeply rooted in your day-to-day that your updates, all the way down to how your process is, are directly correlated with and impacted by user needs, interests and wants.

[00:19:43] What that does is further new and innovative ways to discover and execute novel, nimble or noble ideas. It's where you're not so rigid about, "Well, we can only do it this way," or, "We've already done it, it's a one-off, we don't have to worry about it anymore." Thankfully the CI/CD pipelines have now given us the opportunity to always think forward, to fail forward, to experiment, and to get into purposeful play like you need to, because that's really what it's there for. You don't need a CI/CD pipeline if it just stays the same.

[00:20:25] That's what it's given us as product designers: to be able to see this as an ongoing process within the design ideations, the development considerations, how we manage and strategize. It's completely changed the design strategy overall, in that you're not going from a waterfall perspective. As was shown with the diagram of the UXDX life cycle, it's constantly going in perpetuity; it's like a living process that never ends. And that's how you think. If you're applying game theory, it's an infinite game. You don't think, "OK, now it's here, and that's it." There's great disruption; there's a lot of sustainable innovation, whether it's recombinant or endogenous, that can take place within that. You can contact me on LinkedIn at artist Conte [?]. Thank you for your time.

Q&A

[00:21:36] Host: Awesome, thank you very much, Matthew. Really almost a technical view of how to incorporate design into a CI/CD pipeline. And you used a lot of graphics, which touches on the last conversation as well.

[00:21:53] Matthew: Thank you, thank you so much.

[00:21:56] Host: Great. Just to remind everybody out there, if you do have any questions, please feel free to post them on whichever channel you're watching this on, and I'll put them through to Matthew. The first question I had, and it's a term I've never come across before: the multi-factor and full factor ideation. Can you give me some examples of those, to make it a bit more tangible? What are the differences in your mind between those three different elements?

[00:22:26] Matthew: Let's say you have a use case where you have to submit a form, and you put all of your information in there: first name, last name, email, like you're setting up a profile or an account. First name, last name, creating a username and password. You submit that, and it creates your profile. That's one factor, a single factor. But then let's say it shoots out an email saying, "OK, now that you've got this, we need a confirmation that this is the correct email." You've got that. OK, now that we have that, let's say we now have to set up multi-factor authentication, or you have the option to enroll in it when you confirm your email. So even though you're essentially just creating your credentials, it can then lead into these different factors that you're taking into account. And let's say when they send the confirmation email to you and you select confirm email, then it auto-populates, as you see in society, where all of a sudden you're signing up for subscriptions to their email list. So then you're taking all of those different considerations from just one instance of creating someone's profile or login credentials.

[00:24:03] Full factor is getting into, well, we can use the same instance. Once they have all of those credentials, if you have MFA, then you're getting into single sign-on, where all of a sudden you can have access to five different applications, like Okta or some type of tool that allows you to sign into multiple different applications. Then those credentials branch out into all the different applications as well, and you have to set up different types of validation for those as well. So it's all a level of magnitude, and how that impacts the greater scale, the splash, as I call it. If you're creating something that completely revolutionizes a ticketing system in an application, the more advanced or complex it is, even though it might not look very complex. I've worked with designers and researchers in the past where it might not look like a complicated implementation, or it looks very cutting edge, but then you realize the ripple effect it has across multiple different areas of your system, or system of systems, or product of products, especially if you're getting into these architectures where you have a one-to-many or many-to-many relationship. So it's never just designing for design's sake, which is great, you should, but you also have to think about the associated risks. Hopefully that makes some sense.

[00:25:52] Host: Yeah, it's a great example. I love the "just give me an account" feature, because that's a brilliant one. You said, "OK, so now we need to validate your email," but wait a sec, what happens if somebody's trying to use the system and they haven't validated their email? How does an account that's partially validated work? And what if they forget their password, and what if they... So it just keeps rippling on, from what seems like potentially a simple request.

[00:26:20] Matthew: Absolutely. You know what you know, you know what you don't know, but then you don't know what you don't know. It's a triangulation of the three. "OK, now that we've done this from a primary perspective, now secondarily, from a research perspective, or a primary or secondary design ops perspective, now that we've done that, how do we validate to make sure we're covering all user needs?" If someone says, "I wish, when I create my sign-on credentials, I just want to make it easy," and I'm sure people have heard that response, what does easy mean? Communication: everything's communicated, but not necessarily comprehended. "Just make it easy," but what does that mean? "Oh, well, send out the email, make it simple, so we don't have to go searching for anything in particular." The cost and benefit of that is never one size fits all, unfortunately. But sometimes the novelty is also in challenging assumptions. Let's say we do that knowingly: are we missing any good ideas in the process?

[00:27:47] Host: That's a great run-through. I have another question; we might just have a little bit of time. One of the things that I was watching is all of those different steps. Do you have a process or any kind of techniques for workshopping ideation, or for doing these steps, that make sure that you're touching on all of those different elements?

[00:28:12] Matthew: I always like to advocate, and hopefully this is taken in jest, but it makes a lot of sense, for the human-centered elements. Human-centric design, human-centric computing: sometimes the process also needs to be human-centric. There have been instances where I've had to create a process specifically based on those needs. For me, I do have toolkits, I have frameworks that I use, I have research templates that I've created myself. But no business, no organizational entity, should ever feel like there's one way of doing it. I would say that they should create toolkits or processes that are defined by their models and the way that their ecosystem is structured, because not everyone might use what they need the way that others want to. So I would just have to say: don't feel limited to one way of doing something, as long as you're solving the end goal.

[00:29:36] Because usually, from a design perspective, we're out to solve problems. When I grew up, and I got my degree in fine arts, there's a practice that I studied called process art, where it wasn't about the final outcome all the time, even though that's very important. The process itself was a form of artistic expression. So take that as an opportunity for you to create a CI/CD process for product design in a way that makes sense for your organization and the way your organization is structured.

[00:30:25] Host: I love that analogy. And that actually brings us to time, so thank you very much, Matthew, for sharing.

[00:30:32] Matthew: Thank you so much.