The QA Designer

Diogo Cunha at UXDX Community: AI, Low-Code & Designing Quality. Video: https://youtu.be/aOVO-O6Bt68

Readable transcript: edited from the recording's captions for readability (fillers and false starts removed, punctuation and section headings added). Wording is the speaker's own. Timestamps are positions in the video. Names marked [?] could not be verified against the audio.

Introduction: a Brazilian designer who reads memes

[00:00:07] Hi, Harry[?]. Thanks, and thanks UXDX for having me here today, and thanks to everyone who is watching. I'm here to talk about the QA designer, since QA is a crucial step in digital product development, and we will see how designers can help in this moment.

[00:00:44] First I will introduce myself. I'm Dio. I'm a Brazilian designer, product specialist, consultant and entrepreneur. I've been working with design for around 20 years for many companies, especially startups and tech companies in different areas: advertising, branding and product design.

[00:01:17] I will start first by talking about one thing. As I said, I'm a Brazilian, and I know that some of you know some information about Brazil, more related to the carnival and football. But I want to first talk about one thing that maybe few people know about Brazil: that Brazil loves memes. We use memes to send a message, to talk with friends, to comment on news, on Twitter. It's like a second language for us. We really love memes.

[00:02:06] For me as a designer, I always look at memes with different eyes. I think that memes reflect what is trending in society. Every time that one topic is trending in society you will find a lot of memes about it on Twitter, for example, and if you dive deep on it we can understand the momentum of society by researching these memes. I think that this is the same for different communities.

[00:02:48] As designers, for example, I always see some design memes on the internet. As we can see, we have here three examples of design memes related to how we manage our work on software, about UX and about UI and the relationship with the dev team. Of course memes are jokes, they are fun, they are created to be fun, but even with a joke we can extract some good insights.

[00:03:35] If we start to analyze these three memes we can take some good insights. For example, one insight is that designers who keep layers organized are true heroes. And this is really true. The second one is that the user doesn't care how much time you spent designing the interface. They will use it as they want. And the third one is that devs and designers need a conversation.

[00:04:15] As I said, it's always a joke and it's fun. But if we start to think more about them, we can find some more insights that we can use and we can apply in our daily work. For example, when we see that one meme related to the users using a product in a different way than we designed it, we can already see some insights from it. And the other one is when the final product, after the development, works differently from the original designs. These two cases are related to one specific thing in my vision: the final product.

The product development process, step by step

[00:05:21] It's a good point to start a discussion about what these memes are trying to say to us, and to talk about it. I think that we can start to think not on the final product, but on the process since the beginning that results in the final product. That of course will be different company by company and method by method. But in general, the process to create a digital product is covered by these steps.

[00:06:08] The first step is the discovery and ideation moment, where we start with an idea and we start to create and ideate on it. We have a great many different methods: design sprint, discovery sprint, ideation methods. After that we start to plan and validate the assumptions that we generated in the ideation, in the discovery. So we start to do some research. We start to design the first wireframes and validate these ideas with research, with tests, with user tasks.

[00:07:01] In this moment we start to evolve towards a product and start to create the interface, the user flows and the design system, and it evolves into a whole product on the design side. After that we hand off to development, to the dev team. Of course the designer will in this moment keep providing support, but most part of the work at that moment is related to the dev team. And after that we will launch the product and the product will finally be in the users' hands.

[00:07:57] This process is not a line, as we can see here, but a circle. Because after that we will see how the product works in reality, live, and we will have a lot of data, and this data will be crucial for us to understand the product and start to create new features, iterate the product, and start the ideation step, and after that validation, and then developing new features and launching a new release.

[00:08:36] But as I said, design is more related to the first steps on this line. We are in the discovery, the ideation, the validation. We start to create the interface, the user flows, a lot of research on it. And after that we will hand off to the dev team. That final part is the moment when the product is finished, to go to the users' hands. And if you think that for most part of the process the designers were working on the product to make sure it aligned with the vision about the product, the better user experience that we design, does that make sense, that in the end of this process we just don't get to be part of it?

Why designers should be in the pre-launch steps

[00:09:40] As you can see here, let's dive deep on this last part. Again, it's general tasks, but in general it covers most part of the work. As you can see, we have the UI/UX design, the interfaces and design system, that are finished, and after that it goes to the front end and back end development, API and third party systems, system integrations, QA and performance testing, and after that the launch. This is a very crucial moment, and as a designer I always saw it and thought that designers need to be in this party too, and work on these pre-launch steps.

[00:10:48] In the last three years I've been working with the Realio team. I worked with the Realio team. Realio is a web3 tech company and investment fund that works especially with real world assets. Realio has a lot of great products created based on the Realio Network, that's the multi-chain web3 ecosystem focused on issuance and management of digital and native real world assets.

[00:11:31] And we work on products such as the Realio Network platform. We've been working on Freehold, that's a crypto wallet app, and Districts, that's the metaverse focused on the RWA assets. During this time working on different products, and multi products, with a small team, we faced the challenge to keep the quality of our products, to make sure that the vision that we have in the first moment about providing a web3 ecosystem with products related to RWA, and that create value for the users.

[00:12:44] This is a really great and big challenge and we started to think about how we can make sure that in the end, when this product Districts that we are seeing here, the interface, will work great in the users' hands. And the way that we found to solve this challenge, to face this challenge, is designing a QA process.

Designing the QA process: why, structure and team

[00:13:20] As I mentioned, the Realio team is a small team but a very talented team. We started to think how we can improve the QA process, that for us is very crucial to launch the projects, the products. The first thing that we thought on designing this QA process is why. And it's basically because involving those who planned the product in the pre-launch steps is a better way to ensure that the final product is what we planned.

[00:14:07] Our main idea is that we start in those first steps that we saw in that chart with the people who are thinking about it, have the vision, and start to design the business and start to think and create and plan the product. And the designers come in on the project and start to research and design and test. So for us, the QA process needs to have these people working on it to make sure they are the specialists on the product, on our products. We thought that it will be interesting for our products.

[00:15:12] Diving into this QA process design, we started to think on the process and the structure. For us the better way to do it was to create a QA department that works with rounds of testing for each feature, one feature or product each time. So every time that one new feature will be released, before release we will create one round of the QA testing, and it works for the products too.

[00:15:58] After that we started to think about the team. Okay, if we want to design a new way for us to QA, to test, how can we do it, who will do it? So we designed a department, a team, of the QA manager, that in this case was me as a designer, and I always worked with the dev responsible for the feature or product that we will test. So a new round of QA starts with the QA manager and the dev responsible for the feature working together to create the test.

[00:16:50] And the testers: we don't focus on just the dev team, but as I said, the people that worked on the first steps are important for us in this moment. They bring diversity to the team, not just focusing on development. So it's interesting to have this kind of professional working on the tests.

The tools: guidelines, bug reporting and evaluation

[00:17:30] After that we started to think on the tools. What are the tools that we will need to do this test? First was designing the test guidelines, to provide the information for the testers, all the information the testers need to do their work. And then after that, how we can report the bugs, that are very linked with the task management. So we have a process that when a bug is reported, this report will create a task that will be solved. So we can manage bug by bug to improve the process. And after that, always after a test we will have a test evaluation doc to keep improving, as a product. This QA process needs to be reviewed and always improved.

[00:18:47] I will talk more about the guidelines. I think that is the main part of this process, since the guidelines provide for the users, for the testers, sorry, for the testers, that are sometimes designers, sometimes not, sometimes are the product team, the marketing team. So we try to cover the tests in a way that we can use the professionals' capabilities, to make sure that all the testers involved in the process will provide their best capabilities to improve our product.

[00:19:39] So on the guidelines we have the UI guidelines, that are related to the visual consistency, imagery quality, and if the product is aligned with the design system. These guidelines are, as I said, shared with all the testers, but are very focused on the designers because they have more attention, because they work with the interface. They can find this kind of bug more easily than the other testers. But of course anyone can find these kinds of bugs.

[00:20:25] The UX guidelines are related to the experience, the user flow, evaluating the experience in different contexts, evaluating the performance. So we have guidelines to use the products in different contexts, about the internet connection and devices for example, to make sure that the experience will be great in different situations.

[00:21:03] We have the content guidelines that are especially for the marketing team. They have very much attention on the information, spell checking, links, to make sure that the information is updated during the process. Sometimes we change the information and we need to make sure. The legal team is important to work on these guidelines too.

[00:21:34] And the test cases, that are based on the user flows, user flows designed covering each feature. The test cases usually are created by the designers and reviewed by the person responsible for the feature. So it's the work of these two professionals. The designers are the best ones to create these test cases, since we worked a lot on the user flows and on each detail of the interface. So we are the best professionals in this case to create these test cases. But the devs are very important in this case to make sure that on the function side, on the development, everything is aligned.

Outcomes of the QA process

[00:22:42] After creating it, we started to work with this QA process, and I'm sharing here some insights, some outcomes that we found applying this QA process on the product. We saw that we had a faster bug discovery and repair process. It was easier and faster to find and repair the bugs, because they easily create a task and then we test again. Less rework, reducing the bugs in the release of the product. This is crucial for the user experience in the end.

[00:23:43] Fixing minor adjustments to improve performance and usability before release: minor things that we didn't see in the first steps, on ideating and then designing it, but we find these minor adjustments on the QA test, and this improves the performance and the user experience and the product after launch.

[00:24:20] We found insights for future improvements. Every time that we test we have great insights that will create new features or improvements on the products. And in the end the test cases feed into support material such as tutorials and FAQs. As the test cases are very detailed, because they are based on the user flows, we can easily convert them into a lot of material that will help on tutorials, FAQs, providing support for the users. So it's a collateral benefit that we found on this process.

[00:25:15] I think that's it. I shared a bit of the work of designing the QA process focused on design and how designers can help on this process. Thank you for listening to me. Here are my contacts. Feel free to contact me, and I will be very happy to continue this conversation on other platforms.

Q&A

[00:25:52] Host: Excellent. Thank you very much, Diogo. That was a really great overview of how you've approached a very common problem of designers handing off and not seeing the end results, or the end results not being what we expected. Can I ask a question about cost, because that's often one that comes up with this problem? You mentioned bringing some of the designers back at the end for that kind of QA testing. Was there any pushback or feedback from management that, well, that's going to cost us more? Was there a cost pushback for this approach?

[00:26:32] Diogo: Yeah, sure. When we started to think about it, we had a challenge that I think most part of the companies have. When we start to plan to launch a new product or new feature, the first challenge is the deadline, when we plan to launch the new product. This means that in the end of the process we will use a lot of the main resource that we will use on this pre-launch step, which is the dev team. So for us it's a high cost to take the dev team to the QA, or it needs a lot of rework to solve some bugs that we didn't identify early.

[00:27:38] Diogo: So for us, when we put it in the balance, it's interesting for us to bring the team that worked on it initially back to the process, to have this different eye on the QA, finding bugs early and solving them early, and keep the dev team focused on the final steps of development. But of course the dev team is part of this QA process too.

[00:28:20] Host: Okay, brilliant. But that actually brings us to time. I had loads more questions that I'd love to ask, but unfortunately we're out of time. So I just want to thank you once again for sharing, and I hope everybody out there enjoyed it as much as I did.

[00:28:33] Diogo: Thanks, Ari[?]. Thank you. Thank you UXDX for having me today. It was a pleasure to talk with you guys.