The UX Scorecard: Transforming Research into Measurable Impact Across the Org

“The most expensive usability test that you’ll ever do is the one that you actually don’t do.”

When Experience Quality Becomes a Release Gate

DocuSign is a big organisation with a broad product surface area. That is exactly why Amit Sathe and Marina Lin decided that “research insights” were not enough. If teams cannot measure experience quality in a consistent way, then usability becomes subjective, usefulness becomes a debate, and satisfaction becomes an afterthought that shows up in support tickets.

Amit opened with a familiar reality for many research teams. Research often gets pulled in as a tiebreaker. Competing opinions collide, and researchers are asked to run evaluative work to settle the argument. That is not inherently wrong, but it becomes expensive in a different way: it creates busy work for a function that is already under-resourced. Instead of learning upstream needs, researchers end up validating downstream decisions.

Marina added another pain point: the product ships first, then adoption does not happen, and only then does the organisation ask for research to explain the gap. Their internal name for this pattern was blunt: expensive guessing. Build on a hypothesis, ship, and then pay for the consequences when reality disagrees.

The third problem was harder to pin down but easy to recognise. Qualitative research does not always get a seat at the table. Not because it lacks value, but because it can be hard for stakeholders to compare it, track it, and make trade-offs with it. When success is not measurable, “impact” is too easy to dismiss.

Then a VP of Design asked a deceptively simple question: Are we doing what it takes to delight customers? That question implied something else. If delight matters, how do you know whether you are improving it? If you are not measuring it, what exactly are you managing?

Why They Built the UX Scorecard

Before building anything, Amit and Marina looked inward. They noticed broken links across their product development lifecycle. Researchers were using disjointed methodologies. Measures varied by individual preference. Benchmarking was inconsistent or absent. Even defining success metrics was hard because each team defined “good” differently.

They formed a small task group with UX researchers and a resident research data scientist. The mission was clear: create a repeatable measurement programme that could travel across teams, create shared language, and show progress over time.

They explored existing frameworks, including Google’s HEART, the Single Usability Metric, and other approaches they had tried to institutionalise in past roles. Some were too broad and abstract. Others were too narrow or too restrictive. They wanted something that matched how product teams actually work, and something that could scale.

So they built their own framework with three dimensions that are easy to explain and hard to argue with.

The Three Dimensions That Made Experience Measurable

Amit and Marina chose usability, usefulness, and satisfaction.

Usability answers whether people can complete what they came to do. Usefulness answers whether the experience solves a real need. Satisfaction answers whether the experience feels good enough to continue using.

Those dimensions mattered because any one of them can fail you. A product can be usable but pointless. It can be useful but painful to use. It can be usable and useful yet still leave users cold in a competitive market. Their scorecard forces the organisation to look at all three.

Observed vs Self-Reported, and Why Both Matter

Marina spoke about a pattern every researcher recognises. What participants say and what they do often diverge. Someone can struggle through a task and still rate it highly when asked a self-reported question.

That is why the scorecard was built to include both observed and self-reported metrics. Observed measures included things like task success and time on task. Self-reported measures included tools like the Single Ease Question, Kano, and CSAT. The goal was balance, so the score would not be distorted by confidence, politeness, or a desire to please the moderator.

Amit also dismantled the myth that qualitative work cannot be quantified. Researchers already code behaviours, cluster reactions, and look for recurring patterns. Quantifying qualitative signals is not a betrayal of craft; it is one way of making patterns legible to an organisation that runs on numbers.

One Score, Three Buckets, Flexible Inputs

The UX Scorecard produces a single overall score expressed as a percentage. That score is made up of the three buckets, weighted equally in prototype and live code phases.

The important detail is flexibility. Researchers are not forced into one rigid method. They choose the metrics that fit their study, as long as they include one from each dimension and aim for a mix of observed and self-reported measures.

Their data scientist created a normalisation approach that allows different scales to roll up into one score. That is how percentages, Likert scales, and standardised measures can live in the same programme without breaking comparability.

They also defined a minimum threshold of six participants to generate a score. Six is not a target; it is a floor. They chose it because their organisation tends towards qualitative studies, and because five to eight participants remains a practical industry norm when you are looking for directional usability signals.

The Scorecard Slide That Changed Readouts

Their scorecard is not a hidden dashboard. It is a slide that appears in a readout deck. It includes the experience name, a timestamp, and a clear label for the phase, concept, prototype, or live code. It shows the overall score and pass or fail grade, plus the three component scores, so teams can see where the weakness is.

It also keeps the qualitative core intact. There is a section for key qualitative insights, because the number is not the story. The number is the shared signal that opens the conversation.

There is also space to reference relevant product KPIs, which helps teams connect leading experience indicators with lagging business outcomes. It makes the scorecard easier to integrate into product conversations that already revolve around funnels and outcomes.

The Pilot That Turned It Into a Release Decision Tool

Amit and Marina used a workflow automation project as their pilot. The project had all the warning signs they described earlier. Research was happening, but insights were not landing. The release date was approaching, and the team needed clarity.

The first score was a wake-up call. It gave the team a shared language to admit that the experience was not ready. It stopped being about taste or opinion and became about measurable performance across usability, usefulness, and satisfaction. The beta phase was extended, and the team prioritised usability fixes.

When they retested months later, the score improved significantly. Usability and satisfaction moved into passing territory, with usability even exceeding expectations. Usefulness still lagged, but the overall score reached 70%, which meant the product could launch with a clear plan for what to improve next.

Making Measurement Stick in a Large Org

The strongest part of the talk was not the maths. It was an adoption.

Amit and Marina described how the scorecard programme became part of DocuSign’s release methodology. A scorecard is now required before a product can exit beta. Scorecards are discussed at senior levels, including the C-suite. And because research capacity is limited, they enabled non-research teams to conduct lightweight measurements within guardrails, using enablement rather than “democratisation” as their framing.

They argued that building a culture of measurement is not a company-size problem. It is a process and enablement problem. If it can work at DocuSign, it can work in smaller organisations too.

The Hard Conversations, and the Point of the Score

In the Q and A, Marina was asked about difficult conversations after low scores. She admitted it was hard, but also noted the score rarely surfaced something nobody sensed. Teams usually know when work is not landing. The scorecard simply made the problem undeniable and focused the discussion on what to fix.

Amit was asked why they weighted the three dimensions equally. His answer was practical: they wanted to ship a first version of the programme without getting stuck in debates. Equal weighting avoided analysis paralysis, and it aligned with the reality that all three dimensions matter. He also clarified that concept phase scoring is different, weighted more heavily toward usefulness, because early work is about strategic value before interaction design exists.

The Shift They Were Really Selling

Amit and Marina were not trying to turn research into a single number. They were trying to make experience quality operational. When teams start saying, “We can’t release yet with this usability score,” research is no longer a late-stage validator. It becomes part of how product decisions are made.

That is what the UX Scorecard ultimately offers: a shared language, a repeatable measurement habit, and a way to turn research into measurable impact across the organisation.

Want to watch the full talk?

You can find it here on UXDX: https://uxdx.com/session/the-ux-scorecard-transforming-research-into-measurable-impact-across-the-org1/

Or explore all the insights in the UXDX USA 2025 Post Show Report: https://uxdx.com/post-show-report