Performance Valuation: Using Outcomes to Measure the Impact of Product Changes

Performance Valuation is all about uncovering the user outcomes that most strongly correlate with revenue, and finding ways to deliver those outcomes more effectively. Here we provide an introduction to measuring how well you're currently producing user outcomes and confirming that those user outcomes are valuable for your business, too.

Do not index

Measuring the impact of design changes (6:20)

Moving from the business's timeline to the user's timeline (16:23)

Premise 1: Getting better at producing desired outcomes (19:01)

The key benefit of tracking performance in weekly cohorts (21:35)

Premise 2: Identifying which outcomes are creating the most value (26:06)

Using the user's timeline to create a performance feedback loop (29:42)

Improving performance by amplifying what's working (39:32)

Using Step Drop-off to improve conversion (43:28)

Using in-product behaviors to discover Super-Outcomes (49:50)

Takeaways

Which user outcome power revenue?

Of the many user outcomes that your app supports, it makes sense to identify the ones that are creating the most value for your business and invest in improving your ability to deliver them. (4:44)

Creating outcome-based feedback loops

Good NPS scores, positive shoutouts, and friendly customer tickets can't tell you how effectively your UX is moving users along a workflow. The key is to create an outcome-based feedback loop so you can improve the success rate of an outcome occurring rather than engagement with a particular design asset. (14:26)

Value Paths disrupts the current thinking, which is: if you make a more compelling product, then more users will want to engage with it. Thus, you will have more revenue coming in. In Value Paths, the quality of the product is not as important as the quality of the outcome that the product facilitates. The outcome is what we should be paying attention to, from a design and a measurement standpoint. (38:18)

Using the user timeline in decision making

Most business decisions are based on the business's timeline rather than the user's timeline. For example, Marketing promotions based on what competitors are doing or whether a round of funding is coming up rather than what users need to make progress at that particular point in the path. The idea is to switch this around measure the user's timeline so you can use it in your decision-making. (17:50)

Tracking users in cohorts

The benefit of tracking users in weekly cohorts is visibility into what the conversion curve looks like over time. (22:27)

Engagement metrics vs. outcome metrics

Improving an engagement metric like "day seven retention" in isolation doesn't really impact revenue in a meaningful way. The idea is to, instead, use the correlation as a starting point and ask, "Why do the people who continue to show up on day seven actually convert on a better basis, other than the fact that they're just continuing to engage with the app in general?" (54:56)