Prioritise the riskiest assumptions in big problem spaces
We often fail to test the things we’re uncertain about and instead spend a disproportionate amount of time on the things that are givens. This leads to long explorations, without conclusive evidence to make good, timely investments. We can avoid this by prioritising our assumptions by their risk, then testing them. If you are working in a big problem space, doing this will focus your service team on learning the things with the most value.
In this blog post, I’m sharing how you can use this method with your teams. Plus, you’ll get to make your prioritisation more like being a judge on a reality TV show, Strictly Come Dancing-style. This is way more fun than any effort-and-impact priority matrix.
This is the first of two blog posts. The second blog post is about the value of testing riskiest assumptions. It discusses the benefits of using the method, with some examples of when it’s been valuable.
Why it’s important
Exploration can drag on when we don’t spend enough time learning about the uncertain, especially in a big problem space. We might make progress on solving some problems, yet we’re still no closer to a usable service we can operate, or having an impact.
When starting the discovery phase for a service, we often don’t know where to begin our research. It’s great when people have different hunches. We see this particularly in teams with many disciplines. Yet it can be hard to know what to focus on when trying to solve a whole problem for users.
When a discovery phase throws up lots of problems, we don’t know where to start our alpha phase. We know we need to design a few user journeys from start to finish. We could say start at the beginning or start anywhere, as it all needs designing.
Many alpha phases take much longer than the suggested 6 to 8 weeks. We still don’t feel confident about what our service should look like in the beta phase. We’re left in perpetual exploration, like me digging for records in a shop basement, but less fun. This can happen when we mistakenly spend all of our time according to what is most valuable for the user. Value for users is incredibly important, but there’s a nuance to it.
We should prioritise spending time on things we most need to learn about. The things we most need to learn about might be different from what has the most value for users. We don’t always make this distinction, and in the end, value for users suffers.
How to prioritise assumptions by risk
It’s ok to have hunches. Everyone has them. Hunches are only dangerous when we don’t test them.
We form hunches about how to solve problems from a range of assumptions. Sometimes, our assumptions are informed by spotting patterns in our experiences. Other times, we overreach and that’s when, to quote the old saying, “To assume is to make an ASS out of U and ME”.
Prioritising assumptions by risk helps us focus on where they might be overreaching and prevents never-ending exploration. The first step is called assumption surfacing, then later when testing them, riskiest assumption testing (RAT).
How to do it:
- Everyone writes down their own assumptions as well as assumptions they’ve heard. Cover the service, product, users, business case, market opportunity, technology, delivery approach and stakeholders.
- Everyone writes down what the consequences of their assumptions being wrong would be.
- Everyone writes down what existing knowledge they have to inform their assumptions. This could be things that suggest it to be right or wrong.
- Get everyone’s assumptions in one place and combine any duplicates.
- Privately, each person gives each assumption two different scores out of 10. The first is “impact if we get this assumption wrong”, the second is “confidence I know this is right or wrong”.
- Meet as a team. Read out each assumption. Channel your favourite Strictly Come Dancing judge and reveal your individual scores. Agree consensus scores for impact and confidence. Top tips: time box discussion about each assumption to a minute and gather all the existing knowledge in one place.
- Give each assumption a risk score. Risk score = impact × (10 - confidence). 100 is the riskiest assumption, 0 is the safest assumption. Inverting confidence means assumptions we lack confidence in produce higher risk scores.
- Sort the assumptions by their risk score.
I’ve given a simple example below, with a throwback to a 2004 design icon – the Motorola Razr mobile phone. The overwhelming sense of nostalgia should help the method feel less intimidating. Imagine our team are trying to figure out how to make the next smash hit smartphone…
Assumption that people would buy a folding smartphone
Consequence if wrong: We lose money on our investment
Existing knowledge to mitigate risk: The 2004 Motorola Razr was popular
Impact if wrong: 8
Confidence we know the answer: 4
Risk score: 48
Assumption that we can produce folding glass screens
Consequence if wrong: We don’t have a unique selling point
Existing knowledge to mitigate risk: We have low-resolution prototypes
Impact if wrong: 7
Confidence we know the answer: 2
Risk score: 56
If you’re reading the example, thinking, “no way is that a 4!”, perfect. You know something that I don’t. You can imagine our team discussing how popular the Motorola Razr was. We’d debate how similar our folding smartphone is.
Our team now has a list of assumptions, prioritised by risk. Now we can agree as a team what we need to quickly learn, experiment and test to help mitigate those risks. This will help form a good backlog for a discovery or alpha. This method can also help in other situations when you're working in big problem spaces with lots of uncertainty, and don’t know where to start.
What to remember
- When you’re in a big problem space, prioritise your team’s learning according to your riskiest assumptions.
- Use this method to avoid spending too long on discoveries and alphas.
- Anyone can get started, but do it as a team to learn from each other for a more informed prioritisation.
- Ask your team “what is the quickest way to test our riskiest assumptions?”
- The value of testing riskiest assumptions by me
- The MVP is dead. Long live the RAT by Rik Higham was my first introduction to using the method
- Assumption Surfacing by Mycoted is a good way to look at the risk of decisions already made
Learning about weighting scores
When we discussed the method, Georgina Watts was curious about weighting the impact and confidence scores. For example, we could weight impact higher than confidence. This might be useful with a group of people that tend to be overly-confident, or don’t talk about failure.
I’ve only used a 50/50 weighting. I'd be curious to hear from others that have experimented with other weightings. Share your remix on Mastodon or Twitter.
To help me write better posts, I’d love to get your feedback. If you found this useful, please repost to share it with your network.
Photo on screen: Some rights reserved by Jorge Franganillo.
I’ve recently moved to Copenhagen, Denmark. I'm looking for my next product management role with opportunities for social or environmental impact. If you’re interested, please connect and let’s get kaffe!
Originally published on GOV.UK: Services in Government Blog.
Next: What product managers can do to reduce CO₂
Previous: How to launch a tech product in a large company