Growing Consumer startups [Part 2]: Getting to Empathy Using Experimentsvaidy
March 23, 2016
In this post, we dive into the gory details of performing experiments to cross the Pre-Empathy stage in a Consumer Startup.
[If that makes no sense to you, read Part 1 of this series]
As mentioned in the last post, the four stages of experiments are as follows:
Step 1: Setup the experiment
Step 2: Build
Step 3: Review results
Step 4: Iterate
Here’s how you execute those steps for a Consumer Startup:
Step 1: Set up the Experiment
1. What is your OMTM? This is usually the number of Customer Development interviews you’ve done. These are meetings that you schedule with your target users to increase your empathy for them.
2. What is the current value of the OMTM? This is simply the total number of interviews you’ve conducted.
3. What is your target value for your OMTM? For a consumer product, you should ideally start seeing some kind of early pattern among your target users when you speak with about 5 people (assuming you’ve chosen a narrow enough early adopter group). But to get to a very good understanding of your users, you should plan to at least do 20-30 interviews. Most ideally this should be between 50-100. The more, the better.
Considering that each interview will take about 90 minutes of your time (for the interview and sharing the notes), this is indeed a significant time investment. But nobody said that empathizing with your users is easy :). Of course, having a deep level of empathy with your users will pay off rich dividends as your startup grows into stickiness, virality and further stages. So, it’s worth the upfront investment.
4. What is your experiment hypothesis (or “idea”) to help you move forward toward the OMTM goal? At the empathy stage, the “idea” is almost always the same: Customer Development Interviews. And your goal is to hit your target number of total interviews.
Read on. The next question will help you have more faith :).
5. How will the idea help you to learn more about your customer? Not feeling very good about Cust Devs so far? If it sounds like I’m asking you to do interviews solely for the purpose of hitting an arbitrary number, I wouldn’t blame you. You’d right in asking: “How is this exercise going to help me move my startup forward? I realize that I need to get to know my users and customers better, but how do I ensure that I get the maximum value from these interviews?”
It’s going to get a bit crazy here - so hang on. I’ve said so far that your hypothesis is this: Hitting your OMTM goal of a total number of Cust Dev’s will enable to cross the empathy stage.
But Cust Dev’s also have hypotheses nested within them. And it’s best to see Cust Dev’s as the lowest cost way to validate these hypotheses you have about your market.
How do you come up with these hypotheses? Read the following two chapters from Talking to Humans:
- "Who do you want to learn from?”
- Who are the people who will find your product most useful? Who will eventually want to partner with you? And most importantly, who are those people who will pay to use your product? You need to have hypotheses on who these people might be and talk to them.
- “What do you want to learn”
- Most first time entrepreneurs have a hard time thinking about their “ business as their product”. There’s usually a very strong tendency to focus on the product “idea”, its features, the technology and the user experience. But there’s a bunch of much more important questions that need answering. Here’s a quick starter:
The problem my customer wants to solve is?
Why can’t my customer solve this today?
I will make money (revenue) by?
My primary competition will be?
I will beat my competitors primarily because of?
My biggest risk to financial viability is?
What assumptions do we have that, if proven wrong, would cause this business to fail? (Tip: include market size in this list)
Make sure to read the chapter end to end. That will help you get into the habit of thinking about your “business model as your product” and then get into the habit of systematically de-risking your business model.
[NOTE: If you’re familiar with the Business Model Canvas or the Lean Canvas, then you’ll notice that the type of questions mentioned above are on the same lines.]
“ Talking to Humans” is IMHO the only resource you might need to get a ton of value from your interviews. And its free!
Here’s 2 key reasons why Cust Dev’s are much more efficient at testing hypotheses than, say testing with a real product:
You can test multiple hypotheses in a single interview
You can get much quicker feedback. It only takes a maximum of 90min.
6. How will you measure whether your “idea” is working?
“Cust Dev’s seems like a great way to learn about my market in a super cheap way. But I still want to able to quantify my learnings in some way.”
If those were your thoughts, kudos to you. That’s the kind of exacting discipline and rigor that will help you on the path of constant learning.
Use this Scoring Sheet. It’ll keep you honest and focused on the goal of learning and empathizing with your target users.
As you consistently and diligently follow this process, you will have significant insights into your users’ psychology, motivation and behavior patterns. Dare I say, you might be able to predict how they will behave better than they themselves might!
7. What is the reporting structure for the above information? Here’s why a good reporting structure is important:
Don’t have a team to share your learnings with? You still need this to keep your sanity and ensure that you’re not drowning in a sea of information. You will collect a ton of learnings in a very short period of time - and you need an easy way to navigate through all that. Only then will you be able to discover insights and patterns between various interviews.
Have a team? Great. Everyone on your team should be doing Cust Devs. And if that’s happening, you need a consistent way to share your learnings with each other. Every time that happens, you’re extracting multi-fold value (in terms increased empathy across the team) from each Cust Dev.
There are two things that you need to report on:
The notes from the interview: Here’s a template that you could start with
The score of the interview: Here’s the link once again to the scoring sheet template.
Step 2: Build
To reiterate, the two key parts of this phase are:
Implement the idea
Keeping your measuring process sustainable
Doing Cust Dev’s well requires some key skills:
Probing for “stories”
Avoiding selling your idea
Avoiding confirmation bias
Once again, I’d recommend reading Talking To Humans. Specifically, the following chapters:
“How do you find your interview subjects?”
“How to ensure an effective session”?
2. Is your measuring process sustainable? It’s important to ensure that the process doesn’t come in the way of your learning goals. Otherwise, you may end up losing motivation to continue the interviews.
Use the Scoring Sheet and Notes templates as resources to get you started. If you find them easy to use, that’s great. But if they’re becoming cumbersome, then feel free to tweak them so they work better for you.
Step 3: Review results
1. What is the OMTM value, after the experiment? Once again, this is the total number of Cust Dev’s you’ve finished.
2. Update the Report with the new OMTM value Okay, you don’t need to do this. The number of records in the Scoring Sheet will of course suffice to know the OMTM value.How well did your “idea” work? Which hypotheses were validated and which were
3. How well did your “idea” work? Which hypotheses were validated and which were invalidated ? This is also not very relevant in this case.
4. Is the customer’s behavior more predictable to you after the experiment? This will be reflected in the scores you recorded after each Cust Dev. If you’re tweaking your hypotheses (the ones nested in your Cust Devs) and also getting better at defining your target user persona - after each Cust Dev, you should see your scores rising. And if your scores are rising, that’s an indicator that you’re able to predict your target customer’s behavior better as well.
5. Was there something else (perhaps unexpected, an “insight” perhaps) that you discovered during the experiment? As mentioned earlier, Cust Dev’s are about “listening”. Not listening for what you “want to hear”, but for discovering “unknown unknowns”. Watch the following video, to get a better understanding of this concept.
It is almost certain that once you get to 20+ Cust Devs, you’d have had a good number of insights - on both who are your target users are and also what they want.
6. Are there any new hypotheses that you want to test? Any hypotheses that need updating? With the insights and learnings from each interview, you’ll have new ideas that you’ll want to validate in the upcoming interviews. Keep track of these meticulously and diligently.
Record your thoughts in last 3 columns of the Scoring Sheet.
Step 4: Iterate
You’ll need to go through the experimentation process as many times as it takes to get to the target value of your OMTM.
Feed the updated/new hypotheses that you’ve got as the output from your just finished experiment cycle - as the input for the new experiment cycle.
This process will get you closer to the OMTM over a period of time.
Perfecting the art of doing Customer Development interviews will take time. But it is a skill that is well worth picking up, if you want to make sure that you’ve crossed the empathy stage, before starting to invest more in your idea.
The next and last part of this series dives into experiments at the Stickiness stage. Read on.