When building any technology product, one of the most common pieces of advice is “talk to your users.”
But the default way most of us talk to customers and prospects is unscientific, putting us in danger of being lied to and wasting months building something nobody wants.
I learned this truth the hard way over the past decade founding multiple companies.
But it wasn’t until I was working on my third startup that I came to understand the right way to do user research.
When I first started building software 10 years ago, I never anticipated how applicable Yoda’s wisdom would be as a founder.
See, I was a startup newb. We had an idea, got a visual designer to turn it into a UI, and then hired overseas developers to build the actual product. But after a year and tens of thousands of dollars, we launched it.
The only people anticipating the launch? Us.
Nobody wanted what we’d built. Ghost town. Crickets. It was so bad I can’t even say the company’s name.
I determined that the missing piece of the puzzle was my lack of technical ability, so I enrolled as one of the first 15 students at a now-famous coding bootcamp. Afterward, I got a job as a software engineer at a publicly traded company.
So when I started my next company in 2016, this time we talked to dozens of potential users first. And it worked! Sort of.
With hindsight, I could look back to my original research notes and see I had ignored several fatal warnings.
I heard what they said, but I did not listen.
So if I wanted to avoid failing a third time, I needed to figure out what I was really missing in this user-research process.
What I eventually learned—and what ended up the true missing piece of my startup puzzle—was how the best companies in the world use rigorous, scientific approaches to interviewing customers.
(Yes, Steve Jobs could instinctively know what users want. But none of us are Steve Jobs.)
Marty Cagan, Silicon Valley Product Group founder and former PM at early eBay, says there are “two inconvenient truths about product.”
Truth #1: At least half of our ideas are just not going to work:
That’s pretty challenging on its own. But even worse?
Truth #2: Even the good ideas take several iterations to become viable.
No matter how good your initial launch is, it still takes more than a few tries to deliver on your promises and build a product that the market actually wants.
My experience has been that there’s simply no escaping these inconvenient truths. It doesn’t matter how smart or experienced we may be, if you know how to code or not, or how great our insights are. Statistically speaking, most of our ideas are simply not going to work. And the successful ones take time and hard work to turn into a real product that gets widely adopted by a market.
So we have no choice but to take a humble approach to building products—one that puts an emphasis on truly understanding customers—what we call qualitative user research and customer discovery—before engineering and delivery.
Your ideas are not nearly as important as your process of presenting those ideas to customers and gaining their feedback.
Customer discovery starts with generative research. Your goal is not to validate whether your idea has merit—that comes later. Instead, you need to learn everything you can about what the users in your target market are already doing to solve their problems.
Your generative research should help you to identify potential solutions to problems that you have learned actually exist directly from prospective users and save you from the pain that comes from ignoring Marty’s two inconvenient truths of building products.
Requiring significant behavior change is a death sentence for most new products. Therefore, a deliberate part of the customer discovery process is to deeply understand potential customers’ existing incentives, behaviors, and desires.
We are well served to remember that markets respond to desires they already have and do everything we can to discover what these existing desires are before we presume to solve problems we don’t yet fully understand.
Generative research questions are focused on understanding existing behavior. For example, here are some questions from my interview guide from the early days of Grain to understand how my prospective users already document and share information from live meetings:
Be sure to avoid hypothetical questions about what people might do. Don’t try to validate your future product with questions that begin with “would you use this” or “what do you think about the possibility of”—that’s what we call leading the witness, and it will inevitably bias your data and waste your time building the wrong thing. At this stage, you simply need to observe what users are already doing, not what they might theoretically do.
Most researchers solicit permission from their interviewees to record these interviews and take notes that will help them to identify behavioral patterns that could eventually help to define “jobs to be done” that product, engineering, and design teams can build for with confidence.
After gaining insights about the problems your target market faces in generative research, you may be confident enough to test out a specific product solution to see if these users would actually value it. This is the concept behind evaluative testing.
Evaluative testing involves putting an ultra-lightweight implementation of your product solution in front of your target users to see how they react. It shouldn’t be a fully functional product yet: designs on paper, prototypes, mock-ups—anything like that will work. Your goal at this stage is to get a clear answer to questions like “Does this solve a real problem people actually have?” and “How much would this user and others like them value a solution to this problem if it were to manifest?”
Unfortunately, all too many product teams speed through this testing in a couple of days or weeks. But depending on the complexity of the market and the problem you’re trying to solve, this stage could take months or, in some cases, years. That might sound discouraging and time-consuming, but I know this for certain: the success of your product will be directly proportional to the quality of work done in this initial customer discovery phase. It’s worth doing it, and it’s certainly worth doing it well.
The best product teams never stop this work of generative and evaluative testing for new features. Even as their initial research and testing turns into a real product, they know the importance of creating a customer discovery and product delivery engine that never stops learning and growing.
Even if your team creates something that people want, if customers can’t figure out how to use it, the product is dead in the water. This is why product teams conduct usability interviews during the build process.
The traditional approach to usability interviews is to set up a test environment, where we watch as a user navigates the product. An interviewer encourages a user to explain what they see, think, and observe. The interviewer also offers prompts for what the user might consider next if they get stuck using the product. Usability issues in the product become self-evident in most of these cases.
A less scientific and more agile approach to identifying lower-hanging usability issues is concierge onboarding. In concierge onboarding, someone from your team guides--via video call is best -new users through setting up the product and answers the questions in real-time. Concierge onboarding helps the team member understand the steps users are asked to take and the ways those steps directly lead to value.
We used concierge onboarding at Grain for the first nine months. It was tremendous in helping us to not only improve usability as we watched people struggle using certain parts of the product but it also helped us to get more users further down the activation funnel to learn if they’d retain.
It’s much more common for product teams to continually learn and discover from their existing users than it is for them to gather insights from completely unbiased non-users. But a balance between the two groups—known and unknown—is ideal; it’s only with that balance that you can truly achieve unbiased, continuous discovery. New users can give you a better understanding of your initial product experience, and “power users” can offer you insights that come from living with a product for weeks or months.
Great product teams develop long-standing relationships of trust with their most active users. You’ll often see the people on these teams setting up recurring feedback sessions to gain insight and listen to users’ concerns and ideas. The point of these interviews is to find out what’s delightful and what’s frustrating—what’s there and working well, and what’s still missing.
In the early days of building your product, a standing weekly or biweekly check-in is common for these feedback sessions. These can shift to monthly or even quarterly as time goes on and the product gets closer to reaching market fit.
At every phase of the discovery process, you should be using what you learn to drive decisions. For inexperienced product teams, these decisions are often based on one perspective or option with little data to back it up. The members of the team also often evaluate this information separately to draw their own conclusions—and those conclusions don’t align, resulting in a poorly designed product.
For the sake of your team—the audience of your research—prioritize making your data easily shareable. For your team to ultimately present a case for or against a decision, they need to be able to access the same information as the person who conducted the interview in the first place.
It’s 2020, and with the high quality of today’s video conferencing, you can do this work from everywhere. Because it’s already a digital setting, there’s no excuse not to record it. Save the audio and video, mine them for insights, and look for patterns that can be shared with your team. The easier the notes are to relate back to the relevant part of the recording, the easier it is to get multiple unbiased perspectives and recognize true behavioral patterns together.
Some teams are going one step further to create short audio or video clips of the interview to augment or even replace those text-based verbatim quotes. At Grain, we call this “using the literal voice of the customer,” and it has been the key to developing empathy for our users throughout every level of our product team as every user facing call gets cut into pieces and shared instantly in Slack.
Don’t fall into the trap of collecting data yourself, absorbing it, and sharing your (potentially biased) interpretation only with the team. Make the data you’ve collected readily accessible so you can work together to draw the conclusions that your product needs to be improved.
Whether we like it, the two inconvenient truths of start-ups mean that if we are to build great products that people love, we have no choice but to do the hard work of talking to our users in the right way and the even harder work to gain consensus as a product team about their actual jobs to be done.
When I built my first two startups, I took the conventional advice: I got out of the building, and I talked to plenty of users. But I wanted people to agree with me, and prioritized my fear of rejection over the truth. That’s one of the reasons my launches were met with so much silence. Eventually I found a better way, and if you follow this guide I’m confident you will, too.