What is Validation?

What constitutes validation is one of the first, essential questions we ask in our work with entrepreneurs using the Business Canvas Model (BMC). We also discuss value proposition, targeting customers, and product/market fit. However, validation for all of the components in the Canvas is never really defined.

Validation is the ultimate goal of the process using the BMC. Through validation we are looking to determine the sufficient number of paying customers that creates a market adequate enough to create a business opportunity.

Entrepreneurs do not have the luxury of knowing how many paying customers they have when beginning to pursue an opportunity. Many teachers of entrepreneurship, including Steve Blank and those at the Innovation Corp program at the National Science Foundation, claim that an entrepreneur needs to talk to at least 100 customers in order to reduce the uncertainty surrounding a startup. In fact, the more confirmation achieved, the less the uncertainty. The specific number of 100 is likely derived from qualitative studies performed by social scientists who claim that a population of 100 in a survey makes for a valid survey. In reality, most entrepreneurs find that after talking to about ten customers, the results tend to be the same. So, what constitutes a sufficient number of interviews?

In the past, I have stated that entrepreneurs should interview as many people as necessary to confirm the valid, the uncertainty of the market to a comfortable level. Of course, that omits the concept of confirmation bias. Entrepreneurs need to be mindful to avoid thinking: “My invention is great, so I’ll do anything to make the numbers believable.” Do not fall prey to your own lies, damn lies and statistics.

In research methodology, validity is the soundness of the design of each test and methodology used. Validity shows that the findings/results truly represent the phenomenon claiming to be measured. We cannot talk about validity without discussing reliability. Can the test be repeated or replicated with another population and obtain similar results? Is the test inherently repeatable?

Entrepreneurial validity means using good methods to test hypothesis obtaining data with observable facts that can be measured and are relevant. In addition, the test results must end up with a binary result. The test either passes or fails. There is no “close enough” response. As Yoda says, “Do or do not. There is no try.”

During hypothesis testing, an entrepreneur must “draw the line in the sand.” Ask whether your metric provides you with a level of success that gives rise to doubling down and taking the next action step? Can you tell the difference between complete failure and overwhelming success? Where does your opportunity fall?

What happens if you land close to the line in the sand but do not pass over? There are two possible scenarios: One would come from the norms of your industry. In other words, understanding how your competitors view this metric would provide the necessary knowledge to take action. The second should come from the business model. How many positive responses are required in order to be successful?

Now that you understand what validity is and why it’s important, make sure that you understand exactly how to test for validity. Testing for validity must correspond to the extent to which a concept, conclusion or measurement is well-founded and corresponds accurately to the real world.

 

Restart – Both the Blog and our Program

I’m back from a break in blogging with renewed optimism and an excellent new staff at the Advantage Accelerator. In reviewing our program over the past two years, I note we’ve had a number of successful ventures, but still find issues relying solely on Lean Launch Pad methodology.

The greatest issue is the time it takes entrepreneurs to acquire a product market fit. Product market fit is vital because it shows customer validation and the discovery of a repeatable sales process. In order to develop a successful fit, the entrepreneur must focus on finding a reasonably sized market for growth.

Our current Accelerator program runs five months and operates as an expanded version of the Lean Launch Pad. Many entrepreneurs successfully complete the Accelerator program in record pace; others struggle with product market fit that may cause a client to struggle. Finding the best response to the iteration or pivot while learning new skills can cause the most seasoned veterans of startup cultures to stumble.

The Lean Launch Pad model requires that product market fit must be validated. If product market fit is false, then the operational side of the startup is invalid. Moving forward into the execution side of the canvas would be wasteful based upon an invalid product market fit.

After considerable thought, we will now be dividing our current five-month accelerator program into two parts: “Accelerate” and “Launch!” These programs will be offered in addition to our existing pre-Accelerator program, “Iterate!”

Iterate is a pre-accelerator program that is focused on problem/solution fit. The program has four, 2-hour workshop sessions with outcomes based upon entrepreneurial thinking, value proposition, an introduction to the Business Model Canvas as a tool, a basic understanding of the customer discovery process, and an introduction to hypothesis testing and validation. These are all necessary tools designed to help understand the hard work in the process of becoming an entrepreneur.

The program homework is optional (hint: but it is a great screening tool for us) and all team members are strongly encouraged to attend all workshops. In order to move into the next phase, Accelerate, and gain acceptance into the Accelerator, there is an expectation of progress, hustle, and “grit.” These qualities are not the only qualifications for acceptance into the Accelerator, but they are helpful considerations in our decision-making.

Accelerate is focused on product/market fit. The objectives of the program are to confirm the entrepreneurial opportunity, define and build a minimal viable product, validate product market fit, complete a first sales call and develop a repeatable sales model, complete building the team, and to be ready to execute and build operations. The program is eight weeks and is backed with significant resources including our intern program, mentors, and first looks by our Executives in Residence and early investors.

Entrepreneurs may repeat the Accelerate program twice if an iteration or pivot is required. A major focus of this program is to reduce the uncertainty of the startup through validation of product/market fit. Successful clients may move forward to the Launch! Program.

Launch is a five-month program focused on taking the company to the build and sell level. There are two major goals: Fulfillment of build and execution on an operation growth plan, and realization of a repeatable sale process. Each month’s program is focused on a specific topic toward a deep dive into a milestone based growth plan. There is work for each week of the program that starts with the introduction of a topic, private coaching sessions, a cohort workshop and roundtable with the last session of each month being a formal Advisory Board session that consists of our senior programming staff, Executives in Residence and any mentor or advisor to the company.

Overall, we feel that separating the completion of product market fit until the concept is validated makes more sense than attempting to move on to build a company with less certainty of success.

Customer Development Interviews

A new book by Cindy Alvarez about customer interviews reminded me that it is often important to review the basics of our tradecraft.

There are a number of important points to consider when reaching out to potential customers. In the Lean Methodology, it is essential to complete customer interviews before building your first minimally viable product. Ongoing interviews with clients also helps you to stay in touch and cement relationships with clients as well.

It is best to interview someone face-to-face. Clearly, that is more time consuming approach, yet more effective than Skype, telephone, or online interviews. A key component of these interviews is the observation of body language. That observation is lost with anything less than face-to-face meeting. Video conversations only capture part of the picture, and the nuances of observation cannot be completely captured.

The second important issue is to help the interviewee get past the politeness factor. No one wants to deliver bad news. Your goal is to make them feel comfortable. There are two ways past the politeness factor. The first is to change the way questions are asked. Asking, “What do you dislike about the product?” makes it difficult to yield honest results. Instead, asking “How would you improve this product?” frames the question as a respectful call for assistance. Everyone wants to help. No one wants to be impolite.

Cindy Alvarez starts just about every interview with four basic questions:

Tell me about how you do __________ today?

Do you use any other tools or have any specific tips or tricks you use to help?

Is there anything specific that you always do before or after you do _______?

If you could have a magic wand and be able to do anything else that you can’t do today, what would it be? (Forget about whether or not it’s possible, just anything.)

These questions are open-ended and allow for understanding customer behavior and activity. It is also crucial that there is not a single mention of your product.

Here are a number of other tips to use during customer interviews:

  • Try to have two people attend
  • Work from an outline of 3 – 5 questions
  • Focus on real behavior not hypothetical
  • Shut up
  • Probe
  • Don’t overstay your welcome
  • Debrief the same day
  • FOLLOW UP!
  • Work to their Schedule
  • Disarm the politeness training
  • Ask open ended questions
  • Don’t influence
  • Ask the Right Questions
  • Frame the Questions Correctly
  • Parrott back
  • Get Psyched to hear things you don’t want to hear
  • Ask for Introductions

The goal of your interviews is to position the interviewee as the expert. A good interview avoids yes/no answers, and gives potential clients an opportunity to tell a story – one that may cause them to think of related problems they’re having, or may trigger more questions for you to ask later.

Remember, your goal is to determine how your customer is currently dealing with specific tasks.  What do they like about their current solution or process? Is there some other approach that they have taken in the past that was better or worse?

Attempt to discover what they wish they could do that currently isn’t possible or practical. How would that make their lives better? Who their organization is directly involved with addressing such approaches? How long does it take to make a decision?

Your interviewees’ feelings and state of mind are also important. How do they feel when they are performing this task?  How busy/hurried/stressed/bored/frustrated? You can learn this by watching their facial expressions and listening to their voice as they answer your questions. What are they doing immediately before and after their current task? How much time or money would they be willing to invest in a solution that made their lives easier?

There is a lot more to being good at customer interviews. Like most anything else, practice is crucial to getting better.

Out-Of-The-Building Blocks

I had the opportunity to chat with Steve Blank about the Business Model Canvas and his Stanford class. I suggested that there is much work to be accomplished even before embarking on the Lean Launchpad Class. When I taught at the University of Southern California (USC), the cornerstone class before the Business Plan class was the Feasibility study. In fact, there were two tracks for entrepreneurs: one for technology and another for other businesses, and each had separate feasibility and business plan classes. (Disclaimer: I am not a believer in the traditional business plan, but rather focus on the operational and business-building sides of startups.)

I adopted many of the topics in both courses as a precursor to the Lean Launchpad class for mechanical engineers at Cal State Los Angeles, which I developed. When I taught at USC, the feasibility class was part of the MBA program. Therefore, many of the entrepreneurial and business concepts were already familiar to many of the students. However, the engineers at Cal State had no experience with business and entrepreneurial terminology and concepts.

I found the key to starting a new feasibility class for engineers was to introduce a few creativity concepts and exercises as a way of enriching their way of thinking. Engineers tend to think in a very linear fashion, and are sometimes uncomfortable with ambiguity in business. They weren’t as interested in variations in valuations or marketing—they just wanted the formula.

At some point in any business process, the research stops and the marketing begins. I am also a big subscriber to effectuation – the concept that entrepreneurs must think differently than ongoing businesses; using evolving means to reach new goals. Startups that are resource poor can’t throw money at a problem in order to make it sellable. A large business may be able to afford such a play, but not a startup. Entrepreneurs must use a different toolkit and a different way of thinking in order to succeed. Both effectuation and the business model canvas provide those tools.

In my entrepreneurship class at USC, students were required to meet 25 strangers. Steve Blank’s classic line is “to get entrepreneurs out of the building.” Entrepreneurial knowledge can only be gained through meeting people in their industry, advisors, and anyone else who can help budding entrepreneurs think through the parts of their business, how they compete, what niche does their competitor target, and just about everything required to be known before thinking about spending a dime.

So what kind of research could an entrepreneur do before starting on their entrepreneurial journey? This is secondary research to be completed before getting out the door. This needs to be done before to make the primary research (getting out the door) more valuable and shorter in duration. After all, having better hypotheses will yield more confirmations:

  1. Know thyself. What are you good at, who do you know that can advance the concept, and why is this an important undertaking? Can you articulate the value and benefits? Is the opportunity clear?
  2. Know the industry. Every aspect of the industry, every competitor and their niche, how would you find the pattern of change in the industry to become the leader of the pack, who are the major suppliers, can you successfully target the underserved market?
  3. Know the value chain – who gets paid and how, can you gain access to critical supplies on good terms? Do you know every step of your business process from order taking to fulfillment and post sale customer service? Where is the most value added?
  4. What is the best way to reach the customer? What effects their buying decisions?
  5. Who are your target customers and their demographic data? Are they big enough to constitute a market? What color underwear do they wear? Yes, you need to know everything about the client down to their undergarments.
  6. Financials – Don’t even think of continuing if you can’t do a spreadsheet forecast called a pro-forma. We all know the spreadsheet is a work of fiction, but how can you guestimate the entrepreneur’s bet? What are the premises underlying the data on your targeted financials? What is the delivered cost of the product? What are the multiple streams of revenue? Get the premises right and be convincing to yourself.

Possibly the most important skill to teach engineers and principle investigators is a little marketing and a whole lot of sales. Let’s face it; with few exceptions scientists are not famous for being extroverts. We need to impart to them that sales is not a shady profession, but rather an exercise in building relationships and helping potential clients solve problems. Good sales and marketing skills help build relationships allowing engineers to fix the world and its problems. Sales and engineering make for a good match.

However, relationship building is sometimes difficult for engineers and principle investigators. Social engagement must be built into all classes. In my class, everyone had to do a one-minute sales pitch that you could give someone in an elevator—a classic elevator pitch. After the first three or four intrepid students made their pitches, the whole class understood the key elements. Not everyone ended up being comfortable with selling, but then again not everyone will become a CEO.

So remember that before jumping into the Business Model Canvas, there is still much thought and research to put into your plan. If you are able to do the homework before embarking on validation tests in the canvas, my hypothesis is that you may have better information through good secondary research techniques before embarking on validation testing in each of the nine building blocks.