We all want to succeed. Whether it’s succeeding on an exam in school, succeeding in the business world, on the soccer field or in the dating arena. Success makes us feel good and moves us forward on our journey.
Testing theories and experimenting are important activities because they give us clues toward making successful decisions. And success depends on us making good decisions. Over and over again. When we inevitably make a bad decision, our future success depends on us recovering quickly and changing course before we get bogged down in the consequences of the bad decision. Or worse, we get so discouraged that we simply don’t make any more decisions at all.
So how do we know which decisions are good ones and which aren’t? How do we know a good idea from a bad one?
We base all our decisions on data.
Data we’ve collected ourselves through experience; data others have collected and recorded; and data we collect in the moment — as the decision is playing out.
Many of us also base our decisions on our desires or by guessing. Our imaginations are powerful. If we are presented with two options, our imaginations can run dozens of scenarios for how these options will play out in reality. The rational mind then chooses the most likely outcomes based on prior experience, and we take our best guess as to which option to choose.
In reality, all our decisions are based on using our best guess. Unless you’re a time traveler, you can’t truly know what the outcome of a decision will be unless you have test data to back you up. The more data we have before we make a decision, the more likely we’ll make the correct choice. And the more successful we’ll be.
How do we gather that data? Through testing and real world experience (ours or someone else’s).
3 Traditional Testing Methods
1. Trial and Error: This is also called the “spaghetti method.” You just throw stuff at the wall and see what sticks. You have an idea, and you try it. Either you succeed or you don’t. When you record the results or add it to your mental database of experience, you can use trial and error as data gathering for future ideas. Trial and error can work, but it’s slow, unpredictable and terribly inefficient.
2. Scientific Method: This a lot like trial and error, just more evolved. You define a hypothesis (make a guess), run an experiment, and see if you were right. Usually, whether the test works or doesn’t is irrelevant. Either way, you are gathering data. Unfortunately, the data gathering can go on indefinitely. Most of the time, you’ll need to run many more tests to rule out other possibilities.
This is smarter than straight trial and error, but complicated and risky. And it’s still a guessing game. Experimenting is very important, and everyone should be doing it — especially in business. But it’s still based on guessing, then experimenting to see if your guess was right or wrong. And your results are based on how well you defined the problem to begin with. There’s got to be a faster, more reliable way to get at the right answer.
3. Surveys: These are great for collecting lots of data, and there are innovative ways to use surveys. For a long time, surveys have been considered one of the best ways for businesses to get the data they need to make smarter decisions about consumers, predicting behavior or how to invest money. Unfortunately, surveys are error prone and can be very expensive. The more there is at stake, the more expensive the survey will be. Fortune 500 companies pay hundreds of thousands of dollars to run focus groups and acquire survey data for fairly simple questions.
The main problem with surveys and focus groups is they are not reliable. People will say one thing and do another — not because they are trying to be misleading (usually), but because they truly believe what they’re saying. They don’t even realize that the choices they think they make are not always accurate.
Flip through any issue of Journal of Marketing Research or Journal of the American Statistical Association (or really any serious marketing journal), and you’ll see that researchers are continually seeking better ways to measure consumer behavior. Why? Because surveys are inherently flawed. The data doesn’t always tell the truth; and even when it does, the truth can change from day to day.
Recent technology breakthroughs allow marketers to track purchasing behavior on a granular level. They know everything you bought at the grocery store last week and how often you buy gasoline (and what brand). They even know when big life events are happening like weddings, your first child and milestone birthdays. While all this technology is great, it’s often out of reach for most businesses. They are still forced to rely on flawed surveys or just plain guessing to figure out what their customers really want.
Are surveys worthwhile? Sometimes. Surveys can point you in the right direction, but they may not be conclusive and may even be misleading. The most common problem I’ve found with business surveys is they are surveying their customers instead of prospects. This skews the data in the wrong direction.
Is a survey worth hundreds of thousands of dollars? Maybe. Is there a better and cheaper way to get more conclusive data? Often, the answer is yes.
There’s a No. 4 — Test Before You Test
TBYT adds a new dimension to these testing methods so you know certain data ahead of time. This way, when you go through the real test of an idea, you’re not guessing at the outcome — you’re simply confirming it. The rate of successful outcomes for testing before you test is far better than testing by trial and error or basing decisions on research methods like surveys.
TBYT is all about gathering data involving actual behavior in the most low-risk, low-cost way possible. So you can pre-prove that your idea has legs without going through the time and expense of carrying out the entire idea to see if it’s successful.
Proving real behavior — not filtered behavior. Hopefully, I’ve made this clear by now...People rarely act in reality the way they say they would on a survey or in a focus group.
Testing before you test means you are testing to determine real behavior, not filtered or perceived behavior or imagined scenarios. You’re not “asking” consumers which toothpaste they prefer; you’re following them down the toothpaste aisle and seeing which one they actually buy.
By testing your way to success, you get where you want to go much faster and with much lower risk. And if you’re racing against time, the economy or a competitor, that’s a huge advantage.
This is one of the reasons most business startups fail so miserably. They spend enormous amounts of time and money on a hunch or an idea, then wait for it to play out to see if it was a success. In today’s world, where life moves at the speed of thought, that’s just too slow.
This is a modified excerpt from the book “Failure is Obsolete.”
