Smart Experiments For CEOs

Every leader encounters two types of decisions: decisions based on reliable information, and decisions based on highly uncertain assumptions. These scenarios represent the difference between walking across a well-lighted room, and stumbling through a jungle in the dark.

I have studied entrepreneurial mindset for more than 20 years, including embedded observation of hundreds of entrepreneurs as they navigated situations defined by uncertainty. Some entrepreneurs consistently succeed, and most fail. Why? What are the differences between the two groups? Are there lessons we can learn to become better leaders?

It is easy to lead when the path is well lit. More and more, however, leaders face decisions and actions about highly uncertain futures. This is the realm of entrepreneurial decision making, and experimentation. By definition, an experiment explores the unknown. The life of every leader is essentially a series of experiments. If each experiment is an investment of available resources, then the return on investment is additional (often new) resources. We experiment whether we are cooking a meal, building a skyscraper, or entering a new market in an unstable economic environment. No matter what the experiment is, we can choose to do it poorly or well.

Doing our experiments better means acknowledging that experimentation is a process, and every process has its intricacies. Doing a process well involves knowledge about the process, and proper application of that knowledge. We have come a long way in our understanding about how to do experiments well – a process I refer to as a Smart Experiment.

Every Smart Experiment involves four steps:

1) DESIGN. We must design the experiments we wish to do. This happens naturally and even subconsciously when the world around us is stable and certain, such as the aforementioned walk across the well lighted room. Other decisions – the kinds that make and break executives and their organizations – more often resemble dark spaces, where assumptions and unknowns prevail. In these situations, we benefit from a two-step experimental design process. The first step involves assessing and understanding available resources – of all kinds. The second step is to figure out possible ways to combine these resources as possible experiments to be done.

2) DECIDE. With an ever growing roster of possible experiments, it is critical to know which ones are worthy of consideration. We must know our priorities, and prevent distractions. The best experiments are sometimes the ones we choose not to do. In the Smart Experiment process, we apply a simple two-by-two matrix that balances expected value and potential loss. Such an analysis can be used, either formally or informally, to decide whether any particular experiment should be a) done now, b) delayed, c) partnered, or d) killed.

3) DE-RISK. Successful entrepreneurs de-risk their experiments. Contrary to the notion that entrepreneurs are risk-takers (which they may be, with other people’s resources), successful entrepreneurs are actually expert risk mitigators. They achieve this by de-risking experiments as a last step before conducting them. If you’re going to hike for days in the woods, it makes sense to be prepared, carry a first aid kit, and know how to use it, for example. The benefit of preparing for potential ‘hotspots’ far outweighs the cost. Simple de-risking provides outsized results since we often fail due to easily foreseeable – and fixable – challenges.

4) DELIVER. It is now time to run the experiment. Interestingly, execution of a well designed, prioritized and de-risked experiment is typically straightforward – since all of the critical pieces are in place. In terms of Smart Experiment delivery, one area that benefits from improvement is the harvesting of resulting resources. Remember that every experiment is an investment of resources. The return is an additional set of (often more valuable) resources. We have a tendency to leave valuable resources on the table, especially when results are unexpected.

Leaders and organizations are best served when they perform all of the steps well, however even incremental improvement in any one area has the potential to yield positive results. Going back to the cooking metaphor, doing even a few things better has the potential to improve the entire meal. Expert execution of Smart Experiments produces a compounding effect that results in far superior results over time – by predisposing to succeeding again and again.

With an understanding of Smart Experiments, it is easy to see how business entails a series of experiments, small and large, and that doing better experiments has critical implications for long-term success. So what are some of the lessons that arise from Smart Experiments?

First, Smart Experiments are intended to be done. Many individuals and organizations experience analysis paralysis. Instead, break down experiments into small steps that empower people and teams to get started, and to learn quickly with low risk of loss.

Second, succeeding and failing are parts of the same process. Smart Experiments, using prioritization and risk mitigation, mean that failures become small. That said, they do occur, and are expected to occur. If an organization is not failing, this is clear evidence of not trying, or not pushing the limits of knowledge and potential for the organization.

Finally, and most important of all, if we believe in Smart Experiments as a valuable tool for progress and growth, then we should celebrate them, irrespective of any particular experiment’s outcome. People get results by virtue of their actions. If we want people, including ourselves, to consistently do Smart Experiments, then we must motivate and celebrate their use in pursuit of sustainable, long-term success.

Commentary by Steven K. Gold. Here’s what you’ve missed?
World’s Best Universities.
World’s Best Fashion Schools.
World’s Best Medical Schools.