Intuition isn’t a Substitute for Evidence-based Decision Making
Apr 28, 2009
The idea that good managers routinely gather and analyze information before making decisions is generally accepted as self-evident. Indeed, evidence-based decision making is so much in vogue that books such as Competing on Analytics and Supercrunchers top the list of business bestsellers. One would be hard pressed today to find a manager who does not advocate this approach.
But do people really gather facts before making a decision, or, as Francis Bacon described nearly 400 years ago, do we “adopt an opinion [and] draw all things else to support and agree with it”?
Despite mountains of evidence that analytic decision making results in better outcomes, research seems to suggest that Bacon’s observation was correct. A six month study at financial and engineering companies found that although participants described processes where information was collected and analyzed before decisions were made, what was observed were ritualistic and ceremonial charades that manufactured evidence to support pre-determined outcomes. Managers decided what they wanted to happen and then conducted analysis to document and support it. Despite having listened to the gospel of evidence based decision making, what many managers apparently practise is decision-based evidence making.
Here is an example: Asked to consider an expensive system upgrade, analysts at a financial institution conducted a comprehensive evaluation of two options and identified option “A” as the best choice. Approximately one year later they were again requested to evaluate the options and again determined that option “A” was the ideal choice. Finally, less than six months later, a new group of analysts were assembled and a third evaluation requested. This time, however, one of the participants was told “unless he wanted to participate in a fourth evaluation, he ought to make the outcome come out to be ‘B’ — even if it meant simply changing ‘A’ to ‘B’ throughout the earlier documents. This was what management wanted and no one would notice anyway.”
Sadly this was not an isolated incident. In examples that ranged from the identification of candidates that managers wanted to fire or extensive renovations that they wanted to undertake, managers frequently decided what they wanted to do and then created the analysis that supported it.
Although decision researchers have developed important prescriptions to better improve decision making, in many cases these appear to have been left behind in the class room rather than carried to the boardroom. Advanced tools and techniques that can be used to analyze and assess a myriad of options and enable the “best” choice have become props used by managers like criminal defense lawyers who pick and choose expert witnesses to support a particular position.
This is not to say that decision ceremonies and rituals are always bad or that analysis after the fact should never be done. In fact, confirmatory or post decision analysis can be extremely important, particularly in the context of strategic multi agent game theory or to validate choice. In some situations decision makers cannot wait for incontrovertible proof and choices can only be made on faith. However, managers need to carefully think not just about how they may make decisions but how the processes used to make decisions may be perceived and affect or constrain the results. For example, at the financial institution described earlier, an analyst told me of a creative alternative to the two choices under consideration that was not presented as an option because she felt that it was not what management wanted to hear.
In cases of extreme uncertainty or innovation, no amount of analysis will predict the future and estimating markets for new goods or services is nearly impossible. Discussing the early age of the automobile, Henry Ford famously said that had he asked people what they wanted, they would have responded with “faster horses.” The bureaucracy of many organizations requires that decisions be supported and because managers have been trained in analysis, many may see little choice but to create sophisticated fantasy documents. These can cause significant pr oblems if management comes to believe that this imagination represents reality, participants become cynical of the processes and good ideas are ignored or never considered.
There are many implications to this research. Decision making is by definition difficult and rather than apply scientifically tested principles it is easy to fall into a cycle of intuitive choices that we seek to justify after the fact. Change, of course, is never easy but good decisions have never been more important. In times of trouble, many managers seek the perceived safety of known harbours rather than new approaches, but they need look no further than General Motors to see that incrementalism does not always work. When reviewing your own decision making, whether it involves employee evaluation or project selection, the key question is if information and fact gathering are used to explore ideas that are then used to make the best decisions or if information is used the way that a drunk uses a lamppost — for support rather than illumination.
Financial Post
– Peter Tingling teaches Operational Risk and Sustainability at the Segal School of Business at Simon Fraser University and is the founder and CEO of Octothorpe Software Corporation–A Decision Sciences Company. This article was published in the April 21, 2009 edition of the Financial Post. To read it online, visit http://www.financialpost.com/story.html?id=1516591