Unknown Unknowns

“There are known knowns, known unknowns, and unknown unknowns,” Donald Rumsfeld famously observed. He was talking about an issue close to the heart of every serious project manager: managing risks and making decisions when you don’t always have data to back them up.

Classical risk management is based on the law of large numbers. A creation of famous mathematicians of the 18th century, classical risk is based on statistics. We don’t know if your house will burn down, but in a pool of 100,000 houses, we can make a pretty good prediction about how many houses will. In many ways, classical risk management is at the heart of modern economic civilization. From insurance to interest rates, the ability to analyze and measure risk is essential. At the root of our current economic crisis is the unfortunate fact that risk analysts sometimes get it wrong.

The classic risk formula R=PxI (risk equals probability times impact) measures the severity of risk and provides a guideline for determining how much you should spend in dealing with the risk. If there is a 10% chance, for example, of an event that will cost you $10,000 if it happens, the risk score is $1000. That means if you can get rid of the risk for under $1000, you are better off.

But in project management, you often have no idea what the probability is. “We’ve never done this before! What’s the chance of Event A happening?” Clearly, we have no idea. What do we do now?

Rumsfeld’s answer, basically, was to do nothing. Known knowns and known unknowns fall into the universe of planning, but the set of unknown unknowns was too far outside the box. Unfortunately, that doesn’t work out in practice. If you go to your boss and say, “Sorry that project failed, but it was because of an unknown unknown,” that doesn’t work very well as an excuse. Someone else, after the fact, with the benefit of 20/20 hindsight, gets to decide whether it was reasonable for you to have missed it. You are a hostage to fate.

Fortunately, there are a great many things project managers and planners can do even in the face of unknown unknowns.

1. Establish a reserve. Military planners know this well. As the elder von Moltke famously observed, no battle plan survives first contact with the enemy. No project plan survives first contact with reality. The total reserve should be proportional to the estimated risk, but the real secret is that reserve can be created in three dimensions: extra resources, extra time, and optional scope.
2. Think backwards. The triple constraints – the interplay of time, cost, and performance – form a powerful tool for insight. Any negative event can only do three things to your project: it can make you late, it can drive up your cost, or it can degrade your performance. If it does none of those things, it’s not an issue. If you have generic strategies for recovering lost time, recouping lost resources, or patching up problems in scope, you don’t need to know every possible direction from which trouble can appear. You already know what to do.
3. Define the range of outcomes. There are six possible outcomes for any given project. The baseline outcome is “fully satisfactory.” That’s the level of meaningful good enough. Another outcome is “barely adequate.” That’s the worst you can do and not have to call the outcome of failure. Above “fully satisfactory” is “exceeds expectations,” a traditional operational definition of quality. At the top is “outstanding,” and below “barely adequate” are the levels of “failure” and “catastrophe.” (Failure just takes down your project; catastrophe takes other stuff down along with it.) The reason you want to define failure and catastrophe is for risk management. The reason to define the higher grades is to improve quality. While it’s easy to provide quality as long as you don’t care how much you spend, that’s not so good for business. Find the elements of outstanding that cost very little, and you’ll deliver high quality performance even on a shoestring budget.
4. Work harder and dig deeper. At the beginning of the project just about everything is an unknown unknown, but it doesn’t have to stay that way. Have you really analyzed your project risk environment? In a number of organizations, thinking about failure causes people to be branded as “not a team player.” Unfortunately, the failure to think about failure increases the likelihood of failure. The more you obsess about the reasons for failure, the more powerful you’ll be at preventing it.

SideWise thinkers don't fall to pieces in the face of uncertainty. Uncertainty and the unknown is quite real, and denying its reality doesn't make your life better. Unknown unknowns are part of projects and of life in general. You can prepare even if you can't know.

Next week, Unknown Knowns — What you don't know that you really do know.