Credible Prediction – Expected Value & Uncertainty

Every good prediction has at least 2 parts:

• the expected value (the predicted value itself) and
• the amount of uncertainty about the range of possible values that prediction could take in the data.

When statistical predictions are utilized, a lot of general interest is focused around the first part (predicted value) because that feels like the successful part, like hard facts. In contrast, the uncertainty part can sometimes feel, well, like the falling-short part, the part to be minimized.

Even while being something we may desire to minimize, the estimate of uncertainty is still valuable information. For one, you can’t minimize what you don’t measure. For another, when we know something that is highly certain, we can make confident decisions. So if we are fairly certain that an outcome will be highly (dis)advantageous, knowing this information with certainty will cause us to act more decisively and possibly more intensively to (divest) invest.

So both parts are valuable. In every day life we use both to make informed decisions. And its good when the reporting and consumption of statistical prediction can reflect the way we use prediction all the time.

For example, say, you ask a financial advisor for her opinion on such and such an investment: what is wanted in her assessment of the opportunity is the expected value and how much she is confident that this estimate is likely.

She may be the wisest advisor there is and she could still rightly say, we only have enough information at this point to expect this amount of financial return on investment with this amount of variability in the possible outcomes. In fact, the wiser she is, the more clearly she’s able to articulate the risk part of investments via her estimate of the uncertainty, not just describe expected rewards as if they are certain. Furthermore, as new information comes out, she can likely adjust not only her informed estimate of the expected value but also how confident we can be in the likelihood of that estimate being realized.

My example above shows both levels of confidence or certainty: the certainty a client can have in a professional’s advice and the certainty of the expected outcome the professionals advice includes. A client’s certainty should be increased in their advisors credibility when she includes an uncertainty estimate of an expected outcomes. Experienced advisors will find a way to convey the level of certainty in a estimate without endangering the client’s trust in the credibility of the advisor’s experience or recommendation.

This is the distinction we are wanting to draw. Prediction gives two value pieces of insight:

• insight into what is likely to happen if the expected state of the world prevails and
• insight into how far from that expectations eventualities could vary.

Credible predictive analytics provides both. And consumers of predictive analytics should want both in order to scope and refine their actions.