Making Decisions When Probabilities Are Uncertain

Defining what is uncertain

Consider a board debating entry into a new market. The commercial team expects strong demand; the finance team worries that the evidence comes from expressions of interest rather than signed orders. Asking for a single probability of success may appear to settle the disagreement. It can also conceal it. The teams may be using different definitions of success, different time horizons or different assumptions about what customers will do.

The first task is to define the event or quantity. Is the question whether the project has positive net present value, reaches a sales target or avoids a funding shortfall? Over which period, and under which operating plan? Those are different questions. A forecast can be well supported and still leave a decision finely balanced; a highly uncertain forecast can coexist with a decision that remains attractive across many plausible outcomes.

It also helps to distinguish variation in the world from incomplete knowledge about it. Customer orders may vary from month to month, while the team remains uncertain about their average level or whether a new competitor will change the market. A model can represent the first kind of variation and still omit the second. EPA’s Monte Carlo guidance calls for explaining input distributions, dependencies and numerical stability, while distinguishing variability from uncertainty where possible.²

Making expert judgment explicit

Limited data do not make probability illegitimate. Relevant experts may have information that has never been assembled into a dataset. The challenge is to express that knowledge without disguising its limitations. EFSA’s elicitation guidance treats this as a structured process: framing the question, selecting experts and a method, eliciting uncertainty, handling aggregation and documenting the basis of the result. It is more demanding than asking a meeting to agree on a percentage.¹

In the market-entry example, customer interviews could reveal that adoption requires integration with an existing system. That finding changes the model’s structure: demand may depend on completing the integration first. The interviews do not, by themselves, establish how likely timely completion is. Technical records, comparable projects or a carefully defined elicitation would be needed for that separate estimate.

Disagreement can be informative. One expert may assume competitors keep prices unchanged, while another anticipates a price response. Averaging their estimates before examining those assumptions could create a forecast neither believes. EFSA’s methods guidance recognises the role of both statistical analysis and expert judgment, and of testing how conclusions respond to changes in model structure and inputs.⁵

Comparing choices across plausible futures

When the evidence cannot yet support agreed probabilities, the board can still compare actions. RAND’s robust decision-making approach begins with candidate plans and investigates the conditions under which they perform well or poorly. This changes the immediate research question from predicting one future to finding the vulnerabilities that matter to the choice.³

Consider three hypothetical alternatives: full entry, limited entry or waiting. Full entry costs RM18 million now; limited entry costs RM8 million. Under weak, steady and strong demand, full entry generates annual net cash of RM3 million, RM9 million and RM16 million, respectively. Limited entry generates RM2 million, RM5 million and RM10 million. Each scenario’s cash flow repeats at the end of each of three years. Waiting has no incremental cost or receipt in this simplified comparison.⁶

Discounting those cash flows at an illustrative 10% annual rate gives the values below. All amounts are in constant-price RM million, net of operating costs and before tax and financing, with no terminal value. The rate and cash flows are teaching assumptions. No probabilities have been assigned to the scenarios, and the limited operation is held fixed: a later expansion option has not been valued.

Full entry has the greatest upside and the largest loss. Limited entry performs better under weak demand and slightly better under steady demand. Waiting avoids the modelled investment loss but also forgoes the opportunity. These results do not yet identify one best action. They reveal the trade-off the board must decide how to judge.

Choosing how to judge the alternatives

One approach is to minimise maximum regret: the largest gap between an action’s value and the best value available in the same scenario. Regret is an opportunity-cost comparison, not an additional cash expense. The benchmark uses hindsight within each scenario; the decision itself must be made before that scenario is known.

Maximum regret is RM10.54 million for full entry, RM4.92 million for limited entry and RM21.79 million for waiting. Limited entry therefore wins under this criterion. But a board that instead maximises the worst modelled outcome would choose waiting, whose worst value is zero. Neither rule is automatically correct; each expresses a different attitude to missed opportunity and downside.⁶

A hard constraint can narrow the choice before any ranking. If the board will accept no scenario with an NPV loss greater than RM4 million, full entry fails that test, while limited entry and waiting remain eligible. This is an assumed NPV-loss limit, not a cash reserve or lender covenant. Changing the limit, the feasible options or the scenario boundaries can change the answer.

Robustness is always relative to what was tested. Three demand states do not cover every possibility. A credible further assessment would challenge the model with competitor responses, implementation failure and joint stresses on sales and costs. It should also investigate whether limited entry genuinely limits the commitment, including cancellation terms and obligations that survive withdrawal.

Finding the assumptions that change the choice

Probabilities can now be introduced as a sensitivity exercise. Suppose, purely for illustration, the probability of steady demand is held at 40%. Let the probability of strong demand vary, with the remaining probability assigned to weak demand. On an expected-NPV objective, full entry overtakes limited entry when the strong-demand probability exceeds approximately 36.4%. The other probabilities at that point are 40% steady and 23.6% weak.⁶

Andi SC01. Probability weights are assumptions for sensitivity, not estimates. Expected monetary value uses a risk-neutral ranking; constraints are assessed separately.⁶

The threshold is useful because it makes the disagreement concrete. If credible assessments straddle 36.4%, better information about demand could change an expected-value choice. If a binding loss limit already excludes full entry, refining that probability may be less urgent than understanding the downside of limited entry. The threshold is conditional on holding steady demand at 40%; another probability structure would produce another boundary.

Running a simulation cannot settle which weights are credible. More draws reduce sampling variation within a specified model; they do not supply missing commercial evidence. Nor does an action succeeding in two of three unweighted scenarios mean it has a two-thirds chance of success. In this small example, exact arithmetic is sufficient; the probability curve is a sensitivity analysis, not an empirical forecast.

Preparing to adapt as evidence arrives

A decision can include a plan for changing course. Haasnoot and colleagues’ work on adaptive policy pathways links actions to observable signposts and triggers, with contingency actions prepared in advance. Its original water-management application offers a useful planning principle for other settings, although it does not establish the commercial value of a particular market-entry strategy.⁴

For the board, a useful signpost might be verified order conversion, customer retention or an integration milestone. The trigger should specify the evidence required, the responsible owner and the action available. It must also allow time to respond: a warning that arrives after a non-cancellable order has been placed offers little flexibility. Any later expansion would need its own costs, lead times and financing assessment.

The practical discipline is to separate three questions: what outcomes are plausible, how well their likelihood is supported and whether the choice remains sensible as assumptions change. A board need not settle every uncertainty before acting. It does need to understand what it is committing, which loss it can bear and what new evidence would warrant a different decision.