Predictive PlanningInstitute
The Discipline10 min read

Tacit knowledge and the limits of predictive analytics

Predictive software can price what is measurable. The hardest decisions inside an organization turn on what isn't. The discipline of predictive planning exists because of that gap — and the gap is not closing.

Every enterprise software vendor in the planning category is pitching a version of the same promise. Connect your data, train the model, and the system will predict outcomes with enough accuracy to make planning a solved problem. The pitch has been running for a decade. The discipline of strategic decision-making inside large organizations has gotten harder, not easier, in the same period. There is a reason.

Predictive analytics, no matter how well-tooled, can only price the future on the basis of what is measurable. The decisions that matter most inside an operating company turn on knowledge that is not measurable, not centrally held, and often not articulated by the people who have it. Michael Polanyi called this tacit knowledge. The discipline of predictive planning exists because the gap between what can be measured and what must be decided is not closing.

What predictive analytics is good at

Modern predictive systems are extraordinarily good at three things, and it is worth being precise about them so the limits are visible by contrast.

They are good at regime-stable forecasting. Demand forecasts in stable categories, churn predictions in mature cohorts, equipment-failure prediction with adequate sensor coverage. When the underlying generative process of the data has not changed materially, models built on that data forecast well. This is real value, and it is worth paying for.

They are good at pattern recall at scale. Anomaly detection across millions of transactions, fraud signals across a portfolio, segmentation across a customer base too large for human attention. The point of the model in these cases is not novel insight but disciplined application of known patterns to data volumes humans cannot personally process.

They are good at scenario simulation given assumptions. Given a set of explicit assumptions, a model can rapidly compute the implied outcomes across thousands of variations. Monte Carlo is now a button. The model does not generate the assumptions, but it does the arithmetic at a speed that lets a human team explore a much wider possibility space than they otherwise could.

Predictive planning depends on all three of these capabilities. The signal layer is built on regime-stable forecasts. The Scan phase relies on pattern recall. The Story phase makes heavy use of scenario simulation. The discipline does not exist in opposition to predictive analytics — it sits one layer up.

What predictive analytics is structurally bad at

The capability stops where the measurable stops. And inside an operating company, the most consequential strategic information is not measurable in the system of record.

It is not measurable that the second-largest customer has stopped returning the senior account manager's calls — until it is measurable in the form of a lost contract, by which point the signal has aged into a fact. It is not measurable that the new plant manager in the third region is privately telling colleagues that the equipment will not hold for another quarter — until the equipment fails. It is not measurable that the regulator's chief of staff is preparing a memo that will reframe the policy environment — until the memo lands.

These are not edge cases. They are the modal shape of the information that matters most. The information arrives as human signal — conversation, observation, the absence of a normal pattern — and it arrives early. By the time it has been instrumented into the data layer, the predictive system has lost the part of the signal that was forecasting.

Tacit knowledge and the operator

The reason organizations once tolerated heavy reliance on the intuition of senior operators was not nostalgia. It was that the operator carried a layer of tacit knowledge the data layer could not see. The plant manager who has run the line for fifteen years knows things about the equipment that no sensor will instrument. The veteran salesperson knows things about the buying process that no CRM will record. The legislative aide who has watched the committee for a decade knows things about the next bill that no policy-tracking dashboard will surface.

Two parallel mistakes have eroded this layer over the last two decades. The first is the displacement of senior operators by analytical functions in front of the leadership team. The analyst's job was to bring data; the operator's job was to bring judgment; the leadership team got both. As the analytical function professionalized, the operator's chair migrated. In many firms, the executive layer now hears the data without the tacit translation that used to accompany it.

The second mistake is the assumption — common among technology leaders — that tacit knowledge is a temporary inefficiency eventually solved by better instrumentation. It is not. There is a class of human knowledge that is generated through repeated embodied practice and cannot be encoded in a way that survives the encoding. Tacit knowledge does not become explicit knowledge when you write it down. It becomes a different, weaker thing.

What the discipline actually does

Predictive planning is the discipline that holds both layers in a single operating cadence. It uses predictive analytics for what analytics is good at. It uses structured human judgment for the rest. And it has a posture for the boundary between them — for the moment when the model is producing one read and the operator is producing another, and the question is whose read you trust on which dimension.

The Scan phase ingests both system signal and human signal as first-class inputs. The Story phase uses the model for arithmetic and the operator for the cleavages. The Stake phase uses analytical sizing inside structural conviction. The Steer phase watches for the indicators the model can monitor and the indicators only a human pattern-reader will catch. The discipline is the architecture that makes those layers compose.

A firm that runs predictive analytics without the discipline gets a faster version of the planning it already had. A firm that runs the discipline without predictive analytics gives up leverage that the technology genuinely provides. A firm that runs both gets the closest thing to a continuous, calibrated, operator-grounded read of its own future that any organization has yet been able to assemble.

Why the gap is not closing

The technology will improve. Models will get better at fewer-shot learning, at handling regime change, at integrating unstructured signal. None of that closes the gap. The gap exists because organizational decisions of consequence are made under conditions that are simultaneously novel, contested, and political. Novelty defeats pattern matching. Contestation defeats consensus forecasting. Politics defeats decision-by-spreadsheet.

Those three conditions are the operating environment of the executive layer in any organization that is large enough to need a strategy. The models will serve the layer below. The discipline serves the layer the models cannot reach.

The vendors are right that planning needs better tools. They are wrong that better tools will substitute for the discipline. The firms that get this distinction will buy the software and install the discipline. The firms that don't will buy the software and call it a strategy.

Continue reading

The full library of essays on the discipline.

All articles →
The newsletter

Long-form essays on the discipline. Roughly weekly. Reply anytime.