Forecasts that change what somebody does on Monday
You get a prediction attached to a decision, a record of how well it did against reality, and an honest answer when a simple rule would do the same job.
The short version
4 things that decide this
- 01Start from the decision, not the data. If nobody would act differently on the answer, the model is a report with extra steps.
- 02Every prediction needs a baseline. Beating last month's average is the bar, and plenty of models do not clear it.
- 03A forecast is only trustworthy once you have compared old predictions against what actually happened.
- 04Hashlogics has 22 systems in production, including SmartREI, which projects a property's ROI, cash flow and total cost from real inputs.
The dashboard nobody opens twice
Most predictive projects die the same way. A model gets built, a dashboard gets shipped, people look at it for a fortnight, and then it goes quiet.
The reason is almost never accuracy. It is that the number arrived without a decision attached. A churn score with no rule about who gets called and by when is trivia. The team already suspected which accounts were shaky.
The other killer is silence about the past. If nobody checks last quarter's forecast against what happened, the model can be badly wrong for a year and still look authoritative on a screen.
What we have shipped
Counted, not estimated
22
systems in production
9
of them with AI or scoring inside the product
50+
ESG topics Greenlight scores per company
23
tracked actions in TrialTriage's audit trail
What a useful prediction needs around it
The model is the small part. These are the parts that decide whether anyone uses it.
A decision it feeds, with the action and the owner named.
A baseline to beat, so improvement can be proved rather than asserted.
History of past predictions against what actually happened, kept and reported.
A rule for what happens when the model is unsure, instead of a number with false confidence.
An explanation a person can repeat to a customer, because scores get questioned.
A check that the data behind it has not quietly changed shape.
- DecideWhich action changes. Who takes it.
- BaselineThe simple rule the model must beat.
- BacktestOld data, honest cutoff dates.
- ShipInto the workflow, not a dashboard.
- ScorePredictions against outcomes, monthly.
Baseline sits second because plenty of problems are solved by a rule. If the model cannot beat last month's average, the honest recommendation is the average.
Building a forecast that cheats without meaning to
The most common failure in this work is a model that scores brilliantly in testing and poorly in production. Nothing was faked. The test simply let it see information that would not exist at the moment of the real decision.
A churn model that uses the cancellation date, a demand forecast trained on figures revised afterwards, a risk score that includes an outcome recorded later. Each looks like a triumph until launch. Your test gets built to match the moment the decision is actually made. That usually lowers the score and raises the value.
- Tests use only what was known at the moment the decision would have been taken.
- Late-arriving and corrected data are treated as what they are, not as history.
- The model is compared with a simple rule, every time, and sometimes loses.
- Predictions and outcomes are stored together, so accuracy can be checked later.
What we build on
Modelling
Data
Delivery
Run
“Their attention to detail, quality of employees, and work ethic were outstanding.”
Nicolas de Quesada · CEO, Lexpair
The usual analytics project against ours
Both end with a number on a screen. Only one of them changes what happens next.
| Criterion | The usual approach | How we build |
|---|---|---|
| Where it starts | With the data that happens to be available. | With the decision, the action, and who takes it. |
| The comparison | The model against nothing. | The model against a simple rule, which sometimes wins. |
| Testing | Random split, including data from after the decision. | Cut off at the moment the decision would be made. |
| After launch | Nobody checks the old forecasts. | Predictions and outcomes stored together, scored monthly. |
| Where it lands | A dashboard someone has to remember to open. | Inside the workflow, where the decision already happens. |
01Have you built a dedicated forecasting product for a client?
Our predictive work sits inside products as scoring and ranking rather than as standalone forecasting engagements, and that distinction is worth stating plainly. SmartREI projects a property's ROI, cash flow and total cost from purchase price, renovation, rent, taxes and expenses. Greenlight scores companies across more than 50 ESG topics with 10 to 15 independent sources each. Those two are the shape of work you can check.
02How much history do we need before a forecast is possible?
Enough to cover the patterns you care about, which usually means more than one full cycle of whatever repeats. A seasonal business needs at least a couple of years to separate a trend from a summer. When the history is thin, a simple rule plus honest uncertainty beats a model pretending to precision.
03How do we know the prediction is any good?
Compare it with the simplest rule you could have used instead, on data from before the decision date. Last month's number, or last year's same month, is a serious opponent and it wins more often than vendors admit. If the model cannot beat it, you have saved yourself a build.
04Can the model explain why it scored something the way it did?
It has to, if anyone will be asked to justify the decision. That constraint sometimes means a simpler model, giving up a little accuracy for an explanation your team can repeat to a customer. Where the score affects someone's access to a service, that trade is not optional.
05What happens when the market changes and the forecast stops working?
You find out from the monthly comparison of predictions against outcomes, which is the point of storing them together. A model built on last year's behaviour degrades as behaviour changes, and nothing about that raises an error on its own. Retraining is scheduled work, not an emergency.
06Is this different from the reporting our BI tool already does?
Reporting tells you what happened; a prediction commits to what will happen and can be scored on it. That difference only matters if someone acts on it, which is why your decision comes first. Plenty of teams asking for prediction actually need their existing reports to be trusted first.
07Could a simple rule replace the whole project?
Often, and you should hear that in the free scoping call rather than after a build. Rules are cheaper to run, easy to explain and never wrong in a surprising way. A model earns its place when the pattern is genuinely complex and the volume makes small improvements worth money.
08Does this suit a small company, or only large data teams?
Both, at different depths. A small team usually needs one decision improved, such as which jobs to schedule first, and a rule with good data often does it. A large organisation wants backtesting, monitoring and an explanation trail. More people rely on the number, and someone will eventually have to defend it.
