Calibration, explained.
Three ideas separate real edge from guesswork: calibration, the honest check on your probability estimates; Brier score, the industry grade on how good they are; and the Kelly criterion, the formula that tells you how much to bet. This is the 20-minute version.
What "calibration" actually means.
A weatherman who says "70% chance of rain" every day for a year should see roughly 255 rainy days out of 365. If it rained only 180 times, he was overconfident — his 70% was really 50%. If it rained 300 times, he was underconfident — his 70% was really 82%. Either way, the probability he quoted does not match the frequency of the thing happening.
That mismatch is miscalibration. A probability estimate is calibrated when the frequency of outcomes matches the stated probability across the full range of predictions. Over 1,000 predictions bucketed by confidence, if we look at the ones where the model said "20%", 20% of them should have resolved YES. The ones where it said "80%", 80% should have resolved YES. If you plot predicted probability on the x-axis against actual frequency on the y-axis, a perfectly calibrated model lies on the diagonal.
Prediction market prices are themselves probability estimates. If Kalshi says 54¢ YES on "Fed cuts in December", the market is claiming 54% confidence. If we gather every market that traded at 54¢ in the final hour before resolution and 54% of them resolve YES, the market is calibrated. In practice, prediction markets are decently calibrated at the mid-range (40-60%) and systematically miscalibrated at the tails (favorites underpriced, longshots overpriced — the classic "favorite-longshot bias").
Platt scaling: how to fix a model that's wrong.
Most machine learning models output a number between 0 and 1 that people treat as a probability. The number is usually not a real probability. A deep neural network trained on classification will often output 0.95 for cases where the true frequency is closer to 0.80 — the model is systematically overconfident on its own predictions.
Platt scaling, introduced by John Platt in 1999, is the standard fix. You take your model's raw outputs on a holdout set, pair them with the actual 0/1 outcomes, and fit a one-parameter logistic function:
Where s is the raw model score and A, B are learned from the holdout data. In production, you pass every raw prediction through this logistic function before quoting it as a probability. The result is a model whose outputs track reality.
EDJE applies Platt scaling at the end of M4 (the ML Residual Blender), with weights refit every Monday morning on the prior 30 days of resolved markets. This is why the probability numbers you see on EDJE cards tend to be humbler than the raw model output — the system is actively compressing overconfident predictions back toward reality.
The Brier score: the industry grade.
Everyone needs a single number to compare one forecaster to another. The Brier score is that number. For a set of N predictions with probabilities pi and actual outcomes yi (where y is 1 if the event happened, 0 if not):
Perfect prediction gives Brier = 0. Always predicting 50% on binary markets gives Brier = 0.25. Random guessing on balanced data gives roughly the same. Elite forecasters land around 0.14-0.18 on hard markets. The algorithm's current live Brier is published on the Transparency page and updates in real time.
Brier has a nice decomposition: it breaks into reliability (calibration error) plus resolution (how much your predictions vary from the base rate) minus a constant. A model with great calibration but no resolution — predicting 50% on everything — has Brier = 0.25. A model with great resolution but poor calibration — predicting 100% and 0% confidently but half the time wrongly — is worse. The best models have both.
The Kelly criterion: how much to bet.
You have an edge. The market is at 50¢ YES. Your calibrated model says 60%. The expected value of a $1 YES bet is 0.60 · $1 − 0.40 · $1 = $0.20, which is excellent. The question is: what fraction of your bankroll do you bet?
In 1956, John Kelly derived the fraction that maximizes long-run expected log-growth of the bankroll given a known edge:
Where b is the net odds received on the bet (if the market is 50¢ YES, winning doubles your money, so b=1), p is your calibrated true probability (0.60), and q = 1 − p (0.40). Plugging in: f* = (1·0.60 − 0.40) / 1 = 0.20. The math says bet 20% of bankroll.
Full-Kelly is correct in theory and brutal in practice. It assumes your probability estimate is exact, and when you're off (which you always are), full-Kelly's variance is punishing. Drawdowns of 50%+ are common even for positive-expectancy bettors. The practical rule is half-Kelly — bet half the Kelly fraction. You give up some long-run growth in exchange for dramatically lower variance. EDJE's half-Kelly recommendations are additionally clamped to 20% ceiling, because real traders have real risk tolerance.
How EDJE puts it together.
Every edge on EDJE is the output of this exact chain. The 47 signal sources are blended into a raw probability. The ML residual blender (M4) applies Platt scaling to that raw probability using weights refit every Monday on 30 days of real resolutions. The resulting calibrated probability is compared to the vig-removed market price to produce an edge magnitude in percentage points. The Kelly card then recommends a half-Kelly stake at your bankroll size.
The Transparency page publishes the live Brier, the hit rate, the realized P&L, and the delta between expected and actual. Nothing is hidden. If the model is miscalibrated, you'll see it in the numbers before you see it in the pitch.
FAQ.
What's a good Brier score?
On resolved prediction markets, elite forecasters land in the 0.14-0.18 range. Markets themselves tend to score around 0.18-0.22. EDJE's current live Brier is published on the Transparency page.
Isn't the Kelly criterion risky?
Full-Kelly is risky because it assumes your edge estimate is exact. Half-Kelly cuts the variance roughly in half while sacrificing only a small fraction of long-run growth. For most practical purposes, half-Kelly is the correct default.
How often does EDJE recalibrate?
The M4 ML Residual Blender weights refit every Monday at 04:00 UTC on the prior 30 days of resolved markets. A Brier gatekeeper refuses to publish any candidate whose holdout Brier exceeds the current production model.
What's the connection to "wisdom of crowds"?
Markets are often decent calibrators when their participants are well-informed and diverse. They become miscalibrated when the trader pool is thin, partisan, or inattentive — which happens in the exchange category more often than in liquid financial markets. That's where edge lives.
See a calibrated probability in action.
Watch Platt scaling, Brier scoring, and half-Kelly on a live market.
Open The Engine Room →