What PSI is: Population Stability Index measures how much a feature distribution has shifted from a historical baseline. For each feature, divide the baseline into 10 deciles. Then bin recent observations using the SAME edges. Compute:
PSI = ฮฃ (recent_pct - hist_pct) ร ln(recent_pct / hist_pct)
Thresholds (industry standard):
- < 0.10 STABLE โ no meaningful shift, brain operates normally
- 0.10-0.25 MINOR โ monitor closely, may need retrain soon
- > 0.25 MAJOR โ population has changed, model needs retraining
Why this is a leading indicator: rolling accuracy drift only triggers AFTER the model has been wrong many times. PSI triggers when the INPUT distribution shifts โ before the model has even had a chance to be wrong. Buys you days of warning instead of minutes of regret.