The problem: the model was trained on features collected over the past weeks. If today's features look qualitatively different (different volatility, different sector flows, different time-of-day patterns), the model's predictions are unreliable โ it's extrapolating into territory it doesn't understand.
The technique: a classic Kaggle trick called
adversarial validation. We collect two pools of feature vectors:
- Old pool โ features captured >24h ago
- Recent pool โ features captured in the last 2h
Then we label them (old=0, recent=1) and train a logistic regression to distinguish them. Test set AUC tells us how separable they are:
- AUC โ 0.5 โ features are indistinguishable; no shift
- AUC โ 0.6โ0.7 โ slight shift; still safe
- AUC > 0.70 โ clear shift; model is extrapolating
When shift is detected, the Unified Predictor multiplies position size by 0.60 (more conservative under regime change) and fires a
bpleone:covariate-shift window event for any listeners.
Complements DriftPSI: DriftPSI tracks output drift (are the brain's predictions changing); this tracks input drift (is the world changing). Together they cover both sides.