Machine Learning
Customer Churn Prediction Service
A gradient-boosted churn model served as a real-time API with monitored feature drift and explainable scores.
Time-based
Validation
Per-prediction
Explainability
Real-time API
Serving
Architecture
Hover or tap any stage to see its role in the pipeline.
Data flow
Batch and event data land in the feature pipeline, features are versioned and stored, training runs produce a registered model artefact with evaluation metrics, and the serving API loads the promoted version. Live inputs and predictions are logged for drift and performance monitoring.
Problem statement
Retention teams reacted to churn after it happened, with no ranked view of which customers were at risk or why.
Business objective
Produce calibrated churn probabilities with per-customer explanations, served fast enough for in-journey decisioning.
Solution overview
A reproducible feature pipeline feeding a gradient-boosted classifier, packaged behind a low-latency scoring API with drift monitoring and scheduled retraining.
AI / ML approach
Behavioural and transactional feature engineering, class-imbalance handling, probability calibration and feature-attribution explanations returned alongside each score.
Model / LLM used
XGBoost classifier with probability calibration
Technology stack
Pythonscikit-learnXGBoostSQLAirflowFastAPIDockerAzure ML
Key capabilities
- Calibrated churn probability
- Per-prediction feature attribution
- Versioned features and models
- Drift and performance monitoring
- Scheduled retraining
Evaluation approach
- Time-based validation split to avoid leakage
- ROC-AUC, PR-AUC and calibration curves
- Lift by decile for campaign targeting
- Stability checks across customer segments
Challenges
- Leakage from post-event features
- Severe class imbalance
- Model performance drifting after seasonal shifts
Solution decisions
- Strict feature cut-off relative to the prediction date
- Class weighting plus threshold tuning against business cost
- Automated drift alerts wired to a retraining trigger
Business impact
Gave retention teams a ranked, explainable risk list they can act on before a customer leaves, rather than a post-mortem report.