Customer Churn Prediction
2026-07-20
A churn model framed around a real decision (a retention team that can only call about 200 customers) and evaluated against the heuristics it has to beat, not just on accuracy.
classification scikit-learn model evaluation statistics
Problem
Untargeted retention calling wastes most of its budget. The real question isn't “can I predict churn?” but “given a team that can call about 200 customers a month, which 200, and is a model actually better than a sensible rule of thumb?” I built the project around that decision so the evaluation reflects how the model would really be used.
Data
IBM Telco Customer Churn (Public sample dataset (7,043 customers, 26.5% churn))
Approach
EDA grounded in formal hypothesis tests with effect sizes (tenure and contract type dominate, gender tested null), then a strict protocol: a stratified split touched once, 5-fold cross-validation on the training set only, and preprocessing inside the pipeline so nothing leaks. Two no-model baselines came first, then logistic regression and gradient boosting, judged on precision within the top 200 scores, the metric that matches the calling budget.
Accuracy is deliberately demoted. With a 26.5% base rate, predicting “nobody churns” already scores 73.5% while catching no one. The write-up leads with precision-at-200, log-loss, calibration, and error analysis instead.
Result
The model reaches 74% precision in its top-200 calls, versus 63.5% for the best no-model heuristic (“call the newest month-to-month customers”) and 24.5% for random, about 21 extra genuine churn risks reached per cycle, at ROC-AUC 0.846. The simpler logistic model beat gradient boosting, and the report explains why. Confident misses are long-tenure two-year-contract customers, churn the features cannot see.
demo video
a short recorded walkthrough goes here once the project is complete
What I'd do differently
The data is a single snapshot, so the model ranks who is at risk, not when. The honest next step is uplift modeling, targeting persuadable customers rather than merely likely ones, which needs post-intervention outcome data this dataset doesn't contain.