1. Define churn precisely
Decide what counts: a cancelled subscription, a missed renewal, or a period with no usage. Settle whether paused accounts count and how downgrades are treated. Two teams using two definitions will report two different churn rates, and that argument is better had before any modelling.
2. Look at cohorts before models
Group customers by the month they joined and track how many remain over time. Cohort retention curves show whether churn is improving, whether it is concentrated early in the customer life, and whether a product or pricing change moved it. Many questions are answered at this stage without any machine learning.
3. Separate customer count from revenue
Losing ten small accounts and losing one large account are different problems. Track revenue churn alongside logo churn, and note whether expansion from existing customers is hiding losses.
4. Build the prediction without leakage
Use only information available at the prediction date, and test on a later period than the one used to train. Signals recorded after a customer has already decided to leave, such as a cancellation survey, make a model look excellent and useless.
5. Calibrate the probabilities
If a model says 30 percent, roughly 30 percent of those customers should churn. Calibration checks this and corrects it when it is off. It matters most when you multiply the probability by the revenue at stake to decide where to focus.
6. Ask who can actually be persuaded
The customers most likely to leave are not always the ones a retention offer will change. Some will leave regardless, and some would have stayed anyway. Uplift modelling estimates the difference an intervention makes for each customer, and it needs data from a real experiment or a carefully argued causal setup. Without that, treat any claim about who to target with caution.
7. Turn it into an action list
- Name the segments, not just the scores.
- Tie each to a specific action and an owner.
- Decide in advance how you will measure whether it worked.
See it in a working project
I built a public case study around this topic (read the full write-up), with code and documentation on GitHub: saas-customer-churn-retention-analytics. It runs on synthetic data generated for the project, so it demonstrates the method and the engineering rather than results from a real client.
Further reading
- scikit-learn: common pitfalls, including data leakage: Why preprocessing must happen after the train/test split.
- scikit-learn: probability calibration: How to check that a predicted 20 percent behaves like 20 percent.
- CausalML documentation: An open-source library for uplift modelling.