September 2, 2026
İlkim SEVİNÇ
• Assistant Associate Data Scientist
How enterprise data engineering, LightGBM, and explainable AI came together to turn
HR records into a practical early-warning system
Employee turnover rarely happens overnight.
More often, it is the result of signals that build up gradually: long periods in the same role, limited career movement, repeated managerial changes, or a growing disconnect from the surrounding team. On their own, these signals may not say much. But when they are combined across time, they can start to reveal patterns that are difficult to see in traditional HR reports.
That is where machine learning can become useful—not as a replacement for human judgment, but as a way to help HR teams notice risk earlier and ask better questions. In this project, we developed an explainable employee churn prediction framework for six companies operating under the same corporate group. Although the companies shared a broader governance structure, they differed significantly in industry, workforce composition, HR processes, data quality, and turnover dynamics. The goal was not simply to build a model that classified employees as “leaving” or “staying.”
The goal was to create an operational early-warning system that could:
The resulting framework combined enterprise data engineering, company-specific LightGBM models, time-based validation, SHAP explainability, and an automated production pipeline.
Why Churn Prediction Is Harder Than It Looks
Employee churn is a naturally imbalanced problem. Within any three-month period, most employees stay. Only a relatively small share leave. This means a model can appear accurate simply by predicting “no churn” for almost everyone—while still failing to identify the people HR teams most need to understand. Enterprise HR data introduces further complications. Different companies may use different HR systems, populate fields inconsistently, apply different organizational structures, or interpret seemingly similar variables in different ways. Even within the same company, schemas and business processes may change over time.
A successful enterprise churn model must therefore solve more than a classification
problem. It must also address:
- Incomplete and inconsistent data
- Temporal relationships between employee records
- Differences between organizations
- Potential leakage from post-departure HR processes
- Highly imbalanced target classes
- The need for interpretable predictions
- Integration with existing enterprise systems
This monthly snapshot structure was critical. Instead of treating each employee as a static record, it allowed the model to observe how an employee’s organizational situation changed over time.
This was essential because churn is a temporal event. The relevant question is not only:
“Did this employee leave?”
It is:
“Based on the information available at this moment, will this employee leave within the
next three months?”
A Layered HR Datamart
To create a reliable modeling dataset, data from multiple enterprise HR systems was
consolidated into a centralized datamart.
The architecture followed a layered design:
- Staging layer: Raw data extracted from operational systems with minimal
modification - Transformation layer: Cleaned, standardized, and integrated data
- Analytical layer: Final employee snapshots and features used by the machine learning pipeline
This separation improved data lineage, maintainability, and traceability. It also allowed modeling logic to remain independent from the structures of individual source systems.
Keeping Invalid Records Visible
One of the most important preprocessing mechanisms was an Out-of-Model filtering process. Some records contained structural problems that could not be safely resolved through simple imputation. Examples included:
- Missing internal seniority
- Invalid relationships between hiring and snapshot dates
- Inconsistent employment timelines
- Missing or malformed current-position information
- Organizational hierarchy conflicts
Instead of silently correcting or deleting these observations, the pipeline moved them into a dedicated exclusion layer.
This served two purposes.
First, it prevented invalid records from affecting target generation and model training.
Second, it preserved traceability. HR and data teams could inspect excluded records separately and identify recurring upstream data-quality problems. This distinction matters. In enterprise machine learning, removing a record from modeling should not mean losing visibility into why it was removed.
Turning HR Records Into Predictive Signals
Most operational HR systems are built for payroll, organizational management, reporting, and administrative workflows. Their variables are not created for prediction by default. For that reason, feature engineering became one of the most important parts of the project. Early experiments focused primarily on individual employee attributes such as tenure, job grade, department, and demographic variables. These features provided some predictive signal, but they did not sufficiently capture the organizational context surrounding employee turnover.
The key improvement came from expanding the dataset beyond the individual employee.
Career Progression Features
Career-related variables were created to describe how an employee’s position within the organization evolved over time.
Examples included:
- Total time within the company
- Internal seniority
- Time spent in the current position
- Time spent in the current department
- Time spent at the current grade level
- Indicators of career stability or stagnation
These variables helped represent whether an employee was progressing, remaining stable, or spending unusually long periods in the same organizational position.
Managerial Context Features
Employee behavior does not exist independently from leadership and team structure. To incorporate managerial context, employee snapshots were enriched with information about the employee’s current manager, such as:
- Manager tenure within the company
- Manager’s time in the current department
- Manager’s time at the current grade
- Managerial career history
- Stability of the employee–manager relationship
This allowed the model to identify patterns that individual-level attributes alone could not reveal.
For example, an employee’s tenure might not be particularly informative by itself. However, the proportion of that tenure spent under the same manager may indicate a very different organizational experience.
Relative and Ratio-Based Features
We also created variables that compared employee characteristics with managerial and organizational values.
Examples included:
- The proportion of employee tenure spent with the current manager
- Internal seniority relative to total career length
- Employee tenure relative to manager tenure
- Position duration relative to team or organizational norms
- These relative measures often carried more meaning than absolute values.
Five years in the same role may be normal in one organizational context but may signal career stagnation in another. Ratio-based features allow the model to interpret employee characteristics within their surrounding hierarchy.
One of the clearest lessons from this stage was:
Employee churn cannot be fully understood through employee attributes alone. Managerial relationships and organizational context matter.
Framing Churn as a Three-Month Prediction Problem
The target variable was designed to support early intervention.For every monthly employee snapshot, the system checked whether the employee left the company during the following 90 days.
The target was defined as:
- 1: The employee leaves within the next three months
- 0: The employee remains beyond the three-month prediction horizon.
This produced a forward-looking target rather than a historical label.
An employee could therefore appear in multiple monthly snapshots, with each snapshot representing a new prediction point. The model learned from the conditions observed before departure rather than from a single final employee record.
The three-month horizon was chosen as a practical balance: close enough to support near term HR planning, but still early enough to leave room for a potential retention action.
Avoiding the Trap of Data Leakage
Data leakage is one of the greatest risks in employee churn prediction.
Some administrative variables may be updated after an employee has already initiated a resignation or termination process. If these fields are included in model training, the model can appear highly accurate while learning information that would not actually be available during real-world prediction.
To reduce this risk, we introduced an iterative leakage-detection stage.
Shallow decision trees were trained as diagnostic models using the candidate feature set. Features with unexpectedly high predictive importance were reviewed to determine whether they might indirectly represent post-event information.
Suspicious variables were removed, and the process was repeated until the remaining features represented information that could realistically exist before a potential departure.
This procedure was performed independently for each company because leakage risks differed according to local HR systems and processes.
A high-performing model is not useful if its performance depends on information from the future.
Choosing LightGBM for Large-Scale Tabular HR Data
Both XGBoost and LightGBM were evaluated during model development.
LightGBM produced the strongest and most stable results across the company-specific datasets and was selected as the final algorithm.
During the early stages of model development, Logistic Regression was evaluated as a simpler baseline approach. However, its predictive performance remained below the tree-based boosting models.
The structure of the enterprise HR data also made boosting methods more suitable for the problem. The datasets contained missing values, high-dimensional engineered features, nonlinear relationships, and interactions between employee, managerial, and organizational variables. Logistic Regression required additional preprocessing and imputation to handle these characteristics effectively, while tree-based boosting methods were better suited to capturing the underlying complexity of the data.
Based on these early experiments, the modeling process shifted toward gradient-boosting algorithms.
Both XGBoost and LightGBM were then evaluated during model development. LightGBM produced the strongest and most stable results across the company-specific datasets and was selected as the final algorithm.
It was particularly suitable for this use case because it:
The models were trained with balanced class weights to account for the low frequency of churn events. Parameters controlling tree depth, leaf count, minimum observations, and L1/L2 regularization were tuned to reduce overfitting.
Rather than producing a simple yes-or-no result, LightGBM generated a churn probability for every employee snapshot.
These probabilities were then used to rank employees by relative risk.
Why We Did Not Use One Global Model
At first glance, building one model for all six companies might appear more scalable. However, the companies differed in several important ways:
- Industry and business model
- Workforce size
- Promotion and career structures
- Organizational hierarchy
- Feature availability
- Data-entry practices
- Baseline churn rate
- Managerial dynamics
A pattern that signals elevated churn risk in one organization may be normal in another.
For this reason, a separate model was trained for each company while keeping the broader data engineering and modeling pipeline consistent.
This approach provided a useful balance:
- Shared infrastructure and methodology
- Company-specific learning and feature selection
The most useful feature set was also selected separately for each organization. This allowed each model to retain the variables most relevant to its own workforce dynamics.
Training on the Past, Testing on the Future
Random train–test splitting was not appropriate for this project.
Random splitting could place a later employee snapshot in the training set while an earlier snapshot of the same employee appeared in the test set. It could also mix historical and future workforce conditions, producing an unrealistic estimate of model performance.
Instead, we used a strict time-based split.
The latest three months of available snapshots were reserved for testing, while earlier snapshots were used for training.
Time-series-based validation was also used during model development to simulate repeated scenarios of:
Train on historical workforce data, then predict a later period.
This setup was much closer to production conditions, where a model trained on existing data must score employees in a newly completed monthly snapshot.
Measuring What Actually Matters
ROC AUC was selected as the primary evaluation metric.
Accuracy was not suitable because churn events were rare. A model predicting that almost no employees would leave could achieve high accuracy but fail to identify meaningful risk.
Metrics such as precision, recall, and F1 score are useful, but they depend on a specific classification threshold. In this project, the primary objective was to rank employees according to relative risk rather than force every prediction into a fixed binary category.
ROC AUC measures the probability that an employee who eventually leaves receives a higher risk score than an employee who remains.
This made it suitable for:
- Imbalanced employee datasets
- Comparing models across companies
- Feature selection
- Hyperparameter optimization
- HR prioritization based on ranked risk
What the Models Showed Across Six Companies
The models produced the following ROC AUC scores:
- Company A: 0.83
- Company B: 0.75
- Company C: 0.79
- Company D: 0.75
- Company E: 0.98
- Company F: 0.72
These results showed that the framework could produce meaningful churn-risk rankings across organizations with different workforce structures, data conditions, and HR processes.
However, the scores must be interpreted together with data quality and preprocessing decisions.
Data Quality Strongly Influenced Performance
Companies with more complete and consistent HR records generally achieved stronger results.
Company F, for example, initially produced an AUC close to 0.61. After additional cleaning, feature engineering, and refinement, its performance increased to 0.72.
This improvement was not primarily the result of switching to a more complex algorithm. It came from improving the quality and representation of the underlying data.
Extremely High Performance Required Caution
Company E achieved an AUC of 0.98, but this result was not treated as unquestionable success.
The company used different filtering conditions because of data constraints. Some records were included without the standard exclusions applied elsewhere, increasing the possibility of overfitting or hidden leakage.
An unusually high evaluation score should trigger further validation, not immediate celebration.
Managerial and Career Features Were Consistently Valuable
Across multiple models, features associated with the following themes frequently appeared among the strongest predictors:
- Employee tenure
- Time in the current position
- Time in the current department
- Internal seniority
- Time with the current manager
- Managerial stability
- Promotion and career progression
- Relative employee–manager tenure
This reinforced one of the central findings of the project: turnover risk is shaped by an employee’s position within an organizational system, not only by isolated personal attributes.
Making Risk Scores Usable for HR Teams
Raw probability scores can be difficult to use in HR operations.
To improve interpretability, predicted probabilities were converted into six risk groups:
This segmentation created a clearer interface between the model and HR users.
Rather than examining thousands of decimal values, teams could monitor how employees were distributed across risk groups and prioritize cases requiring closer review.
Risk categories could also be aggregated by:
- Department
- Job family
- Manager
- Company
- Organizational unit
- Reporting period
These categories were not designed to make automatic employment decisions. They were designed to help human decision-makers identify areas that deserved closer review.
Explaining Predictions With SHAP
Accuracy alone was not sufficient for an HR application.
A model may identify an employee as high risk, but HR professionals still need to understand what influenced that result.
SHAP values were therefore integrated into the prediction pipeline.
For each employee, SHAP estimated how individual features moved the prediction above or below the model’s baseline risk.
A positive contribution increased the predicted churn risk, while a negative contribution reduced it.
An employee’s explanation might indicate that risk was influenced by factors such as:
- Extended time in the same position
- Limited recent career movement
- A change in managerial structure
- An unusual employee–manager tenure relationship
- Department-level instability
SHAP supported two different levels of interpretation.
Local Explainability
Local explanations described why the model assigned a particular risk score to an individual employee.
Global Explainability
Aggregated SHAP values showed which features were most influential across an entire company or workforce segment.
Both the probability predictions and feature-level explanations were stored in structured enterprise tables so that reporting and analytics systems could use them together.
This turned the model from a black-box scoring mechanism into an interpretable decision support tool.
From Notebook Experiment to Production Pipeline
A machine learning model provides limited business value if it only runs manually in a development notebook.
The churn framework was therefore designed as an automated production pipeline.
Its main stages were:
The workflow was orchestrated through Apache Airflow and deployed within a containerized environment.
Airflow represented the process as a series of dependent tasks. This made it possible to control execution order, monitor failures, schedule monthly runs, and ensure that upstream HR data processes completed before model scoring began.
Model artifacts were versioned so historical predictions could be associated with the model version and workforce snapshot that generated them.
This operational layer was a major part of the project. The final outcome was not just a predictive model, but a repeatable workforce analytics system.
Key Lessons From the Project
1. Data Quality Matters More Than Algorithm Complexity
A sophisticated model cannot compensate for inconsistent employment dates, incomplete position information, or unreliable organizational hierarchies.
Several of the largest performance improvements came from better data preparation rather than more advanced modeling.
2. Organizational Context Improves Churn Prediction
Employee-level variables alone did not explain turnover sufficiently.
Manager characteristics, hierarchical relationships, and relative career-position features added important predictive information.
3. Every Company Has Its Own Churn Dynamics
Companies within the same corporate group may still require different models, features, thresholds, and validation strategies.
A shared technical framework does not require a single global model.
4. Temporal Validation Is Essential
A churn model must be evaluated on a future period, not on a random sample drawn from the same historical distribution.
Otherwise, reported performance may not reflect real deployment conditions.
5. Explainability Must Be Part of the Architecture
Adding explanations after deployment is not enough.
Explainability should be calculated, stored, monitored, and delivered as a standard component of the prediction pipeline.
6. Very High Scores Can Be Warning Signs
An AUC of 0.98 may indicate excellent predictive signal, but it may also point to leakage, overfitting, or inconsistent preprocessing.
Model evaluation always requires context.
What This Framework Still Cannot Solve
The framework still has important limitations.
First, the company-specific models cannot automatically be transferred to unrelated organizations. Each company has its own processes, workforce distributions, and data practices.
Second, model performance remains dependent on upstream HR data quality. Changes in schemas, delayed pipelines, or inconsistent field population can affect prediction reliability.
Third, the target captures only departures within the next 90 days. It does not directly model long-term disengagement, internal mobility, or the exact expected time of departure.
Fourth, major external events such as economic shocks, mergers, restructurings, or changes in labor-market conditions may alter employee behavior faster than the model can adapt.
Finally, explainability does not remove ethical risk. Features involving age, gender, seniority, organizational status, or related proxies require careful governance. Churn predictions should support human review—not automate employment decisions or label employees as disloyal.
Where the Framework Can Go Next
Several extensions could strengthen the system.
Sequence-aware or survival-analysis models could be used to estimate not only whether an employee will leave, but also when departure is most likely.
Automated drift monitoring could track changes in:
- Feature distributions
- Churn rates
- Prediction distributions
- Model performance
- SHAP explanation patterns
Probability calibration could improve the relationship between predicted risk and observed churn frequency.
Fairness audits could compare model behavior across employee groups and identify potential disparities.
The final step would be to connect prediction with intervention measurement. Instead of only identifying risk, organizations could track which retention actions were taken and whether they changed subsequent outcomes.
This would move the system from predictive analytics toward evidence-based, prescriptive workforce intelligence.
Final Thoughts
Employee churn prediction is not just a machine learning problem.
It is a combination of:
- Data integration
- Temporal modeling
- Feature engineering
- Organizational understanding
- Explainable AI
- Model governance
- Production engineering
The strongest outcome of this project was not a single AUC score.
It was the creation of a common enterprise framework that could adapt to six different companies while preserving traceability, interpretability, and operational consistency.
By combining monthly workforce snapshots, company-specific LightGBM models, SHAP explanations, and an automated data pipeline, the system helped shift HR analytics from retrospective turnover reporting toward proactive workforce analysis.
Machine learning cannot determine why a person truly chooses to leave an organization. It should not replace managerial judgment, and it should never be used as an automatic decision mechanism.
But when it is designed responsibly, validated over time, and supported by strong data governance, it can help organizations notice meaningful patterns earlier—and ask better questions before valuable employees walk out the door.