Icon

01_​Churn_​Prediction_​Solution

SOLUTION workflow of the KNIME AI Course (university workshop): telco customer churn prediction across the full data-science lifecycle with GenAI support.

1 Access & blend: Excel Reader (CallsData.xls) + CSV Reader (ContractData.csv) from the workflow data area, Joiner on Area Code + Phone.
2 Explore & prepare: Data Explorer, K-AI generated Expression features (Total Mins / Charge / Calls, Many Service Calls flag), nominal target, Phone as RowID, State and Area Code dropped, color palette on Churn, stratified 80/20 split (seed 42).
3 Algorithm selection: Decision Tree, Random Forest, Gradient Boosted Trees, Logistic Regression; one comparison table with accuracy, Cohen's kappa, recall / precision / F1 of churn = 1.
4 Hyperparameter optimization: Bayesian (TPE) Parameter Optimization Loop with 5-fold stratified cross-validation maximizing Cohen's kappa; best parameters bound via flow variables to the retrained model.
5 Evaluation: confusion matrix, ROC curve (AUC), Random Forest based feature ranking.
6 GenAI (KNIME AI Extension, needs your own OpenAI key or an OpenAI-compatible local model): personalised retention emails for the top-10 at-risk customers and a plain-language management summary.
7 Interactive dashboard: data-app component with linked KPI text, bar charts, histogram and at-risk table.

Data: classic KNIME telco churn example data, shipped in the workflow data folder.

KNIME AI Course - Telco Customer Churn Prediction (SOLUTION)

One use case, the full data-science lifecycle with GenAI support: 1 access & blend - 2 explore & prepare - 3 algorithm selection - 4 hyperparameter optimization - 5 evaluation - 6 GenAI with the KNIME AI Extension - 7 interactive dashboard. Read the top row (sections 1-3) from left to right, then the bottom row (sections 4-7). Everything except section 6 is executed; section 6 needs your own API key.

1. Access & blend

What: read call usage (Excel) and contract data (CSV) from this workflow's own data folder (knime://knime.workflow/data/...), so the workflow is self-contained when exported or uploaded to KNIME Hub. The Joiner matches both tables on Area Code + Phone (inner join, 3,333 customers).

Why: join keys must match in value AND type. Both readers deliver Area Code as Integer and Phone as String, so no conversion node is needed - always check the key types in the output table first.

2. Explore & prepare

What: Phone becomes the RowID (an identifier has no predictive value, but we keep it to find customers later). The Expression node engineers Total Mins, Total Charge, Total Calls and the flag Many Service Calls (4+ calls), turns the plan flags into yes/no and Churn into a nominal target ('0'/'1'). This expression was generated with K-AI inside the Expression node; prompt: "Add the columns Total Mins, Total Charge and Total Calls as the sums of the day, evening, night and international minutes, charges and calls. Add Many Service Calls = 'yes' if CustServ Calls is 4 or more, otherwise 'no'. Replace Int'l Plan and VMail Plan (1/0) by 'yes'/'no' and convert Churn to a string."

Why: State (51 values) and Area Code (3 codes that do not even match State in this data) are dropped - high-cardinality or non-causal columns invite overfitting. The color palette (churn = red) carries through every later view. The stratified 80/20 split with a fixed seed keeps the 14.5% churn share in both parts and makes results reproducible. Open the Data Explorer view to check distributions and missing values.

How to work with this workflow

Green traffic light = executed: right-click a node and choose the table or view to inspect results. Hover over a node to read its description. Section boxes explain WHAT is done and WHY. Ask K-AI (the AI assistant side panel) in Q&A mode to explain any node, or in Build mode to add nodes for you. To re-run everything: right-click the dashboard component and choose Execute - the GenAI nodes stay unexecuted until an API key is entered.

3. Algorithm selection

What: Decision Tree, Random Forest, Gradient Boosted Trees and Logistic Regression learn from the same training set and predict the same test set. Each Scorer's statistics get the model name and the overall Accuracy / Cohen's kappa (read from the Scorer's flow variables) in an Expression node; Concatenate stacks them and the churn = 1 row is kept -> ONE comparison table (last node, sorted by kappa).

Why accuracy is misleading: only ~14.5% of customers churn, so a model that always predicts 'no churn' already reaches ~85% accuracy. Logistic Regression scores 85% here yet finds only 13% of the churners. Cohen's kappa corrects for chance agreement; recall and F1 of the churn class show how many churners you actually catch. Note: Logistic Regression uses the de-duplicated feature set (minutes, no charges or totals) because charge = minutes x tariff - perfectly correlated inputs make its coefficients unstable.

4. Hyperparameter optimization

What: Bayesian optimization (TPE, 20 warm-up + 20 guided trials) tunes number of trees (50-300), tree depth (2-8) and learning rate (0.02-0.3) of Gradient Boosted Trees. Each trial is scored by 5-fold stratified cross-validation on the TRAINING set; the Loop End maximizes Cohen's kappa. The best values become flow variables that drive the final learner (Flow Variables tab), retrained on the full training set. Why: never tune on the test set - it must stay unseen until the final evaluation.

5. Evaluation & interpretation

Confusion matrix: rows = actual, columns = predicted. Missed churners (actual 1, predicted 0) cost revenue; false alarms only cost a retention offer. ROC curve: recall vs. false-positive rate for every threshold; AUC 1 = perfect, 0.5 = random. The default threshold is 0.5 - if a missed churner is more expensive than an offer, lower it to catch more churners at the price of more false alarms. Feature ranking: how often the Random Forest chose a feature for its top splits relative to how often it was available.

6. GenAI with the KNIME AI Extension (not executed: needs your API key)

Credentials Configuration -> OpenAI Authenticator -> OpenAI LLM Selector (gpt-4.1-mini). Double-click OpenAI API key and paste YOUR key into the password field - never type a key into a node setting or annotation. Branch A: the 10 customers with the highest churn probability get one prompt each (customer facts + churn drivers) -> LLM Prompter writes a personalised retention email. Branch B: the model comparison and the tuned model's confusion matrix become one prompt -> plain-language management summary. Swap the provider without touching the prompts: point the Authenticator's base URL to a local OpenAI-compatible server such as Ollama (http://localhost:11434/v1), or use another provider's Authenticator + LLM Selector.

7. Interactive dashboard

A component turned into a data app (right-click > Open View): KPI summary, churn rate by international plan and by the service-call flag, the churn-probability distribution and a table of at-risk customers. All views share one input table, so selecting bars or rows highlights the same customers everywhere. It is fed from section 4/5 only, so it runs without an API key. Once branch A has run, join the 'Retention email' column on the RowID (phone) to show the emails in the table.

5b · Threshold vs. business cost

6b / 6c · LLM on the evaluation

What: the tuned model's test-set probabilities P(Churn=1) are compared with 19 thresholds (0.05 to 0.95, step 0.05) in ONE vectorised pass, no loop: Cross Joiner (667 customers x 19 thresholds) -> Expression flags TP / FP / FN / TN -> GroupBy counts them per threshold -> Expression adds recall, precision, contacted customers and total cost = FN x cost_FN + (TP + FP) x cost_offer.

Cost assumptions (Table Creator 'Cost assumptions - EDIT HERE' -> flow variables cost_FN, cost_offer): missed churner (FN) = 500 EUR lost customer value; retention offer per contacted customer (TP + FP) = 50 EUR. Change them and re-execute this section.

Chosen threshold: the cheapest one; on a tie the HIGHEST threshold wins (same cost, fewer offers). It becomes the flow variable best_threshold (default costs: 0.45 -> 9,350 EUR vs. 9,800 EUR at 0.5), is applied to the predictions and scored: compare 'Confusion matrix (cost-optimal)' with 'Confusion matrix (tuned GBT)' in section 5. Line Plot = cost curve; Binary Classification Inspector = interactive slider for any threshold.

Caveat: for simplicity the threshold is chosen on the TEST set, so its cost is optimistic. In practice choose it on a separate validation set or on cross-validated (out-of-fold) predictions, and report the cost on untouched test data.

6b · LLM explains the metrics
6c · LLM advises on model choice

The two LLM Prompters are configured but NOT executed; everything else here is executed, so both prompts can be read without a key. Both reuse the existing OpenAI LLM Selector.

6b: pairwise AUC (every churner vs. every non-churner; share ranked correctly = AUC 0.936, identical to the ROC Curve) + the tuned Scorer's churn statistics and confusion matrix at 0.5 + top-5 features + best_threshold and costs from 5b -> ONE fact row -> 'Metrics prompt' -> LLM Prompter. The same row is transposed into the FACT TABLE (verify the LLM): check every number the LLM writes against it - anything not in the table was invented.

6c: the section-3 comparison text (4 models: accuracy, kappa, recall, precision, F1) + data facts (3,333 customers, 14.5% churn, 2,666 / 667 split) -> 'Model choice prompt' -> the LLM compares Random Forest vs. Gradient Boosted Trees, recommends one and names what to check next. Ask yourself: is a 1-2 point difference on 667 customers meaningful?

WHAT THIS SHOWS: two files are joined on Area Code + Phone: CallsData.xls (usage) and ContractData.csv (contract).

KEY NUMBERS / CLICK HERE:

  • Result: 3,333 customers, one row each; 14.5% churn.

  • Click: Joiner output → scroll to the Churn column.

WHAT THIS SHOWS: one Expression node creates the features, then a stratified split sets aside an unseen test set.

KEY NUMBERS / CLICK HERE:

  • Expression: Total Mins / Total Charge / Total Calls; Many Service Calls (4 or more); plans recoded to yes/no; Churn as a string.

  • Phone becomes the RowID: it is an identifier, not a feature. State and Area Code are dropped.

  • Table Partitioner: 80/20 stratified on Churn, seed 42 → 2,666 training / 667 test. Stratified = both parts keep 14.5% churn.

WHAT THIS SHOWS: Bayesian search tunes the Gradient Boosted Trees, scored by cross-validation on the training set only.

KEY NUMBERS / CLICK HERE:

  • Never run this live: it takes minutes.

  • Outer loop: Parameter Optimization Loop, Bayesian (TPE), 40 trials (20 warm-up), seed 42, early stopping. Ranges: trees 50–300, depth 2–8, learning rate 0.02–0.30.

  • Inner loop: 5-fold stratified cross-validation on the 2,666 training customers only. Objective: Cohen's kappa.

  • Best: 152 trees, depth 5, learning rate 0.12; CV kappa 0.908.

  • Flow variables nrModels / maxDepth (→ maxLevels) / learningRate go to the final learner via Table Row to Variable.

  • Click: Loop Start dialog → the learner's Flow Variables tab → Loop End outputs (best setting, all trials).

PRESENTER NOTES · 3 (← section box) ▶ LIVE 2

WHAT THIS SHOWS: four learners trained and tested on the same split, compared in one table.

KEY NUMBERS / CLICK HERE:

  • All four learners use the same split: top port = training, bottom port = test.

  • Test-set kappa: GBT 0.937, Decision Tree 0.930, Random Forest 0.917, Logistic Regression 0.149.

  • Logistic Regression has 0.850 accuracy, below the 0.855 "always no churn" baseline.

  • Random Forest in KNIME 5.12: bootstrap sampling must be on.

  • Click: comparison table → sort by kappa; GBT Learner dialog shows the hyperparameters.

WHAT THIS SHOWS: how well the tuned GBT does on the 667 unseen test customers, and which features drive it.

KEY NUMBERS / CLICK HERE:

  • Tuned GBT on 667 test customers: TN 570, FP 0, FN 11, TP 86.

  • Recall 88.7%, precision 100%, kappa 0.930, AUC 0.936.

  • Top features (Random Forest split statistics): Total Charge, Day Charge, Day Mins, Many Service Calls, Total Mins.

  • Click: Scorer view, ROC Curve view.

WHAT THIS SHOWS: which probability threshold gives the lowest business cost, instead of the default 0.5.

KEY NUMBERS / CLICK HERE:

  • Cost = missed churners × 500 EUR + contacted customers × 50 EUR. Edit the costs in the "Cost assumptions (EUR) – EDIT HERE" table.

  • The sweep covers 0.05–0.95. Best threshold 0.45: 9,350 EUR, versus 9,800 EUR at 0.5 (one more churner caught).

  • 10 churners score below 0.05, so no threshold reaches them.

  • Caveat: the threshold is chosen on the test set for simplicity.

  • Click: cost table → Line Plot → Binary Classification Inspector slider.

WHAT THIS SHOWS: the LLM turns numbers that KNIME has already computed into explanations, advice and text.

KEY NUMBERS / CLICK HERE:

  • Not executed: it needs an API key in Credentials Configuration, or a local Ollama model (authenticator base URL http://localhost:11434/v1). Never type a key on screen.

  • Four branches: LLM explains the metrics (FACT TABLE → prompt → LLM Prompter), LLM advises on model choice, retention emails for the top 10 customers, management summary.

  • Rule: KNIME computes, the LLM explains, people check every number against "FACT TABLE (verify the LLM)".

PRESENTER NOTES · 7 (section box below ↓)

WHAT THIS SHOWS: the results as an interactive data app for the business.

KEY NUMBERS / CLICK HERE:

  • Open the component's interactive view.

  • KPI tiles: 667 test customers, 97 churned (14.5%), 86 flagged, 86 of 97 caught (88.7% recall).

  • Churn rate by Int'l Plan: 37% vs 12%. By 4+ service calls: 56% vs 11%.

  • Probability distribution: churners (red) cluster at 90–100%. Ranked table below; clicks are linked across views.

Score test set
Scorer
Score test set
Scorer
Score test set
Scorer
Add model name+ metrics
Expression
Add model name+ metrics
Expression
Score test set
Scorer
Add model name+ metrics
Expression
Stack results
Concatenate
Stack results
Concatenate
Add model name+ metrics
Expression
Stack results
Concatenate
Keep Churn +P (Churn=1)
Column Filter
Thresholds0.05 ... 0.95
Table Creator
Interactive data app(right-click > Open view)
Churn Dashboard
Keep metriccolumns
Column Filter
Count perthreshold
GroupBy
MODEL COMPARISONby kappa: GBT 0.937
Sorter
Recall, precision,total cost
Expression
Every customer xevery threshold
Cross Joiner
Keep churn=1 row
Expression Row Filter
Flag TP / FP /FN / TN
Expression
Predictheld-out fold
Gradient Boosted Trees Predictor
Cost vs.threshold
Line Plot
GBT with trialparameters
Gradient Boosted Trees Learner
Bayesian TPE, 40 trials(20 warm-up), seed 42
Parameter Optimization Loop Start
Cost assumptions(EUR) - EDIT HERE
Table Creator
5-foldstratified CV
X-Partitioner
cost_FN, cost_offeras variables
Table Row to Variable
best: 152 / 5 / 0.12CV kappa 0.908
Parameter Optimization Loop End
Collect foldpredictions
X-Aggregator
CV score(Cohen's kappa)
Scorer
Read call usage(CallsData.xls)
Excel Reader
Read contracts(ContractData.csv)
CSV Reader
Area Code + Phone-> 3,333 customers
Joiner
Engineer features(K-AI generated)
Expression
Phone -> RowID(ID, not a feature)
RowID
Drop State andArea Code
Column Filter
Palette: churn 1 = red,0 = blue
Color Palette Designer
Prefixbest_
Column Renamer
best_thresholdas variable
Table Row to Variable
Cost asc,threshold desc
Sorter
Cheapest: 0.45= 9,350 EUR
Expression Row Filter
Retrain tuned GBT(see Flow Variables tab)
Gradient Boosted Trees Learner
Threshold slider(open the view)
Binary Classification Inspector
TN 570 FP 0 FN 11 TP 86(tuned GBT)
Scorer
Churners vs.non-churners
Row Splitter
nrModels / maxDepth /learningRate -> variables
Table Row to Variable
Applybest_threshold
Expression
Predict test set(+ probabilities)
Gradient Boosted Trees Predictor
Confusion matrix(cost-optimal)
Scorer
FEATURE RANKINGtop: Total Charge
Sorter
Row0 -> Value
Column Renamer
OpenAI API key(enter your own)
Credentials Configuration
AUC 0.936(churn = 1)
ROC Curve
All pairs(97 x 570)
Cross Joiner
LLM explainsthe metrics
LLM Prompter
Importance scorefrom RF splits
Expression
Pair rankedcorrectly?
Expression
Metrics prompt(one row)
Expression
Model choiceprompt
Expression
FACT TABLE(verify the LLM)
Column Resorter
Authenticatewith OpenAI
OpenAI Authenticator
Select model(gpt-4.1-mini)
OpenAI LLM Selector
LLM advises onmodel choice
LLM Prompter
AUC = mean
Row Aggregator
+ churn stats(threshold 0.5)
Cross Joiner
A: top 10 at-riskcustomers
Top k Row Filter
+ AUC
Cross Joiner
Top 5 featuresas text
Expression
Keep row 1
Expression Row Filter
B: comparisontable -> text
Expression
RowID = Row0
RowID
top = train 2,666bottom = test 667
Table Partitioner
Keep first row
Expression Row Filter
One factper row
Table Transposer
Build emailprompt per row
Expression
Facts(one row)
Expression
Write retentionemails
LLM Prompter
Fact columnsonly
Column Filter
Color rowsby Churn
Color Designer (Apply)
Train GBT(dialog: hyperparameters)
Gradient Boosted Trees Learner
Combine texts
Cross Joiner
Profile the data(open the view)
Data Explorer
Train LogisticRegression
Logistic Regression Learner
Build managementprompt
Expression
TrainDecision Tree
Decision Tree Learner
B: keep churn=1statistics
Expression Row Filter
Fact namecolumn
RowID
Train Random Forest(bootstrap ON)
Random Forest Learner
Tuned model-> text
Expression
Predict test set
Gradient Boosted Trees Predictor
Predict test set
Logistic Regression Predictor
Predict test set
Decision Tree Predictor
Write managementsummary
LLM Prompter
Predict test set
Random Forest Predictor

Nodes

Extensions

Links