Icon

05_​Cohen's_​Kappa_​for_​Evaluating_​Classification_​Models

There has been no title set for this workflow's metadata.

This workflow demonstrates how Cohen's kappa can be used to evaluate the performance of a classification model when dealing with imbalanced data. We also show how Cohen's kappa obtains greater values not only due to better model performance, but also because of a more balanced target class distribution.

URL: Cohen's Kappa: Learn It, Use It, Judge It https://www.knime.com/blog/cohens-kappa-an-overview
URL: Scoring Metrics eBook https://www.knime.com/knimepress/scoring-metrics-evaluating-machine-learning-models

Train two models with unbalanced data: One with and the other without resampling the training set.

Case 1: Model training without resampling

Case 2: Model training with resampling

Apply the model to test data with different distributions of the target class. Collect their Cohen's kappa values and plot them in a line plot.

DecTree.pmml
PMML Reader
col_mean
Math Formula
Cohen's kappa
Variable To Table Row
Scorer
Concatenate
Class proportions
Collect proportion of bad and Cohen's kappa
Variable Loop End
Table Row To Variable
70/30 on score
Partitioning
Varying class proportions
Counting Loop Start
Read GermanCredit.csv
CSV Reader
Loop End
Create 100 test sets with the same class distribution
Counting Loop Start
Decision Tree Predictor
Decision Tree Predictor
Math Formula
Decision Tree Learner
Positive classprobability vs.Cohen's kappa
Line Plot
Nominal Value Row Splitter
good
Bootstrap Sampling
good: 900
Bootstrap Sampling
bad: 100
Bootstrap Sampling
bad
Bootstrap Sampling
Concatenate
Target column: remove String: unknown
Missing Value
Nominal Value Row Splitter
Target column: remove String: unknown
Missing Value
customer rating
Rule Engine
Decision Tree Learner
test.table
Table Writer
Decision Tree Predictor
Missing Value (Apply)
Read test.table
Table Reader
DecTree.pmml
PMML Writer
Missing Value (Apply)
Scorer (JavaScript)
SMOTE
Target column: remove String: unknown
Missing Value
Scorer (JavaScript)

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Maarit Laukkanen
Created at: 2020-05-12
On NodePit since: 2026-01-05
Last update: 2026-11-08
Created with KNIME version: v5.1.0
Tags: model evaluationresamplingCohen's kappaoverall accuracybootstrapclassificationimbalanced data

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!