Icon

Cross Validation example

Cross validation example

This example shows how to calculate non-standard evaluation metrics on each fold and then estimate fluctuation of the performance evaluation.

Read the data Basic cross validation example. The last X-Aggregator contains the out-of-fold predictions for eachinput row on the top output port and some evaluation metric (error rate for classification) for each ofthe folds on the bottom port Evaluate further metrics on the fullout-of-fold prediction. This estimateis more robust than individualestimates on a single fold, but doesnot give an estimate of variance The data is a local copy of the data used in user training and available inhttps://api.hub.knime.com/repository//Users/knime/Education/01%20KNIME%20User%20Training/data/CurrentDetailData.table:data Calculate evaluation metrics on each fold. Select a subset (precision, recall, F_1 and accuracy in this example) and thenuse Groupby to calculate the mean and standard deviation for some of those.We see that we can estimate the model to have Recall = 0.802 +- 0.012. Evaluate onwhole out-of-foldpredictionsIterate over foldsGet metricsfor the first classSubset ofmetrics to trackCollect resultsfrom each foldMean and STDof precision and recall Random ForestLearner Table Reader Random ForestPredictor X-Partitioner X-Aggregator Scorer Scorer Group Loop Start Table Rowto Variable Variable toTable Row Loop End GroupBy Read the data Basic cross validation example. The last X-Aggregator contains the out-of-fold predictions for eachinput row on the top output port and some evaluation metric (error rate for classification) for each ofthe folds on the bottom port Evaluate further metrics on the fullout-of-fold prediction. This estimateis more robust than individualestimates on a single fold, but doesnot give an estimate of variance The data is a local copy of the data used in user training and available inhttps://api.hub.knime.com/repository//Users/knime/Education/01%20KNIME%20User%20Training/data/CurrentDetailData.table:data Calculate evaluation metrics on each fold. Select a subset (precision, recall, F_1 and accuracy in this example) and thenuse Groupby to calculate the mean and standard deviation for some of those.We see that we can estimate the model to have Recall = 0.802 +- 0.012. Evaluate onwhole out-of-foldpredictionsIterate over foldsGet metricsfor the first classSubset ofmetrics to trackCollect resultsfrom each foldMean and STDof precision and recallRandom ForestLearner Table Reader Random ForestPredictor X-Partitioner X-Aggregator Scorer Scorer Group Loop Start Table Rowto Variable Variable toTable Row Loop End GroupBy

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: knime
Created at: 2014-06-17
On NodePit since: 2023-02-13
Last update: 2024-11-21
Created with KNIME version: v4.0.1
Tags: tree ensembleclassificationcross validationintermediate

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!