Icon

Conformal prediction classification

This demo describes how to enrich the classification workflow with conformal prediction methods that allows to estimate model prediction certainity and control the desired error rate. The workflow is also implemented with Integrated deployment extension, so it automatically creates production code. Conformal prediction is agnostic to prediction algorithm, so users can easily replace it with any other classification algorithm.

"Advanced" conformal prediction classification "Simple" conformal prediction classification Optimizing error rate Training + calibrationTestTrain modelGet predictionsfor calibration tableCreate several modelsand calibration tablesGather modelsand calibration tablesExtract modelfrom tablePut model intotableSync modelsandcalibration tablesAggregate p-valuesby Median functionApply model totest dataCalibrate predictionsand get rankCalculate p-valuesMake predictionbased on desirederror rateEstimate predictionsTraining + calibrationTrain modelMake predictionbased on desirederror rateGet predictionsfor calibration tableApply model totest dataEstimate predictionsCaptureconformal predictionDefine portsSave and deployworkflowTest runCaptureconformal predictionDefine portsRead deployedworkflowCombine capturedworkflow partsSave and deployworkflowCombine capturedworkflow partsCapture processingDefine portsTest runDeployment runProvide desirederror rateRead deployedworkflowDeployment runper rice typeAdderror rateGet onlytotalEfficiency / ValidityNormalizing dataReadrice dataTraininig/calibration/testcontrolNormalizingfeaturesAdderror rateOptimizeerror rateGather resultsFeaturescorrelation analysisRemovehighly correlatedfeatures Partitioning Random ForestLearner Random ForestPredictor Conformal CalibrationLoop Start Conformal CalibrationLoop End Cell To Model Model to Cell Conformal PredictionLoop Start ConformalPrediction Loop End Random ForestPredictor ConformalCalibrator Conformal Predictor ConformalClassifier Conformal Scorer ConformalPartitioning Random ForestLearner ConformalClassification Random ForestPredictor Random ForestPredictor Conformal Scorer CaptureWorkflow Start CaptureWorkflow End Workflow Writer Workflow Executor CaptureWorkflow Start CaptureWorkflow End Workflow Reader Workflow Combiner Workflow Writer Workflow Combiner CaptureWorkflow Start CaptureWorkflow End Workflow Executor Workflow Executor Error rate Workflow Reader Workflow Executor Bar Chart ConstantValue Column Row Filter Line Plot (Plotly) Normalizer Table Reader Partitioning Normalizer (Apply) ConstantValue Column Parameter OptimizationLoop Start Loop End Linear Correlation Correlation Filter "Advanced" conformal prediction classification "Simple" conformal prediction classification Optimizing error rate Training + calibrationTestTrain modelGet predictionsfor calibration tableCreate several modelsand calibration tablesGather modelsand calibration tablesExtract modelfrom tablePut model intotableSync modelsandcalibration tablesAggregate p-valuesby Median functionApply model totest dataCalibrate predictionsand get rankCalculate p-valuesMake predictionbased on desirederror rateEstimate predictionsTraining + calibrationTrain modelMake predictionbased on desirederror rateGet predictionsfor calibration tableApply model totest dataEstimate predictionsCaptureconformal predictionDefine portsSave and deployworkflowTest runCaptureconformal predictionDefine portsRead deployedworkflowCombine capturedworkflow partsSave and deployworkflowCombine capturedworkflow partsCapture processingDefine portsTest runDeployment runProvide desirederror rateRead deployedworkflowDeployment runper rice typeAdderror rateGet onlytotalEfficiency / ValidityNormalizing dataReadrice dataTraininig/calibration/testcontrolNormalizingfeaturesAdderror rateOptimizeerror rateGather resultsFeaturescorrelation analysisRemovehighly correlatedfeatures Partitioning Random ForestLearner Random ForestPredictor Conformal CalibrationLoop Start Conformal CalibrationLoop End Cell To Model Model to Cell Conformal PredictionLoop Start ConformalPrediction Loop End Random ForestPredictor ConformalCalibrator Conformal Predictor ConformalClassifier Conformal Scorer ConformalPartitioning Random ForestLearner ConformalClassification Random ForestPredictor Random ForestPredictor Conformal Scorer CaptureWorkflow Start CaptureWorkflow End Workflow Writer Workflow Executor CaptureWorkflow Start CaptureWorkflow End Workflow Reader Workflow Combiner Workflow Writer Workflow Combiner CaptureWorkflow Start CaptureWorkflow End Workflow Executor Workflow Executor Error rate Workflow Reader Workflow Executor Bar Chart ConstantValue Column Row Filter Line Plot (Plotly) Normalizer Table Reader Partitioning Normalizer (Apply) ConstantValue Column Parameter OptimizationLoop Start Loop End Linear Correlation Correlation Filter

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: artem
Created at: 2022-11-19
On NodePit since: 2023-02-21
Last update: 2024-04-24
Created with KNIME version: v4.6.4
Tags: Conformal predictionClassificationIntegrated deploymentMachine learningPrediction analysis

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!