Icon

Conformal prediction regression (advanced)

This demo describes how to enrich the regression workflow with conformal prediction methods that allows to estimate model prediction certainity and control the desired error rate. The workflow is also implemented with Integrated deployment extension, so it automatically creates production code. Conformal prediction is agnostic to prediction algorithm, so users can easily replace it with any other classification algorithm.

Beta optimization Error rate optimization Train models and get calibration tables Deploy conformal prediction Train modelGet predictionsfor calibrationdata settraining + calibrationtesttrainingcalibrationGather models andcalibration tablespairsSerializing modelDeserializing modelPair-wise iterationover models and calibration tablesAggregating P-values(median)Get predictionsGet calibrationdata setGet conformalpredictionEstimate predictionsSigma - absolute errorSigma - absolute errorRead dataCapture predictionpartDeploy the workflowSet up globalparametersfor conformal predictionOptimizeerror rateSet onlybeta herethe same value as wasused during trainingor go without normalizationGather optimization resultsAssign currenterror rateAssign currenterror rateVisualize predictionsfor selected producerIterate over producer and error rateCreate dynamiccolumn nameAssign new RowIDsRun deployedworkflowRead deployed workflowSelect producerVisualize producersand modelsCreate new IDsAssign new RowIDsCreate new IDsRun deployedworkflowOptimize betavaluesRead deployed workflowGather optimization resultsVisualize predictionsfor selected producerIterate over producer and betaAssign currentbeta value Create dynamiccolumn nameSerializing modelTrain modelGet predictionsfor calibrationdata setGet calibrationdata setSigma - absolute errorSet onlyerror rate heretrainingcalibrationGather models andcalibration tablespairsUpdate tablespecReduce data setsizeEstimate predictionsEstimate predictionsAssign currentbeta value Normalize features andtarget variableSelect producerinterval sizesvsbetaRandom Forest Learner(Regression) Random Forest Predictor(Regression) Partitioning Conformal CalibrationLoop Start Conformal CalibrationLoop End Model to Cell Cell To Model Conformal PredictionLoop Start ConformalPrediction Loop End Random Forest Predictor(Regression) Conformal Calibrator(Regression) Conformal Predictor andClassifier (Regression) Conformal Scorer(Regression) Math Formula Math Formula Table Reader CaptureWorkflow Start Workflow Writer Conformal predictionconfiguration Parameter OptimizationLoop Start Conformal predictionconfiguration Loop End ConstantValue Column ConstantValue Column Line Plot (Plotly) Group Loop Start Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) RowID Workflow Executor Workflow Reader Select producer Data visualization Column Aggregator RowID Column Aggregator Workflow Executor Parameter OptimizationLoop Start Workflow Reader Loop End Line Plot (Plotly) Group Loop Start ConstantValue Column Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) Model to Cell Random Forest Learner(Regression) Random Forest Predictor(Regression) Conformal Calibrator(Regression) Math Formula Conformal predictionconfiguration Conformal CalibrationLoop Start Conformal CalibrationLoop End CaptureWorkflow End Domain Calculator Row Sampling Conformal Scorer(Regression) Conformal Scorer(Regression) ConstantValue Column Normalizer Select producer Line Plot (Plotly) Error rateoptimization analysis Beta optimization Error rate optimization Train models and get calibration tables Deploy conformal prediction Train modelGet predictionsfor calibrationdata settraining + calibrationtesttrainingcalibrationGather models andcalibration tablespairsSerializing modelDeserializing modelPair-wise iterationover models and calibration tablesAggregating P-values(median)Get predictionsGet calibrationdata setGet conformalpredictionEstimate predictionsSigma - absolute errorSigma - absolute errorRead dataCapture predictionpartDeploy the workflowSet up globalparametersfor conformal predictionOptimizeerror rateSet onlybeta herethe same value as wasused during trainingor go without normalizationGather optimization resultsAssign currenterror rateAssign currenterror rateVisualize predictionsfor selected producerIterate over producer and error rateCreate dynamiccolumn nameAssign new RowIDsRun deployedworkflowRead deployed workflowSelect producerVisualize producersand modelsCreate new IDsAssign new RowIDsCreate new IDsRun deployedworkflowOptimize betavaluesRead deployed workflowGather optimization resultsVisualize predictionsfor selected producerIterate over producer and betaAssign currentbeta value Create dynamiccolumn nameSerializing modelTrain modelGet predictionsfor calibrationdata setGet calibrationdata setSigma - absolute errorSet onlyerror rate heretrainingcalibrationGather models andcalibration tablespairsUpdate tablespecReduce data setsizeEstimate predictionsEstimate predictionsAssign currentbeta value Normalize features andtarget variableSelect producerinterval sizesvsbetaRandom Forest Learner(Regression) Random Forest Predictor(Regression) Partitioning Conformal CalibrationLoop Start Conformal CalibrationLoop End Model to Cell Cell To Model Conformal PredictionLoop Start ConformalPrediction Loop End Random Forest Predictor(Regression) Conformal Calibrator(Regression) Conformal Predictor andClassifier (Regression) Conformal Scorer(Regression) Math Formula Math Formula Table Reader CaptureWorkflow Start Workflow Writer Conformal predictionconfiguration Parameter OptimizationLoop Start Conformal predictionconfiguration Loop End ConstantValue Column ConstantValue Column Line Plot (Plotly) Group Loop Start Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) RowID Workflow Executor Workflow Reader Select producer Data visualization Column Aggregator RowID Column Aggregator Workflow Executor Parameter OptimizationLoop Start Workflow Reader Loop End Line Plot (Plotly) Group Loop Start ConstantValue Column Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) Model to Cell Random Forest Learner(Regression) Random Forest Predictor(Regression) Conformal Calibrator(Regression) Math Formula Conformal predictionconfiguration Conformal CalibrationLoop Start Conformal CalibrationLoop End CaptureWorkflow End Domain Calculator Row Sampling Conformal Scorer(Regression) Conformal Scorer(Regression) ConstantValue Column Normalizer Select producer Line Plot (Plotly) Error rateoptimization analysis

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: artem
Created at: 2023-02-26
On NodePit since: 2023-03-27
Last update: 2024-04-18
Created with KNIME version: v4.6.4
Tags: Conformal predictionRegressionIntegrated deploymentMachine learningPrediction analysis

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!