Icon

Conformal predictive systems (advanced)

This demo describes how to enrich the regression workflow with conformal prediction methods that allows to estimate model prediction certainity and control the desired error rate. The workflow is also implemented with Integrated deployment extension, so it automatically creates production code. Conformal prediction is agnostic to prediction algorithm, so users can easily replace it with any other classification algorithm.



Beta optimization Error rate optimization Train models and get calibration tables Deploy conformal prediction Train modelGet predictionsfor calibrationdata settraining + calibrationtesttrainingcalibrationGather models andcalibration tablespairsSerializing modelDeserializing modelPair-wise iterationover models and calibration tablesAggregating P-values(median)Get predictionsEstimate predictionsSigma - absolute errorSigma - absolute errorRead dataCapture predictionpartDeploy the workflowGather optimization resultsAssign currentintervalIterate over producer and error rateCreate dynamiccolumn nameAssign new RowIDsRun deployedworkflowRead deployed workflowSelect producerVisualize producersand models;filter correlated featuresCreate new IDsAssign new RowIDsCreate new IDsRun deployedworkflowOptimize betavaluesRead deployed workflowGather optimization resultsIterate over producer and betaAssign currentbeta value Create dynamiccolumn nameSerializing modelTrain modelGet predictionsfor calibrationdata setSigma - absolute errortrainingcalibrationGather models andcalibration tablespairsUpdate tablespecReduce data setsizeEstimate predictionsEstimate predictionsAssign currentbeta value Normalize features andtarget variableSelect producerGet calibrationdata setNode 906CalculateCDFAssign percentileintervalsOptimizesiginicance levelCreating significancelevel values foroptimizationSet up globalparametersfor conformal predictionSet up globalparametersfor conformal predictionAssign currentintervalUpperpercentileLowerpercentileRandom sample is shownSet up globalparametersfor conformal predictionRandom Forest Learner(Regression) Random Forest Predictor(Regression) Partitioning Conformal CalibrationLoop Start Conformal CalibrationLoop End Model to Cell Cell To Model Conformal PredictionLoop Start ConformalPrediction Loop End Random Forest Predictor(Regression) Conformal Scorer(Regression) Math Formula Math Formula Table Reader CaptureWorkflow Start Workflow Writer Loop End ConstantValue Column Group Loop Start Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) RowID Workflow Executor Workflow Reader Select producer Data visualization Column Aggregator RowID Column Aggregator Workflow Executor Parameter OptimizationLoop Start Workflow Reader Loop End Group Loop Start ConstantValue Column Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) Model to Cell Random Forest Learner(Regression) Random Forest Predictor(Regression) Math Formula Conformal CalibrationLoop Start Conformal CalibrationLoop End CaptureWorkflow End Domain Calculator Row Sampling Conformal Scorer(Regression) Conformal Scorer(Regression) ConstantValue Column Normalizer Select producer Error rateoptimization analysis Predictive SystemsCalibrator (Regression) Predictive SystemsCalibrator (Regression) Predictive SystemsPredictor (Regression) Predictive SystemsClassifier (Regression) Table Row ToVariable Loop Start Table Creator Conformal predictionconfiguration Define intervals Conformal predictionconfiguration Define intervals Define intervals ConstantValue Column Column Rename(Regex) Column Rename(Regex) Beta optimizationresults Significance leveloptimization results CDF and intervalsexplanation Conformal predictionconfiguration Beta optimization Error rate optimization Train models and get calibration tables Deploy conformal prediction Train modelGet predictionsfor calibrationdata settraining + calibrationtesttrainingcalibrationGather models andcalibration tablespairsSerializing modelDeserializing modelPair-wise iterationover models and calibration tablesAggregating P-values(median)Get predictionsEstimate predictionsSigma - absolute errorSigma - absolute errorRead dataCapture predictionpartDeploy the workflowGather optimization resultsAssign currentintervalIterate over producer and error rateCreate dynamiccolumn nameAssign new RowIDsRun deployedworkflowRead deployed workflowSelect producerVisualize producersand models;filter correlated featuresCreate new IDsAssign new RowIDsCreate new IDsRun deployedworkflowOptimize betavaluesRead deployed workflowGather optimization resultsIterate over producer and betaAssign currentbeta value Create dynamiccolumn nameSerializing modelTrain modelGet predictionsfor calibrationdata setSigma - absolute errortrainingcalibrationGather models andcalibration tablespairsUpdate tablespecReduce data setsizeEstimate predictionsEstimate predictionsAssign currentbeta value Normalize features andtarget variableSelect producerGet calibrationdata setNode 906CalculateCDFAssign percentileintervalsOptimizesiginicance levelCreating significancelevel values foroptimizationSet up globalparametersfor conformal predictionSet up globalparametersfor conformal predictionAssign currentintervalUpperpercentileLowerpercentileRandom sample is shownSet up globalparametersfor conformal predictionRandom Forest Learner(Regression) Random Forest Predictor(Regression) Partitioning Conformal CalibrationLoop Start Conformal CalibrationLoop End Model to Cell Cell To Model Conformal PredictionLoop Start ConformalPrediction Loop End Random Forest Predictor(Regression) Conformal Scorer(Regression) Math Formula Math Formula Table Reader CaptureWorkflow Start Workflow Writer Loop End ConstantValue Column Group Loop Start Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) RowID Workflow Executor Workflow Reader Select producer Data visualization Column Aggregator RowID Column Aggregator Workflow Executor Parameter OptimizationLoop Start Workflow Reader Loop End Group Loop Start ConstantValue Column Loop End (ColumnAppend) Column Filter Column Rename(Regex) String Manipulation(Variable) Model to Cell Random Forest Learner(Regression) Random Forest Predictor(Regression) Math Formula Conformal CalibrationLoop Start Conformal CalibrationLoop End CaptureWorkflow End Domain Calculator Row Sampling Conformal Scorer(Regression) Conformal Scorer(Regression) ConstantValue Column Normalizer Select producer Error rateoptimization analysis Predictive SystemsCalibrator (Regression) Predictive SystemsCalibrator (Regression) Predictive SystemsPredictor (Regression) Predictive SystemsClassifier (Regression) Table Row ToVariable Loop Start Table Creator Conformal predictionconfiguration Define intervals Conformal predictionconfiguration Define intervals Define intervals ConstantValue Column Column Rename(Regex) Column Rename(Regex) Beta optimizationresults Significance leveloptimization results CDF and intervalsexplanation Conformal predictionconfiguration

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: artem
Created at: 2023-02-26
On NodePit since: 2023-11-14
Last update: 2024-04-27
Created with KNIME version: v4.7.6
Tags: Conformal predictionRegressionIntegrated deploymentMachine learningPrediction analysis

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!