Icon

Monitoring

There has been no title set for this workflow's metadata.

Monitoring workflow. This workflow monitors the performance of the model based on different metrics. For the data where the ground truth is available, it monitors the performance. For the data where the ground truth is not available, it monitors the percentage of predictions for an applicant to be risky. The retraining will be triggered by the CDDS automatically if the retraining decision exported by this workflow is positive.

Session 4 - Best Practices when Productionizing Data ScienceExercise 01 CDDS Continuous deployment Learning objective: In this part of the exercise you need to finalize the monitoring workflow.Workflows description: Monitoring workflow. This workflow monitors the performance of the model based on different metrics. For the data where the ground truth is available, it monitors the performance. For thedata where the ground truth is not available, it monitors the percentage of predictions for an applicant to be risky. The retraining will be triggered by the CDDS automatically if the retraining decision exported by thisworkflow is positive.You'll find the instructions to the exercises in the yellow annotations. Step 2. Monitor the performanceOpen the metanode andcomplete the performancemonitoring Step 1. Access the training data from the databaseConnect to the databaseProvide the hostname and database name shared bythe trainers as well as your username (userXX),password (userXX), and schema (userXX)predictions and customer_data tables will be read fromthe database Step 3. Monitor the predictionsdistributionOpen the metanode andcomplete the distributionmonitoring Step 4. Export the retraining decision to CDDSConfigure the component Configure Monitoring Output. Thecomponent is provided together with the CDDS and is available in theCDDS Administration space under 00_Components/MonitoringSelect Retraining column as a column with the retraining decisionSelect log_final as a column with the log messageExport the JSON column with the retraining decision with theWorkflow Service Output node. The node doesn't require configurationReset the workflow predictionsProvide the DBparameters, youruser & passwordand schemacustomer_dataAdd ground truthwhere avaiablewith ground truthwithout ground truth0: refernece data(240-120 days ago)1: new data(120 days ago) ConfigureMonitoring Output Predictionsdistribution WorkflowService Output AUC percentagechange DB Table Selector PostgreSQLConnector DB Table Selector DB Joiner DB Row Filter DB Row Filter DB Reader Select referenceand new data Retraining decision Session 4 - Best Practices when Productionizing Data ScienceExercise 01 CDDS Continuous deployment Learning objective: In this part of the exercise you need to finalize the monitoring workflow.Workflows description: Monitoring workflow. This workflow monitors the performance of the model based on different metrics. For the data where the ground truth is available, it monitors the performance. For thedata where the ground truth is not available, it monitors the percentage of predictions for an applicant to be risky. The retraining will be triggered by the CDDS automatically if the retraining decision exported by thisworkflow is positive.You'll find the instructions to the exercises in the yellow annotations. Step 2. Monitor the performanceOpen the metanode andcomplete the performancemonitoring Step 1. Access the training data from the databaseConnect to the databaseProvide the hostname and database name shared bythe trainers as well as your username (userXX),password (userXX), and schema (userXX)predictions and customer_data tables will be read fromthe database Step 3. Monitor the predictionsdistributionOpen the metanode andcomplete the distributionmonitoring Step 4. Export the retraining decision to CDDSConfigure the component Configure Monitoring Output. Thecomponent is provided together with the CDDS and is available in theCDDS Administration space under 00_Components/MonitoringSelect Retraining column as a column with the retraining decisionSelect log_final as a column with the log messageExport the JSON column with the retraining decision with theWorkflow Service Output node. The node doesn't require configurationReset the workflow predictionsProvide the DBparameters, youruser & passwordand schemacustomer_dataAdd ground truthwhere avaiablewith ground truthwithout ground truth0: refernece data(240-120 days ago)1: new data(120 days ago)ConfigureMonitoring Output Predictionsdistribution WorkflowService Output AUC percentagechange DB Table Selector PostgreSQLConnector DB Table Selector DB Joiner DB Row Filter DB Row Filter DB Reader Select referenceand new data Retraining decision

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Lada.Rudnitckaia
Created at: 2022-10-07
On NodePit since: 2023-10-05
Last update: 2024-04-27
Created with KNIME version: v5.1.0
Tags: EducationIntegrated DeploymentMachine LearningModel DeploymentCredit ScoringKNIME Business HubRESTful ServiceServiceModel as a ServiceService DeploymentCDDSContinuous DeploymentModel MonitoringModel RetrainingMLOpsAutomated Retraining

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!