Classification Model Evaluation using Giskard

<p>This workflow uses the <strong>Giskard Scanner node</strong> to scan a machine-learning workflow for common weaknesses.</p><p>The idea of the Giskard Scanner node is to allow <strong>fine-tuning of your ML model while developing</strong> it in KNIME Analytics Platform. Hence, the example mimics a data scientist's different attempts and modifications to their models and workflows before putting them into production.</p><p>Additionally, it is also possible to use Giskard after the model is deployed in production. Workflow monitoring using Giskard allows the data scientist to be aware of issues with production data flowing into the model, such as data shift phenomena.</p><p>You can download and run the workflow directly in your KNIME Analytics Platform. We recommend using the latest version of KNIME AP for optimal performance.</p><p>The goal of the model is to predict the grade class of a group of students based on different features.<br><br>Workflow Steps</p><ol><li><p><strong>Reading the Data</strong></p><ul><li><p>The data is embedded into the workflow, so you can just execute the node.</p></li></ul></li><li><p><strong>Partitioning into Train and Test</strong></p><ul><li><p>The partitioning node splits the data into 70% training data and 30% test data, which is used later to evaluate the model with Giskard.</p></li></ul></li><li><p><strong>Various Fine-Tuning ML Models Attempts</strong></p><ul><li><p>The workflow represents various moments of fine-tuning ML models to make clear how to use Giskard while developing your ML workflow in KNIME.</p></li></ul></li></ol><p>Attempt 1</p><ol><li><p><strong>Model Training</strong></p><ul><li><p>Train a Random Forest classification model on the dataset using GradeClass as the target column.</p></li></ul></li><li><p><strong>Capture Inference Workflow</strong></p><ul><li><p>The inference workflow is surrounded by the Capture Workflow Start and End nodes that extract the contained workflow snippet into a dedicated workflow port object.</p></li><li><p>The inference workflow uses the model predictor to create the classification output. Besides the predicted class, the predictor also outputs the class probabilities, which are used by Giskard.</p></li></ul></li><li><p><strong>Evaluate Inference Workflow with Giskard</strong></p><ul><li><p>The Giskard Scanner node checks the inference workflow for common weaknesses in robustness, spurious correlation, performance, and overconfidence categories. Please refer to the workflow annotations for further details.</p></li></ul></li></ol><p>Attempt 2</p><ol><li><p><strong>Model Training</strong></p><ul><li><p>Train a Random Forest classification model on the dataset using GradeClass as the target column. Based on previous learnings, remove Absence and Include GPA variables.</p></li></ul></li><li><p><strong>Capture Inference Workflow</strong></p><ul><li><p>The same as in attempt 1.</p></li></ul></li><li><p><strong>Evaluate Inference Workflow with Giskard</strong></p><ul><li><p>Please refer to the workflow annotations for further details.</p></li></ul></li></ol><p>Final Attempt</p><ol><li><p><strong>Model Training</strong></p><ul><li><p>Train a Random Forest classification model on the dataset using GradeClass as the target column. Based on previous learnings, remove the GPA variable.</p></li></ul></li><li><p><strong>Capture Inference Workflow</strong></p><ul><li><p>The captured workflow part includes some <strong>preprocessing operations</strong>, binning, and outliers handling. The inference workflow applies the preprocessing operations followed by the model predictor to create the classification output. In addition to the predicted class, the predictor also outputs the class probabilities, which are used by Giskard.</p></li></ul></li><li><p><strong>Evaluate Inference Workflow with Giskard</strong></p><ul><li><p>Please refer to the workflow annotations for further details.</p></li></ul></li></ol><p>This workflow demonstrates how to fine-tune and evaluate ML models using the Giskard Scanner node in KNIME, ensuring robust and reliable models before deployment.</p>