ML Python 200 - Impute Categorical and Numeric with Mice LightGBM

Two-Step MICE Imputation Workflow in KNIME with miceforest

This process trains a multiple imputation model on an initial dataset using miceforest (Step 1) and then applies the trained model to new data (Step 2).

The training step prepares the schema, fits the LightGBM-based MICE kernel, and saves both the model and metadata.

The application step ensures new data exactly matches the training schema before imputing missing values, guaranteeing consistent and reliable imputations across datasets.

Handling “Missing Data” Like a Pro — Part 3: Model-Based & Multiple Imputation Methods

https://medium.com/p/bdfe85f93087

Training & Saving the MICE Imputer

Purpose:
This script runs inside a KNIME Python node to prepare data, train a miceforest multiple-imputation model, and export both the trained kernel and key metadata for later use.

Key steps:

• Reads KNIME’s input table and identifies:

  • Features to impute

  • Categorical vs. numeric columns

  • Column category levels

• Converts categoricals to pd.Categorical and stores their levels

• Trains a miceforest.ImputationKernel with LightGBM backend

• Optionally completes the training data’s missing values

• Outputs:

  • Output Table 0: Imputed training data

  • Output Object 0: The trained MICE kernel

  • Output Object 1: Metadata dictionary (schema info, categories, etc.)

Applying the Trained MICE Imputer to New Data

Purpose:
This script takes new incoming data in KNIME and applies the trained MICE imputer from step 1, making sure the new data’s schema matches the training schema exactly before imputation.

Key steps:

• Loads the trained kernel and metadata from KNIME object ports

• Reads the new KNIME input table

• Aligns columns to the exact order, names, dtypes, and category levels used during training

• Coerces column dtypes (categorical/numeric/etc.) to match training exactly

• Runs impute_new_data() and completes the imputation

• Reattaches untouched “rest” columns

• Outputs:

  • Output Table 0: The fully imputed new dataset

MEDIUM BLOG

Data preparation for Machine Learning with KNIME and the Python “vtreat” package

https://medium.com/lp/efcaf58fa783

Download the whole workflow group "Missing Value Imputation":

https://hub.knime.com/s/smId4VcHVVVL5D_t