Icon

Feature Engineering with GenAI for Classification - Technical Research

<p><strong>Feature Engineering with GenAI for Classification</strong></p><p>This workflow uses GenAI to engineer features for <strong>supervised machine learning</strong>. The tasks is a binary classification problem to decide whether a bank should <strong>approve or reject a loan request</strong> advanced by an applicant. For comparison, the workflow displays two machine learning pipelines:</p><ol><li><p>ML pipeline to predict loan status</p></li><li><p>ML pipeline to predict loan status enriched with AI-engineered features</p></li></ol><p>Both pipelines use the <strong>XGBoost Tree Ensemble </strong>and its performance is optimized by selecting relevant features and tuning hyper-parameters.</p>

URL: Loan approval dataset https://www.kaggle.com/datasets/taweilo/loan-approval-classification-data/data

AI-Engineered Feature Creation

Data Cleaning, Normalization and Feature Selection

Model Training & Testing

Data Cleaning, Normalization and Feature Selection

1: Regular Machine Learning Pipeline

2: Machine Learning Pipeline with AI-Engineered Features

Model Training & Testing

Feature Engineering with GenAI for Classification

This workflow uses GenAI to engineer features for supervised machine learning. The tasks is a binary classification problem to decide whether a bank should approve or reject a loan request advanced by an applicant. For comparison, the workflow displays two machine learning pipelines:

  1. ML pipeline to predict loan status

  2. ML pipeline to predict loan status enriched with AI-engineered features

Both pipelines use the XGBoost Tree Ensamble and its performance is optimized by selecting relevant features and tuning hyper-parameters. In the "Supervision" component, the performance of the model with and without AI-generated features is compared, AI-generated features can be accepted (and saved), or rejected and technical support is requested per email.

Data access

- Duplicate check- Encoding ofcategorical features
Data Cleaning
Compute features
Feature Creation
Extract AI-generatedfeature name, descriptionand syntax
Parse responses
XGBoost Predictor
Handledin targetcolumn
Class Imbalance
Optimizing accuracy
Feature Selection
Top: train (70%)Bottom: test (30%)
Table Partitioner
- boostingRounds- maxDepth
HP Optimization
Accuracy:90.5% approx
Scorer
- boostingRounds- maxDepth
HP Optimization
Accuracy:91.2% approx
Scorer
Promptengineering
Expression
Z score
Normalizer
Normalizer (Apply)
Top: train (70%)Bottom: test (30%)
Table Partitioner
Feature Selection Filter
Feature Selection Filter
OpenAI Authenticator
- Remove features withmissing values - Duplicate check- Encoding ofcategorical features
Data Cleaning
XGBoost Predictor
Normalizer (Apply)
OpenAI APIkey
Credentials Configuration
Model: Gpt-4o
OpenAI LLM Selector
XGBoost Tree Ensemble Learner
Feature Selection Filter
Handledin targetcolumn
Class Imbalance
ROC Curve
Documentation tocorrectly reference columns inthe Expression node
Table Creator
ROC Curve
Loan approvaldataset 1 = approved; 0 = rejected
CSV Reader
LLM Prompter
XGBoost Tree Ensemble Learner
Feature Selection Filter
Z score
Normalizer
Optimizing accuracy
Feature Selection

Nodes

Extensions

  • No modules found

Links