Spark Normalizer

This node normalizes the values of all selected (numeric) columns.

Options

Min-max normalization

Linear transformation of all values such that the minimum and maximum in each column are as given.

Z-score normalization (Gaussian)

Linear transformation such that the values in each column are Gaussian-(0,1)-distributed, i.e. mean is 0.0 and standard deviation is 1.0.

Normalization by decimal scaling

The maximum value in a column (both positive and negative) is divided j-times by 10 until its absolute value is smaller or equal to 1. All values in the column are then divided by 10 to the power of j.

Input Ports

Spark DataFrame/RDD requiring normalization of some or all columns.

Output Ports

Spark DataFrame/RDD with normalized columns.

PMML document containing normalization parameters, which can be used in the "Spark Compiled Transformations Applier" node to normalize test data the same way as the training data has been normalized.

Best Friends (Incoming)

• Table to Spark (21 %) Streamable
• Spark Category To Number (21 %) 
• Spark GroupBy (14 %) 
• CSV to Spark (7 %) 
• Spark to Table (7 %) Streamable
• Spark Column Filter (7 %) 
• Spark Missing Value (7 %) 
• Spark SQL Query (7 %) 
• Persist Spark DataFrame/RDD (7 %) 
• Show all 9 recommendations

Best Friends (Outgoing)

• Spark k-Means (24 %) 
• Spark Predictor (10 %) 
• Spark Transformations Applier (10 %) 
• Spark to H2O (10 %) 
• Spark Cluster Assigner (5 %) 
• Spark Random Forests Learner (5 %) 
• Spark Logistic Regression Learner (5 %) 
• Spark Linear Regression Learner (5 %) 
• Spark Category To Number (5 %) 
• Spark Column Filter (5 %) 
• Spark Partitioning (5 %) 
• Spark SQL Query (5 %) 
• Spark Correlation Matrix (5 %) 
• Persist Spark DataFrame/RDD (5 %) 
• Show all 14 recommendations

Workflows

• 09_​Big_​Data_​Irish_​Meter_​on_​Spark_​only
• 17108_​Big_​Data_​Irish_​Meter_​on_​Spark_​only

Installation

To use this node in KNIME, install KNIME Extension for Apache Spark from the following update site:

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.