Spark Lag Column

This component copies column values from preceding rows into the current row in a Spark DataFrame/RDD. The component can be used to

- make a copy of the selected column and shift the cells I steps up (I = lag interval)
- make L copies of the selected column and shift the cells of each copy I, 2I, 3I, ... L*I steps up (L = lag)

Required extensions:
KNIME Data Generation
(https://hub.knime.com/knime/extensions/org.knime.features.datageneration/latest)
KNIME Extension for Apache Spark
(https://hub.knime.com/knime/extensions/org.knime.features.bigdata.spark/latest)
KNIME Quick Forms
(https://hub.knime.com/knime/extensions/org.knime.features.js.quickforms/latest)

Options

Skip incomplete rows

If checked, rows containing missing values will be removed.

Value Column

The column to create the lag for. This column has to be a numeric column.

Sort Column

The column to sort by. This column has to be a numeric or timestamp column.

Lag

How many lagged columns to be generated.

Lag Interval

The interval between the lagged columns.

Nodes

• Column Selection Configuration2 ×
• Component Input2 ×
• Component Output2 ×
• Empty Table Creator2 ×
• Integer Configuration2 ×
• Show all 21 nodes

Extensions

• FeatureKNIME Base nodes
• FeatureKNIME Data Generation
• FeatureKNIME Extension for Apache Spark
• FeatureKNIME Extension for Apache Spark (legacy)
• FeatureKNIME Javasnippet
• Show all 7 modules

Links

• No links available