PySpark Script (2 to 1)

Options

PySpark Script Node

Flow Variable List

The list contains the flow variables that are currently available at the node input. Double clicking any of the entries will insert the respective identifier at the current cursor position (replacing the selection, if any).

Snippet text area

Enter your Python code here.

The SparkSession can be accessed via the global variable spark. The input Dataset<Row> can be accessed via the method input parameter dataFrame1 and dataFrame2. The output Dataset<Row> must be called resultDataFrame1.

The editor optionally provides autocompletion (CTRL + Space) using the local Python installation, if the KNIME Python Extension and the Python module jedi is installed. The path to the Python executable has to be configured in Preferences → KNIME → Python.
Note: The completion suggestions are made based on the local Python environment, which may differ from the Python setup in the cluster!

Views

This node has no views

Workflows

• No workflows found

Links

• No links available

Developers