KNIME Extension for Apache Spark core infrastructure version 4.3.1.v202101261633 by KNIME AG, Zurich, Switzerland
This node allows you to execute Python code on Spark (See PySpark documentation). The code has to put the desired output in the data frame with the name resultDataFrame1 and resultDataFrame2
Enter your Python code here.
The SparkSession can be accessed via the global variable spark. The input Dataset<Row> can be accessed via the method input parameter dataFrame1 and dataFrame2. The output Dataset<Row> must be called resultDataFrame1 and resultDataFrame2.
The editor optionally provides autocompletion (CTRL + Space) using the local Python installation,
if the KNIME Python Extension and the Python module jedi is installed.
The path to the Python executable has to be configured in Preferences → KNIME → Python.
Note: The completion suggestions are made based on the local Python environment, which may differ from the Python setup in the cluster!
To use this node in KNIME, install KNIME Extension for Apache Spark from the following update site:
A zipped version of the software site can be downloaded here.
You don't know what to do with this link? Read our NodePit Product and Node Installation Guide that explains you in detail how to install nodes to your KNIME Analytics Platform.
You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.
Do you have feedback, questions, comments about NodePit, want to support this platform, or want your own nodes or workflows listed here as well? Do you think, the search results could be improved or something is missing? Then please get in touch! Alternatively, you can send us an email to firstname.lastname@example.org, follow @NodePit on Twitter, or chat on Gitter!
Please note that this is only about NodePit. We do not provide general support for KNIME — please use the KNIME forums instead.