Avro to Spark

Creates a Spark DataFrame/RDD from given Avro file.

Notice: This feature requires at least Apache Spark 1.5.

This node can access a variety of different file systems. More information about file handling in KNIME can be found in the official File Handling Guide.

Options

Settings

Mode
Select if you like to read a file or folder.
File/Folder
Enter the input path. The required syntax of a path depends on the connected file system. The node description of the respective connector node describes the required path format. You can also choose a previously selected file from the drop-down list, or select a location from the "Browse..." dialog. Note that browsing is disabled in some cases:
  • Browsing is disabled if the connector node hasn't been executed since the workflow has been opened. (Re)execute the connector node to enable browsing.
The location can be exposed as or automatically set via a path flow variable.
Driver
Upload data source driver or depend on cluster side provided driver.

Input Ports

Icon
Spark compatible connection (HDFS, WebHDFS, HttpFS, S3, Blob Storage, ...)
Icon
Spark context

Output Ports

Icon
Spark DataFrame/RDD

Popular Predecessors

Popular Successors

Views

This node has no views

Workflows

  • No workflows found

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Installation

To use this node in KNIME, install the extension KNIME Extension for Apache Spark (legacy) from the below update site following our NodePit Product and Node Installation Guide:

v5.3

A zipped version of the software site can be downloaded here.

Plugin provider: KNIME AG, Zurich, Switzerland
Plugin version: 5.3.0.v202406141453
On NodePit since: 2024-07-09
Last update: 2024-11-21
KNIME versions: Since v4.3

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!