0 ×

HDFS Connector (KNOX)

KNIME Extension for WebHDFS via Apache KNOX Filehandling version 4.3.0.v202012020404 by KNIME AG, Zurich, Switzerland

This node connects to Hadoop Distributed File System (HDFS) using an Apache KNOX gateway. The resulting output port allows downstream nodes to access the files of the remote file system, e.g. to read or write, or to perform other file system operations (browse/list files, copy, move, ...).

Path syntax: Paths for HDFS are specified with a UNIX-like syntax, for example /myfolder/file.csv, which is an absolute path that consists of:

A leading slash (/).
The name of a folder (myfolder), followed by a slash.
Followed by the name of a file (file.csv).

SSL: This node uses the JVM SSL settings.

Options

Settings

URL

KNOX URL including cluster name, for example https://<server>:8443/gateway/<default> (replace <server> with the fully qualified hostname and <default> with your cluster name). The protocol should be https and port 8443 on most deployments.

Authentication

Specify the authentication method to use:

Credentials: Authenticate with the username and password of the selected credentials flow variable.
Username & password: Authenticate with the provided Username and Password. The password is persistently stored (in encrypted form) in the settings of this node and will be saved with the workflow.

Working directory

Specify the working directory of the resulting file system connection, using the Path syntax explained above. The working directory must be specified as an absolute path. A working directory allows downstream nodes to access files/folders using relative paths, i.e. paths that do not have a leading slash. The default working directory is the root "/", under which all the document libraries are located.

Advanced

Connection timeout: Timeout in seconds to establish a connection or 0 for an infinite timeout.
Read timeout: Timeout in seconds to read data from connection or 0 for an infinite timeout.

Output Ports

: HDFS (via KNOX) File System Connection.

Workflows

09_Data_Transfer_between_Clouds (KNIME Hub)
11_Working_with_Utility_Nodes (KNIME Hub)
12_Incremental_processing_Parquet_file (KNIME Hub)

Installation

To use this node in KNIME, install KNIME Big Data Connectors from the following update site:

KNIME 4.3

A zipped version of the software site can be downloaded here.

You don't know what to do with this link? Read our NodePit Product and Node Installation Guide that explains you in detail how to install nodes to your KNIME Analytics Platform.

Wait a sec! You want to explore and install nodes even faster? We highly recommend our NodePit for KNIME extension for your KNIME Analytics Platform. Browse NodePit from within KNIME, install nodes with just one click and share your workflows with NodePit Space.

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.