HDFS Connector (KNOX)

This node connects to Hadoop Distributed File System (HDFS) using an Apache KNOX gateway. The resulting output port allows downstream nodes to access the files of the remote file system, e.g. to read or write, or to perform other file system operations (browse/list files, copy, move, ...).

Path syntax: Paths for HDFS are specified with a UNIX-like syntax, for example /myfolder/file.csv, which is an absolute path that consists of:

  1. A leading slash (/).
  2. The name of a folder (myfolder), followed by a slash.
  3. Followed by the name of a file (file.csv).

SSL: This node uses the JVM SSL settings.

Options

Settings

URL
KNOX URL including cluster name, for example https://<server>:8443/gateway/<default> (replace <server> with the fully qualified hostname and <default> with your cluster name). The protocol should be https and port 8443 on most deployments.
Authentication
Authenticate with a username and password.
  • Either enter a username and password, in which case the password will be persistently stored (in encrypted form) with the workflow.
  • Or check Use credentials and a select a credentials flow variable to supply the username and password.
Working directory
Specify the working directory of the resulting file system connection, using the Path syntax explained above. The working directory must be specified as an absolute path. A working directory allows downstream nodes to access files/folders using relative paths, i.e. paths that do not have a leading slash. The default working directory is the root "/", under which all the document libraries are located.

Advanced

Connection timeout
Timeout in seconds to establish a connection or 0 for an infinite timeout.
Read timeout
Timeout in seconds to read data from connection or 0 for an infinite timeout.

Input Ports

This node has no input ports

Output Ports

Icon
HDFS (via KNOX) File System Connection.

Popular Predecessors

Popular Successors

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Installation

To use this node in KNIME, install the extension KNIME Big Data Connectors from the below update site following our NodePit Product and Node Installation Guide:

v5.3

A zipped version of the software site can be downloaded here.

Plugin provider: KNIME AG, Zurich, Switzerland
Plugin version: 5.3.0.v202406180713
On NodePit since: 2024-07-09
Last update: 2024-11-15
KNIME versions: Since v4.3

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!