Databricks File System Connector

This node connects to the Databricks File System (DBFS) of a Databricks deployment. The resulting output port allows downstream nodes to access DBFS as a file system, e.g. to read or write files and folders, or to perform other file system operations (browse/list files, copy, move, ...).

Path syntax: Paths for DBFS are specified with a UNIX-like syntax, for example /myfolder/file.csv, which is an absolute path that consists of:

  1. A leading slash (/).
  2. The name of a folder (myfolder), followed by a slash.
  3. Followed by the name of a file (file.csv).

Options

Settings

Databricks URL
Full URL of the Databricks deployment, e.g. https://<account>.cloud.databricks.com on AWS or https://<region>.azuredatabricks.net on Azure.
Authentication
Username and password or a personal access token can be used for authentication. Databricks strongly recommends tokens. See authentication in Databricks AWS or Azure documentation for more informations about personal access token.
  • Username & password: Authenticate with a username and password. Either enter a username and password, in which case the password will be persistently stored (in encrypted form) with the workflow. Or check Use credentials and a select a credentials flow variable to supply the username and password.
  • Token: Authenticate with the provided personal access token. If entered here, the token will be persistently stored (in encrypted form) with the workflow. Alternatively, if Use credentials is selected, the password of the selected credentials flow variable will be used as the token for authentication (username of the flow variable will be ignored).
Working directory
Specifies the working directory using the path syntax explained above. The working directory must be specified as an absolute path. A working directory allows downstream nodes to access files/folders using relative paths, i.e. paths that do not have a leading slash. If not specified, the default working directory is "/".

Advanced

Connection timeout
Timeout in seconds to establish a connection, or 0 for an infinite timeout.
Read timeout
Timeout in seconds to read data from an established connection, or 0 for an infinite timeout.

Input Ports

This node has no input ports

Output Ports

Icon
Databricks File System Connection

Popular Predecessors

Popular Successors

Views

This node has no views

Workflows

  • No workflows found

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Installation

To use this node in KNIME, install the extension KNIME Databricks Integration from the below update site following our NodePit Product and Node Installation Guide:

v5.2

A zipped version of the software site can be downloaded here.

Plugin provider: KNIME AG, Zurich, Switzerland
Plugin version: 5.2.2.v202403071725
On NodePit since: 2023-12-06
Last update: 2024-04-23
KNIME versions: Since v4.3

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!