Databricks File System Connector

This node connects to the Databricks File System (DBFS) of a Databricks deployment. The resulting output port allows downstream nodes to access DBFS as a file system, e.g. to read or write files and folders, or to perform other file system operations (browse/list files, copy, move, ...).

Path syntax: Paths for DBFS are specified with a UNIX-like syntax, for example /myfolder/file.csv, which is an absolute path that consists of:

  1. A leading slash (/).
  2. The name of a folder (myfolder), followed by a slash.
  3. Followed by the name of a file (file.csv).

Options

Databricks URL
Full URL of the Databricks deployment, e.g. https://<account>.cloud.databricks.com on AWS or https://<region>.azuredatabricks.net on Azure.
Authentication method
Specify the authentication method to use. Personal access token or Username and password can be used for authentication. Databricks strongly recommends tokens.
  • Personal access token: Authenticate with a personal access token. The token will be persistently stored (in encrypted form) with the workflow if not provided by a flow variable. Databricks strongly recommends tokens.
  • Username & password: Authenticate with a username and password. The password will be persistently stored (in encrypted form) with the workflow.
Personal access token
Enter a personal access token for authentication. The token will be persistently stored (in encrypted form) with the workflow.
Username & Password
Credentials for username and password authentication.
Working directory
Specifies the working directory using the path syntax explained above. The working directory must be specified as an absolute path. A working directory allows downstream nodes to access files/folders using relative paths, i.e. paths that do not have a leading slash. If not specified, the default working directory is "/".
Connection timeout (seconds)
Timeout in seconds to establish a connection, or 0 for an infinite timeout.
Read timeout (seconds)
Timeout in seconds to read data from an established connection, or 0 for an infinite timeout.

Input Ports

Icon
Databricks Workspace Connection, that can be connected to the Databricks Workspace Connector.

Output Ports

Icon
Databricks File System Connection

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.