Amazon S3 Connector

This node configures the connection information that will be used to connect to Amazon S3. Using this connection the other KNIME remote file handling nodes such as Excel Reader and Excel Writer can download and upload files from and to Amazon S3.

For further documentation please take a look at the AWS Documentation.

Path syntax: Paths for Amazon S3 are specified with a UNIX-like syntax, /mybucket/myfolder/myfile. An absolute for S3 consists of:

A leading slash ("/").
Followed by the name of a bucket ("mybucket" in the above example), followed by a slash.
Followed by the name of an object within the bucket ("myfolder/myfile" in the above example).

URI formats: When you apply the Path to URI node to paths coming from this connector, you can create URIs with the following formats:

Presigned https:// URLs which contain credentials, that allow to access files for a certain amount of time (see AWS documentation).
s3:// URLs to access Amazon S3 with the aws command line interface, or inside Hadoop environments.

Options

Settings

Working directory: Specifies the working directory using the path syntax explained above. The working directory must be specified as an absolute path. A working directory allows downstream nodes to access files/folders using relative paths, i.e. paths that do not have a leading slash. If not specified, the default working directory is "/".
Normalize paths: Determines if the path normalization should be applied. Path normalization eliminates redundant components of a path like, e.g. /a/../b/./c" can be normalized to "/b/c". When these redundant components like "../" or "." are part of an existing object, then normalization must be deactivated in order to access them properly.

Advanced

Read/write timeout in seconds: The socket read/write timeout. For further details see the S3 documentation.
Server-side encryption (SSE): If selected, all data written to S3 will be encrypted with Server-side encryption (SSE) using SSE-S3, SSE-KMS or SSE-C.
Use default AWS managed key: If SSE-KMS is selected as the SSE method, then this option specifies whether or not to encrypt data with the default AWS managed CMK.
KMS key id: If SSE-KMS is selected as the SSE method and the default AWS managed CMK should not be used, then this option allows to choose the KMS key with which to encrypt data written to S3. Click on List keys to fetch the list of available keys (requires permissions kms:ListKeys, kms:DescribeKey and optionally kms:ListAliases).
Customer key: If SSE-C is selected as the SSE method it is necessary to provide an encryption key. There are multiple ways the key could be provided. The base64 encoded key could be entered directly or provided via credentials variable using the encoded key as password (username can be anything or left empty). The third way is to select a file that contains exactly 32 bytes (256 bit) that should be used as key.

Input Ports

: Port object containing the AWS connection information.
…: A file system connection to read/write the customer key, when SSE-C encryption mode is enabled.

Output Ports

: S3 File System Connection

Popular Predecessors

Popular Successors

Views

This node has no views

Workflows

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.

Installation

To use this node in KNIME, install the extension KNIME Amazon Cloud Connectors from the below update site following our NodePit Product and Node Installation Guide:

v5.2

A zipped version of the software site can be downloaded here.

Plugin provider: KNIME AG, Zurich, Switzerland

Plugin version: 5.2.0.v202310301554

On NodePit since: 2023-12-06

Last update: 2024-04-18

KNIME versions: Since v4.1

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!