In order to connect to the service, Amazon S3 credentials are required.
Folders created on root level are created as buckets on S3. The virtual file system functionality is fully supported by the file handling nodes.
In the first step a bucket and a directory inside that bucket is created. After that a file is uploaded to the directory inside the bucket (path: /examplebucket***/exampledirectory). Finally all the files and folders inside the example bucket are listed. The result is a list of URIs which can be used to download files/directories.
In the second step we download the iris.csv which we uploaded in the step before. This saves the file to disk; this file can then be read using a csv reader. Since it might be unnecessary to cache the file to disk, the Amazon S3 File Picker provides the functionality to create URLs from which files can be read directly. This works with all file readeres that support reading files from a URL. These URLs have an expiration date which can be set in the dialog.
Stream reading also becomes useful in conjunction with the KNIME Streaming Executor as the (possibly large) data is read from the remote side and is immediately processed without being downloaded to the local machine.
After having seen how we upload and download files and how we use the File Picker to read files directly we can delete our example bucket from S3.
To use this workflow in KNIME, download it from the below URL and open it in KNIME:
Download WorkflowDeploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.
Try NodePit Runner!Do you have feedback, questions, comments about NodePit, want to support this platform, or want your own nodes or workflows listed here as well? Do you think, the search results could be improved or something is missing? Then please get in touch! Alternatively, you can send us an email to mail@nodepit.com, follow @NodePit on Twitter or botsin.space/@nodepit on Mastodon.
Please note that this is only about NodePit. We do not provide general support for KNIME — please use the KNIME forums instead.