Icon

DuckDB Get Data From S3 (S3 Inventory File)

Options

File Limit:
Set limit on how many files to load at once.%%00010%%00010Limit of 0 will load all files
Read Type:
How source files should be read into DuckDB%%00010%%00010https://duckdb.org/docs/stable/data/overview
File Sort:
Set order in which files will sorted.%%00010%%00010ASC = Ascending%%00010DESC = Descending%%00010%%00010Examples: %%00010%%000101. Setting "File Limit" of 1 and "File Sort" to 'DESC' will only get the most recently created file.%%000102. Setting "File Limit" of 1 and "File Sort" to 'ASC' will get the oldest file, useful if performing incremental load operations.%%00010
Get files from timestamp (string):
String indicating from when to download files%%00010%%00010Format "2021-08-23 22:31:06"
S3 Inventory Location:
Path to S3 inventrory%%00010%%00010e.g. s3:////
Glob Pattern:
Glob pattern to filter S3 Inventory File
Custom Read Type:
Only used when "Read Type" is set to 'CUSTOM'%%00010%%00010Custom DuckDB read_*() statement.%%00010%%00010Omit file(s) path

Input Ports

This node has no input ports

Output Ports

This node has no output ports

Nodes

Extensions

Links