Parquet Writer

Options

Mode

Depending on the selected mode the node writes the input data into a single file or splits it up into several files of the defined size which are then stored in the specified folder.

Output location

Select a file system and location where you want to store the file(s).

Create missing folders

If enabled, missing folders in the specified path will be created automatically.

If exists

Specify the behavior of the node in case the output file already exists.

• Fail: Will issue an error during the node's execution (to prevent unintentional overwrite).
• Overwrite: Will replace any existing file.
• Ignore: Will ignore if a file already exists and continues the copying process.

File Compression

The compression codec used to write the Parquet file.

Split data into files of size (MB)

Splits up the input data into files of the specified maximum size in megabytes. This option is only available if the folder mode is selected.

File name prefix

The prefix to use for the file within the selected folder. A running index is appended starting with 0 e.g. part_00000.parquet, part_00001.parquet. This option is only available if the folder mode is selected.

Within file row group size (MB)

Defines the maximum size of a row group within a file in megabyte. For more details see the Parquet documentation.

Name

Columns that match the given name (or regular expression) and KNIME type will be mapped to the specified database type.

• Column selection type: The option allows you to select how the column is matched.
  • Manual: Use the exact name of the column
  • Regex: Allow regex expressions to select multiple columns
• Column name: The column name or regex expression.
• KNIME type: KNIME data type to map from.
• Mapping to: Parquet data type to map to.

Type

Columns that match the given KNIME type will be mapped to the specified Parquet data type.

• KNIME type: KNIME data type to map from.
• Mapping to: Parquet data type to map to.

Views

This node has no views

Workflows

• 02.1 Databricks Integration OverviewKNIME Hub
• 06_Connecting_to_DatabricksKNIME Hub
• 07_Microsoft_Sharepoint_Data_AccessKNIME Hub
• 12_Incremental_processing_Parquet_fileKNIME Hub
• 2. Data PreparationKNIME Hub
• Show all 89 workflows

Links

• No links available

Developers