This node uses database specific bulk loading functionality that only some databases
(e.g. Hive, MySQL, PostgreSQL and H2) support to load large amounts of data into an existing
database table.
Notice:
Most databases do not perform data checks when loading the data into the table which
might lead to a corrupt data table.
The node does some preliminary checks such as checking that the column order and column names are
compatible. However it does not check the column type compatibility. So before using this node please
make sure that the column types of the KNIME columns and the database columns are compatible.
Depending on the database an intermediate file format is often used for efficiency which might be required to upload the file to a server. If a file needs to be uploaded, any of the protocols supported by the file handling nodes can be used, e.g. for Apache Hive HDFS or webHDFS. After the loading of the data into a table, the uploaded file gets deleted if it is no longer needed by the database.
If there is no need to upload or store the file for any reason, a remote file input connection prevents execution.
Some databases such as MySQL and PostgreSQL support file based and memory based uploading which require different rights in the database. If this is the case and if you do not have the rights to execute the file based loading of the data try the memory based method instead. If supported the different modes can be changed in the Loader mode section of the "Options" tab which is otherwise hidden.
Depending on the connected database the dialog settings may change. For example, MySQL and PostgreSQL use a CSV file for the data transfer. In order to change how the CSV file is created go to the "Advanced" tab.
This node can access a variety of different file systems. More information about file handling in KNIME can be found in the official File Handling Guide.
Popular Predecessors
- DB Table Creator13 %
- DB SQL Executor9 %
Row Filter (deprecated)6 %- Column Resorter5 %
- Column Filter4 %
Column Rename4 %- Concatenate3 %
- Joiner3 %
- Excel Reader2 %
- String to Date&Time2 %
- Constant Value Column2 %
- DB Writer1 %
- CSV Reader1 %
- Rule Engine1 %
- DB Query Reader1 %
- DB Loader1 %
- GroupBy1 %
- Duplicate Row Filter1 %
- Hive Connector1 %
- Rule-based Row Filter1 %
- Table Manipulator1 %
- Missing Value1 %
- Google BigQuery Connector< 1 %
- Double to Integer< 1 %
- Microsoft SQL Server Connector< 1 %
- DB Reader< 1 %
- Partitioning< 1 %
- String to Number< 1 %
- Table Creator< 1 %
- Date&Time to legacy Date&Time< 1 %
- File Reader< 1 %
- DB Connector< 1 %
- String Manipulation< 1 %
- MySQL Connector< 1 %
- Loop End< 1 %
- CSV Writer< 1 %
- Cross Joiner< 1 %
- Snowflake Connector< 1 %
- HDFS Connector< 1 %
- Chunk Loop Start< 1 %
- Reference Row Filter< 1 %
- Nominal Value Row Filter< 1 %
- Excel Writer< 1 %
- DB Insert< 1 %
- Java Snippet< 1 %
- Number to String< 1 %
- PostgreSQL Connector< 1 %
- Legacy Date&Time to Date&Time< 1 %
- Date&Time-based Row Filter< 1 %
- Date&Time Shift< 1 %
- Top k Row Filter< 1 %
- Python Script (legacy)< 1 %
Row Splitter (deprecated)< 1 %- RowID< 1 %
- DB Merge< 1 %
- Math Formula< 1 %
- Date&Time to String< 1 %
- String Manipulation (Multi Column)< 1 %
- HDFS Connector (KNOX)< 1 %
- Amazon S3 Connector< 1 %
- DB Connection Extractor< 1 %
- DB Table Selector< 1 %
- SSH Connector< 1 %
- Google Sheets Reader< 1 %
- Table Row to Variable< 1 %
- Parallel Chunk Start< 1 %
File Reader< 1 %- Row Sampling< 1 %
- Unpivot< 1 %
- Databricks File System Connector< 1 %
- Parquet Reader< 1 %
- SQLite Connector< 1 %
- Local File System Connector< 1 %
- DB Delete (Filter)< 1 %
- Cell Splitter< 1 %
- Wait...< 1 %
- Variable to Table Column< 1 %
- Table Reader< 1 %
Loop End< 1 %- Column Auto Type Cast< 1 %
- Column Renamer (Dictionary)< 1 %
Joiner< 1 %- Reference Column Resorter< 1 %
- Number Rounder< 1 %
- Sorter< 1 %
- Rule-based Row Splitter< 1 %
- Data Generator< 1 %
- Impala Connector< 1 %
Parquet Reader< 1 %- Create Local Big Data Environment< 1 %
- Amazon Redshift Connector< 1 %
- H2 Connector< 1 %
- Microsoft Access Connector< 1 %
- Oracle Connector< 1 %
- DB Delete (Table)< 1 %
- Parameterized DB Query Reader< 1 %
- DB Update< 1 %
- DB Transaction Start< 1 %
- DB Table Remover< 1 %
DB Table Creator< 1 %- Counter Generation< 1 %
- Column Expressions (legacy)< 1 %
- Math Formula (Multi Column)< 1 %
- Modify Time< 1 %
- Create Date&Time Range< 1 %
- Empty Table Switch< 1 %
- Table Column to Variable< 1 %
- DB Query< 1 %
- Counting Loop Start< 1 %
- Column Renamer< 1 %
- Table Validator (Reference)< 1 %
- Extract Table Spec< 1 %
- Parallel Chunk End< 1 %
- DB Looping< 1 %
- String to Number (PMML)< 1 %
- Create Table Structure< 1 %
- InfluxDBConnector< 1 %
- String Manipulation (Variable)< 1 %
- End IF< 1 %
- Merge Variables< 1 %
- Column Combiner< 1 %
- IF Switch< 1 %
- DB Type Mapper< 1 %
- DB Joiner< 1 %
- Credentials Configuration< 1 %
- Date&Time Part Extractor< 1 %
- Try (Data Ports)< 1 %
- Try (Variable Ports)< 1 %
- Inject Variables (Data)< 1 %
- Recursive Loop Start< 1 %
- Column Name Extractor< 1 %
- String Replacer< 1 %
- Table View (JavaScript)< 1 %
- String to Date/Time (legacy)< 1 %
- KNIME Server Connector< 1 %
- String to URI< 1 %
- Variable to Table Row< 1 %
Database Connection Table Reader< 1 %Database Update< 1 %Table Reader< 1 %- Group Loop Start< 1 %
- Numeric Binner< 1 %
- Column Appender< 1 %
- Many to One< 1 %
- One to Many< 1 %
- Linear Correlation< 1 %
- Table Validator< 1 %
- Reference Column Splitter< 1 %
- Missing Value Column Filter< 1 %
- Rank< 1 %
- CASE Switch Start< 1 %
- ORC Reader< 1 %
- HDFS File Permission (legacy)< 1 %
- Hive to Spark< 1 %
- Vertica Connector< 1 %
- DB Column Rename< 1 %
- DB Query Extractor< 1 %
- Google Cloud Storage Connector< 1 %
- List Files/Folders< 1 %
- Create Temp Folder< 1 %
- JSON to Table< 1 %
- GET Request< 1 %
- Modify Time Zone< 1 %
Views
- This node has no views
Workflows
[01.CX] Combining Siloed Legacy Data with Running App's Back-End DatabaseKNIME Hub- 00_Setup_Hive_TableKNIME Hub
- 00_Setup_Hive_TableKNIME Hub
- 01_Big_Data_Preprocessing_ExampleKNIME Hub
- 01_Big_Data_Preprocessing_ExampleKNIME Hub
- Show all 118 workflows
Links
- Read and process HUGE csv Fileby izaychik63 on 2021-03-22forum.knime.com/p/119121
- 1Tb File to be loaded “fastly”by izaychik63 on 2023-10-20forum.knime.com/p/238737
Developers
You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.