Icon

kn_​example_​hive_​db_​loader_​45

Hive - how to get from DB-Connectors to Hive (or Impala) tables - KNIME 4.5+

Hive - how to get from DB-Connectors to Hive (or Impala) tables - KNIME 4.5+
There does not seem to be a direct way to get from the comfortable (brown) Database nodes (up to KNIME 3.7) writing SQL code to store the data in a big data (Hive) table. Via Spark there is a way.
You also want to consult the "School of Hive" (https://kni.me/w/1q-JwD0cwmuEkWpy)

Hive - how to get from DB-Connectors to Hive (or Impala) tables - KNIME 4.5+There does not seem to be a direct way to get from the comfortable (brown) Database nodes (up to KNIME 3.7) writing SQL code to store the data in a big data (Hive) table. Via Spark there is a way.You also want to consult the "School of Hive" (https://kni.me/w/1q-JwD0cwmuEkWpy) Test if a multiline string can be handled with Hive and the uploaderhttps://forum.knime.com/t/controling-file-reader-via-flowvariables/30216/2?u=mlauber71 test_multilinecreate dummy datatest_multilineadd line breaksstring($column0$ + "\nnewline\nnewline")Group byuse Sparkto Hive"test1_group"SELECT * FROM default.test1_groupdefault.test1make sure thetable is goneand the system is aware of thatCluster_Membershipno_entrieswrite table within Hive"test2_group"test2_groupload datainto hivetable"test1"default.test1test_multilinetest_multilinetest_multilinetest_multiline1st cretae (empty)hive table"test1"1st cretae (empty)hive table"test_multiline"test_multiline.parquetDESCRIBE EXTENDED `default`.`test2_group`DESCRIBE EXTENDED `default`.`test1_group`=> deletes the wholelocal big data folder/big_datadecide in the configuration if you want sub-folders or parent foldersif you encouter any problems, closeKNIME and delete all data from the folder/big_data/test_multiline.parquetTable Creator Data Generator String Manipulation DB GroupBy Hive to Spark Spark to Hive DB Query Reader DB Table Remover DB SQL Executor Column Rename DB Column Rename DB ConnectionTable Writer DB Reader DB Loader DB Table Selector DB Table Remover DB Loader DB Table Selector DB Reader DB Table Creator DB Table Creator Parquet Reader DB Query Reader DB ConnectionExtractor DB Query Reader DB ConnectionExtractor local big datacontext create Parquet Writer Hive - how to get from DB-Connectors to Hive (or Impala) tables - KNIME 4.5+There does not seem to be a direct way to get from the comfortable (brown) Database nodes (up to KNIME 3.7) writing SQL code to store the data in a big data (Hive) table. Via Spark there is a way.You also want to consult the "School of Hive" (https://kni.me/w/1q-JwD0cwmuEkWpy) Test if a multiline string can be handled with Hive and the uploaderhttps://forum.knime.com/t/controling-file-reader-via-flowvariables/30216/2?u=mlauber71 test_multilinecreate dummy datatest_multilineadd line breaksstring($column0$ + "\nnewline\nnewline")Group byuse Sparkto Hive"test1_group"SELECT * FROM default.test1_groupdefault.test1make sure thetable is goneand the system is aware of thatCluster_Membershipno_entrieswrite table within Hive"test2_group"test2_groupload datainto hivetable"test1"default.test1test_multilinetest_multilinetest_multilinetest_multiline1st cretae (empty)hive table"test1"1st cretae (empty)hive table"test_multiline"test_multiline.parquetDESCRIBE EXTENDED `default`.`test2_group`DESCRIBE EXTENDED `default`.`test1_group`=> deletes the wholelocal big data folder/big_datadecide in the configuration if you want sub-folders or parent foldersif you encouter any problems, closeKNIME and delete all data from the folder/big_data/test_multiline.parquetTable Creator Data Generator String Manipulation DB GroupBy Hive to Spark Spark to Hive DB Query Reader DB Table Remover DB SQL Executor Column Rename DB Column Rename DB ConnectionTable Writer DB Reader DB Loader DB Table Selector DB Table Remover DB Loader DB Table Selector DB Reader DB Table Creator DB Table Creator Parquet Reader DB Query Reader DB ConnectionExtractor DB Query Reader DB ConnectionExtractor local big datacontext create Parquet Writer

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: A9699459
Created at: 2019-12-02
On NodePit since: 2022-06-11
Last update: 2024-12-24
Created with KNIME version: v4.7.8
Tags: big datahiveimpaladb nodeslegacyknime

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!