Icon

02_​HDFS_​and_​File_​Handling_​Example

HDFS file handling

This workflow demonstrates the HDFS file handling capabilites using the file handling nodes in conjunction with an HDFS connection.

To run this workflow on a remote cluster, use an HDFS Connection node (available in the KNIME Big Data Connectors Extension) in place of the Create Local Big Data Environment node.

Requirements:
- KNIME File Handling Nodes
- KNIME Big Data Connectors Extension
- KNIME Extension for Local Big Data Environments

Setup:
The directory /tmp needs to exist in the HDFS file system and the user needs to have read/write rights to it.


Upload csv file to HDFS file system Download csv file from HDFS file system HDFS File Handling This workflow demonstrates the HDFS file handling capabilites using the file handling nodes in conjunction with an HDFSconnection.For more information see the workflow metadata. Find it here: View -> Description random datathe file namefrom HDFStest file contentdelete the remote filewrite fileto HDFSFiles only Data Generator Java EditVariable (simple) String to URI List Remote Files Download Create Temp Dir CSV Reader Table DifferenceChecker Delete Files CSV Writer Create Temp Dir Upload Create Local BigData Environment Variable toTable Row Table Rowto Variable Row Filter Table Rowto Variable Upload csv file to HDFS file system Download csv file from HDFS file system HDFS File Handling This workflow demonstrates the HDFS file handling capabilites using the file handling nodes in conjunction with an HDFSconnection.For more information see the workflow metadata. Find it here: View -> Description random datathe file namefrom HDFStest file contentdelete the remote filewrite fileto HDFSFiles only Data Generator Java EditVariable (simple) String to URI List Remote Files Download Create Temp Dir CSV Reader Table DifferenceChecker Delete Files CSV Writer Create Temp Dir Upload Create Local BigData Environment Variable toTable Row Table Rowto Variable Row Filter Table Rowto Variable

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: koetter
Created at: 2015-02-11
On NodePit since: 2019-07-12
Last update: 2024-04-18
Created with KNIME version: v4.1.0
Tags: HDFSHadoopBig Data

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!