Icon

017002_​HDFS_​File_​Handling

017002_HDFS_File_Handling
HDFS file handling Upload csv file from local URL to HDFS file system Download csv file from HDFS file system to local path Description:This workflow demonstrates the HDFS file handling capabilites using the file handling nodes in conjunction with the HDFSConnection node, which is part of the KNIME Big Data Extension. Requirements:- KNIME File Handling Nodes (KNIME & Extensions)- KNIME Big Data Connectors (https://www.knime.com/knime-big-data-connectors)- Hortonworks Sandbox (http://hortonworks.com/products/hortonworks-sandbox/) generate a bunch of fake datatemp_path +temp.csvtemp_filetempfilefind nowthe file namein HDFSdownloadHDFS file tolocal temp pathfile URIon HDFScreate localtemp_pathread filefrom local pathdelete the remote HDFS filewrite temp.csv to local temp folderupload temp_fileto /Node 56Data Generator Java EditVariable (simple) Variable toTable Row String to URI List Remote Files Download Table Rowto Variable Create Temp Dir Table Rowto Variable CSV Reader Delete Files CSV Writer Create Temp Dir Upload Create Local BigData Environment HDFS file handling Upload csv file from local URL to HDFS file system Download csv file from HDFS file system to local path Description: This workflow demonstrates the HDFS file handling capabilites using the file handling nodes in conjunction with the HDFSConnection node, which is part of the KNIME Big Data Extension. Requirements:- KNIME File Handling Nodes (KNIME & Extensions)- KNIME Big Data Connectors (https://www.knime.com/knime-big-data-connectors)- Hortonworks Sandbox (http://hortonworks.com/products/hortonworks-sandbox/) generate a bunch of fake datatemp_path +temp.csvtemp_filetempfilefind nowthe file namein HDFSdownloadHDFS file tolocal temp pathfile URIon HDFScreate localtemp_pathread filefrom local pathdelete the remote HDFS filewrite temp.csv to local temp folderupload temp_fileto /Node 56Data Generator Java EditVariable (simple) Variable toTable Row String to URI List Remote Files Download Table Rowto Variable Create Temp Dir Table Rowto Variable CSV Reader Delete Files CSV Writer Create Temp Dir Upload Create Local BigData Environment

Nodes

Extensions

Links