0 ×

06_​Modularized_​Spark_​Scripting

Workflow

Spark Java snippet nodes
SparkHadoopBig Data
Modularize and Execute Your Spark Code Modularized Spark Scripting This workflow demonstrates the usage of the different Spark Java Snippet nodes to read a text file from HDFS, parse it, filter it and write the result back toHDFS. Execute FIRST Execute LAST split into columnsfilter by typefilter invalid rowsrename columnswrite to hdfsRead filefrom HDFSContext is destroyed on closeConnect to HDFSDelete Demo FileExecute last!!!this node uploads the example file included in the workflow to the hdfs file system. Spark RDDJava Snippet Spark RDDJava Snippet Spark RDDJava Snippet Spark Column Rename Spark RDD JavaSnippet (Sink) Spark to Table Spark RDD JavaSnippet (Source) Create Spark Context(Jobserver) HDFS Connector DeleteFiles/Folders Upload Demo File Modularize and Execute Your Spark Code Modularized Spark Scripting This workflow demonstrates the usage of the different Spark Java Snippet nodes to read a text file from HDFS, parse it, filter it and write the result back toHDFS. Execute FIRST Execute LAST split into columnsfilter by typefilter invalid rowsrename columnswrite to hdfsRead filefrom HDFSContext is destroyed on closeConnect to HDFSDelete Demo FileExecute last!!!this node uploads the example file included in the workflow to the hdfs file system. Spark RDDJava Snippet Spark RDDJava Snippet Spark RDDJava Snippet Spark Column Rename Spark RDD JavaSnippet (Sink) Spark to Table Spark RDD JavaSnippet (Source) Create Spark Context(Jobserver) HDFS Connector DeleteFiles/Folders Upload Demo File

Download

Get this workflow from the following link: Download

Nodes

06_​Modularized_​Spark_​Scripting consists of the following 15 nodes(s):

Plugins

06_​Modularized_​Spark_​Scripting contains nodes provided by the following 4 plugin(s):