0 ×

04_​Control_​Workflow_​for_​Performance_​and_​Scalability_​Measurements

Workflow

Generate Context to remove time fromoverall time measurement. Control Workflow for Performance and Scalability MeasurementsThis workflow was constructed to be able to compare different performance and scalability measurements. First a data set of the defined size is created.This dataset is sent forward to multiple workflows. We are here comparing three modes, using Native KNIME and adding Spark or H2O nodes. Finally thesemeasurements are repeated mutliple times to ensure that the results are independent from effects outside of the workflow. Define Data Table Size Define Number of ValidationsBy using multiple validations youreduce effects from outside of KNIME Execute in each of the three worlds Automatically create the datatable withdefined row sizeexecute workflowAll possibleexecution modes(Native KNIME, H2O andSpark)For each modeCollect speedmeasurementsfor each modeStart H2OLocal ContextInitalize LocalBig Data EnvironmentGeneratecorrect filepathEnsureMemoryis freedGenerate Test Data Tables For each defined data sizefor eachvalidationNumberof Validationsfor each defined data size Call Workflow(Table Based) Call Workflow(Table Based) Table Creator Table Row ToVariable Loop Start Loop End H2O Local Context Create Local BigData Environment String Manipulation Java EditVariable (simple) Variable toTable Column Table Creator Table Row ToVariable Loop Start Loop End Counting Loop Start Loop End Generate Context to remove time fromoverall time measurement. Control Workflow for Performance and Scalability MeasurementsThis workflow was constructed to be able to compare different performance and scalability measurements. First a data set of the defined size is created.This dataset is sent forward to multiple workflows. We are here comparing three modes, using Native KNIME and adding Spark or H2O nodes. Finally thesemeasurements are repeated mutliple times to ensure that the results are independent from effects outside of the workflow. Define Data Table Size Define Number of ValidationsBy using multiple validations youreduce effects from outside of KNIME Execute in each of the three worlds Automatically create the datatable withdefined row sizeexecute workflowAll possibleexecution modes(Native KNIME, H2O andSpark)For each modeCollect speedmeasurementsfor each modeStart H2OLocal ContextInitalize LocalBig Data EnvironmentGeneratecorrect filepathEnsureMemoryis freedGenerate Test Data Tables For each defined data sizefor eachvalidationNumberof Validationsfor each defined data size Call Workflow(Table Based) Call Workflow(Table Based) Table Creator Table Row ToVariable Loop Start Loop End H2O Local Context Create Local BigData Environment String Manipulation Java EditVariable (simple) Variable toTable Column Table Creator Table Row ToVariable Loop Start Loop End Counting Loop Start Loop End

Download

Get this workflow from the following link: Download

Nodes

04_​Control_​Workflow_​for_​Performance_​and_​Scalability_​Measurements consists of the following 15 nodes(s):

Plugins

04_​Control_​Workflow_​for_​Performance_​and_​Scalability_​Measurements contains nodes provided by the following 5 plugin(s):