Icon

01_​Simple_​Streaming_​and_​Components

Simple Streaming (i.e. row-wise concurrent node execution)

Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits are less I/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executed concurrently. This workflow demonstrates the streaming capability.

Simple Streaming (i.e. row-wise concurrent node execution)Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits are lessI/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executedconcurrently.The first component reads a csv file and does some basic processing on it.The second component generates a lot of data and processes it as well. Here the streaming execution is more obvious. How to create a component?- select multiple nodes- right click - 'Create Component...'- right click on the meta node - 'Metanode > Convert to Component'In order to run all nodes within a component in streamable fashion, configure the node >'Job ManagerSelection'> select 'Simple Streaming' How to tell whether a particular node is streamable?By using the drop-down menu (the little triangle) in the Node Repository you can either filter for streamablenodes ("Show Streamable Nodes Only") or can display it as an additional information ("Show Additional Info").Within a streamed component, nodes with green arrows on it are streamable, those with a red cross aren't.Non-streamable nodes can still be executed within a streamed component, however, they they'll be executed innode-by-node fashion and won't benefit from the advantages of streaming. Sub Workflow as Component.To open it:- right click > Component > Open- Ctrl + Double Click Streamed Component - Readingdata from a file and process Streamed Component -Generate a lot of data Simple Streaming (i.e. row-wise concurrent node execution)Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits are lessI/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executedconcurrently.The first component reads a csv file and does some basic processing on it.The second component generates a lot of data and processes it as well. Here the streaming execution is more obvious. How to create a component?- select multiple nodes- right click - 'Create Component...'- right click on the meta node - 'Metanode > Convert to Component'In order to run all nodes within a component in streamable fashion, configure the node >'Job ManagerSelection'> select 'Simple Streaming' How to tell whether a particular node is streamable?By using the drop-down menu (the little triangle) in the Node Repository you can either filter for streamablenodes ("Show Streamable Nodes Only") or can display it as an additional information ("Show Additional Info").Within a streamed component, nodes with green arrows on it are streamable, those with a red cross aren't.Non-streamable nodes can still be executed within a streamed component, however, they they'll be executed innode-by-node fashion and won't benefit from the advantages of streaming. Sub Workflow as Component.To open it:- right click > Component > Open- Ctrl + Double Click Streamed Component - Readingdata from a file and process Streamed Component -Generate a lot of data

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Berthold
Created at: 2015-12-08
On NodePit since: 2020-04-09
Last update: 2024-11-17
Created with KNIME version: v4.0.0

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!