Icon

01_​Simple_​Streaming_​and_​Wrapped_​Nodes

Simple Streaming (i.e. row-wise concurrent node execution)

Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits are less I/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executed concurrently. This workflow demonstrates the streaming capability.

Simple Streaming (i.e. row-wise concurrent node execution)Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits areless I/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executedconcurrently.The first wrapped node (that encapsulates a workflow) reads a csv file and does some basic processing on it.The second wrapped node generates a lot of data and processes it as well. Here the streaming execution is more obvious. How to create a Wrapped Node?- select multiple nodes- right click - 'Collapse into Meta Node'- right click on the meta node - 'Meta Node > Wrap'In order to run all nodes within a wrapped node in streamable fashion, configure the node >'Job ManagerSelection'> select 'Simple Streaming' How to tell whether a particular node is streamable?By using the drop-down menu (the little triangle) in the Node Repository you can either filter for streamablenodes ("Show Streamable Nodes Only") or can display it as an additional information ("Show Additional Info").Within a streamed wrapped node, nodes with green arrows on it are streamable, those with a red cross aren't.Non-streamable nodes can still be executed within a streamed wrapped node, however, they they'll beexecuted in node-by-node fashion and won't benefit from the advantages of streaming. Sub Workflow as Wrapped Node.To open it:- right click > Wrapped Node > Open- Ctrl + Double Click Streamed Wrapped Node - Readingdata from a file and process Streamed Wrapped Node -Generate a lot of data Simple Streaming (i.e. row-wise concurrent node execution)Streaming execution is another way to execute nodes and is different to the default "node-by-node" execution. Benefits areless I/O and faster runtime at the expense of limited explorativity and traceability. Nodes that are streamable are executedconcurrently.The first wrapped node (that encapsulates a workflow) reads a csv file and does some basic processing on it.The second wrapped node generates a lot of data and processes it as well. Here the streaming execution is more obvious. How to create a Wrapped Node?- select multiple nodes- right click - 'Collapse into Meta Node'- right click on the meta node - 'Meta Node > Wrap'In order to run all nodes within a wrapped node in streamable fashion, configure the node >'Job ManagerSelection'> select 'Simple Streaming' How to tell whether a particular node is streamable?By using the drop-down menu (the little triangle) in the Node Repository you can either filter for streamablenodes ("Show Streamable Nodes Only") or can display it as an additional information ("Show Additional Info").Within a streamed wrapped node, nodes with green arrows on it are streamable, those with a red cross aren't.Non-streamable nodes can still be executed within a streamed wrapped node, however, they they'll beexecuted in node-by-node fashion and won't benefit from the advantages of streaming. Sub Workflow as Wrapped Node.To open it:- right click > Wrapped Node > Open- Ctrl + Double Click Streamed Wrapped Node - Readingdata from a file and process Streamed Wrapped Node -Generate a lot of data

Nodes

Extensions

Links