Icon

03 OPTIONAL Production data pipeline - Triggered execution - Exercise

This workflow:
Step 1. Extract raw inspection data from KNIME Business Hub and JSON files when a new inspection report appears in a KNIME Business Hub space

  1. Use the Container Input (Repository Event) node to import the event details when the workflow gets triggered (when a new report gets uploaded)*

  2. Connect to KNIME Business Hub with the Space Connector node:

    • Set knime-edu-hubas the Hub mountpoint and select your exercise space

    • Set the working directory as /Inspection

    • Connect the Container Input (Repository Event) and the Space Connector nodes to the PDF report text metanode


*During the development of this workflow, we use example event details.


Learning objective: Learn how to trigger workflow execution on the KNIME Business Hub


Workflow description: This workflow:

  • Reads the data from real data sources, e.g., new inspection reports once they appear in KNIME Business Hub space,

  • Blends and transforms raw inspection files and inspection reviews,

  • Loads the transformed data,

  • Finally, this workflow is triggered each time when a new report appears in the selected KNIME Business Hub space.


You'll find the instructions to the exercises in the yellow annotations.

Part 2 - Introduction to KNIME Business Hub

Exercise 03 (Optional) Production data pipeline - Triggered execution

Step 0. Upload an example PDF file to your space on Hub to be able to execute this workflow

Before configuring this workflow:

  1. Connect to your user space on KNIME Business Hub and run the Transfer Files node to create an Inspection folder in your user space and upload an example PDF report to it.

  2. Once you executed these two nodes, delete them!


Transform
Extract
Load (production DB)
Step 4. Upload this production data pipeline to KNIME Business Hub and deploy it as a trigger

  1. Save this workflow, upload it to your user space on KNIME Business Hub (make sure to reset all the nodes), and version it.

  2. On the workflow page, create the trigger deployment by clicking Deploy and Trigger buttons:

    • Call the deployment:<your username>-triggered-execution, select the latest version and a valid execution context

    • Configure the trigger options:

      • Team & space: course team and your private space

      • Item type: Data file

      • Action: Added

      • Additional filtering: By subfolder

      • Subfolder:/Inspection

    • In Advanced settings, select the User execution scope, and create the deployment.

  3. Add one of the PDF files to your Inspection folder in your space

    • You can find the PDF files in the exercises data folder ../../../data/Inspection

    • The execution of the deployed data pipeline should get triggered by this event.

Append:inspections
DB Writer
Transform
SQLite Connector
Inspection folder inyour user space
Space Connector
PDF report text
Read Inspection Reviews
JSON Reader
Create Inspectionfolder & uploadAdriano.pdf to it
Transfer Files
Your user space
Space Connector

Nodes

Extensions

Links