Icon

01 Development data pipeline - Integrated deployment - Exercise

This workflow reads a sample of restaurant orders data, cleans and transforms it, and loads the transformed data to the development database. Besides, it captures the data transformation segment and saves it as a separate Data Transformation workflow.
Step 2 : Write the transformed data to database

  1. The Connector and DB Writer node are pre-configured. Execute them.


Note. When deploying on KNIME Business Hub, you'll use the PostgreSQL database.



Step 1 : Capture the workflow segment using Integrated Deployment

  1. Capture the Data Transformation segment of the workflow

    • After reading the data, add the Capture Workflow Start node to begin the workflow capturing process

    • Use the Capture Workflow End node after the Summarize TimeStamp-Hour component to end the capturing process



Development Database


Data Transformation

Session 1 - Preparing a Data Pipeline for Deployment

Exercise 01 Development data pipeline - Integrated deployment

Learning objective: In this exercise you'll practice using the Integrated Deployment extension to capture the data transformation workflow segment and save it as a new workflow


Workflow description: This workflow reads a sample of restaurant orders data, cleans and transforms it, and loads the transformed data to the development database. Besides, it captures the data transformation segment and saves it as a separate Data Transformation workflow.


You'll find the instructions to the exercises in the yellow annotations.

Read Data from Recent Week

Step 3 : Save the Data Transformation workflow


  1. In the Workflow Writer node:

    • Use the workflow relative path to write the workflow to /00 Deployment workflows folder

    • Enable overwriting of the workflow

    • Use custom workflow name: Data_Transformation

    • In the Inputs and outputs tab:

      • Select Workflow Service Input and Workflow Service Output nodes for the input and output nodes to be added automatically to the newly created workflow

  2. Explore the automatically created workflow



"Price" Column to Number
String to Number
Summarize Timestamp-Hour
"Table ID" Column to Number
String to Number
Append:orders_cleaned
DB Writer
by time, restaurant
Sorter
Date&Time
String to Date&Time
Remove Duplicates
Duplicate Row Filter
Filter relevant cols
Column Filter
Sample data
CSV Reader
SQLite Connector

Nodes

Extensions

Links