Icon

Session 4

2 workflows in this folder and its subfolders

Icon
01 Data validation & Statistics Process Control

This workflow implements the input data validation and checks: Whether input data table is empty, Whether input data has all the required columns, Statistical properties of the input data, And raises error if any of the tests fails. TAGS: education,data engineering,data pipeline,ETL,deployment,integ…

Icon
02 Production data pipeline with data validation & logging

August 2023 · This workflow: Reads and validates the data from real data sources, Calls the developed and tested Data Transformation workflow to transform the raw orders data, Loads the transformed data to the production database, Besides, it logs the validation and transformation steps to a log file and sends an…