Icon

demo-dqa-rules1

* Customer details e.g. name, date of birth, gender* Contact details e.g. Address, telephone number(s), email* Agreement holdings i.e. which customer owns which (new) product agreement* Relationships with other customers* Contact History, e.g. a summarised record of conversations* Social Media data e.g. tweets, comments* Agent details e.g. territory covered/* Organisational hierarchy e.g. visibility into agent’s portfolio information */* Dedicated contact centre agent to field agent relationship * Effective date of Change e.g. this could be used to provide transparency of which agent sold/inherited whichcustomer The SCV may also publish key information on behalf of other systems that may not have appropriate* Households* integration capabilities ??? * Summarised agreement information e.g. previous day balance* Customer involvement in fraudulent case activities* Calculated scores e.g. behavioural scores, credit scores CUSTOMER DATA TOPICS Sample data preparation for Data Validation demo Amazon Athena - Creation of ACIDbased tables in S3 bucketshttps://docs.aws.amazon.com/athena/latest/ug/work-with-data.html LANDING ZONE RAW ZONE VALIDATION - Phone Numbers ALERT HANDLING (WIP) VALIDATION -- Name(s) CLEANSING & STANDARDIZATION INGESTION LANDING ZONE Creating the raw data in SQL server ( DB Connection and DBConnection Table Writer) and extracting it into KNIIME workspace(DB Reader Node) SIMULATEDusing SQLServer NON-OPERATIONAL (currently)Depicted for academic purposes only) Node 1Node 224CRM Data from MS-DynamicsSQL Server databaseNode 226Node 227Node 228CRM Data from MS-DynamicsSQL Server databaseIncoming data with email-addressLocalizing the incoming data within KNIME workspaceIsolate the customers with Missing email IDsEmail ID Validation using regular expressionsRecord the missing email ID customer records to a delta tableConnect to AthenaNode 910Node 911Name Validation regular expressionsName Validation Cleansing/StandardizationAmazonAuthentication DB Table Selector Microsoft SQLServer Connector DB Column Filter DB Column Rename DB ConnectionTable Writer Microsoft SQLServer Connector DB Table Selector DB Reader Rule-basedRow Splitter String Manipulation DB Writer Amazon AthenaConnector DB Reader DB Writer String Manipulation String Manipulation * Customer details e.g. name, date of birth, gender* Contact details e.g. Address, telephone number(s), email* Agreement holdings i.e. which customer owns which (new) product agreement* Relationships with other customers* Contact History, e.g. a summarised record of conversations* Social Media data e.g. tweets, comments* Agent details e.g. territory covered/* Organisational hierarchy e.g. visibility into agent’s portfolio information */* Dedicated contact centre agent to field agent relationship * Effective date of Change e.g. this could be used to provide transparency of which agent sold/inherited whichcustomer The SCV may also publish key information on behalf of other systems that may not have appropriate* Households* integration capabilities ??? * Summarised agreement information e.g. previous day balance* Customer involvement in fraudulent case activities* Calculated scores e.g. behavioural scores, credit scores CUSTOMER DATA TOPICS Sample data preparation for Data Validation demo Amazon Athena - Creation of ACIDbased tables in S3 bucketshttps://docs.aws.amazon.com/athena/latest/ug/work-with-data.html LANDING ZONE RAW ZONE VALIDATION - Phone Numbers ALERT HANDLING (WIP) VALIDATION -- Name(s) CLEANSING & STANDARDIZATION INGESTION LANDING ZONE Creating the raw data in SQL server ( DB Connection and DBConnection Table Writer) and extracting it into KNIIME workspace(DB Reader Node) SIMULATEDusing SQLServer NON-OPERATIONAL (currently)Depicted for academic purposes only) Node 1Node 224CRM Data from MS-DynamicsSQL Server databaseNode 226Node 227Node 228CRM Data from MS-DynamicsSQL Server databaseIncoming data with email-addressLocalizing the incoming data within KNIME workspaceIsolate the customers with Missing email IDsEmail ID Validation using regular expressionsRecord the missing email ID customer records to a delta tableConnect to AthenaNode 910Node 911Name Validation regular expressionsName Validation Cleansing/StandardizationAmazonAuthentication DB Table Selector Microsoft SQLServer Connector DB Column Filter DB Column Rename DB ConnectionTable Writer Microsoft SQLServer Connector DB Table Selector DB Reader Rule-basedRow Splitter String Manipulation DB Writer Amazon AthenaConnector DB Reader DB Writer String Manipulation String Manipulation

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: anant
Created at: 2021-10-18
On NodePit since: 2021-11-14
Last update: 2024-04-27
Created with KNIME version: v4.4.2

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!