Icon

wkflw_​dd_​check_​lz

1. Source file checks File-name parsing: <market>-<ss prefix>-<file type index>-<ingestion type code>-<storage area>-<layout type code> <market>-<ss prefix>-<file type index>-<ingestion type code>-<storage area>-<layout type code>-cnt Sample data files RO-MSD-2-DBBTCH-LZ-1-20220127174355.txt RO-MSD-2-DBBTCH-LZ-1-20220127174355-cnt.txt RO,MSD => Ref-table-1 (market, ss-prefix) 2 => Ref-table-5 (File-Type-code) DBBTCH => Ref-table-2 (ingestion-Type-code) LZ => Ref-table-1(Storage-area) 1 => Ref-table-3 (Layout-Type-code) Acceptance criteria: All the parts of the file name delimited by "-" should match with the main column of the reference tables mentioned against each part. 2. Data Dictionary checks if (Ref-table-3 (Layout-Type-code)) == 1 { CSV file: first line = header => read the field names from row-1 Acceptance Criteria for field-name: Field-names from the header should match field names from the Ref-table-5 get the length of each field Acceptance Criteria for field-length: field length computed should not exceed field length in the Ref-table-5
String Manipulation
String Widget
Ref_table_3
Table Reader
Duplicate Row Filter
meta_dwnld_extract_split_filename_lz
meta_conn_s3_bkt_lz
Column Filter
Rule-based Row Splitter
Ref-table-3 (Layout-Type-code)
Reference Row Filter

Nodes

Extensions

Links