Icon

Data Blending

Starting from different data sources this workflow shows how to blend data (extract value from multiple data sources) and how to make efficient ETL (Extract, Transform and Loading) with KNIME Analytics Platform.

URL: KNIME's Learning page http://www.knime.com/learning

Data Blending and ETL

This workflow illustrates how to blend different data sources within KNIME Analytics Platform. The resulting blended table is the source of the Bar Chart node used to visualize the average age of the customers for each product.

Find more information on KNIME’s Learning page at http://www.knime.com/learning (courses, tutorials, cheatsheets, books, and more)

Task

Blend together various data sources in an effective data integration process using ETL (Extract, Transform and Load). Visualize the final result.

Data Reading

6 data sources:

- CSV file - Webdata Old System.csv (Old Web System)

- SQLite Database - WebActivity.sqlite (New Web System)

- .table file - Sentiment Analysis.table

- CSV file - Sentiment Rating.csv

- CSV file - Demographics.csv

- Excel file - Prooduct Data2.xls

The files are located in TheData/Misc/

Data Blending & ETL

- Concatenate web activity data from old and new systems

- Replace sentiment labels (text) with numeric scores

- Set all product names to lowercase

- Join all data together by the customerKey column

- Group by products, and aggregate by age (mean)

Visualize

Create a bar chart showing the average age of customers for each product

Reading WebActivity.sqlite
DB Query Reader
Add Demographics data
Joiner
Joinweb activity &sentiment scores
Joiner
Exclude Sentiment Rating and Web Activity
Column Filter
Reading SentimentAnalysis.table
Table Reader
Products in lower case
String Manipulation
Add product data
Joiner
Replace sentimentlabels with numeric sentiment ratings
Value Lookup
Concatenate the two data tables
Concatenate
RenameFirst(WebActivity)as WebActivity
Column Renamer
Reading Demographics.csv
CSV Reader
Connect to SQL database
SQLite Connector
Calculate average customer agebyproduct: blended table
GroupBy
Reading ProductData2.xls
Excel Reader
Reading WebdataOldSystem.csv
CSV Reader
Reading SentimentRating.csv
CSV Reader
Visualize average age vs.product
Bar Chart

Nodes

Extensions

Links