Icon

07.02 Outlier Detection exercise

<p>07.02 Outlier Detection - exercise<br><br>[L4-ML] Machine Learning Algorithms - Specialization</p><p>07 Data Cleaning Preparation<br>Detect and remove outliers in the data using the following techniques:<br>- Numeric outliers outside the upper/lower whiskers of a box plot<br>- Outliers in the distribution tails (z-score)</p>

URL: Ames Housing Dataset on kaggle https://www.kaggle.com/prevek18/ames-housing-dataset
URL: Description of the Ames Iowa Housing Data https://rdrr.io/cran/AmesHousing/man/ames_raw.html

07 - Data Cleaning & Preparation

07.02 Outlier Detection

Learning objective: In this exercise you learn how to clean the data to build a better model, in particular by applying oulier detection techniques.


Workflow description: In this workflow, outlier detection will be implemented in both the training and test datasets, employing numeric outlier identification. Additionally, there is an option to normalize the data using the z-score.


You'll find the instructions to the exercises in the yellow annotations.

Step 1. Numeric Outliers

Remove houses that have a sales price lying outside the interquartile range of all sales prices (Numeric Outlier and node)

  • Select the SalePrice column

  • Set the interquartile range parameter to 1.5


Step 2. Outliers in Distribution Tails (Optional)

Remove the 5 % of the houses that are the most extreme in terms of size (Normalizer and Expression Row Filter nodes)

  • Normalize the Lot Area column using z-score

  • Filter out houses whose normalized lot size is outside the range [-1.96, 1.96]


Data Preparation


Missing Value Handling
Read AmesHousing.csv
CSV Reader
Preprocessing

Nodes

Extensions

Links