Icon

01 Taxi Demand Prediction on Spark Training

Taxi Demand Prediction on Spark Training

This workflow uses the NYC taxi dataset and a Random Forest to train a simple time series prediction model to predict taxi demand in the next hour based on data from past hours.

Given the large size of the dataset, we train and deploy the machine learning model on a Spark cluster. The KNIME Big Data Extension allows you to run a KNIME workflow on the big data platform you prefer, via in-database processing or via Spark.

Taxi Demand Prediction (Training)

This workflow trains a taxi demand prediction model on Spark

Connect to a big data environment
Connect to a real Spark Cluster
Explore the seasonalities
Preprocess the data
Train and test the model
Evaluate the model
lag=24
Spark Lag Column
Spark to Table
Spark Random Forest Learner (Regression)
Create Spark Context (Livy)
Data Exploration
Amazon Authenticator
lag=24
Spark Lag Column
Parquet to Spark
View line plot
Path to training set
Spark Predictor (Regression)
Split by date and time
Model Writer
Amazon S3 Connector
Spark Numeric Scorer
Create Local Big Data Environment

Nodes

Extensions

Links