Icon

01 ML on Databricks Unity data

<p>This workflow demonstrates how to <strong>start a Databricks cluster</strong>, load data <strong>from Unity Catalog into a Spark DataFrame</strong>, <strong>train</strong> a decision tree <strong>model</strong> using Spark, and <strong>save the model</strong> for deployment.</p>

URL: Connect to all-purpose and job compute https://docs.databricks.com/en/compute/use-compute.html

01 Machine learning on Databricks Unity data

This workflow demonstrates how to start a Databricks cluster, load data from Unity Catalog into a Spark DataFrame, train a decision tree model using Spark, and save the model for deployment.

Upload Unity data to a Databricks Spark cluster and train a machine learning model on Spark. The KNIME Spark connectivity framework allows you to visually construct and execute Spark queries on a Databricks cluster.

Authenticate and connect to a Databricks Workspace

Connect to Databricks Unity Volumes

Connect with and start a Databricks all-purpose compute cluster for interacting with in Spark

Save the models for deployment.

Select datafor training
Spark Column Filter
Free up frees upcluster resources &avoid unnecessary costs
Destroy Spark Context
Select datafor training
Spark Row Filter
Preprocessing
Model Writer
Preprocessfor ML algorithm
Spark Missing Value
Spark Missing Value (Apply)
Apply
Spark Predictor (Regression)
Split intotraining andtesting set
Spark Partitioning
Evaluate
Spark Numeric Scorer
Import Unity datato a Spark DataFrame
Parquet to Spark
Select theworking directory
Databricks Unity File System Connector
Connect and start yourall-purpose computecluster on Databricks
Create Databricks Environment
Model
Model Writer
Train
Spark Decision Tree Learner (Regression)
Connect to yourDatabricks workspace
Databricks Workspace Connector

Nodes

Extensions

Links