Icon

01_​TRAIN

The monitor decides and the monitor calls. 02_MONITOR sees the drift and starts 01_TRAIN itself. The registry/ folder is the only thing the two of them share.

1 Data

Sixteen sensors, one number each. train_upto is the last batch this run may learn from.

Read it, cut it at train_upto

2 Label and split

One question: is this reading Acetone. Stratified 70 / 30 on it, seed 42.

70 % to learn on, 30 % to be judged on

3 Two fits, on purpose

Fit A on the 70 % gives the accepted accuracy; fit B on the whole range is what ships.

The number you report and the model you ship

4 Register

One model file and one registry row. Nothing else leaves this workflow.

Hand it to MLOps: a file and a row

One workflow, five versions. Four of the five are started by 02_MONITOR through Workflow Service Input, and nothing in here knows that. Send nothing and the declared workflow variables are used instead, which is why it still runs on its own.

It makes its own registry. Both writers sit on the LOCAL file system and work their absolute path out at run time, so the folder, the model file and registry.csv all appear on a first press with no registry/ beside them.

Why two fits, why rows_trained is not rows_measured, why the solver tolerance is 1e-8 and why created_at is the one column verify.py skips: README.md.

Data: UCI Machine Learning Repository, record 224, "Gas Sensor Array Drift Dataset"https://doi.org/10.24432/C5RP6WVergara, A., Vembu, S., Ayhan, T., Ryan, M. A., Homer, M. L., & Huerta, R. (2012).Chemical gas sensor drift compensation using classifier ensembles.Sensors and Actuators B: Chemical, 166-167, 320-329.Collected January 2007 to February 2011 specifically to study sensor drift.data/gas_sensor_16.csv keeps one column per sensor - the normalised steady-stateresponse f002, f010 ... f122 - plus batch_id, row_id and gas_name.

The reference profile — the other thing this workflow registers

Accuracy needs labels and labels are weeks late, so every version also leaves profile_v<N>.csv behind: one row per sensor — mean and sd — plus the share of "yes" the deployed model gave on its own training window. 02_MONITOR measures the next batch against it with no labels at all. Why Unpivoting: README.md.

Node 113,910 rows16 sensors, 10 batches
CSV Reader
Node 2Keep batches 1 .. train_upto
Rule-based Row Filter
Node 19feature, mean, sd, share
Expression
Node 3The label: Acetone, yes or no
Expression
Node 4Stratified 70 / 30, seed 42
Table Partitioner
Node 5Size of the whole range
Extract Table Dimension
Node 6Fit A: the 70 %to be measured
Logistic Regression Learner
Node 7Score the held-out 30 %
Logistic Regression Predictor
Node 20One row per sensor
GroupBy
Node 10Fit B: all of itto be deployed
Logistic Regression Learner
Node 21registry/profile_v<N>.csv
CSV Writer
Node 8The registry row, in one node
Expression
Node 13The variables already here
Extract Context Properties
Node 9Roll it up to one row
GroupBy
Node 14What a caller may send in
Workflow Input
Node 11registry/model_v<N>.model
Model Writer
Node 12Append to registry.csv
CSV Writer
Node 17Fit B scores its own window
Logistic Regression Predictor
Node 1816 columns -> 16 rows
Unpivot
Node 16Where am I? Absolute paths
Variable Expression

Nodes

Extensions

Links