Gradient Boosted Trees Learner (Regression)

Options

Target column

Select the column containing the value to be learned. Rows with missing values in this column are ignored during the learning process.

Training attributes

Choose whether to derive attributes from a fingerprint vector column or from ordinary table columns.

• Use column: Select ordinary columns and configure include/exclude rules for training attributes.
• Use fingerprint: Expand a fingerprint (bit, byte, or double vector) column into individual attributes during training.

Fingerprint attribute

Use a fingerprint (bit, byte, or double vector) column to learn the model. Each entry of the vector is treated as a separate attribute. All vectors must share the same length.

Attribute selection

Missing value handling

Specify how missing values should be handled during training.

• XGBoost: If this is selected (it is also the default), the learner will calculate which direction is best suited for missing values, by sending the missing values in each direction of a split. The direction that yields the best result (i.e. largest gain) is then used as default direction for missing values. This method works with both, binary and multiway splits.
• Surrogate: This approach calculates for each split alternative splits that best approximate the best split. The method was first described in the book "Classification and Regression Trees" by Breiman et al. (1984). NOTE: This method can only be used with binary nominal splits.

Use mid-point splits

For numerical splits, use the mid-point between two class boundaries. Otherwise the split value corresponds to the lower boundary with a ≤ comparison.

Use binary splits for nominal columns

If selected, nominal columns also produce binary splits instead of multiway splits in which each nominal value corresponds to one child node.

Limit number of levels (tree depth)

Limit the maximal number of tree levels. When disabled the tree depth is unbounded. For instance, a value of 1 would only split the (single) root node resulting in a decision stump

Enable row sampling

Enable sampling of training rows for each individual tree. If disabled, each tree learner gets the full dataset, otherwise each tree is learned with a different data sample. A data fraction of 1 (=100%) chosen "with replacement" is called bootstrapping. For sufficiently large datasets this bootstrap sample contains about 2/3 different data rows from the input, some of which are replicated multiple times. Rows that are not used in the training of a tree are called out-of-bag.

Fraction of data to learn single model

Fraction of data rows to sample for learning each individual tree. A value of 1 means 100% of the data rows are sampled.

Sample with replacement

Draw sampled rows with replacement (bootstrap sampling). When disabled, rows are sampled without replacement.

Number of models

Number of decision trees to learn. Larger ensembles generally provide more stable results but increase runtime. For most datasets, a value between 100 and 500 yields good results; however, the optimal number is data dependent and should thus be subject to hyperparameter tuning.

Learning rate

The learning rate influences how much influence a single model has on the ensemble result. Usually a value of 0.1 is a good starting point but the best learning rate also depends on the number of models. The more models the ensemble contains the lower the learning rate has to be.

Alpha fraction

Alpha controls what percentage of the data will be considered as outliers. The higher Alpha the smaller the fraction of outliers. If Alpha is set to 1.0, the algorithm will consider no point to be an outlier. This is discouraged however because outliers can have fatal effects on regression.

Attribute sampling (columns)

Defines the sampling of attributes to learn an individual tree. This can either be a function based on the number of attributes (linear fraction or square root) or some absolute value. The latter can be used in conjunction with flow variables to inject some other value derived from the number of attributes (e.g. Breiman suggests starting with the square root of the number of attributes but also to try to double or half that number).

• All columns: Disable column sampling and use all available attributes for every tree.
• Square root: Sample the square root of the number of available attributes for each tree (default random forest behaviour).
• Linear fraction: Sample a fraction of the available attributes for each tree.
• Absolute number: Sample a fixed number of attributes for each tree.

Linear fraction

Fraction of attributes to sample for each tree.

Absolute number

Number of attributes to sample for each tree.

Attribute selection

Use the same set of attributes for each tree means that the attributes are sampled once for each tree and this sample is then used to construct the tree.

Use a different set of attributes for each tree node samples a different set of candidate attributes in each of the tree nodes from which the optimal one is chosen to perform the split. This is the option used in random forests.

Use static random seed

Provide a seed to obtain deterministic results. Leave disabled to use a time-dependent seed.

Generate new seed

Generate a random seed and apply it to the field above for reproducible runs.

Views

This node has no views

Workflows

• 02_AutoML_Regression_and_Classification_ExamplesKNIME Hub
• 02_AutoML_Regression_and_Classification_ExamplesKNIME Hub
• 02_AutoML_Regression_and_Classification_ExamplesKNIME Hub
• 02_SHAP_and_Shapley_ValuesKNIME Hub
• 02_SHAP_and_Shapley_ValuesKNIME Hub
• Show all 188 workflows

Links

• No links available

Developers