Random Forest Learner

Learns a random forest*, which consists of a chosen number of decision trees. Each of the decision tree models is built with a different set of rows (records) and for each split within a tree a randomly chosen set of columns (describing attributes) is used. The row sets for each decision tree are created by bootstrapping and have the same size as the original input table. The attribute set for an individual split in a decision tree is determined by randomly selecting sqrt(m) attributes from the available attributes where m is the total number of learning columns. The attributes can also be provided as bit (fingerprint), byte, or double vector. The output model describes a random forest and is applied in the corresponding predictor node.

This node provides a subset of the functionality of the Tree Ensemble Learner corresponding to a random forest. If you need additional functionality please check out the Tree Ensemble Learner.

Experiments have shown the results on different datasets are very similar to the random forest implementation available in R.

The decision tree construction takes place in main memory (all data and all models are kept in memory).

The missing value handling corresponds to the method described here. The basic idea is that for each split to try to send the missing values in every possible direction; the one yielding the best results (i.e. largest gain) is then used. If no missing values are present during training, the direction of the split that the most records are following is chosen as the direction for missing values during testing.

Nominal columns are split in a binary manner. The determination of the split depends on the kind of problem:

  • For two-class classification problems the method described in section 9.4 of "Classification and Regression Trees" by Breiman et al. (1984) is used.
  • For multi-class classification problems the method described in "Partitioning Nominal Attributes in Decision Trees" by Coppersmith et al. (1999) is used.


(*) RANDOM FORESTS is a registered trademark of Minitab, LLC and is used with Minitab’s permission.

Options

Target Column
Select the column containing the value to be learned. Rows with missing values in this column are ignored during the learning process.
Attribute Selection

Select the attributes on which the model should be learned. You can choose from two modes.

Fingerprint attribute Uses a fingerprint/vector (bit, byte, and double are possible) column to learn the model by treating each entry of the vector as separate attributes (e.g. a bit vector of length 1024 is expanded into 1024 binary attributes). The node requires all vectors to be of the same length.

Column attributes Uses ordinary columns in your table (e.g. String, Double, Integer, etc.) as attributes to learn the model on. The dialog allows you to select the columns manually (by moving them to the right panel) or via a wildcard/regex selection (all columns whose names match the wildcard/regex are used for learning). In case of manual selection, the behavior for new columns (i.e. that are not available at the time you configure the node) can be specified as either Enforce exclusion (new columns are excluded and therefore not used for learning) or Enforce inclusion (new columns are included and therefore used for learning).

Enable Hightlighting (#patterns to store)
If selected, the node stores the selected number of rows and allows highlighting them in the node view.
Save target distribution in tree nodes (memory expensive - only important for tree view and PMML export)
If selected, the model stores the distribution of the target category values in each tree node. Storing the class distribution may increase memory consumption considerably and we therefore recommend disabling it if your use-case doesn't require it. Class distribution is only needed if
  • You want to see the class distribution for each tree node in the node view.
  • You want to export individual decision trees to PMML.
  • You want to use soft-voting (i.e. aggregation of probability distributions instead of votes) in the predictor node.
Split Criterion
Choose the split criterion here. Gini is usually a good choice and is used in "Classification and Regression Trees" (Breiman et al, 1984) and the original random forest algorithm (as described by Breiman et al, 2001); information gain is used in C4.5; the information gain ratio normalizes the standard information gain by the split entropy to overcome any unfair preference for nominal splits with many child nodes.
Limit number of levels (tree depth)
Number of tree levels to be learned. For instance, a value of 1 would only split the (single) root node resulting in a decision stump.
Minimum child node size
Minimum number of records in child nodes. In the original paper this is set to 1, which ensures that each learned tree fits its training data perfectly (that is if it doesn't contain equivalent rows with differing labels).
Number of models
The number of decision trees to be learned. For most datasets, a value between 100 and 500 yields good results but the optimal number is data dependent and should thus be subject to parameter tuning.
Use static random seed
Choose a seed to get reproducible results.

Input Ports

Icon
The data to be learned from. They must contain at least one nominal target column and either a fingerprint (bit-vector/byte-vector) column or another numeric or nominal column.

Output Ports

Icon
The input data with the out-of-bag predictions, i.e. for each input row this is the majority vote of all models that did not use the row during their training. The appended columns are equivalent to the columns appended by the corresponding predictor node. There is one additional column model count, which contains the number of models used for the voting (number of models not using the row throughout learning.) The out-of-bag predictions can be used to get an estimate of the generalization error of the random forest by feeding them into the Scorer node.
Icon
A statistics table on the attributes used in the different trees. Each row represents one training attribute with these statistics: #splits (level x) as the number of models, which use the attribute as split on level x (with level 0 as root split); #candidates (level x) is the number of times an attribute was in the attribute sample for level x (in a random forest setup these samples differ from node to node). If no attribute sampling is used #candidates is the number of models. Note, these numbers are uncorrected, i.e. if an attribute is selected on level 0 but is also in the candidate set of level 1 (but will not be split on level 1 because it has been split one level up), the #candidate number will still count the attribute as candidate.
Icon
The trained model.

Popular Predecessors

Views

Tree Views
An decision tree viewer for all the trained models. Use the spinner to iterate through the different models.

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.