Spark Association Rule (Apply)

This node applies association rules created by a Spark rule learner*. The rules are applied to a given column with item sets (transactions). A rule matches an item set, if the item set contains all antecedent items of the rule. The consequent items of the matching rules are then added to the output set of predicted items if they are not already included in the input items collection (the output set contains only new items).

To avoid memory and computations explosions, the number of used association rules can be limited. A warning will be added to the node if the limit is reached and only the first n rules are used (this means randomly chosen on a unsorted set of rules).

The Spark Association Rule Learner Node can be used to generate the association rules. The Spark GroupBy or Spark SQL Node can be used to generate a collection of items as input.

Rules with missing values in the selected antecedent or consequent column and item rows with missing values in the selected item column are removed.

This node requires at least Apache Spark 2.0.

(*) RULE LEARNER is a registered trademark of Minitab, LLC and is used with Minitab’s permission.