Rule-based Row Filter (Dictionary)

This node takes a list of user-defined rules and tries to match them to each row in the input table. If the first matching rule has a TRUE outcome, the row will be selected for inclusion. Otherwise (i.e. if the first matching rule yields FALSE) it will be excluded. If no rule matches the row will be excluded. Inclusion and exclusion may be inverted, see the options below.

Each rule is represented by a row. To add comments, start a line in a (condition) cell with // (comments can not be placed in the same line as a rule). Anything after // will not be interpreted as a rule. Rules consist of a condition part (antecedent), which must evaluate to true or false, and an outcome (consequent, after the => symbol) which is either TRUE or FALSE.

If no rule matches, the outcome is treated as if it was FALSE.

Columns are given by their names surrounded by $, numbers are given in the usual decimal representation. Note that strings must not contain (double-)quotes (for those cases use the following syntax: /Oscar Wilde's wisdom: "Experience is simply the name we give our mistakes."/). The flow variables are represented by $${TypeCharacterAndFlowVarName}$$. The TypeCharacter should be 'D' for double (real) values, 'I' for integer values and 'S' for strings.

The logical expressions can be grouped with parentheses. The precedence rules for them are the following: NOT binds most, AND, XOR and finally OR the least. Comparison operators always take precedence over logical connectives. All operators (and their names) are case-sensitive.

The ROWID represents the row key string, the ROWINDEX is the index of the row (first row has 0 value), while ROWCOUNT stands for the number of rows in the table.

Some example rules (each should be in one row):

// This is a comment
$Col0$ > 0 => TRUE
When the values in Col0 are greater than 0, we select the row (if no previous rule matched with FALSE outcome).
$Col0$ = "Active" AND $Col1$ <= 5 => TRUE
You can combine conditions.
$Col0$ LIKE "Market Street*" AND
    ($Col1$ IN ("married", "divorced")
        OR $Col2$ > 40) => FALSE
With parentheses you can combine multiple conditions.
$Col0$ MATCHES $${SFlowVar0}$$ OR $$ROWINDEX$$ < $${IFlowVar1}$$ =>
    FALSE
The flow variables, table constants can also appear in conditions.

The following comparisons result true (other values are neither less, nor greater or equal to missing and NaN values):

  • ? =,<=,>= ?
  • NaN =,<=,>= NaN

Options

Dictionary mode
Choose how the rules table encodes the rules.
  • Rule: A single column contains the full rule expression including the outcome (e.g. "$Col$ > 0 => TRUE").
  • Condition and value: Two separate columns are used: one for the condition part of the rule and one for the outcome value.
Rule column
The column in the rules table that contains the full rule expressions (condition and outcome combined, e.g. "$Col$ > 0 => TRUE").
Condition column
The column in the rules table that contains the condition part of each rule (e.g. "$Col$ > 0").
Value column
The column in the rules table that contains the outcome value for each rule (e.g. "TRUE" or "FALSE").
Treat values starting with $ as references
When checked, values in the value column that start with $ will be treated as references (for example, column, flow variable, or table property references) rather than literal string values.
Filter behavior
Choose whether to output matching or non-matching rows.
  • Output matching rows: Rows where the first matching rule evaluates to TRUE are included in the output.
  • Output non-matching rows: Rows where the first matching rule evaluates to FALSE or no rule matches are included in the output.

Input Ports

Icon
Any data table from which to filter rows
Icon
Table containing the rules

Output Ports

Icon
Data table with the included rows

Popular Successors

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.