Rule-based Row Filter (Dictionary)

This node takes a list of user-defined rules and tries to match them to each row in the input table. If the first matching rule has a TRUE outcome, the row will be selected for inclusion. Otherwise (i.e. if the first matching rule yields FALSE) it will be excluded. If no rule matches the row will be excluded. Inclusion and exclusion may be inverted, see the options below.

Each rule is represented by a row. To add comments, start a line in a (condition) cell with // (comments can not be placed in the same line as a rule). Anything after // will not be interpreted as a rule. Rules consist of a condition part (antecedent), which must evaluate to true or false , and an outcome (consequent, after the => symbol) which is either TRUE or FALSE .

If no rule matches, the outcome is treated as if it was FALSE .

Columns are given by their names surrounded by $, numbers are given in the usual decimal representation. Note that strings must not contain (double-)quotes (for those cases use the following syntax: /Oscar Wilde's wisdom: "Experience is simply the name we give our mistakes."/). The flow variables are represented by $${ TypeCharacterAndFlowVarName }$$ . The TypeCharacter should be 'D' for double (real) values, 'I' for integer values and 'S' for strings.

The logical expressions can be grouped with parentheses. The precedence rules for them are the following: NOT binds most, AND, XOR and finally OR the least. Comparison operators always take precedence over logical connectives. All operators (and their names) are case-sensitive.

The ROWID represents the row key string, the ROWINDEX is the index of the row (first row has 0 value), while ROWCOUNT stands for the number of rows in the table.

Some example rules (each should be in one row):

// This is a comment
$Col0$ > 0 => TRUE
When the values in Col0 are greater than 0, we select the row (if no previous rule matched with FALSE outcome).
$Col0$ = "Active" AND $Col1$ <= 5 => TRUE
You can combine conditions.
$Col0$ LIKE "Market Street*" AND 
    ($Col1$ IN ("married", "divorced") 
        OR $Col2$ > 40) => FALSE
With parentheses you can combine multiple conditions.
$Col0$ MATCHES $${SFlowVar0}$$ OR $$ROWINDEX$$ < $${IFlowVar1}$$ =>
    FALSE
The flow variables, table constants can also appear in conditions.

The following comparisons result true (other values are neither less, nor greater or equal to missing and NaN values):

  • ? =,<=,>= ?
  • NaN =,<=,>= NaN

Options

Rules column
Name of the column in the second input table containing the rules or just the conditions if an outcome column is also selected. In case the rules are available during configuration, the type of the outcome column is shown at the right end of this configuration row.
=>
In case your rules are in two separate columns (condition and outcome), this should be the column containing the outcome values. (Ignored for the comment -starting with //- conditions. It should not contain missing values or other than the logical/boolean values or the TRUE/FALSE values in a String column.
Treat values starting with $ as references
When checked, the values in the string -outcome- column starting with $ are not treated as string constants with that value, but are tried to be parsed as references to flow variables, columns or table properties.
Include TRUE matches
A row will be included in the output if the first rule that matches has TRUE outcome. If the first matching rule has a FALSE outcome or no rule matches the row will be excluded.
Exclude TRUE matches
A row will be excluded in the output if the first rule that matches has TRUE outcome. If the first matching rule has a FALSE outcome or no rule matches the row will be included.
Errors
The errors found in the input table parsing the rules column
Warnings
The warnings found in the input table parsing the rules column

Input Ports

Icon
Any data table from which to filter rows
Icon
Table containing the rules

Output Ports

Icon
Data table with the included rows

Popular Predecessors

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.