Stop Word Filter

Filters all terms of the input documents, which are contained in the specified stop word list and/or in the second input table. The node provides built-in stop word lists for various languages. Additionally, stop words can be passed to the second input port. At least one stopword list (built-in or custom list) has to be selected in the dialog.

Options

Filter options

Use built-in list
Specifies if built-in stop word list is used or not. If checked the second input table will be ignored.
Stopword lists
Available built-in stop word lists. If "Use built-in list" is checked, the built-in stop word list to use can be specified here.
Use custom list
If checked, the custom stop word list from the second input port is used.
Stopword column
The string column from the second input port which contains one stop word per row.
Case sensitive
If checked the stop words are matched case sensitive otherwise not.

Preprocessing options

Document column
The column containing the documents to preprocess.
Replace documents
If checked, the documents will be replaced by the new preprocessed documents. Otherwise the preprocessed documents will be appended as new column.
Append column
The name of the new appended column, containing the preprocessed documents.
Ignore unmodifiable tag
If checked, unmodifiable terms will be preprocessed too.

Input Ports

Icon
The input table which contains the documents to preprocess.
Icon
The input table which contains a string column with one stop word per row.

Output Ports

Icon
The output table which contains the preprocessed documents.

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.