Spacy Tokenizer

This Node Is Deprecated — This version of the node has been replaced with a new and improved version. The old version is kept for backwards-compatibility, but for all new workflows we suggest to use the version linked below.
Go to Suggested ReplacementSpacy Tokenizer

The node converts a string column with raw text to a Knime Document format, using its own tokenizer based on the selected spaCy model.

Options

Select column
Select a String column that will be tokenized and converted to Document.
Replace column
If checked, the document column will be replaced by the new preprocessed documents. Otherwise the preprocessed documents will be appended as a new column.
Append column
The name of the new appended column, containing the preprocessed documents.

Model

spaCy model
Pick one of the official spaCy models, or refer to a custom model stored in the filesystem. In the latter case refer to a folder with meta.json and config files.

Python

Python
Select one of Python execution environment options:
  • use default Python environment for the Redfield NLP nodes
  • use Conda environment

Input Ports

Icon
The input table which contains the documents to preprocess.

Output Ports

Icon
The output table which contains the preprocessed documents.

Views

This node has no views

Workflows

Links

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.