N-Gram Extractor

Options

Text input

Input column which contains text for which to create n-grams.

Drop input column

Enable to exclude the input column in the output table.

Set output column name (*)

Override the default name for the appended output column. Leave empty to auto-generate the name based on the feature settings

n-gram type

The type of n-grams to be used. n-grams can be created on character and word level.

Min. n-gram length

The minimum length of n-grams to create (i.e. the number of characters or words, depending on n-gram type).

Max. n-gram length

The maximum length of n-grams to create.

Min. term length

(Only effective for n-gram type “word”) The minimum length of a word n-gram in characters to be considered.

Max. term length

(Only effective for n-gram type “word”) The maximum length of a word n-gram in characters to be considered.

Max. term count

The maximum number of terms to extract from each document (useful to speed up processing of huge documents, or to reduce model size in general).

Case sensitive

Activate to treat text documents case sensitively (can improve accuracy in certain cases, but increases model size).

Border padding

Create padded character n-grams at text document’s beginning and end (e.g. for a document starting with “The”, and n-gram length 3, we additionally create features the “##T”, “#Th”. This setting can improve accuracy when classifying very short phrases, but increases model size).

Create skip-grams

When in word mode and n-grams length >= 3, additionally create skip grams. Skip grams allow to model gaps between word groups by leaving out words inside the n-gram. E.g. for the consecutive 3-gram “the quick brown”, the skip gram is “the brown”.

Language-specific settings

Language

The language to use for the language-specific processing (see below; only in case the n-gram type is “word”)

Remove stopwords

Removes stopwords based on a predefined stopword list for the given language.

Stem

Performs stemming using the Snowball stemmer.

Views

This node has no views

Workflows

• 10-K Parsing 2 PDFs rev 1KNIME Hub
• n-gram-extractor-31902NodePit Space
• Salud ExampleKNIME Forum
• Salud ExampleKNIME Forum
• Salud ExampleKNIME Forum

Links

• Ngram Creator splits tagged termsby izaychik63 on 2021-05-13

  forum.knime.com/p/123591
• Bigram for LDAby izaychik63 on 2021-08-31

  forum.knime.com/p/133230

Developers