Document Vector Hashing

This node creates a document vector for each document representing it in the terms space. The values of the feature vectors can be specified as boolean values or as values of either the relative frequency or the absolute frequency of the terms. The advantages of using this node instead of the normal document vector node is that the dimension of the vectors is always fixed and therefore this node is streamable.

Options

Document column

The column containing the documents to use.

Dimension

The dimension of the output document vector. The bigger the dimension, the less likely collisions would tend to happen. However, be aware of the curse of dimensionality.

Seed

Seed for the hashing function.

Hashing function

Choose which hashing function should be used to hash the document terms.

Vector type

There are three ways to fill the values in the document vector.
Binary : The vector will be a bit vector.
TF-Absolute : At each index where a term is hashed to, the value of the absolute term frequency of that term will calculated and stored at the index.
TF-Relative : At each index where a term is hashed to, the value of the relative term frequency of that term will calculated and stored at the index.

As collection cell

If checked all vector entries will be stored in a collection cell consisting of double cells. If not checked all double cells will be stored in corresponding columns. The advantage of the column representation is that most of the regular algorithms in KNIME can be applied. The disadvantage is (which is on the other hand the advantage of the collection representation) that processing of subsequent nodes will be slowed down, due to the many columns that will be created (dependent on the input data of course).

Input Ports

The input table containing the documents.

Output Ports

An output table containing the input documents with the corresponding document vectors.

The model output containing the specifications that have been used for document vector creation.

Best Friends (Incoming)

• Strings To Document (21 %) Streamable
• TF (8 %) 
• Document Vector (8 %) 
• Document Vector Applier (6 %) 
• String Manipulation (5 %) Streamable
• Show all 41 recommendations

Best Friends (Outgoing)

• Document Vector Hashing Applier (15 %) Streamable
• Category To Class (14 %) 
• Document Vector Applier (9 %) 
• PCA (6 %) 
• Row Filter (5 %) Streamable
• Show all 35 recommendations

Workflows

• 03_​Streaming_​DocumentVector (KNIME Hub) 
• 13_​DocumentVector_​Hashing (KNIME Hub) 
• Exercise 3. DocumentVectorHashing_​Creation_​on_​Large_​Movie_​Review_​Dataset (KNIME Hub) 

Installation

To use this node in KNIME, install KNIME Textprocessing from the following update site:

A zipped version of the software site can be downloaded here.

You don't know what to do with this link? Read our NodePit Product and Node Installation Guide that explains you in detail how to install nodes to your KNIME Analytics Platform.

Developers

You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.