This node analyses documents and extracts relevant keywords
using cooccurrence statistics as described in
"Keyword extraction from a single document using word co-occurrence
statistical information" by Y.Matsuo and M. Ishizuka.
First, the most frequent terms (see node settings) are extracted and
then clustered together using the pointwise mutual information and
a normalized version of the L1 norm as measures of distance between
their cooccurrence probability distributions.
A term can be considered as member of a cluster if it is similar to
all the terms inside it according to at least one of the similarity
measures. If more than one cluster meets this condition, the
one with the highest average score will be used. If no cluster
is similar, a new one is created.
Once this is done, each term is ranked
in decreasing order of the deviation between their expected cluster
cooccurrence and the actual observed cooccurrence value. The terms
with the highest divergence are returned as keywords.
Setting the console's output level to DEBUG will make this node
display the set of frequent terms, the distance between them during
the clustering phase and the final clusters.
terms.
Options
Popular Successors
Document vector8 %- Sorter4 %
- Reference Row Filter4 %
- Joiner3 %
- CSV Writer3 %
- Interactive Table (legacy)3 %
- Document Vector2 %
- Document Viewer2 %
- Tag Cloud2 %
CSV Writer2 %- IDF2 %
- TF2 %
- String Matcher2 %
Joiner1 %- Set Operator1 %
- Top k Row Filter1 %
- Term Document Entropy1 %
- Bag Of Words Creator1 %
- Document Data Assigner1 %
- Term Neighborhood Extractor1 %
- Term to String1 %
- Hierarchical Clustering1 %
- Scorer1 %
Row Filter (deprecated)1 %- GroupBy1 %
- Partitioning1 %
- Value Counter1 %
- Color Manager1 %
Excel Writer (XLS)1 %- Frequency Filter1 %
- Topic Extractor (Parallel LDA)1 %
- Keygraph Keyword Extractor1 %
- Tag Cloud (JavaScript)1 %
Table Writer< 1 %- Constant Value Column< 1 %
- Column Filter< 1 %
- Excel Writer< 1 %
- Word Vector Apply (legacy)< 1 %
- NGram Creator< 1 %
- Tags to String< 1 %
- t-SNE (L. Jonsson)< 1 %
- Decision Tree Learner< 1 %
- Column Appender< 1 %
- N Chars Filter< 1 %
- Tag Filter< 1 %
- Document Data Extractor< 1 %
- Meta Info Inserter< 1 %
- Hierarchical Cluster Assigner (local)< 1 %
- Key-Collection HiLite Translator< 1 %
Column Rename< 1 %- GroupBy Bar Chart (JFreeChart)< 1 %
Excel Writer (XLS)< 1 %- Term Co-Occurrence Counter< 1 %
- DF< 1 %
- Category to Class< 1 %
Bag of Words Creator< 1 %Document Data Extractor< 1 %- Document Vector Hashing< 1 %
- Term Vector< 1 %
- Table View (JavaScript)< 1 %
Database Writer< 1 %- Java Snippet< 1 %
Loop End< 1 %- k-Means< 1 %
- Decision Tree Predictor< 1 %
Scorer< 1 %- SVM Predictor< 1 %
- Concatenate< 1 %
Concatenate (Optional in)< 1 %- Cell Splitter< 1 %
- Number to String< 1 %
Column Appender< 1 %- Pivot< 1 %
- Rank< 1 %
- String Manipulation< 1 %
- Rule-based Row Filter< 1 %
- Empty Table Switch< 1 %
- 3D Scatter Plot (Plotly)< 1 %
- Column Expressions (legacy)< 1 %
Table to PDF< 1 %- DL4J Feedforward Learner (Classification) (legacy)< 1 %
- HeatMap (JFreeChart)< 1 %
- Rule Viewer (local) (Legacy)< 1 %
XLS Writer< 1 %Word Vector Learner (legacy)< 1 %- ICF< 1 %
- StanfordNLP Open Information Extractor< 1 %
- Brat Document Writer< 1 %
- Chi-Square Keyword Extractor< 1 %
- Number Filter< 1 %
- RegEx Filter< 1 %
Snowball Stemmer< 1 %- Stanford Lemmatizer< 1 %
Stop word Filter< 1 %- Dictionary Tagger< 1 %
- Stanford Tagger< 1 %
- StanfordNLP NE Learner< 1 %
- Document Vector Applier< 1 %
Document vector hashing< 1 %- Sentence Extractor< 1 %
- Strings to Document< 1 %
- Unique Term Extractor< 1 %
- Table Editor (JavaScript)< 1 %
- Text Classifier Learner< 1 %
Views
- This node has no views
Workflows
01_Emil_the_TeacherBotKNIME Hub- 01_Initial_Model_TrainingKNIME Hub
- 02_AL_First_Try_Assign_Classes_via_DistanceKNIME Hub
- 02_Document_ClassificationKNIME Hub
- 02_Document_ClassificationKNIME Hub
- Show all 14 workflows
Links
- No links available
Developers
You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.