Icon

01_​Document_​clustering

Importing, preprocessing, and clustering of textual data
Document Clustering Example The goal of this workflow is to cluster a set of newsgroup documents into their corresponding topic. Thedata is taken from the 20 newsgroups dataset[1]. The workflow starts with a data table containing somenewsgroup documents, divided into two categories, politics.guns and sport.baseball. First, the data are converted into documents, whose category is the class politics or sport. The documentsare then preprocessed by filtering and lemmatizing. After that, the documents are transformed into a bag ofwords, which is filtered again. Only terms that occur at least in 1% of the documents (at least in 2documents) will be used as features and not be filtered out. Then the documents are transformed intodocument vectors.The document vectors are a numerical representation of documents and are in the following used forhierarchical clustering based on Manhattan, Euclidean, and Cosine distance measures.[1] http://qwone.com/~jason/20Newsgroups/ Data Import Preprocessing Transformation Clustering ManhattanEuclideancolor by category (class)Cosinek=2Distance thresholdBased on documentfrequencyVector creationConvert todocumentsNews groupsdatasetNode 813 Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Distance MatrixCalculate Distance MatrixCalculate Punctuation Erasure N Chars Filter Number Filter Case Converter Color Manager Category To Class Distance MatrixCalculate Column Filter HierarchicalCluster View Hierarchical Clustering(DistMatrix) k-Medoids HierarchicalCluster Assigner Term Filtering Document Vector Markup Tag Filter Stanford Lemmatizer Strings To Document Column Filter Table Reader Stop Word Filter Document Clustering Example The goal of this workflow is to cluster a set of newsgroup documents into their corresponding topic. Thedata is taken from the 20 newsgroups dataset[1]. The workflow starts with a data table containing somenewsgroup documents, divided into two categories, politics.guns and sport.baseball. First, the data are converted into documents, whose category is the class politics or sport. The documentsare then preprocessed by filtering and lemmatizing. After that, the documents are transformed into a bag ofwords, which is filtered again. Only terms that occur at least in 1% of the documents (at least in 2documents) will be used as features and not be filtered out. Then the documents are transformed intodocument vectors.The document vectors are a numerical representation of documents and are in the following used forhierarchical clustering based on Manhattan, Euclidean, and Cosine distance measures.[1] http://qwone.com/~jason/20Newsgroups/ Data Import Preprocessing Transformation Clustering ManhattanEuclideancolor by category (class)Cosinek=2Distance thresholdBased on documentfrequencyVector creationConvert todocumentsNews groupsdatasetNode 813 Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Distance MatrixCalculate Distance MatrixCalculate Punctuation Erasure N Chars Filter Number Filter Case Converter Color Manager Category To Class Distance MatrixCalculate Column Filter HierarchicalCluster View Hierarchical Clustering(DistMatrix) k-Medoids HierarchicalCluster Assigner Term Filtering Document Vector Markup Tag Filter Stanford Lemmatizer Strings To Document Column Filter Table Reader Stop Word Filter

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
On NodePit since: 2019-10-31
Last update: 2024-11-17
Created with KNIME version: v4.0.2
Tags: NLPNatural Language Processing

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!