0 ×

01_​Document_​clustering

Workflow

Importing, preprocessing, and clustering of textual data
The goal of this workflow is to cluster a set of newsgroup documents into their corresponding topic. The data is taken from the 20 newsgroups dataset. The workflow starts with a data table containing some newsgroup documents, divided into two categories, politics.guns and sport.baseball. First, the data are converted into documents, whose category is the class politics or sport. The documents are then preprocessed by filtering and lemmatizing. After that, the documents are transformed into a bag of words, which is filtered again. Only terms that occur at least in 1% of the documents (at least in 2 documents) will be used as features and not be filtered out. Then the documents are transformed into document vectors. The document vectors are a numerical representation of documents and are in the following used for hierarchical clustering based on Manhattan, Euclidean, and Cosine distance measures.
NLPNatural Language Processingclustering
This workflow shows how to import textual data, preprocess documents by filtering and stemming, transform documents into a bag of words and document vectors, and finally cluster the documents based on their numerical representation. Data Import Preprocessing Transformation Clustering ManhattanEuclideancolor by category (class)Cosinek=2Distance thresholdBased on documentfrequencyVector creationConvert todocumentsNews groupsdataset Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Distance MatrixCalculate Distance MatrixCalculate Punctuation Erasure N Chars Filter Number Filter Case Converter Color Manager Category To Class Distance MatrixCalculate Column Filter HierarchicalCluster View Hierarchical Clustering(DistMatrix) k-Medoids HierarchicalCluster Assigner Term Filtering Document Vector Markup Tag Filter Stanford Lemmatizer Strings To Document Column Filter Table Reader Stop Word Filter This workflow shows how to import textual data, preprocess documents by filtering and stemming, transform documents into a bag of words and document vectors, and finally cluster the documents based on their numerical representation. Data Import Preprocessing Transformation Clustering ManhattanEuclideancolor by category (class)Cosinek=2Distance thresholdBased on documentfrequencyVector creationConvert todocumentsNews groupsdataset Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Column Filter Hierarchical Clustering(DistMatrix) HierarchicalCluster View Distance MatrixCalculate Distance MatrixCalculate Punctuation Erasure N Chars Filter Number Filter Case Converter Color Manager Category To Class Distance MatrixCalculate Column Filter HierarchicalCluster View Hierarchical Clustering(DistMatrix) k-Medoids HierarchicalCluster Assigner Term Filtering Document Vector Markup Tag Filter Stanford Lemmatizer Strings To Document Column Filter Table Reader Stop Word Filter

Download

Get this workflow from the following link: Download

Resources

Nodes

01_​Document_​clustering consists of the following 35 nodes(s):

Plugins

01_​Document_​clustering contains nodes provided by the following 4 plugin(s):