Icon

04 Cleaning and Filtering

04 Cleaning and Filtering
Exercise: Document Cleaning and FilteringIn this exercise you'll reduce and clean the Microsoft RSS feeds text.1) Execute the Tagged document metanode. It applies POS and NE tagging to the agenda text.2) Remove punctuation from the text3) Convert all tokens to lower case4) Filter out stop words using the built-in list and a custom list with the following words: "session", "word","agenda", "nofollow","<p>","</p>", "rel", "href", etc. Check the Ignore unmodifiable flag option.5) Filter the document to nouns (POS tags NN, NNP, NNPS and NNS). Check the Ignore unmodifiable flagoption.6) Exclude locations, organizations and "unknown" NE tags. Check the Ignore unmodifiable flag option.7) Inspect the result with the Tagged Document Viewer node. What are the first three tokens in the document?closing, cybersecurity, and skills remove punctuationto lowerstopwordsfilter tonounsexclude locations,organizations, and "unknown"NE tags Tagged document Punctuation Erasure Case Converter Stop Word Filter Table Creator Tag Filter Tag Filter Tagged DocumentViewer Exercise: Document Cleaning and FilteringIn this exercise you'll reduce and clean the Microsoft RSS feeds text.1) Execute the Tagged document metanode. It applies POS and NE tagging to the agenda text.2) Remove punctuation from the text3) Convert all tokens to lower case4) Filter out stop words using the built-in list and a custom list with the following words: "session", "word","agenda", "nofollow","<p>","</p>", "rel", "href", etc. Check the Ignore unmodifiable flag option.5) Filter the document to nouns (POS tags NN, NNP, NNPS and NNS). Check the Ignore unmodifiable flagoption.6) Exclude locations, organizations and "unknown" NE tags. Check the Ignore unmodifiable flag option.7) Inspect the result with the Tagged Document Viewer node. What are the first three tokens in the document?closing, cybersecurity, and skills remove punctuationto lowerstopwordsfilter tonounsexclude locations,organizations, and "unknown"NE tags Tagged document Punctuation Erasure Case Converter Stop Word Filter Table Creator Tag Filter Tag Filter Tagged DocumentViewer

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: Lada.Rudnitckaia
Created at: 2020-07-31
On NodePit since: 2022-03-29
Last update: 2024-04-28
Created with KNIME version: v4.5.2
Tags: Text miningText processingdocument cleaningnlpnatural language processingstop word filtercase convertertag filter

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!