Icon

Sentiment Analysis - Predictor from Scratch

<p>Building a Sentiment Analysis Predictive Model - Supervised Machine Learning</p><p>This workflow uses a Kaggle Dataset (https://www.kaggle.com/crowdflower/twitter-airline-sentiment) including thousands of customer social media posts towards six US airlines. Contributors annotated the valence of the tweets as positive, negative and neutral. Once users are satisfied with the model evaluation, they should export (1) the Vector Space and (2) the Trained Model for deployment over non-annotated data.</p><p><br>If you use this workflow, please cite:

1. Read annotated dataset. Besides the node to read CSV files below, KNIME provides a wide range of nodes to read different datastet formats (e.g., parquet, json, images etc.).

2. Data Manipulation/Preparation. Simplified data manipulation process. Users working on text mining might want to add a "Spell Checker" node to handle grammar issues (e.g., happy = hapy). Here the most important node is "strings to document", which formats several string columns (e.g., author, text, title) into a single document that can be text-mined in KNIME.

3. Use of Text Mining to Transform Data into Numbers. Enrichment means using dictionaries (e.g., LIWC) to tag words into determined categories. This might serve for purposes such as making sure these words are not removed, or to create intensity measures (word category percentages) per document. Preprocessing allows users to simplify the analysis by (1) removing punctuation and stopwords, (2) performing steeming, and (3) executing other preprocessing tasks. Based on the preprocessed documents, a document vector is created to represent each document in a vector space. This blog post describes different document encoding options https://www.knime.com/blog/text-encoding-a-review

4. Use and Evaluation of Supervised Machine Learning. XGBoost is used as an example in this workflow.

Building a Sentiment Analysis Predictive Model - Supervised Machine Learning

This workflow uses a Kaggle Dataset including thousands of customer social media posts towards six US airlines. Contributors annotated the valence of the tweets as positive, negative and neutral. Once users are satisfied with the model evaluation, they should export (1) the Vector Space and (2) the Trained Model for deployment over non-annotated data.

Common steps (e.g., lowercase, stopwords, etc.)
Enrichment and Preprocessing
Transform words to term frequency or other measures
BoW and Vector Space
75% (3 classtask)
Scorer
Convert strings to to documents
Strings to Document
Save models of each algorithm for production
Supervised Machine Learning
Kaggle DatasetSocial media posts fromconsumers toairlines. The posts arefrom Twitter, from 2015.
CSV Reader
Duplicate Row Filter
Export Vector Space for deployment
Model Writer

Nodes

Extensions

Links