Icon

Sentiment Analysis - LLM via API

Generative AI for Sentiment Analysis: Classifying Customer Reviews

Data Reading and Cleaning

CSV Reader:

The CSV Reader Node will import the dataset, which consists of Airline reviews sourced from Twitter.


Column Filter:

  • The top Column Filter Node will isolate the column containing the text to be classified.

  • The bottom Column Filter Node will retain the 'airline_sentiment' column, which will be used to evaluate how accurately the LLM classified the tweets as positive, negative, or neutral.


LLM-Based Processing

OpenAI Nodes:

  • Enter the API Key into the 'Password' field of the Credential Configuration Node.

  • The LLM Prompter Node will classify the Twitter reviews based on the prompt provided by the Column Expressions Node.

Column Expression:

This node contains the prompt for the LLM Prompter, providing instructions and examples on how to classify the sentiment of each tweet.

Data Cleaning

The nodes in this section all contribute to cleaning the output from the LLM's sentiment analysis

  1. Column Expressions:

    • The output from the LLM Prompter is as follows: "Sentiment Classification: {positive/negative/neutral}"

    • Removes "Sentiment Classification:" and replaces it with ""

  2. String Cleaner:

    • Removes the leading white space from the replacement action in the Column Expressions Node.

  3. Column Filter:

    • Keeps only the classification of the twitter review. (i.e. positive/negative/neutral)

  4. Joiner:

    • Merges both tables, allowing the 'airline_sentiment' column and the 'Response' column (containing the LLM-based classification) to be compared and evaluated for accuracy.

Evaluation

The Scorer Node will evaluate the accuracy by calculating the true positives, true negatives, false positives, and false negatives between the LLM output and the 'airline_sentiment' column.


From the Virgin Airlines dataset of tweets, the LLM is classified the tweets correctly 82% of the time.

Removes Whitespace
String Cleaner
Accuracy: ~83%
Scorer
Row Filter
Joins Both Tables
Joiner
Examine mismatches
Table View
OpenAI Chat Model Connector
[Evaluation] Keeps Airline_SentimentColumn
Column Filter
Prompt
Expression
Keeps ResponseColumn
Column Filter
Match =1Non-match = 0
Rule Engine
Sentiment Analysis
LLM Prompter
Keepzeroes
Row Filter
Tweets ForAmerican Airline Companies
CSV Reader
OpenAI Authenticator
Removes"Sentiment Classification:"
Expression
Filters For TheReview Column
Column Filter
Insert Credentials
Credentials Configuration

Nodes

Extensions

Links