02 Beispiel Sentiment Analysis über LLM Generative AI

Generative AI for Sentiment Analysis: Classifying Customer Reviews

This workflow uses a Kaggle Dataset including thousands of customer social media posts towards six US airlines. Contributors annotated the valence of the tweets as positive, negative and neutral. The generative AI-based approach prompts to an LLM, requesting to classify the airline reviews. Here, prompt engineering plays a key role. The response returned by the LLM Prompter node is then cleaned (post-processing) and then the GenAI-based predictions are evaluated.

Data Reading and Cleaning

CSV Reader:

The CSV Reader node will import the dataset, which consists of Airline reviews sourced from Twitter.

Column Filter:

• The top Column Filter node will isolate the column containing the text to be classified.

• The bottom Column Filter node will retain the 'airline_sentiment' column, which will be used to evaluate how accurately the LLM classified the tweets as positive, negative, or neutral.

LLM-Based Processing

OpenAI Nodes:

• Enter the API Key into the 'Password' field of the Credential Configuration node.

• The LLM Prompter node will classify the Twitter reviews based on the prompt provided by the Column Expressions Node.

Expression:

This node contains the prompt for the LLM Prompter, providing instructions and examples on how to classify the sentiment of each tweet.

Data Cleaning

The nodes in this section all contribute to cleaning the output from the LLM's sentiment analysis

1. Expression:

   • The output from the LLM Prompter is as follows: "Sentiment Classification: {positive/negative/neutral}"

   • Removes "Sentiment Classification:" and replaces it with ""

2. String Cleaner:

   • Removes the leading white space from the replacement action in the Column Expression node.

3. Column Filter:

   • Keeps only the classification of the Twitter review. (i.e. positive/negative/neutral)

4. Joiner:

   • Merges both tables, allowing the 'airline_sentiment' column and the 'Response' column (containing the LLM-based classification) to be compared and evaluated for accuracy.

Evaluation

The Scorer node will evaluate the accuracy by calculating the true positives, true negatives, false positives, and false negatives between the LLM output and the 'airline_sentiment' column.

From the Virgin Airlines dataset of tweets, the LLM is classified the tweets correctly 82% of the time.