NLP
Sentiment Analysis
Score text polarity with VADER and transformers.
By EZ4Code Team
sentimentvadertransformers
Code
import nltk
from nltk.sentiment.vader import SentimentIntensityAnalyzer
nltk.download("vader_lexicon", quiet=True)
sia = SentimentIntensityAnalyzer()
text = "I absolutely love this product! Best purchase ever."
print(sia.polarity_scores(text))
for text in ["It was okay.", "This is terrible and broken.", "I love it!"]:
print(text, sia.polarity_scores(text)["compound"])
# HuggingFace transformer sentiment
from transformers import pipeline
clf = pipeline("sentiment-analysis",
model="distilbert/distilbert-base-uncased-finetuned-sst-2-english")
results = clf(["I love this!", "I hate this."])
for r in results:
print(r)
# TextBlob polarity
from textblob import TextBlob
print(TextBlob("The movie was not bad at all.").sentiment)Explanation
VADER is a rule-based analyzer tuned for social media text that returns a compound score from -1 to 1. HuggingFace sentiment pipelines use fine-tuned transformers that generalize better to longer or nuanced text. TextBlob offers a quick polarity/subjectivity score for prototyping.
More NLP Snippets
Tokenization
Tokenize text with NLTK, spaCy, and regex.
Stopwords
Remove common words with NLTK and spaCy.
Stemming and Lemmatization
Reduce words to roots with Porter, Snowball, and lemmatizers.
TF-IDF
Vectorize text with TF-IDF and n-grams.
Word2Vec
Train and use word embeddings with gensim.
Named Entity Recognition
Extract entities with spaCy and transformers.