Increasing Accuracy of Sentiment Classification Using Negation Handling
A novel approach towards increasing accuracy of Sentiment Classification quickly and efficiently
Introduction
The layperson’s notion is that the accuracy of a Sentiment Classification model is mainly dependent on the quality and functionality of the ML/DL model itself. However, this is not entirely true. The quality of the data and its usability for a model is the main defining characteristic of any prediction driven task. Data Scientists all over the world spend at least 80% of their time in Data Wrangling (as said by Andrew Ng, CEO of Landing AI). Exploratory Data Analysis, Feature Extraction, removing noise, choosing the right model for the data and then making the data ready as input for the model are the most time-taking steps in any process aiming to derive insights from data. Moreover, these take place even before any Machine Learning or Deep Learning algorithms are chosen, which can get extremely complex in nature.
The research done here shows that after employing Negation Handling (which increases the quality of data) the accuracy of Sentiment Classification using simple algorithms like Logistic Regression and Naive Bayes classifier show comparable results to complex Word Embedding models.
Methodology
The function for the negation handler is available at my Github repo. An example of the function output is shown below. ‘Negation’ is the main function being called on the tokenized sentence as shown.
In the function, whenever a negation word (like ‘not’, “n’t”, ‘non-‘, ‘un-‘, etc) is encountered, a set of cognitive synonyms called synsets are generated for the word next to the negation. These synsets are interlinked by conceptual semantic and lexical relations to each other in a lexical database called WordNet. WordNet is a part of the NLTK python library. After the synsets are created, the method checks if an antonym of that word exists in the WordNet.
If an antonym doesn’t exist, it implies that the word is either a verb or an entity that has no opposite word available in the WordNet. So, it is left as it is, and the context of the sentence is preserved. But if an antonym exists in the lexical database, then a list of antonyms is created. Then the dissimilarity coefficient of each of these antonyms are found using the function, wup_similarity. The dissimilarity coefficient of two words would be: -
dissimilarity = (1 — word1.wup_similarity(word2))
Out of all the antonyms, one with the highest dissimilarity is considered and the first word next to the Negation is replaced with that antonym. Then the negation word is removed, and we get a sentence with inverted polarity as a result. Replacing with the antonym that has the maximum dissimilarity ensures that the maximum inversion of polarity takes place. This prevents words like “beautiful” being replaced by something like “bad”, which is an indirect antonym. Consider another example as shown below:
Now, What is a Synset?
It is a special sort of interface residing in the NLTK library which helps in looking up words in the WordNet lexical database. Synset instances are groupings of synonymous words that express the same concept. Using Synsets for negation handling gives a much simpler way to find relationships between different words than training complex models or using heavy duty word embeddings, which take much higher processing power and other resources.
Results and Analysis
Extensive experimentation was conducted using Naïve Bayes classifier, Random Forest Classifier and GloVe. However, the goal here, is to highlight the impact of data quality on Sentiment Analysis even when using the simplest Machine Learning models. Therefore, we will be focusing on Logistic Regression and Naïve Bayes Classifier mainly in the following observations.
As seen in the above table, after employing the negation handling function, the accuracy increases. The main point to be noted here is that, only by improving the quality of the data, we were able to increase the accuracy of Sentiment Classification, without making any changes in the ML/DL pipeline.
The dataset used is the IMDB movie review dataset which has 50000 labelled movie reviews with Binary Sentiment. The dataset is available on Kaggle [source: Here]. Table 1 shows observations calculated using Logistic Regression.
As observed in the bar graphs above, models incorporating Vectorizers and simple algorithms like Logistic Regression show almost equal performance to Word Embedding models, when the negation handling function is employed.
Conclusion
The quality of data is one of the most important factors in Sentiment Classification. Good Data can give good results when combined with the appropriate model. For industrial applications of Sentiment Analysis, where a quick turnaround time is required with minimum cost and maximum resource utilization, using heavy and complex algorithms often incur a lot of overhead and resources. Instead of spending time and resources on complex ML and DL algorithms which lie figuratively in the last leg of a Data Pipeline, we should focus on the first leg of the Data Pipeline which is preparing the data and increasing the quality of data. The function implemented here does precisely the same and increases the data quality to an extent that the results are almost comparable to a complex algorithm but uses less resources and simpler Machine Learning models.
References —
- U. Lal and P. Kamath, “Effective Negation Handling Approach for Sentiment Classification using synsets in the WordNet lexical database,” 2022 First International Conference on Electrical, Electronics, Information and Communication Technologies (ICEEICT), 2022, pp. 01–07, doi: 10.1109/ICEEICT53079.2022.9768641.
- Data Source(License: CC BY 4.0): Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning Word Vectors for Sentiment Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon, USA. Association for Computational Linguistics
