Music genre analysis — Understanding emotions and topics in different genres
Understanding the topics in different music genres and creating simple applications for predictions and recommendations
Contributors: Dimple Singhania, Adwait Toro, Sushrut Shendre, Chavi Singal, Anuja Dixit
Introduction
Music is one of the things that connect people everywhere. Good music can be recognized globally regardless of the language of the music lyrics. But even though the music has no language, the lyrics in the music plays an important role in defining the genre. It is essential to understand the themes behind the songs so that it can be leveraged by the upcoming singing talents to write a song which aligns with the current trending theme.
Many musical platforms such as Spotify, Amazon music have thrived on genre predictions using it to better recommend similar songs to their users. Spotify uses genre targeting in which it allows us to deliver the messages immediately after a user has listened to a song of a specific genre.
Therefore, we decided to build a model to predict the genre of a song from the lyrics.
Data
The dataset that we had consisted of around 100k records with 4 columns: Artist, Song, Lyrics and Genre.
EDA
To dig out some interesting facts about the data, we did some exploratory data analysis.
- To look at the sample for genres, the dataset was heavily skewed towards Rock (52k songs; 52%). R&B, Indie, Folk and Electronic covered only 6% of the data with close to 6,000 songs in total.
This bias in the data resulted in our model predicting Rock genre more accurately than other genres.
2. Next, we wanted to look at the top artist in the dataset. Surprisingly, even though Rock dominated the dataset, America’s iconic Country singer Dolly Parton topped the list.
3. Next, we wanted to look at how the length of characters and words differ for different genres. Hip-Hop had the highest average number of characters as well as words. This might be because of the reason that Hip-hop has rap songs which have a lot of words sung in a very short amount of time.
Custom features
First, we extracted some of the features from the lyrics that we thought would help in genre prediction.
- Emotions
We used an external dataset with music lyrics classified into different emotions. We trained a logistic regression classifier on this dataset and predicted the emotions for our given dataset. Below is the function for fitting the classifier on external data and downloading it as .pkl to use for predictions.
https://gist.github.com/chikorita5/7b70be4746e54a23bf4e227a4aeed63e
An alternative approach to extract the emotions can also be tried in the absence of external dataset to train. IBM watson tone analyzer API can be used to get the emotions.
https://gist.github.com/chikorita5/359625fe595dc0c0e34fe65e5d922f58
2. Number of words in the song lyrics
3. Number of lines in the song lyrics
Classification
First, we split our dataset into train and test. We created a simple Pipeline object used TFIDF vectorizer, custom features, and classifier. We used Logistic Regression, Decision tree, and Naive Bayes for prediction.
As you can see, Logistic had the highest accuracy. Decision tree had lower accuracy but it had a bit higher recall and f-1 score than logistic.
Oversampling
Since our data had imbalanced classes for genres. To increase the accuracy of the models, we oversampled the minority classes in the train dataset. We performed Naive Bayes and Logistic Regression again with the oversampled data and the performance increased quite a lot.
Performance of Naive Bayes model increased significantly after oversampling with an accuracy of 73% and recall of 74%. In the above confusion matrix, it can be seen that Rock genre is predicted incorrectly as Pop, country, and metal several times.
Logistic Regression along with CountVectorizer gave the highest accuracy after oversampling. Though, there is still some confusion between rock and pop genres.
Topic modeling
We also did some topic modeling to identify the frequent topics in different genres. For topic modeling, we used the Latent Dirichlet Allocation (LDA) from Gensim package along with the Mallet’s implementation (via Gensim). Mallet has an efficient implementation of the LDA. It is known to run faster and gives better topics segregation.
After applying topic modeling, we found that we had 6 dominant topics and the best coherence score for the model was 53% with a Perplexity score of -10. Perplexity is the measure that the model doesn’t get confused in classifying the documents into topics. The lower the perplexity score, the better is the model.
After looking at the keywords for the 6 topics, we decided that the genres had 6 underlying themes such as Flirting, Love, Youth, Sad, Happy/Dance, Devotion/Death. To depict these into a more interactive and user-friendly way, we created a Tableau dashboard which shows the themes and their associated keywords. The dashboard also has a provision to select your favorite artist, their genres and the song to see the underlying theme and the keywords associated along with the provision to alter the frequency of keywords.
For instance, we chose the artist as Akon with his song ‘Love you more’. It was classified to have a Sad theme beneath its Pop genre which was shown on the tableau dashboard.
Application
Using our final classifier, we created an application using Python’s tkinter library. Lyrics of any song can be pasted in the app to check its genre.
Recommender system
Many times, it happens that we listen to a song and we like it very much, but somehow we miss its name and only have one of its lyrical line lingering in our mind the whole day! In order to solve this problem, we used a distance measurement technique to find out the most similar songs to the one entered by the user. We provide a text box wherein the user can type out the lyrics of the song and the system will recommend the top 10 songs closest to the entered song in terms of lyrics. We designed this using the Jaccard similarity wherein first the user enters the song, then it is tokenized into a list of words and it is compared to every song lyrics present in the dataset. The similarity is calculated as the intersection of two tokenized lists/union of two lists and the songs with similarity close to 1 are displayed.
Github
https://github.com/chikorita5/nlp-genre-classification
References
- Bird, Steven, Edward Loper and Ewan Klein (2009), Natural Language Processing with Python. O’Reilly Media Inc
- https://towardsdatascience.com/the-triune-pipeline-for-three-major-transformers-in-nlp-18c14e20530
- Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825–2830, 2011
- https://www.machinelearningplus.com/nlp/topic-modeling-gensim-python/
