The world’s leading publication for data science, AI, and ML professionals.

Lyrics by Taylor Swift ft. A.I

Taylor Swift's lyrical dataset has grown enormously over the past years, and GPT-2-simple has made it easier than ever to train it.

A.I writes Taylor Swift songs ft. GPT-2

A simple guide to training an A.I model to generate Taylor Swift lyrics

Photo by Raphael Lovaski on Unsplash
Photo by Raphael Lovaski on Unsplash

No one can deny just how commendable and powerful Taylor Swift is. Her influence runs deep and transcends generations. And as she grew in front of the public eye, we saw her music range from country to pop to electropop and very recently, with Folklore and Evermore, to indie and folk.

With nine full-length albums with their own defined eras, there is a fun dataset to work with here. This article will explore how to use GPT-2 to generate lyrics based on all her released songs until February 2021. I compiled the data with the help of azlyrics and decided to use Google Colaboratory and the GPT-2-simple python package to train this data.

Brief Technical Context

GPT-2 (Generative Pre-trained Transformer 2) is an open-source A.I based on transformer architecture. GPT-2-simple is a python package that wraps finetuning to text generation scripts from GPT-2 to make text generation easier. This python package by Max Woolf has made GPT-2 very accessible, and he has this superb blog detailing how it works and how to use it. If you want to get a more thorough technical understanding, please check out the readings in the References section at the end.

Without further ado, these are the steps used to generate new Taylor Swift lyrics with GPT-2.

Setting Up

  • Create a new Google Colaboratory notebook. You will need a Google account, and you have to use chrome.
  • Go to Edit -> notebook settings. Make sure you have "GPU" selected for the hardware accelerator.
  • Import dependencies and load the package
%tensorflow_version 1.x
!pip install -q gpt-2-simple
import gpt_2_simple as gpt2
from google.colab import files
  • Download GPT-2
gpt2.download_gpt2()
  • Upload your dataset.txt file on the files menu on the left side

Finetuning and training

  • The next step will start to finetune and train the model based on the dataset. I set it to run the training for 1000 steps in the smallest available model size of 124M.
sess = gpt2.start_tf_sess()
gpt2.finetune(sess, dataset=dataset.txt,
model_name='124M',
steps=1000)

Here are some of the interesting snippets from the samples generated during the training:

There was some emo stuff:

I'm so sad, so sad, so sad 
I'm so sad, so sad, so sad 
Oh, so sad, so sad  
I'm so sad, so sad, so sad 
I'm so sad, so sad, so sad  
My name is whatever you throw at me 
It's all just a blur 
I'm so sad, so sad, so sad 
I'm so sad, so sad, so sad  
I'm so sad, so sad, so sad 
I'm so sad, so sad, so sad  
What died didn't stay dead, it died alive, oh, oh

Some mildly motivational stuff:

I was throwing stones, trying to stop you 
But you got mine And the long haul is free 
I threw them all at me 
Every day is a choice, even when you aren't a choice 
You're still trying to figure it out, just like I was 
You throw these things at me, but I'm throwing back stones 
You see I'm brave 
And the long haul is free I threw them all at me, yeah 
Every day is a choice, even when you aren't a choice 
You're still trying to figure it out, just like I was

Some confusing stuff:

You talked shit, you said sorry, 
you were just kidding 
You were losing it, losing it, losing it 
They said the end is near, 
but we'll mark it with a smile 
We were wrong and come back around here  
So it goes... - You started it - 
It expired just now - 
It's been a long time coming, can you tell me now I don't wanna miss you like this

Some self-love stuff:

Life was a willow and it bent right to your wind 
So you gotta belong somewhere  
Hey, that's my arse 
That's my dress 
Hey, that's my skirt 
Hey, that's my hat 
That's my hat  
It's not that easy in your world 
It's that simple 
But I got that arse

Some angst:

I'm sick and tired of your reasons 
I've got no one to follow me 
You tell me that you want me then cut me down 
And I'm sick and tired of your reasons 
I've got no one to follow me 
You ask me for my love then you lead me to your grave  
It's your turn, yours is good 
Turn around and find something else 
It's your turn, ours is good

And towards the end, it was starting to make more sense:

I promise you I'm not a saint 
But I'll do anything to get you 
Ridiculed by your shapeshifting urges 
And she's passing laws that will make you her end game 
All that I know is that you're my number one priority 
And I'm starting to see how you feel about me

Lyrics Generation and Results

Finally, after training is done, you can generate lyrics by running this:

gpt2.generate(sess)

You can create as many songs as you like by running this command again and again. Here are some of the ones I generated:

  1. I’m going to call this song Are you still at school? For obvious reasons.
I'm just another thing for you to roll your eyes at, honey 
You might have him but haven't you heard 
I'm just another thing for you to roll your eyes at, honey 
You might have him but I always get the last word  
Just another day at school, now why are you here? 
Because I, I know, I know.  
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school? 
Are you still at school?  
Hey, all you had to do was...  Stay. 
I don't wanna do this to you. 
I don't wanna do this to you. 
I don't wanna do this to you. 
I don't wanna do this to you. 
I don't wanna do this to you. I, I, I...  
Iculus 
The screen is your screen. 
I'm your furniture. 
I'm your skin and my bones. 
I'm my lifeline. 
I'm my queen. 
I'm my new self.  
Reconnect with my family and I'll show you what you know. 
Music starts playin' in the living room. 
Hang up, don't fall into anyone's trap. 
This is our place. 
Let you know that this is not what we had been. 
Music starts playin' in the kitchen. 
Sit and watch what we've been up to. 
Play back what we learned.  
And you'll realize that what you see is what you know. 
In the wildest winter, just know that I've been spending the last eight months "on edge" 
I mean it, I'm just gonna shake it off, I'm just gonna shake it off. I mean it, I'm just gonna shake it off, I'm just gonna shake it off.  Well, maybe you and I are just a couple of the many nicknames 
I'm afraid I'm getting 
Just know I'm not letting up until the day I die
  1. This one was a bit of an emotional roller coaster. I can’t think of an appropriate name for it, other than maybe, Please don’t be in love with me, I’m never gonna leave you alone.
And there's a dazzling haze, a mysterious way about you 
In dreams I've been yours before  
In wonderland 
Back when we were still changing for the better 
I reached for you like a rarefied love 
You kissed me on the cheek and said we were forever dreaming  
[In dream sequence:] 
You're in the car on the way to the movies 
You're in the car on the way to the movies 
You're in the car on the way to the movies 
And the fakers run wild Screaming, "Give me a reason" 
Why don't I just drive around 
And see what's under that attitude 
What's under that attitude 
I don't want you like a patchwork of other people's tears 
I don't want you like a prize for me 
I don't want you like a prize for me  
'Cause you're my best friend 
I'm always dreaming about you 
And I'm never leaving you alone 
I'm never leaving you alone  
So, please don't be in love with me 
I'm never gonna leave you alone 
I'm never gonna leave you alone 
Nothing's gonna make me run again 
I'm never gonna leave you alone 
I'm never gonna leave you alone  
'Cause, baby, now I'm scared 
I don't know what's down this road 
I don't know what to think about 
I don't know what to say 
I'm never gonna leave you alone 
I'm never gonna leave you alone  
Singing, baby, sing me down 
Don't know what's down this road 
I don't know what to think about 
I don't know what to say  
Let me hope you never lose me 
Let me hope you never lose me  
You're my best friend 
I'm always dreaming about you 
And I'm never leaving you alone 
I'm never leaving you alone  
Singing, baby, 
sing me down 
Don't know what's down this road 
I don't know what to think about 
I don't know what to say 
I'm never gonna leave you alone
[Instrumental] Oh, I remember you taking flight 
Taking off with me, honey 
Just in case you hadn't noticed 
Just in case you hadn't noticed  
[Instrumental] 
Have you ever thought just maybe would it be nice if I was yours? Would it be cool if I was theirs?
Would it be cool if I was yours?  
'Cause it's been years of hoping 
Never seen it this way. 
And we've been walking alone 
We're not speaking to each other, 
no matter what we say. 
It's been years of hoping. It's been years of hoping.
  1. And the final one I’m including could maybe be a sad or moody teenage anthem? I’m going to call this one Ghost in the stands.
This playful trend is gonna keep you laughing 
Ever since I met you 
Oh, I'm only seventeen 
I don't know anything new to say 
I sit and watch you 
But I know you're making friends 
And I'm only seventeen 
So, playful thing 
Play it cool, stay out of your eyes 
And I'll poke your fancy car 
If it's a trend 
And you're gonna follow suit 
And I promise that nobody's gonna love you like me  
In the winter, in the icy outdoor pool 
In the dead of night, life was just a dream 
But you knew it when you saw it,  
I was dancing when the phone rings, 
And I'd show you what you don't see 
I'd hit the ground running, 
And I say, "I'm sorry for that night."  
The night we frosted up 
There's a ghost in the stands 
The one and only me 
Who isn't afraid 
To break my heart 
Be honest with you 
I'm not afraid at all 
I'm only seventeen 
I don't know anything 
Now I'm walking down the hall 
I should've known from the first look 
That I'm not the only one 
Who feels that way 
So I'm taking this chance to say 
That I'm sorry for that night.  

Final Remarks

And these were all the steps I took to train an A.I model to generate Taylor Swift lyrics. GPT-2-simple is a great and easy way to play with GPT-2 for free with minimal effort, and there are many ways to customize it. You can change the model size, add more steps, add various checkpoints to restart training on, etc., to yield different types of results on various text data.

I had a fun time reading through some of these lyrics, and I definitely think they had Taylor Swift vibes, minus her lyrical profoundness. And while there were some deep lines in there, will these A.I generated lyrics win a grammy like T.S? Probably not.

You can find the dataset and all the code used here.

References

Max Woolf’s blog: How to make custom A.I-generated text with GPT-2

GPT-2-simple

OpenAI: Better Language Models and Their Implications

Transformer: A Novel Neural Network Architecture for Language Understanding


Related Articles