A.I writes Taylor Swift songs ft. GPT-2
A simple guide to training an A.I model to generate Taylor Swift lyrics
No one can deny just how commendable and powerful Taylor Swift is. Her influence runs deep and transcends generations. And as she grew in front of the public eye, we saw her music range from country to pop to electropop and very recently, with Folklore and Evermore, to indie and folk.
With nine full-length albums with their own defined eras, there is a fun dataset to work with here. This article will explore how to use GPT-2 to generate lyrics based on all her released songs until February 2021. I compiled the data with the help of azlyrics and decided to use Google Colaboratory and the GPT-2-simple python package to train this data.
Brief Technical Context
GPT-2 (Generative Pre-trained Transformer 2) is an open-source A.I based on transformer architecture. GPT-2-simple is a python package that wraps finetuning to text generation scripts from GPT-2 to make text generation easier. This python package by Max Woolf has made GPT-2 very accessible, and he has this superb blog detailing how it works and how to use it. If you want to get a more thorough technical understanding, please check out the readings in the References section at the end.
Without further ado, these are the steps used to generate new Taylor Swift lyrics with GPT-2.
Setting Up
- Create a new Google Colaboratory notebook. You will need a Google account, and you have to use chrome.
- Go to Edit -> notebook settings. Make sure you have “GPU” selected for the hardware accelerator.
- Import dependencies and load the package
%tensorflow_version 1.x
!pip install -q gpt-2-simple
import gpt_2_simple as gpt2
from google.colab import files- Download GPT-2
gpt2.download_gpt2()- Upload your dataset.txt file on the files menu on the left side
Finetuning and training
- The next step will start to finetune and train the model based on the dataset. I set it to run the training for 1000 steps in the smallest available model size of 124M.
sess = gpt2.start_tf_sess()gpt2.finetune(sess, dataset=dataset.txt,
model_name='124M',
steps=1000)
Here are some of the interesting snippets from the samples generated during the training:
There was some emo stuff:
I'm so sad, so sad, so sad
I'm so sad, so sad, so sad
Oh, so sad, so sad
I'm so sad, so sad, so sad
I'm so sad, so sad, so sad
My name is whatever you throw at me
It's all just a blur
I'm so sad, so sad, so sad
I'm so sad, so sad, so sad
I'm so sad, so sad, so sad
I'm so sad, so sad, so sad
What died didn't stay dead, it died alive, oh, ohSome mildly motivational stuff:
I was throwing stones, trying to stop you
But you got mine And the long haul is free
I threw them all at me
Every day is a choice, even when you aren't a choice
You're still trying to figure it out, just like I was
You throw these things at me, but I'm throwing back stones
You see I'm brave
And the long haul is free I threw them all at me, yeah
Every day is a choice, even when you aren't a choice
You're still trying to figure it out, just like I wasSome confusing stuff:
You talked shit, you said sorry,
you were just kidding
You were losing it, losing it, losing it
They said the end is near,
but we'll mark it with a smile
We were wrong and come back around here
So it goes... - You started it -
It expired just now -
It's been a long time coming, can you tell me now I don't wanna miss you like thisSome self-love stuff:
Life was a willow and it bent right to your wind
So you gotta belong somewhere
Hey, that's my arse
That's my dress
Hey, that's my skirt
Hey, that's my hat
That's my hat
It's not that easy in your world
It's that simple
But I got that arseSome angst:
I'm sick and tired of your reasons
I've got no one to follow me
You tell me that you want me then cut me down
And I'm sick and tired of your reasons
I've got no one to follow me
You ask me for my love then you lead me to your grave
It's your turn, yours is good
Turn around and find something else
It's your turn, ours is goodAnd towards the end, it was starting to make more sense:
I promise you I'm not a saint
But I'll do anything to get you
Ridiculed by your shapeshifting urges
And she's passing laws that will make you her end game
All that I know is that you're my number one priority
And I'm starting to see how you feel about meLyrics Generation and Results
Finally, after training is done, you can generate lyrics by running this:
gpt2.generate(sess)You can create as many songs as you like by running this command again and again. Here are some of the ones I generated:
- I’m going to call this song Are you still at school? For obvious reasons.
I'm just another thing for you to roll your eyes at, honey
You might have him but haven't you heard
I'm just another thing for you to roll your eyes at, honey
You might have him but I always get the last word
Just another day at school, now why are you here?
Because I, I know, I know.
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school?
Are you still at school? Hey, all you had to do was... Stay.
I don't wanna do this to you.
I don't wanna do this to you.
I don't wanna do this to you.
I don't wanna do this to you.
I don't wanna do this to you. I, I, I... Iculus
The screen is your screen.
I'm your furniture.
I'm your skin and my bones.
I'm my lifeline.
I'm my queen.
I'm my new self.
Reconnect with my family and I'll show you what you know.
Music starts playin' in the living room.
Hang up, don't fall into anyone's trap.
This is our place.
Let you know that this is not what we had been.
Music starts playin' in the kitchen.
Sit and watch what we've been up to.
Play back what we learned.
And you'll realize that what you see is what you know.
In the wildest winter, just know that I've been spending the last eight months "on edge"
I mean it, I'm just gonna shake it off, I'm just gonna shake it off. I mean it, I'm just gonna shake it off, I'm just gonna shake it off. Well, maybe you and I are just a couple of the many nicknames
I'm afraid I'm getting
Just know I'm not letting up until the day I die
2. This one was a bit of an emotional roller coaster. I can’t think of an appropriate name for it, other than maybe, Please don’t be in love with me, I’m never gonna leave you alone.
And there's a dazzling haze, a mysterious way about you
In dreams I've been yours before
In wonderland
Back when we were still changing for the better
I reached for you like a rarefied love
You kissed me on the cheek and said we were forever dreaming [In dream sequence:]
You're in the car on the way to the movies
You're in the car on the way to the movies
You're in the car on the way to the movies
And the fakers run wild Screaming, "Give me a reason"
Why don't I just drive around
And see what's under that attitude
What's under that attitude
I don't want you like a patchwork of other people's tears I don't want you like a prize for me
I don't want you like a prize for me
'Cause you're my best friend
I'm always dreaming about you
And I'm never leaving you alone
I'm never leaving you alone
So, please don't be in love with me
I'm never gonna leave you alone
I'm never gonna leave you alone
Nothing's gonna make me run again
I'm never gonna leave you alone
I'm never gonna leave you alone
'Cause, baby, now I'm scared
I don't know what's down this road
I don't know what to think about
I don't know what to say
I'm never gonna leave you alone
I'm never gonna leave you alone Singing, baby, sing me down
Don't know what's down this road
I don't know what to think about
I don't know what to say
Let me hope you never lose me
Let me hope you never lose me
You're my best friend
I'm always dreaming about you
And I'm never leaving you alone
I'm never leaving you alone
Singing, baby,
sing me down
Don't know what's down this road
I don't know what to think about
I don't know what to say
I'm never gonna leave you alone[Instrumental] Oh, I remember you taking flight
Taking off with me, honey
Just in case you hadn't noticed
Just in case you hadn't noticed [Instrumental]
Have you ever thought just maybe would it be nice if I was yours? Would it be cool if I was theirs?
Would it be cool if I was yours?
'Cause it's been years of hoping
Never seen it this way.
And we've been walking alone
We're not speaking to each other,
no matter what we say.
It's been years of hoping. It's been years of hoping.
3. And the final one I’m including could maybe be a sad or moody teenage anthem? I’m going to call this one Ghost in the stands.
This playful trend is gonna keep you laughing
Ever since I met you Oh, I'm only seventeen
I don't know anything new to say
I sit and watch you
But I know you're making friends And I'm only seventeen
So, playful thing
Play it cool, stay out of your eyes
And I'll poke your fancy car
If it's a trend And you're gonna follow suit
And I promise that nobody's gonna love you like me
In the winter, in the icy outdoor pool
In the dead of night, life was just a dream
But you knew it when you saw it,
I was dancing when the phone rings,
And I'd show you what you don't see
I'd hit the ground running,
And I say, "I'm sorry for that night." The night we frosted up
There's a ghost in the stands
The one and only me
Who isn't afraid
To break my heart
Be honest with you I'm not afraid at all
I'm only seventeen
I don't know anything
Now I'm walking down the hall
I should've known from the first look
That I'm not the only one
Who feels that way
So I'm taking this chance to say
That I'm sorry for that night.
Final Remarks
And these were all the steps I took to train an A.I model to generate Taylor Swift lyrics. GPT-2-simple is a great and easy way to play with GPT-2 for free with minimal effort, and there are many ways to customize it. You can change the model size, add more steps, add various checkpoints to restart training on, etc., to yield different types of results on various text data.
I had a fun time reading through some of these lyrics, and I definitely think they had Taylor Swift vibes, minus her lyrical profoundness. And while there were some deep lines in there, will these A.I generated lyrics win a grammy like T.S? Probably not.
You can find the dataset and all the code used here.
References
Max Woolf’s blog: How to make custom A.I-generated text with GPT-2
OpenAI: Better Language Models and Their Implications
Transformer: A Novel Neural Network Architecture for Language Understanding
