The world’s leading publication for data science, AI, and ML professionals.

Writing Songs with GPT-4: Part 3, Melodies

How to use the latest language model from OpenAI to help write melodies for new songs

"a woman standing, wearing headphones, looking at a computer screen, while singing into a professional microphone," Image created using an AI image creation program, Midjourney, and edited by the author
"a woman standing, wearing headphones, looking at a computer screen, while singing into a professional microphone," Image created using an AI image creation program, Midjourney, and edited by the author

This is the third and final article in my series on using GPT-4 [1] to compose music. My first article covered writing lyrics for songs, and my second article delved into composing chords to go along with lyrics. In this article, after a brief introduction to GPT-4, I’ll show you how to coax the model to create melodies for songs with chords and lyrics. I’ll wrap up with a general discussion and offer some next steps for composing music with AI.

Intro to GPT-4

GPT-4 is the latest large language model from OpenAI. It’s available in the paid version of their ChatGPT service, which costs US$20 per month. The model is an improvement on their previous iteration, GPT-3.5. Here’s an excerpt of what OpenAI says about the new model on their GPT4 landing site.

GPT-4 is more creative and collaborative than ever before. It can generate, edit, and iterate with users on creative and technical writing tasks, such as composing songs, writing screenplays, or learning a user’s writing style. … We spent 6 months making GPT-4 safer and more aligned. GPT-4 is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5 on our internal evaluations – OpenAI

OK, they listed "composing songs" as one of their examples of creative tasks. They also mentioned the model being more "aligned," which is OpenAI’s buzzword for making AI models follow human values and intent. You can read more about this in their technical report.

As I mentioned in my earlier articles on GPT-4, OpenAI collects users’ prompts and responses by default and may use them to improve future models. However, you can opt out of this data collection by submitting this form, here.

Creating Melodies with GPT-4

Currently, large language models only deal with text in and text out. However, OpenAI is working on extending GPT-4 to use images in prompts. This model is not available to the public yet, but it is available in a beta version of an app called Be My Eyes. The app helps people who are blind or have low vision with daily tasks by analyzing pictures and reading a text description of the scene.

For this article, I explored various ways to get the model to compose melodies using text formats. I found the best format for this is using guitar tablature (or tab) notation.

Guitar Tabs

Musicians use guitar tabs to transcribe notes and chords for stringed instruments as plain text. Instead of the usual musical notes on a staff, guitar tabs show which frets are used over time for the strings on an instrument. For example, here’s the first line of "Happy Birthday to You" as a six-stringed guitar tab.

Guitar Tab of the Melody for "Happy Birthday" Image by Author
Guitar Tab of the Melody for "Happy Birthday" Image by Author

Time is on the horizontal axis; the strings are on the vertical axis from the lowest at the bottom to the highest at the top. The numbers represent which fret to play at points in time. A bigger number corresponds to a higher note. After experimenting with this, I found that there were enough guitar tabs with the corresponding commentary found in the web-based training data to allow language models to compose in this format.

Interacting with GPT-4

Using the example song from my first two articles, I asked GPT-4 to compose a Melody one part at a time. Here’s how the first interaction went.

RobG: Write a melody for the song Dogs and Cats in guitar tab format. Play one note for each syllable.

[G] Dogs and cats, they [C] roam the streets [Em] Furry friends with [D] hearts that beat [G] Claws and paws, they [C] rule the land [Em] Tails a-waggin’, [D] they understand

GPT-4: I’m an AI text-based model, so I can’t create or play a sound. But I can give you an example of how you might put these chords to a simple melody using guitar tab format, each note for each syllable.

The basic chords provided are G, C, Em, and D. I’ll use the G major scale as the melody foundation.

Remember, in tab format, the lines represent the strings of a guitar (from low E on bottom to high E on top), and the numbers represent which fret to play on that string. Here’s a simple example:

e|----------------0------------------|-----------------3------------------|
B|---------1---3-----3-1---------1--|-------0---1-3-------3-1------------|
G|-------0---0-----------2-0--------|-----0---0---------------2-0--------|
D|-----2----------------------2------|---2-------------------------------|
A|----------------------------------|-----------------------------------|
E|----------------------------------|-----------------------------------|
     G          C                   Em             D

e|----------------0------------------|-----------------3------------------|
B|---------1---3-----3-1---------1--|-------0---1-3-------3-1------------|
G|-------0---0-----------2-0--------|-----0---0---------------2-0--------|
D|-----2----------------------2------|---2-------------------------------|
A|----------------------------------|-----------------------------------|
E|----------------------------------|-----------------------------------|
     G          C                   Em             D

This is just a very simple example and the melody might not fully capture the spirit of your lyrics, but I hope it helps as a starting point. You may want to adjust the rhythm and melody to suit your song better.

This is interesting. It’s a two-part musical phrase that is repeated twice. But on closer inspection, you can see that there are too many syllables in the phrases; it has twelve+eleven when it should be seven+seven. Also, the vertical lines don’t line up. I repeated the process for the second verse and cleaned up the phrases a bit in a text editor to look like this.

e|-------------0-----|-------------------|
B|-------1---3---3---|-------0---1-3-----|
G|-----0---0---------|-----0---0-----0---|
D|---2---------------|---2---------------|
A|-------------------|-------------------|
E|-------------------|-------------------|
     G       C             Em      D

e|-------------------|-------------------|
B|-------0-3-1-0-----|-------0---3-1-----|
G|-----1---------2---|-----0---0-----2---|
D|---2---------------|---2---------------|
A|-------------------|-------------------|
E|-------------------|-------------------|
     Am      C           G       D

I wrote some Python code here to convert guitar tabs to MIDI files, so I could play them back to hear how they sounded. The code is based on a similar project by Brian Caffey [2]. Here’s the complete score.

The Score for Dogs and Cats, Image by Author
The Score for Dogs and Cats, Image by Author

And here’s what the song sounds like. I used Band-in-a-Box by PG Music to arrange the song with piano, guitar, bass, and drums. Feel free to sing along. 😀

It’s simple but catchy, kinda like a nursery song. The first note is a bit odd in that it is an E, which would make the G chord effectively a G6. And this occurs with every instance of the G chord, too. I think the best thing about the tune is the variation the third time through with the Am and G. Up next, I’ll show you how I created melodies for a ska song.

"album cover for a ska band, black and white," Midjourney Image, Edited by Author
"album cover for a ska band, black and white," Midjourney Image, Edited by Author

Creating a Vocal Melody for a Full Song

To test this method further, I created lead melodies for the song "Dance with the Fear," which was developed in the first two articles in this series. The lyrics and chords are based on the song "Grow" by the ska band The Agents, which I used with their permission. I managed to get GPT-4 to create a melody for the new song, one section at a time.

Creating the Verse

Like the method I used above, I asked GPT-4 to create a melody for the verse and provided it with the chords and lyrics. I hit the regenerate button several times and examined the results before settling on this one.

RobG: Write a guitar tab for the melody of the verse for this ska song, playing only one note at a time.

[Em] Waking up to a [G] brand new day, [D] Feeling the fear but [A] not turning away, [Em] Facing the unknown, [G] that’s where we’ll start, [D] With an open mind and [A] a fearless heart.

GPT-4: Here is a simple guitar tab for the melody of the verse, playing one note at a time

e|-----------------|-----------------|-----------------|-----------------|
B|-----7-8-------8-|-8-8-7-7---------|-5-5-7-7-------8-|-8-8-7-7-5--5-5--|
G|---7-------7-7---|---------7--7-7--|-----------7-7---|-----------------|
D|-5---------------|-----------------|-----------------|-----------------|
A|-----------------|-----------------|-----------------|-----------------|
E|-----------------|-----------------|-----------------|-----------------|

e|-----------------|-----------------|-----------------|-----------------|
B|-----7-8-------8-|-8-8-7-7---------|-5-5-7-7-------8-|-8-8-7-7-5--5-5--|
G|---7-------7-7---|---------7--7-7--|-----------7-7---|-----------------|
D|-5---------------|-----------------|-----------------|-----------------|
A|-----------------|-----------------|-----------------|-----------------|
E|-----------------|-----------------|-----------------|-----------------|

Please note that this tab is a simplified version of the melody, and you can further modify and embellish it as you see fit for your performance or arrangement.

Here’s what the melody sounds like.

In general, it’s pretty good. It established a theme for the first phrase and has a nice resolution for the second phrase. But, once again, it created too many notes for the lyrics.

Creating the Pre-Chorus

A pre-chorus is a melody that leads into the chorus of a song. Note that the ChatGPT user interface included the context of previous interactions in the session, so I could be a bit more brief in my prompt to generate the melody for the pre-chorus.

Write a melody for the pre-chorus.

[Am] No more hiding, [C] no more doubts, [G] It’s time to let our [D] courage out.

GPT-4 Sure, I can provide a simple melody for the pre-chorus using the provided chord progression. Here’s the tab:

e|------0-0-0-0----|-0---------------|-----------------|-0---------------|
B|----1----------|----3-1-1-1-1---|----1-3-3-1-3-3-|-------3-3-1-3-3-|
G|--0--------------|---------------|----------------|----------------|
D|-----------------|----------------|----------------|----------------|
A|-----------------|----------------|----------------|----------------|
E|-----------------|----------------|----------------|----------------|

This melody uses the A minor, C Major, G Major, and D Major scales respectively. As always, feel free to modify this as you see fit for your performance.

I’m sure it meant the word "chords" instead of "scales" in its response since the melody it generated is in G Major. Just to be sure, I asked for clarification.

RobG: Did you mean to use the word "scales"?

GPT-4: My apologies for any confusion. I meant to say that the melody notes are based on the respective chords, not scales.

OK, that cleared things up. As for the melody, I got the sense that the many notes it wrote could be interpreted as held notes. In other words, a string of four eighth notes could be played as a one half note. This reflects a limitation of the guitar tab format in that the note lengths are not expressed. Seen through this lens, the melody is good. Here is the melody for the pre-chorus, as written originally and after I converted some of the repeated notes to single ones with longer durations.

OK, the second take sounds better with the held notes. Here are the lyrics and chords again for reference.

[Am] No more hiding, [C] no more doubts, [G] It’s time to let our [D] courage out.

All Four Parts

I repeated the process for the chorus and the bridge and assembled the parts into a single score with the chords played on a piano. Here are the four parts of the song.

For this version of the song, I cleaned up all of the repeated notes to make them match the rhythm and meter of the lyrics. Overall, the quality of the melodies is good. There were no sour notes and the phrases gell with the underlying chords.

Up next, you can hear the song with an actual human singer.

Kate Li Sings Dance with the Fear, Image by Author
Kate Li Sings Dance with the Fear, Image by Author

I am very grateful to my friend and colleague, Kate Li, who sang the tune for this article. She transposed the song up six half-steps to go from G to D-flat to accommodate her vocal range. I used Band-in-a-box to fill out the instrumentation with piano, guitar, horns, bass, and drums. Here’s the finished song.

Kate did great singing the main vocal track and adding harmonies for the choruses. She brought a lot of emotional punch to the bridge with the rising melody and provided some nice adlibs for the outro. Please let me know your thoughts on the song here or on SoundCloud.

Discussion

It took a little work, but I got GPT-4 to generate melodies for new songs. The guitar tab format is tricky to work with, but it’s currently the best way to go. When OpenAI releases its version of GPT-4 with vision, I hope it had seen pages of sheet music to help assist with composing melodies.

Comparing the three Music Composition tasks I set out to do in this series, analyzing and generating chords in the second article worked the best. Although imperfect, the language model has deep knowledge of music theory and can easily create new and innovative chord progressions.

Analyzing and creating lyrics for songs, discussed in the first article, was also easy to do with GPT-4, but the generated lyrics were often simple and clichéd. Although the system knows how to rhyme, it takes some work to direct the system to create unique, exciting, and meaningful lyrics.

Source Code

The source code for this project is available on GitHub.

Creative Commons Attribution Sharealike
Creative Commons Attribution Sharealike

Acknowledgments

I want to thank Kate Li for singing the song for this article. I also thank Pete Fayan from the Agents for his engagement with this project. Finally, I thank Jennifer Lim for reviewing the article and providing feedback.

More on Writing Songs with GPT-4

Here are the first two articles in this series.

Writing Songs with GPT-4: Part 1, Lyrics

Writing Songs with GPT-4: Part 2, Chords

References

[1] OpenAI, GPT-4 Technical Report (2023)

[2] B. Caffey, Generating MIDI files from guitar tablatures with Python and regular expressions (2018)


Related Articles