The world’s leading publication for data science, AI, and ML professionals.

Writing Code to Produce Images with ChatGPT

If images are just matrices of pixel values, can ChatGPT write code to produce matrices corresponding to meaningful imagery?

Photo by Jonathan Kemper on Unsplash
Photo by Jonathan Kemper on Unsplash

Like many, I’ve found myself enthralled with OpenAI’s ChatGPT and have used it for a wide variety of activities. From serious tasks like debugging code, to creative applications like writing poetry and short stories, it’s been fun to explore what ChatGPT can do. At the same time, it’s also been enlightening to look at its shortcomings. Being a language model, it cannot generate direct images like Midjourney AI or DALL-E can, as it was not trained on a large body of images, but rather text. However, when you consider that mathematically speaking, images are just 2D (or 3D in the case of colored images, i.e., RGB) arrays, you could have it write code to generate images. Thus, I asked ChatGPT to generate arrays corresponding to images.

Image by Author
Image by Author

With no modifications, I copied and pasted this code into Jupyter notebook to run and indeed obtained a white square (albeit not exactly in the middle):

Image by Author (code written by ChatGPT)
Image by Author (code written by ChatGPT)

So, it can generate images technically. However, I wanted to see if it could generate something more complex like a basketball.

Image by Author
Image by Author

Like last time, I ran this code directly in Jupyter Notebook.

Image by Author
Image by Author

As you can see, ChatGPT clearly didn’t generate a basketball, but rather a fuzzy object that gives the illusion of a 3D sphere. I then gave it some additional instructions, noting that it should have black lines running through it (though in hindsight, I probably should have specified black "curves" instead).

Image by Author
Image by Author
Image by Author
Image by Author

This time, I just got a purely black square.

Image by Author
Image by Author
Image by Author
Image by Author

We’re getting close in that we now have a thick black line through the circle, but it’s still nothing close to a basketball (almost looks like a Japanese riceball, if you ask me).

Image by Author
Image by Author
Image by Author
Image by Author

Now we just have a thin, black horizontal line running through our circle. To make this more like an actual basketball, we want some black curves running through it.

Image by Author
Image by Author

This time, when I ran the code, I actually got an IndexError which I asked ChatGPT to debug and revise accordingly

---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In [7], line 14
     11         image[y.astype(int) + i, x] = 0
     12     return image
---> 14 basketball = basketball_image()
     15 plt.imshow(basketball, cmap='gray')
     16 plt.show()

Cell In [7], line 11, in basketball_image()
      9 y[y < 64] = 64
     10 for i in range(0, 256, 8):
---> 11     image[y.astype(int) + i, x] = 0
     12 return image

IndexError: index 256 is out of bounds for axis 0 with size 256

When I reported this error to ChatGPT, it provided a probable explanation and a rewrite of the code:

Image by Author
Image by Author

This time, when running ChatGPT’s code, I got a ValueError:

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In [9], line 14
     11                 image[y.astype(int) + i, j] = 0
     12     return image
---> 14 basketball_image()

Cell In [9], line 10, in basketball_image()
      8 for i in range(0, 256, 8):
      9     for j in range(256):
---> 10         if y[j].astype(int) + i < 256:
     11             image[y.astype(int) + i, j] = 0
     12 return image

ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

I shared the error with ChatGPT once again, which explained that it was comparing an array to an int, and proposed the following revision that resulted in another IndexError:

Image by Author
Image by Author
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
Cell In [10], line 16
     13                 image[y[j].astype(int) - i, j] = 0
     14     return image
---> 16 basketball = basketball_image()
     17 plt.imshow(basketball, cmap='gray')
     18 plt.show()

Cell In [10], line 11, in basketball_image()
      9 for j in range(256):
     10     if y[j].astype(int) + i < 256:
---> 11         image[y[j].astype(int) + i, j] = 0
     12     if y[j].astype(int) - i >= 0:
     13         image[y[j].astype(int) - i, j] = 0

IndexError: index -9223372036854775808 is out of bounds for axis 0 with size 256
Image by Author
Image by Author

Turns out third time was the charm, as I got no errors with this code. However, I didn’t get anything that looked like a basketball or even a ball in general (it at least knew to draw curves):

Image by Author
Image by Author

I asked to give another shot and this time it tried to use the sine function to generate the image.

Unfortunately, this just gave me a black square:

Image by Author
Image by Author

I tried giving it some encouragement, but after so many failed attempts and several errors, it decided to just throw in the towel:

Image by Author
Image by Author

For my final exercise, I asked it to generate a smiley face, with an added twist that it be an RGB image, which would require it to generate a 3D array instead of 2D.

Image by Author
Image by Author
Image by Author
Image by Author
Image by Author
Image by Author

First attempt was clearly awful. It actually took chatGPT eight more attempts to finally get something coherent. For sake of brevity, I’ll just show the top 3 best images it drew, along with the corresponding code.

Image by Author
Image by Author
Image by Author
Image by Author

The main thing I kept emphasizing was drawing a large circle, and within that circle two smaller circles corresponding to eyes, and an upright half-circle to represent a mouth.

The code snippet to generate this next image was too long to show the full screenshot, so I have the code pasted separately.

import matplotlib.pyplot as plt
import numpy as np

def smiley_face(size=512):
    # Create an empty image with the given size
    image = np.ones((size, size, 3), dtype=np.uint8) * 255

    # Create the circle for the face
    center = (size // 2, size // 2)
    radius = size // 2 - 10
    for i in range(size):
        for j in range(size):
            if (i - center[0]) ** 2 + (j - center[1]) ** 2 < radius ** 2:
                image[i, j, :] = [255, 255, 102]

    # Create the circle for the left eye
    eye_radius = size // 8
    eye_center = (size // 2 - size // 6, size // 2 - size // 6)
    for i in range(size):
        for j in range(size):
            if (i - eye_center[0]) ** 2 + (j - eye_center[1]) ** 2 < eye_radius ** 2:
                image[i, j, :] = [0, 0, 0]

    # Create the circle for the right eye
    eye_center = (size // 2 + size // 6, size // 2 - size // 6)
    for i in range(size):
        for j in range(size):
            if (i - eye_center[0]) ** 2 + (j - eye_center[1]) ** 2 < eye_radius ** 2:
                image[i, j, :] = [0, 0, 0]

    # Create the smile
    smile_center = (size // 2, size // 2 + size // 4)
    smile_radius = size // 4
    for i in range(size):
        for j in range(size):
            if (i - smile_center[0]) ** 2 + (j - smile_center[1]) ** 2 < smile_radius ** 2 and j > smile_center[1]:
                image[i, j, :] = [255, 0, 0]

    return image

smiley = smiley_face()
plt.imshow(smiley)
plt.axis('off')
plt.show()
Image by Author
Image by Author

This was the closest to a smiley face, except for some reason, it was rotated to the side. After two more prompts, it produced this as the final image:

Image by Author
Image by Author
Image by Author
Image by Author

At that point, after 8 attempts, I declared this a win compared to the basketball!

Conclusion

In summary, ChatGPT has the capability to generate arrays corresponding to meaningful images using the numpy library in Python, but with limitations. Despite the lack of training data, ChatGPT is still able to produce simple images such as a smiley face. However, it struggles to generate more complex images like a basketball and requires multiple iterations to produce code for a smiley face. With further development, it’s possible that ChatGPT could generate basic images with less assistance. In the near future, we could see a combination of ChatGPT and an AI image generator. In my next article, I’ll discuss how ChatGPT can be utilized to provide prompts for AI image generators.


If you enjoyed this article and are new to Medium, consider becoming a member. If you join at this referral link, I will earn a portion of your membership at no extra cost to you, while you enjoy full access to all that Medium has to offer.

Jamshaid Shahir – Medium


Related Articles