
Like many, I’ve found myself enthralled with OpenAI’s ChatGPT and have used it for a wide variety of activities. From serious tasks like debugging code, to creative applications like writing poetry and short stories, it’s been fun to explore what ChatGPT can do. At the same time, it’s also been enlightening to look at its shortcomings. Being a language model, it cannot generate direct images like Midjourney AI or DALL-E can, as it was not trained on a large body of images, but rather text. However, when you consider that mathematically speaking, images are just 2D (or 3D in the case of colored images, i.e., RGB) arrays, you could have it write code to generate images. Thus, I asked ChatGPT to generate arrays corresponding to images.

With no modifications, I copied and pasted this code into Jupyter notebook to run and indeed obtained a white square (albeit not exactly in the middle):

So, it can generate images technically. However, I wanted to see if it could generate something more complex like a basketball.

Like last time, I ran this code directly in Jupyter Notebook.

As you can see, ChatGPT clearly didn’t generate a basketball, but rather a fuzzy object that gives the illusion of a 3D sphere. I then gave it some additional instructions, noting that it should have black lines running through it (though in hindsight, I probably should have specified black "curves" instead).


This time, I just got a purely black square.


We’re getting close in that we now have a thick black line through the circle, but it’s still nothing close to a basketball (almost looks like a Japanese riceball, if you ask me).


Now we just have a thin, black horizontal line running through our circle. To make this more like an actual basketball, we want some black curves running through it.

This time, when I ran the code, I actually got an IndexError
which I asked ChatGPT to debug and revise accordingly
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In [7], line 14
11 image[y.astype(int) + i, x] = 0
12 return image
---> 14 basketball = basketball_image()
15 plt.imshow(basketball, cmap='gray')
16 plt.show()
Cell In [7], line 11, in basketball_image()
9 y[y < 64] = 64
10 for i in range(0, 256, 8):
---> 11 image[y.astype(int) + i, x] = 0
12 return image
IndexError: index 256 is out of bounds for axis 0 with size 256
When I reported this error to ChatGPT, it provided a probable explanation and a rewrite of the code:

This time, when running ChatGPT’s code, I got a ValueError:
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Cell In [9], line 14
11 image[y.astype(int) + i, j] = 0
12 return image
---> 14 basketball_image()
Cell In [9], line 10, in basketball_image()
8 for i in range(0, 256, 8):
9 for j in range(256):
---> 10 if y[j].astype(int) + i < 256:
11 image[y.astype(int) + i, j] = 0
12 return image
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
I shared the error with ChatGPT once again, which explained that it was comparing an array to an int, and proposed the following revision that resulted in another IndexError
:

---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In [10], line 16
13 image[y[j].astype(int) - i, j] = 0
14 return image
---> 16 basketball = basketball_image()
17 plt.imshow(basketball, cmap='gray')
18 plt.show()
Cell In [10], line 11, in basketball_image()
9 for j in range(256):
10 if y[j].astype(int) + i < 256:
---> 11 image[y[j].astype(int) + i, j] = 0
12 if y[j].astype(int) - i >= 0:
13 image[y[j].astype(int) - i, j] = 0
IndexError: index -9223372036854775808 is out of bounds for axis 0 with size 256

Turns out third time was the charm, as I got no errors with this code. However, I didn’t get anything that looked like a basketball or even a ball in general (it at least knew to draw curves):

I asked to give another shot and this time it tried to use the sine function to generate the image.

Unfortunately, this just gave me a black square:

I tried giving it some encouragement, but after so many failed attempts and several errors, it decided to just throw in the towel:

For my final exercise, I asked it to generate a smiley face, with an added twist that it be an RGB image, which would require it to generate a 3D array instead of 2D.



First attempt was clearly awful. It actually took chatGPT eight more attempts to finally get something coherent. For sake of brevity, I’ll just show the top 3 best images it drew, along with the corresponding code.


The main thing I kept emphasizing was drawing a large circle, and within that circle two smaller circles corresponding to eyes, and an upright half-circle to represent a mouth.
The code snippet to generate this next image was too long to show the full screenshot, so I have the code pasted separately.
import matplotlib.pyplot as plt
import numpy as np
def smiley_face(size=512):
# Create an empty image with the given size
image = np.ones((size, size, 3), dtype=np.uint8) * 255
# Create the circle for the face
center = (size // 2, size // 2)
radius = size // 2 - 10
for i in range(size):
for j in range(size):
if (i - center[0]) ** 2 + (j - center[1]) ** 2 < radius ** 2:
image[i, j, :] = [255, 255, 102]
# Create the circle for the left eye
eye_radius = size // 8
eye_center = (size // 2 - size // 6, size // 2 - size // 6)
for i in range(size):
for j in range(size):
if (i - eye_center[0]) ** 2 + (j - eye_center[1]) ** 2 < eye_radius ** 2:
image[i, j, :] = [0, 0, 0]
# Create the circle for the right eye
eye_center = (size // 2 + size // 6, size // 2 - size // 6)
for i in range(size):
for j in range(size):
if (i - eye_center[0]) ** 2 + (j - eye_center[1]) ** 2 < eye_radius ** 2:
image[i, j, :] = [0, 0, 0]
# Create the smile
smile_center = (size // 2, size // 2 + size // 4)
smile_radius = size // 4
for i in range(size):
for j in range(size):
if (i - smile_center[0]) ** 2 + (j - smile_center[1]) ** 2 < smile_radius ** 2 and j > smile_center[1]:
image[i, j, :] = [255, 0, 0]
return image
smiley = smiley_face()
plt.imshow(smiley)
plt.axis('off')
plt.show()

This was the closest to a smiley face, except for some reason, it was rotated to the side. After two more prompts, it produced this as the final image:


At that point, after 8 attempts, I declared this a win compared to the basketball!
Conclusion
In summary, ChatGPT has the capability to generate arrays corresponding to meaningful images using the numpy
library in Python, but with limitations. Despite the lack of training data, ChatGPT is still able to produce simple images such as a smiley face. However, it struggles to generate more complex images like a basketball and requires multiple iterations to produce code for a smiley face. With further development, it’s possible that ChatGPT could generate basic images with less assistance. In the near future, we could see a combination of ChatGPT and an AI image generator. In my next article, I’ll discuss how ChatGPT can be utilized to provide prompts for AI image generators.
If you enjoyed this article and are new to Medium, consider becoming a member. If you join at this referral link, I will earn a portion of your membership at no extra cost to you, while you enjoy full access to all that Medium has to offer.