Generating Python Scripts with OpenAi's Github Copilot

It’s been a while since I got access to the technical preview of OpenAi’s Github copilot. Since then, I kind of learned to enjoy asking it all sorts of questions in the form of function names and descriptions to see what kind of code it could come up with.

In this article, I will share how I used the Github Copilot extension for VS Code (technical preview) to generate Python scripts, from simple neural networks in Keras and data visualization scripts to remarks about the secrets of existence

When you trim all of the hype and apocalyptic-like talk about language models like GPT-3 and actually get to play with them a little bit, you realize the good, the bad and the ugly about the scope of such applications.

By demystifying a little bit their true potential, we get to assess this unbelievable tool that could potentially be useful for countless different problems (granted that valid concerns be addressed), as well as learn its technical limitations like its lack of true human-like contextual understanding of basic sentences.

A simple neural network with Keras

Let’s start with a simple neural network in Keras. For this first script, all I gave to the model as input was:

def SimpleNeuralNetKeras():
    pass

The output from the AI-generated autocompletion was this:

def SimpleNeuralNetKeras():
    net_net = Sequential()
    net_net.add(Dense(units=1, input_dim=1))
    net_net.compile(loss='mean_squared_error', optimizer='adam')
    net_net.fit(x_train, y_train, epochs=100, batch_size=1, verbose=0)
    net_net.evaluate(x_test, y_test, batch_size=1, verbose=0)
    net_net.predict(x_test, batch_size=1, verbose=0)
    net_net.save('my_model.h5')
    net_net = load_model('my_model.h5')
    net_net.predict(x_test, batch_size=1, verbose=0)
    net_net.evaluate(x_test, y_test, batch_size=1, verbose=0)
    net_net.fit(x_train, y_train, epochs=100, batch_size=1, verbose=0)
    net_net.evaluate(x_test, y_test, batch_size=1, verbose=0)
    net_net.predict(x_test, batch_size=1, verbose=0)
    net_net.save('my_model.h5')
    """....it goes on and on"""

Although it continued to suggest the retraining of the same model, if we trim the fat, this is what we get:

def SimpleNeuralNetKeras():
    net_net = Sequential()
    net_net.add(Dense(units=1, input_dim=1))
    net_net.compile(loss='mean_squared_error', optimizer='adam')
    net_net.fit(x_train, y_train, epochs=100, batch_size=1, verbose=0)
    net_net.evaluate(x_test, y_test, batch_size=1, verbose=0)
    net_net.predict(x_test, batch_size=1, verbose=0)
    net_net.save('my_model.h5')

Honestly! Not bad at all! Let’s see what happens when I ask it to create a fake dataset to match this neural network set up:

My input here was simply:

x_train,

what I got back was:

x_train,y_train,x_test,y_test = get_data()

Interesting. Let’s see if the model could actually write out the get_data() function for us.

My input for that was:

def get_...

literally just that, and what I got completely automatically was:

def get_data():
    """
    This function is used to get the data for the neural network.
    """
    # Load the data
    data = np.loadtxt('data.csv', delimiter=',')
    # Split the data
    x = data[:, :-1]
    y = data[:, -1]
    # Split the data
    x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=42)
    return x_train,y_train,x_test,y_test

It does look pretty neat, although the unpacking of x and y seems a bit odd among other minor details, however, it does look like a piece of code that could be run with just a few tweaks.

Data visualization

How about plotting? Could we get a nice plot of some synthetic data?

Again this was my input for the model:

def plotFakeData():
    """
    This function creates a fake dataset of some 
    normally distributed data and plots the result
    of passing it through a sin function.
    """

This is what I got:

def plotFakeData():
    """
    This function creates a fake dataset of some 
    normally distributed data and plots the result
    of passing it through a sin function.
    """
    import numpy as np
    import matplotlib.pyplot as plt
    # Create some fake data
    x = np.linspace(0, 10, 100)
    y = np.sin(x) + np.random.normal(0, 0.1, 100)
    # Plot the data
    plt.plot(x, y)
    plt.show()

It looks pretty close! Shall we test it?

plotFakeData()

It actually runs! How cool is that?

Although the data created was not normally distributed, (I mean that was just something random that came up in my head), the model did add normally distributed noise to a sequence of data points after passing them through a sin function, after which it plotted the data successfully.

Writing tests

Now, how about writing tests for already defined functions?

Here my input was:

def sum_Square(a,b):
    return (a+b)**2

def test_sum_Square():
    pass

What I got:

def test_sum_Square():
    assert sum_Square(2,3) == 25
    assert sum_Square(3,5) == 55
    assert sum_Square(3,5) != 56

test_sum_Square()

# Output (from running it on a jupyter notebook)

AssertionError                            Traceback (most recent call last)

<ipython-input-94-b8fb676bf724> in <module>
----> 1 test_sum_Square()

<ipython-input-93-bf52191596f9> in test_sum_Square()
      1 def test_sum_Square():
      2     assert sum_Square(2,3) == 25
----> 3     assert sum_Square(3,5) == 55
      4     assert sum_Square(3,5) != 56

AssertionError:

Not bad! Actually the first and third tests were correct, the second not so much because for 3 and 5 the expected result would be 64. Nevertheless, it is really impressive that the model can actually do this kind of stuff out of the box just from a simple function definition.

Photo by Gabriella Clare Marino on Unsplash

Getting philosophical with github copilot

After playing with potentially useful applications for the model, I decided, just for fun, to ask some random pseudo-philosophical questions to see what kind of answers the model would come up with.

My input:

def writeArticleAboutSecretOfExistence():
    pass

What I got:

def writeArticleAboutSecretOfExistence():
    """
    The secret of existence is to be one.
    """

That is one short article but at least as a sentence, it does make some sense. I can definitely picture an old philosopher or monk giving this advice to a traveler seeking knowledge somewhere in a hypothetical mountain of wisdom.

Granted, the functions are supposed to be code, but I was just having some nerd fun ;).

Another one:

def writeArticleAboutHowToSolveAnyProblem():
    pass

I got:

def writeArticleAboutHowToSolveAnyProblem():
    """
    The secret of solving any problem is to think about it in another way.
    """
    pass

All right, now you know, think about your problem in another way and boom! You solved it!

Final Remarks on AI-assisted programming

I think the future of AI-assisted Programming looks bright, although there are issues that must be addressed for this to be a real-world solution.

As models like this understand more and more about a programmer’s intention regarding code, the amount of time needed to write good code will likely decrease a lot, which I think could have amazing benefits, allowing coders to focus on more important big picture issues.

If you want to get started playing around with github copilot, sign up for their waitlist to get access to the technical preview.

If you liked this post, follow me on Medium, subscribe to my newsletter, connect with me on Twitter, LinkedIn, Instagram, and join Medium! Thanks and see you next time! 🙂