Beyond The Hype: How Generative AI Is Transforming Software Development

Image by Google DeepMind on Unsplash

Introduction

Artificial intelligence (AI) has come a long way since its inception in the 1950s. From rule-based expert systems to deep learning, AI has evolved into a powerful technology that has transformed various industries, from healthcare to finance, from transportation to manufacturing. The latest hype in AI is Generative AI. Yes, it’s a hype, but it undoubtedly has the potential to revolutionize many aspects of our personal and professional lives. In this blog post, we’ll explore one aspect which many people in the technology industry are thinking about: the way we build software.

One subgroup of Generative AI algorithms, a class of models called Large Language Models (LLMs) that can create text based on the user’s input data, has gained significant interest over the last couple of months. LLMs are, in their most basic description, multi-layer deep neural networks that are trained on large — very large — datasets from which the model learns the underlying patterns and generates new data that follow those patterns. Writing to an LLM and asking for a response is called “prompting” and knowing how to write effective prompts is a skill in demand, often referred to as prompt engineering.

To tune LLMs to give a humanlike response, a machine learning training framework called Reinforcement Learning with Human Feedback (RLHF) has been shown to be particularly powerful. This requires a large number of people to repeatedly prompt early versions of an LLM, read its response, and tell the model whether it passed their “humanlike” litmus test. The group of “prompters” are often told to pay attention, for example, to unintentional insults of the user by the model coming from gender or racial biases.

LLMs are changing the way we create, giving everyone the tools they need to write poetry, essays, marketing copies and more. The use cases are extensive, but in this blog post, we’ll explore the impact of Generative AI on the future of software development. Prompts are a key concept for Generative AI, so let us define it before we go into more detail.

Prompts: Prompts serve as the inputs or queries provided by a user or program to an LLM, aiming to elicit a targeted response from the model. These prompts encompass various forms such as natural language sentences, questions, code snippets, or commands. They can be a combination of text and code, allowing for versatile interactions. Furthermore, prompts can be nested or chained, enabling the utilization of the output from one prompt as the input for another. This capability fosters intricate and dynamic exchanges between the user and the LLM, or even among multiple LLMs.

Image by Christina on Unsplash

How LLMs are changing the way we build software

Although LLMs were initially developed for language generation tasks, they have proven to be versatile and can be applied to various tasks beyond generating text. We’ll begin with exploring one capability that has been especially interesting for anyone building software: Code Completion.

LLMs can assist developers by predicting the next lines of code based on the code already written or based on a prompt that describes the functionality of a certain piece of the software. For example, a prompt like “write a Python function that transforms a NumPy array into a dictionary where the keys are the strings of the range over the first dimension of the array” would return the code below.

import numpy as np

def array_to_dict(arr):
    """
    Convert a NumPy array into a dictionary where the keys are the strings of the range over the first dimension of the array.
    
    :param arr: The NumPy array to convert to a dictionary.
    :return: The resulting dictionary.
    """
    dict_ = {}
    for i in range(arr.shape[0]):
        key = str(i)
        dict_[key] = arr[i]
    return dict_

Most software developers are used to the concept of “pair programming”, where one developer keeps the overall software structure in mind whereas the other writes the code. Products built on top of specialized LLMs, like GitHub’s Copilot can be an adequate alternative to a programming partner — helping to accelerate the software development process. Nonetheless, with this level of assistance, the user must understand how to prompt the LLM to achieve their desired output. This may sound simple at first, but in fact, it requires knowledge of data structures, functions and many other concepts in the relevant programming languages.

The next leap in the software revolution will occur when developers do not need to break apart the overarching functionality of the software into smaller pieces that can be distilled in prompts that are understood by LLMs, like the array-to-dictionary example above. Generative AI will fundamentally change how software is developed when LLMs are capable of comprehending the main functionality themselves and providing suggestions for a feasible implementation of it, through a conversational experience.

In essence, this will allow not only developers to create software but anyone who is able to accurately describe the functionality of a desired software product in natural language.

Currently, LLMs cannot return the entire code repository that would be necessary to create advanced software products. Let’s take an example, I want to build an Android mobile application that suggests recipes for smoothies based on a picture I take of a fruit basket in my living room.

Exact prompt submitted to ChatGPT: “Write code for an Android mobile application that suggests recipes for smoothies based on a picture I take of the fruit basket in my living room.”

Today, an LLM would return an outline of how one could approach building the application — potentially with sample code — but not the fully functional software.

Although returning the entire code repository would be of great help to anyone building such an application, it would still not be enough to launch it and be usable by anyone with an Android phone, because the code would most likely need to be hosted in Google Firebase and submitted for validation to Play Store (although the last step can be achieved through Firebase).

Nonetheless, the current capabilities of LLMs are not far away from achieving these functionalities. It’s a popular misconception that LLMs are the bottleneck in this chain of executions, for example, from writing the code for a mobile application to having it operational and downloadable on the Play Store. In fact, the current bottleneck is the safe and reliable integration of LLMs into third-party systems. For the mobile application example above, one would need at least two integrations: LLM to GitHub to store the code, and LLM to Firebase to configure the application and submit it for publication on Play Store.

If LLMs were capable of distilling high-level software product descriptions and operationalizing the product, the development process would become a conversational experience akin to the interactions between a product manager and a full-stack software engineer. Speaking in an overly simplified manner, the creative head of the development efforts, the product manager, would provide the core functionality of the product in natural language, whereas her counterpart, the “full-stack software engineer-turned LLM” would ask clarification questions and technically execute the product manager’s descriptions.

Safe and reliable integrations of LLMs to third-party systems

The safety and reliability of letting LLMs directly interact with third-party systems are crucial for the benefit of using LLMs. We will describe three important requirements to integrate LLMs with third-party systems safely and reliably, but this list is by no means exhaustive.

1. Model Validation: Before integrating an LLM with a third-party system, it is important to thoroughly validate the model’s accuracy. Here, accuracy refers to the output of the LLM given a prompt by its user. The LLM’s output causes an action in the third-party system. It’s crucial to validate in a test environment whether the executed actions are repeatedly the same over a range of variations in the prompts with the same underlying meaning.
2. Transparency and Explainability: LLMs are complex and it is often impossible to understand how a certain response was created. If their decision-making processes, i.e., how they arrive at any given response, would be made transparent by explaining it in natural language, the human counterpart in the conversational software creation experience would know more about how to prompt the LLM so it produces the desired outcome.
3. Version Control: LLMs can be updated or replaced over time, so it’s important to have a clear version control strategy in place because the response to the same prompt might change from version to version. Hence, a repository of previous model versions, a thorough testing and monitoring phase with any model update, as well as contingency plans for unexpected issues or failures with new model updates are crucial.

Conclusion

In summary, Large Language Models (LLMs) have outstanding capabilities that are suitable for assisting with tasks which come from a large pool of well-structured information and require great attention to detail. By automating many repetitive and time-consuming tasks, LLMs are enabling their users, for example product managers, to focus on higher-level tasks that require creativity and problem-solving skills. While LLMs can automate many routine tasks, they do not have the same level of creativity, intuition, and empathy that humans possess.

For the success of LLMs in many business applications, safe and reliable integrations to industry-specific third-party systems are important. Users of LLMs need to be able to execute certain actions that they would normally not be able to perform without support from a dedicated software developer. No-code tools are already on the rise across many industries and job functions, but their capabilities need to mature to build complex software beyond a specific and narrow functionality (for example the functionalities that can be built with Levity).

In conclusion, LLMs are transforming software development in many ways, and what we see today may just be the beginning. From automating routine tasks to enabling new forms of creativity and innovation, LLMs are poised to play a major role in the future of software development. As the Generative AI boom continues to rumble on, we must look beyond the hype to see a seminal moment in technology development forming. Generative AI has changed how we interact with software, and soon it could change how software is developed — enabling anyone with a product vision to build it through the medium of conversation.