Computational Creativity
An experimental AI pipeline to integrate lateral thinking in computational tools of Architecture
The transition of architectural tools from traditional to modern computational has focused on efficiency and productivity. This is so as many of the computational tools used by architects have their legacy in industrial disciplines such as automobile and manufacturing. CATIA, for example, a software used by Frank Gehry and his team to achieve complex designs, was initially developed for the aeronautical industry.
As a result of being inherited from these fields, the values and standards of these other disciplines have discretely trickled into architecture. Consequently, notions of optimization, standardization, and efficiency, all qualities of vertical thinking, or industrial era reasoning, are inevitably prioritized in these tools over lateral thinking qualities like inaccuracies, uncertainty, and accidents, qualities often associated with artistic practices.
Yet, as the work of academics such as Bryan Lawson, Nigel Cross, and Kees Dorst states, there is a clear relationship between creativity and design excellence. Their work points out that successful architectural design must have creative design processes and that this creative process is manifested in a designer’s ability to balance divergent and convergent thinking — with divergent thinking closely related to “lateral thinking”, the term coined by Edward de Bono. Such a creative design process is what often brings about unexpected design opportunities.
Hence, as our tools became more and more digital, the features and methods imposed by these tools, which greatly focus on vertical thinking, have automatically and inadvertently constrained the possible design space.
This article targets the need to re-integrate lateral thinking strategies into architectural design tools. We argue that today’s computational tools that lack these specific qualities could benefit from embedding artificial intelligence at the early stages of the architectural design process as a method to address lateral thinking approaches in our processes. Specifically, we present a 4-step AI-driven pipeline, based on Generative Adversarial Networks (GANs), that draws from the ability to access the latent space of a machine and use this space as an experimental digital environment to design 3D architectural structures.
Related Work
Architectural tools for the last decade or so have mostly focused on parametric approaches. These approaches are characterized by assigning values to relationships between elements and manipulating those values to control and organize these relationships. On the other hand, machine learning algorithms work by exploring data and identifying patterns within them without relying on predetermined equations as models. Several architects and researchers have recently explored the potential of machine learning algorithms, particularly Generative Adversarial Networks (GANs), in improving creative possibilities. These GAN algorithms can statistically learn and imitate any distribution of data presented to them and are commonly used for generative purposes.
Predictable Outputs
Huang et al. applied GANs in recognizing and generating architectural drawings. The authors used pix2pixHD, a modified GAN that learns image data in pairs and generates new images based on the input. They used diagrams of rooms marked in different colors as input to produce apartment plans. Chang et al. proposed a graph-constrained generative adversarial network to automatically generate multiple house layout options from architectural bubble diagrams.
These works clearly demonstrate the potential and ability of GANs to generate architectural plans and possibly optimize architectural processes and workflows. The results are accepted based on how familiar or similar to the current state of architecture they were. Hence, in terms of lateral thinking approaches, these methods do not perform well as their output tends to always be rather predictable, expected, and constrained to what is already known.
Yet, as Philippe Morel states, “the task of any architect is not about using computers to replicate or to automatize what has already been thought of and produced, it is about allowing computers to reveal a fully new form of architectural intelligence.” He elaborates that “Machines, […] shall logically give birth to a kind of architecture that is also beyond our usual capacities”, and states that such an architecture is still to be produced. Our work aims to expand on this and allow the machine to reveal a new kind of “unthinkable architecture.”
“The task of any architect is not about using computers to replicate or to automatize what has already been thought of and produced, it is about allowing computers to reveal a fully new form of architectural intelligence.”
Unpredictable Outputs
Chaillou proposed an approach of using nested GAN models to generate an entire apartment building. Using this method, the author demonstrates how nesting different models results in unique outputs and new architectural styles that fall in-between currently known styles. Although such an approach reveals great results, the process still involves the heavy curation of image pairs to train on.
Our work builds on this body of research and proposes a series of steps and approaches through which we discuss the potential of AI-driven lateral thinking in architectural tools.
Methodology
We propose a pipeline to generate 3D structures that uses StyleGAN and explores its latent space as a 512-dimensional virtual environment to design in. A latent space is an n-dimensional space where, after training a GAN model, the generator of this model learns to map points in this space to specific generated images similar to the ones of the dataset it was initially trained on. These mappings change every time the model is trained.
The latent space has a structure that can be queried and navigated where every point in this space is a 512D vector that represents an image. Our 4-step process draws from the abilities in and properties of this space to design a 3D structure. (1) It consists first of the initialization of this virtual latent space of points through the training of the StyleGAN model. (2) The following step is the entry into this space and access into its dimensions by choosing or generating a 512D vector that represents an image. (3) Further, after getting a point (an image), we navigate through the space by using vector arithmetic operations and interpolations to generate new points that are translated to images. (4) Finally, we generate a voxel 3D structure by using the pixel values of these images. We showcase this pipeline and suggest how it can push architectural digital tools into the grounds of experimentation.
The virtual environment represents the digital design space in which architects currently design, analyze, edit and generate 3D structures. The current tools used, such as Rhino3D, focus on orthographic approaches and rely on curves, lines, surfaces, NURBS, etc., and the manipulation of those elements to create designs in a 3D virtual space. In this article, we initialize instead a 512D virtual design environment through the training of a machine and propose different approaches to designing within that space.
I. Initializing the Virtual Environment
To initialize the virtual environment, a StyleGAN model is trained on 5000 architectural plans scraped from several different online sources such as ArchDaily and Artstor. The scraped dataset consists of mainly housing plans of different styles and periods. This allows for better control over the possible image features that the machine might learn. After training, a latent space of 512D is generated where data points (which represent images) with similar features are placed closer together in this space. We use this generated “virtual environment” as a search and a design ground for design possibilities.
II. Entering the Space
The following step is the entry into this space. We access the dimensions of this latent space using two main methods: through point generation, by randomly using a set of values for a 512-dimensional vector (an image), or through image embedding of a drawing chosen or designed by the user.
In the first method, the machine goes through the latent space and generates an image that corresponds to the specified vector values. In the second method, a StyleGAN embedding algorithm can be used wherein the machine maps the embedded image to a specific point in the space closest to images that have similar features.
III. Navigating the Space
After generating or selecting points in this space, we use these points in vector arithmetic operations to create new points that are then translated to images. To illustrate, by adding vectors, we combine different points in space that have different features and generate a new point. Specifically, in terms of architectural plans, by adding different points (image vectors) in space, we are adding vectors that correspond to plans of different attributes and possibly styles in architectural design, depending on what the initial dataset is. This allows for the targeted yet unpredictable generation of images. Figure below shows one of these outputs in our process.
Moreover, apart from combining vectors, we interpolate linearly between two or more points. By interpolating from one point to the other, we create a series of transformational, or transitional, images with a different number of steps or degrees of similarity, as illustrated in the figure below.
IV. Generating 3D Voxelized Structures
Finally, to create the 3D forms, we stack the series of images generated in Z-space and generate a voxel structure. To do this, in this step, the pixel values of the interpolated images are read and translated to a 3D array of Boolean values which define the solid voxel volume.
This virtual environment (latent space) generated serves as an alternative space to the environments of our current computational tools such as Rhino and AutoCAD. Working within the space of a machine imposes new biases of its own. Yet, we argue that the current “mechanistic” operations usually performed in the current 3D modeling software, such as slicing, lofting, trimming, etc., produce relatively predictable output by following a series of steps and rules.
However, through our integrated non-sequential and non-logical pipeline and latent space directional navigation techniques, we aim to escape from the established methods of manipulating curves, lines, and surfaces of our tools and move toward working with vector arithmetic operations and pixel and voxels as building blocks as a lateral thinking approach. Such an approach may open new opportunities to design and inspire new spatial and formal concepts. Additionally, in this process, the machine acts as an active player in the design output rather than as merely a means to an end. We demonstrate this in the following section through a series of unique 3D forms generated using 4 initial starting points in the latent space.
Results
To test our pipeline, we select 4 random points (images of plans: A, B, C, D) in the generated latent space and get their vector representations. We then test 4 different methods of interpolating and navigating between those points and generate a total of 200 images that we later use to generate 3D voxel structures.
I. Navigation Method 1: Equal Ratios
In the first approach, we test the effects on the generated form of changing ratios between 2 plans during interpolation. As a first test, we give an equal opportunity to both plans (50:50).
II. Navigation Method 2: Unequal Ratios
In the second approach, we change the ratios of the points in each interpolation. By changing the ratio, we give a higher percentage to one of the images to generate more similar images to itself than to the other one.
III. Navigation Method 3: Mixing
For the third approach, we find the average vectors of the 2 points between which we are interpolating and interpolate between those new vectors. A 50:50 interpolation between images is done in this step. In the figure below, we show the results of mixing with different percentages of each plan image.
IV. Navigation Method 4: Combining Mixing & Unequal Ratios
Finally, in the last approach, we combine the last two methods: mixing and unequal ratio interpolations.
The images above demonstrate the performance of the four approaches presented. The order of interpolation in all these methods is from point A to point B then C then D with 50 steps taken between each interpolation.
V. Changing Order
However, interpolating in this order is just one of the possible ways to do so. We visualize, in the figure below, the results of interpolating in different orders between points and show how this simple change drastically affects the generated output.
VI. Changing Points
Furthermore, all these 3D structures are the result of interpolating between four chosen points. Yet, in such a wide latent space, we have infinite options to interpolate between. The figure below shows a gallery of outputs generated with different sets of 4 initial points.
Discussion
The use of GANs and their internal properties in this article presents us with a new set of interaction methods that normally lie outside the traditional boundaries of the architectural discipline. By incorporating these discrete steps into our processes, we demonstrate an alternative experimental space where the form is indirectly generated through a series of different sequential 2D images. We argue that using the pixel and the voxel as the building blocks of architecture can free architectural drawings from tools of precision and also retrieve a drawing’s potential for generative, lateral thinking. Although generated from architectural plans, the resulting highly complex gallery of forms falls at the intersection between conceptual models and architectural structures, portrays distinctive aesthetic features, and transcends the architectural outputs we are familiar with today.
Although the forms are derived from 2D plans, this methodology is not meant to be the sole driver for an architectural project. Also, after generating the 3D structures, the expertise of an architect is required to translate those forms into meaningful architecture proposals. Hence, in order to be translated into a functional tool, further exploration is required to more closely understand the differences and changes in the results produced by the methods applied.
However, the introduced pipeline, if integrated into our digital processes, allows for the generation of several different outputs which may not necessarily be immediately related to each other, qualities that align well with lateral thinking approaches. Such an approach, with its numerous variant outputs and experimental nature, might allow the designer to view a problem from a new light or inspire oblique and new ways of thinking. Thus, this new research has the potential to not only generate new mechanisms for latent thinking in design processes but also to allow designers to move away from established ways of thinking and to question the current techniques of designing embedded in the tools that we normally take for granted.
Conclusion
This article presents a pipeline for future explorations into designing tools that encourage and highlight experimentation and lateral thinking approaches in computational tools of architecture, qualities that are not as prioritized in our tools today. Specifically, in this article, we introduce a 4-step AI-driven pipeline that uses the latent space of a machine as a digital environment to design in, and we demonstrate examples of navigating in this space to generate 3D voxel structures. The results of this process aim to act as examples of possible architectural process scenarios and as transitions between current ways to speculative ways of designing.
We hypothesize that such a pipeline, if fully integrated into digital architectural tools, may enable us to incorporate lateral thinking into our current digital processes and pull our architectural tools out of the industrial era thinking. Hence, by reinforcing the uncertain and the unpredictable, the course of the discipline can be readjusted with our digital futures.
The article also brings forth the use of computers as not only tools but also as collaborators in the process of designing and the next step toward human-computer interactions. Unlike the digital tools we have today, the pipeline presented, instead of following a direct input-output relationship, allows the machine the freedom to work with pixels and contribute its own understanding of the input to generate drawings. This particular space of imprecision and inaccuracy based on machine perception and learned features is where the potential for imagination and lateral thinking lies. This presents us with opportunities of working with architectural drawings in new and novel ways and could allow the discovery of new spatial and previously “unthinkable” concepts. This also introduces a move from design decisions made by only human logic and biases and allows us to reimagine the ways we design and how we articulate our built environment in new and exciting ways.
References
Austin, M., & Matthews, L. (2018, January). Drawing imprecision: The digital drawing as bits and pixels. In Recalibration on Imprecision and Infidelity-Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture, ACADIA 2018.
Bono, E.D. (1985). Six Thinking Hats. Little, Brown and Company.
Chaillou, S. (2019, July 17). ArchiGAN: a Generative Stack for Apartment Building Design. Retrieved from https://devblogs.nvidia.com/archigan-generative-stack-apartment-building-design/
Claypool, M. (2019). The Conference 2019. The Conference 2019. Retrieved from https://www.youtube.com/watch?v=PxAQL7y9wCw
Claypool, M. (2020, January 9). The Digital in Architecture: Then, Now and in the Future. Retrieved from https://space10.com/project/digital-in-architecture/
Cross, N. (2006). Designerly Ways of Knowing. Springer London. doi:10.1007/1–84628–301–9.
Dorst, K. & Cross, N. (2001). Creativity in the design process: co-evolution of problem-solution. Design studies, 22(5), 425–437.
Huang, W., & Zheng, H. (2018, October). Architectural drawings recognition and generation through machine learning. In Proceedings of the 38th Annual Conference of the Association for Computer Aided Design in Architecture(ACADIA), Mexico City, Mexico (pp. 18–20).
Karras, T., Laine, S., & Aila, T. (2019). A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (pp. 4401–4410).
Klemmt, C., Pantic, I., Gheorghe A., Sebestyen, A., (2019). Discrete vs. Discretized Growth: Discretized Fabrication of Geometries Generated with Cellular Growth Simulations. In Proceedings of ACADIA, 2019.
Lawson, B. (2006). How designers think: The design process demystified. Routledge.
Liu, H., Liao, L., Srivastava, A. (2019). An Anonymous Composition. In Proceedings of ACADIA, 2019.
May, J. (2018). Signal. Image. Architecture. Columbia University Press.
Miller, A. I. (2019). The Artist in the Machine: The World of AI-Powered Creativity. MIT Press.
Nauata, N., Chang, K. H., Cheng, C. Y., Mori, G., & Furukawa, Y. (2020). House-GAN: Relational Generative Adversarial Networks for Graph-constrained House Layout Generation. arXiv preprint arXiv:2003.06988.
Newton, D. (2019). Generative Deep Learning in Architectural Design. Technology| Architecture+ Design, 3(2), 176–189.
Isola, P., Zhu, J. Y., Zhou, T., & Efros, A. A. (2017). Image-to-image translation with conditional adversarial networks. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1125–1134).
Retsin, G. (2019). Discrete: Reappraising the Digital in Architecture. John Wiley & Sons.
Steenson, M. W. (2017). Architectural Intelligence: How Designers and Architects Created the Digital Landscape. MIT Press.
