Towards Artificial Intelligence in Architecture: How machine learning can change the way we approach design

Advances in artificial intelligence and machine learning are often perceived as a threat to the creative process. But at Foster + Partners, the Applied Research and Development group is rethinking this mindset and embracing machine learning, not to replicate or replace designers, but to enhance our knowledge, instincts and sensitivities, free us from routine tasks, and to optimise and push the boundaries of our designs.

Architecture today is a multi-skilled profession, calling on disciplines from structural and environmental engineering, to social and material sciences. At Foster + Partners, we have a long history of exploring, adapting and harnessing available technologies in order to expand and improve our design capabilities. At the forefront of this is the Applied Research and Development (ARD) team, who have recently been investigating the potential of using artificial intelligence (AI) in the creative process.

As a field of academic study, AI was established in the 1950s and can be broadly defined as the development of machines that display human intelligence and behaviours. Since its genesis, it has been a fertile topic of debate, repeatedly making headlines – both good and bad – and becoming enshrined in popular culture and science fiction. Thankfully, AI has continued its evolution with mostly altruistic innovations: it can help us to understand and react to human speech, screen for breast cancer, develop robots and self-driving cars, or just recommend us films we’d like to watch.

The cutting-edge area of ARD’s research investigates the potential for these systems to provide genuine design assistance.

There are many systems and processes that need to be perfected in order to accurately mimic human intelligence; an example would be the subset of AI known as machine learning, summarised by computer scientist Tom Mitchell as ‘the study of computer algorithms that allow computer programs to automatically improve through experience.’

In the context of architecture and design, machine learning has great potential to analyse our designs more quickly and at less cost. The ARD team have been exploring two ways in which it can be incorporated into the design process, as well as some potentially more revolutionary applications.

The first is known as surrogate modelling; a direct replacement for analytical engineering simulations (such as structural deformation, solar radiation, pedestrian movement), which take valuable hours, or even days to complete.

There is an enormous scale and complexity to many architectural projects today, typified by a varied array of intersecting expertise and technologies. To assist in this diversified field, machine learning might enable us to solve problems and detect patterns historically dependent on the complex analytical simulations and programmes. Design optimisation based on their findings has been nearly impossible due to the time and work involved, undermining the benefit these tools can provide.

To solve this problem, we need to provide designers with results in near real-time. For that reason, the ARD team investigated surrogate models, where a computationally ‘cheaper’ predictive model is constructed based on a number of intelligently chosen simulation results. The designer can then have results from the surrogate model’s approximation of a simulation in real-time, which is good enough for them to make a quick decision.

The second – and more cutting-edge – area of ARD’s machine learning research we call ‘design assistance’ modelling, and the potential is for these systems is to work alongside the intuition of designers in the creative process.

We call this design-assistance modelling because it helps facilitate architectural processes for which we do not necessarily have an analytical answer – one that can be derived from simulations. This could be, for instance, the optimal spatial layout of furniture within a space or providing designers with document control assistance in real-time while they are working.

Imagine, for example, a child learning to balance a pen on its finger, getting a feel for how the weight of the pen is distributed as they try to balance it. Unremarkably, at the beginning, it will fall – a lot – but every time the pen falls, the child gets closer to identifying the ideal balancing point.

However, when the child is presented with a new pen, with a different shape, the exact rules established for the previous pen will likely no longer apply. So, in order to learn how to balance any pen (and become an expert ‘pen-balancer’), the logical next step is to set about trying with hundreds, if not thousands, more. The resulting expertise allows a person to infer – subconsciously analysing factors like shape, profile and cross-section – how the weight of a new pen is distributed and its ideal balancing point. The quality of our guesses will be highly dependent on the amount and quality of our data and training. In the case of our child example, this sort of training occurs naturally over the course of childhood, as we pick up and interact with all sorts of objects. Appropriately, in machine-learning terms, this is called the ‘training’ phase.

Now imagine that we want a robot to do the same task for us. We would need to collect data about different pens – their length, cross-section and weight distribution – and then test for the optimal balancing point. We would go on to write a procedure that gives explicit conditional instructions to the robot, manually encoding the knowledge about the pens we surveyed. The program would be long but, at the end, given a pen from those we collected, our robot can potentially balance it on its finger.

But what happens if we want to use a pen from outside our original survey, or maybe a paint brush? Since we have had to explicitly define the balancing point (output) for every possible pen in our survey (input), we would have to rewrite or expand the code for every new object. This is not always possible, and certainly isn’t efficient.

An artificial neural network avoids this situation by going through its own training phase – the learning process of trial and error. Once trained, the robot should be able to perform the task on its own, without us having to give explicit and exhaustive instructions. Not only that, but if the training is successful, the network will even generalise, and the robot will be able to balance any object with properties similar to those of a pen. It will have learnt to infer – like the child – how to do so; this is called the ‘inference’ phase.

However, despite its noticeable benefits, the Palladian villa example can only go so far in demonstrating the potential for machine learning to assist in the creative process. What makes machine learning interesting is its potential to effectively respond to a ‘creative’ question, one to which there is not an objectively correct response. The Palladian villa typology – based as it is on a finite number of built examples – can be defined through a description of its fixed architectural rules. We were asking the system a question to which we could already find an accurate answer (admittedly at a cost in terms of time and labour).

What if we wanted our system to capture the more subjective, elusive or unpredictable qualities of architecture? We can know what the defining characteristics of a Palladian villa are, but can we universally pinpoint all the characteristics of a successful public plaza, for example? If we train a machine learning system with a set of thousands of public spaces and point out the successful ones (it is much easier to identify an already successful public space than it is to comprehensively define what it is that makes it so), that system could then be tasked with generating other spaces that have similar traits.

Architectural datasets are rich, complex and often produced in great numbers. They can provide huge amounts of invaluable information and are ideally suited to train AI systems.

In many cases, this training process comes up with invaluable results: the new spaces would be embedded with characteristics and correlations that may be too obscure or complicated for the user to infer, formulate and encode independently. It is worth pointing out that one of the downsides of these systems is that we cannot interrogate the machine about said characteristics, despite the fact that they are usually incorporated, we can only review its results.

Data is a necessity in both surrogate and design assistance models. Generally, there are two types of data we can work with: original or synthesized. At Foster + Partners, original data exists as our archive of drawings, models, sketches, details and so on, produced over the practice’s more than five decades of existence. Synthesized data, on the other hand, is automatically generated with the help of a generative design system. Examples of such datasets that we’ve used during our research are discussed further below.

Currently, each data type presents its own challenges. For original data, for example, it can be time consuming to sift through decades of digitally archived work in different file formats, produced by thousands of employees, for thousands of different projects, to identify the data that would be applicable to our task. While synthesized data may not always be an option either, as creating generative models with the richness of information required may prove an almost impossible task.

The team was interested in the material’s morphological deformation: a controlled transition from an initial state to an end state and back. However, there is a non-linear relationship between the laminates’ internal forces and their displacements. This is opposed to a more common linear analysis, where the effect of tweaking input parameters on the output is predictable. For example, a cantilevered beam of known dimensions and material will always deform in the same way under a particular constant force. But it will deform in a non-linear way during an earthquake. As a result, non-linear analysis requires a sophisticated and time-consuming simulation strategy.

In our research every laminate had an initial, non-deformed state (a ‘fully open’ façade in the adaptive shading example) and a target deformation (a ‘fully closed’ façade). However, faced with a non-linear problem, it is difficult to predict how different parameters – such as the pattern of the thermo-active material over the laminates – would affect the resulting deformation under a given temperature.

An adaptive, passively actuated facade … would self-deform under external light conditions – like an eye’s dilating iris – to provide shading, prevent overheating or increase privacy

One way to approach this problem would be to repeatedly change the laminate’s layering and analyse the results, hoping that the changes would steadily yield results closer to the target deformation. However, given the complexity of the non-linear simulation, this process of trial and error would be extremely time consuming. So, we decided to use machine learning to build a surrogate model.

Before the system could begin its ‘training’, Foster + Partners and Autodesk had to produce enough synthesised data from which it could learn. We developed a parametric model that generated hundreds of laminates and simulated their deformation. This process was run in parallel and distributed (using our in-house custom-written software called Hydra, which builds and analyses the data tens-of-times faster than commercial modelling software) on our computer cluster. Subsequently, we took this dataset (the deformations derived from the initial states) and fed it into two artificial neural networks competing against each other.

In this instance, rather than providing a single neural network with data, such as pens, and it training to balance them, we are asking two networks to learn from and improve each other. In our example, one network would be a teacher – a ‘pen balancer’ – that has data on both pens and balancing points, and it ‘trains’ the other network to design new pens that balance from a specific point of its choosing. As the latter network’s training progresses, it gets better at creating pens that can balance wherever it intends. At the same time, the teacher-network is also improving and getting a better understanding of the physics behind how pens balance. This process is called adversarial learning and, ultimately, the pair reach an equilibrium where both networks cannot improve anymore.

[Our] workflow not only completely challenges the way these laminates are designed, it also suggests a methodology that reduces the material’s costly and time-consuming prototyping phase

In our research, we trained this system – known as a generative adversarial network (GAN) – on deformed laminates (the ‘inputs’) and the pattern of these laminates required to cause a particular deformation (the ‘outputs’). After its training, the machine learning system was then able to create accurate laminate layering for a known deformation result within milliseconds.

The result allowed the research team to prototype a simple yet novel application where a designer could design a laminate in a target deformed state – such as a fully shaded facade – and be presented with the necessary cut-out patterns that would produce this result.

This has resolved the inversed problem with which we started: instead of a designer repeatedly adjusting the laminates to produce an acceptable deformed state, the system allows the designer to conceive the output deformed state first, and then be presented with the required material pattern. We are effectively using the initial surrogate machine learning model as a design-assist one.

This workflow not only completely challenges the way these laminates are designed, it also suggests a methodology that reduces the material’s costly and time-consuming prototyping phase, opening up the potential for its practical inclusion within an architects’ advanced and sustainable material swatch book.

As part of the system’s ‘training’, it is continuously trying to improve itself, comparing the results it creates against those it was provided with. Its results at the beginning are horribly inaccurate. But, as time goes by, and as the artificial neural network learns from feedback, it starts to correct its mistakes and its answers get closer to being correct. By the end, the model finds its own ‘recipe’ to map from input (a floor plan) to output (a floor plan with the correct spatial and visual connectivity analysis) – and it’s not a recipe that we provided. The system can then analyse new floor plans based on its completed analysis of thousands of examples, training itself over and over again through trial and error.

Once the machine learning model has completed its training, it can be put to work running both spatial and visual analyses on any of an architect’s floor plans, producing results in less than 0.03 seconds. Equipped with such a powerful tool, the designer can now see the effectiveness of a change in office arrangement on screen as they make the adjustment.