Evolution is Cleverer Than You

Using Simulators to Mechanize Design.

Design is tricky because it’s rarely obvious how to phrase a problem such that there’s a technology solution that can be built to address it. Design is a lot of work because when low level details interact with each other, unforeseen problems start to crop up. Even something as simple as designing a basic web page in html requires a short period of defining layout, color schemes, etc. followed by a potentially longer period of tuning the html to account for the minor incompatibilities of every browser (and each of their versions). When designing a dishwasher or a nuclear reactor, the number of small problems to take care of just gets larger and larger.

This is actually a pretty well worn subject, so let’s not belabor the point that working through the details of a design takes a lot of effort.

Software Simulators

One of the best ways to deal with low-level incompatibilities in new product designs is to build software simulators to predict how a design will work in a given situation, without actually building and deploying the products into the real world.

There are two kinds of simulators to consider:

Logical Simulators: use defined rules about what will happen in certain situations.

Data-driven Simulators: evaluate a design by performing calculations against a data set previously seen in the real world.

A virtual wind tunnel that can calculate the performance of an airplane design without actually building the plane is a logical simulator because it applies laws of physics to the design and aggregates the results over the course of a simulated flight. An example of a data-driven simulator is the application of a stock-trading strategy to historical market data and aggregating the simulated profits that would have occurred if that strategy had been used over that time period.

Logical simulators are actually fairly common, particularly physical simulators that engineers use to develop things like machinery and buildings.

Data-driven simulators, however, have more potential. Data-driven simulators don’t necessarily require the operator to develop a deep understanding of a system ahead of time, and can solve many problems by evaluating designs against real world data and simply choosing the design that demonstrates the best quality metrics. Furthermore, as new real-world data comes in, the data-driven simulator can re-evaluate designs using the new data and update the quality metrics of the available designs. This is critical for situations where the quality of a design is influenced by changing market conditions. For instance, if the cost of an input to a manufacturing process changes or if a competitor drops their price, the evaluation criteria for a product’s design could change overnight.

Building quality simulators requires considerable costs, however. So much so, that for many applications it hasn’t been worth the expense. Logical simulators require a very detailed model of complex real-world situations that has taken decades to develop in the few areas they are widely used such as physics simulations used by architects. For data-driven simulators, the machine learning development alone can be prohibitively expensive because the software tools available are immature. These development costs will hopefully come down considerably in the near future with improvements to practical engineering techniques for statistical analysis.

Simulators and Automated Design

While the costs of building simulators can be expected to fall due to somewhat predictable improvements in the needed software tools, the benefits of investing in simulators will explode due to the rise of automated design. It’s already possible to automate design with existing machine learning techniques applied to the design process, in fact. They’re just not very widespread.

For example, consider a stack of blueprints with 100 different versions of a product. If we score each blueprint by a set of quality metrics, we can apply statistical methods to find patterns that relate design choices and quality. To automate this type of analysis, a machine-learning algorithm must be able to explore the “design vs. metrics” space by generating designs, submitting them to a simulator to be scored, and using the output information to influence the next generation of designs. At each iteration, the generated designs have an increasingly high probability of being the “best yet.” Over large numbers of iterations, a decent design algorithm has a very high chance of reaching a design with good quality metrics.

Until very recently, the aforementioned “large number of iterations” was a deal-breaker for all but the most well-funded projects in finance and defense. The computations needed are ridiculously parallelizable, though, so to evaluate 1,000 designs using a software simulator, you can run the simulations at the same time if you have 1,000 machines. These types of infrastructures are now common due to an increasing number of cores per CPU and the ability to rent large numbers of CPUs from cloud computing vendors.

Harnessing Evolution is Hard

Regardless of the costs, automated design techniques like Evolutionary Algorithms can still seem like they’re more trouble than they’re worth. Machine learning algorithms tend to give you answers that are exactly what you asked for, but not what you really meant. In the case of automated design, this comes in the form of high scoring designs that cheat by exploiting flaws in the simulator or evaluation metrics. It’s similar to lowering your tax bill by exploiting loopholes in the tax code rather than donating money to charity. It’s technically valid but not what lawmakers actually intended when crafting the tax laws.

Evolutionary biologists have a saying to describe how extremely complex designs emerge in biological systems when survival of the fittest is the only mechanism in place: “Evolution is cleverer than you are.” That is, humans inherently lack the ability to design, say, a chimpanzee, even if a machine existed that could fabricate it. The human designers wouldn’t be able to define the DNA sequence to plug into the machine. Only a trial-and-error system of evolution could continually improve an existing design (like an ancestral primate) to create something so complex in this way.

That’s why it’s reasonable to expect that in the long term, a well-tuned evolution machine will make designs faster and actually better than a human could. The difficulty remains in the short term, where the person who is “less clever than evolution” is working with an evolution machine, generating designs that have high simulated scores but still aren’t quite what she was looking for. The original problem of hand-crafting the design itself is traded for the problem of fine tuning the simulators and scoring mechanisms being used by the evolution machine. The cleverness of evolution is therefore at the heart of both the long-term promise and short-term pain of trying to conjure the creative force of evolution in the design process.

Machines are for Speed, Humans are for Acceleration

The implications for what automated design systems can do to amplify productivity in design are quite profound. When engineers and designers have simulators that reflect their model of the world, and use evolutionary machines to construct designs in that virtual world, product development becomes automatic. Human effort then becomes focused on improving that virtual world (by way of improving the simulators and design elements). The act of running the mechanized design program also provides immediate feedback: if the optimal design given by your simulator is a bad design, then there’s a problem with your virtual world.

In the end, it is expensive and time consuming to build up simulators and libraries of design elements that work with automated design systems. More so to get to a first product designed by machines that is competitive with existing products designed by humans. But ultimately it’s a question of velocity vs. acceleration. An organization that designs products with a high velocity using human ingenuity will inevitably fall behind one that uses its human ingenuity for acceleration, no matter how fast the high-velocity company is moving or how much of a head start it has.

Video: Automated Design of Trading Strategies