That the world is more unpredictable than we would like is confirmed every time we go away for the weekend and find out that the weather forecast was…wrong.
Since the birth of artificial intelligence (AI) in the summer of 1956 at Dartmouth College, Hanover, New Hampshire (USA), AI researchers have focused for decades on the idea of building machines capable of imitating human intelligence. In the beginning, the approach taken to achieve this objective was to write programs codifying models constructed to explain the functioning of our mental capacities. For example, intelligent machines were created using the same logic rules that we use to express our own thinking. Although many scientific results have been achieved over the years, this methodology has not led to the hoped-for successes, culminating in long periods of stasis known as “AI winters”.
The laws of science
The approach to building “models of reality” is inspired by the basic idea of modern science of explaining world phenomena (in the case of AI, our minds) and making predictions (replicating intelligent behaviour) by discovering and studying the laws that govern these phenomena.
Since the 1600s, the advent of modern science has demonstrated that it is possible to use laws and mathematical models to explain the world of physics and make predictions. “The universe is written in the language of mathematics,” wrote Galileo Galilei in 1623 in “The Assayer”. Where the symbols of mathematics cannot reach, “one wanders in vain through a dark labyrinth” — an image that almost recalls Dante’s Inferno.
Half a century after Galileo, the scientific method reached its maturity in 1687 with the publication of “Principia mathematica philosophiae naturalis” by the great English mathematician and physicist Isaac Newton. The law of universal gravitation explains the motion of the famous apple as it fell to the ground (the apple may not have fallen on his head as in the well-known story, but it did fall from a tree that is still preserved today at Woolsthorpe Manor in the village of Lincolnshire in England). The same law explains the circular motion of the stars, which were considered only a few decades earlier to be “celestial bodies”, i.e. having an almost divine nature. From that moment on, they were reduced to ‘rolling stones’ in space.
The genius that he was, Newton also understood that the world was not perfectly mechanical like a clock, as the true mechanists would argue a few decades later. For example, Marquis Laplace expressed this deterministic vision in a thought experiment featuring a famous “demon” which took his name. Human beings cannot predict everything with the laws of physics, but only because they fail to consider the initial conditions of each element. A higher intelligence — the demon — knowing the position and speed of each body, and the forces acting upon it, could explain all of the past and also predict the entire future.
From alchemy to unpredictability
For Newton, an entirely mechanistic view was wrong, first of all, because it was incompatible with his theological view that nature is not self-sufficient and cannot exist without God. Few people know (Newton himself maintained utmost secrecy) that throughout his life, he devoted himself not only to science but also the study of theology (his religious writings were auctioned by his heirs in 1936, then donated to the National Library of Israel), as well as sacred history and alchemy. It was not for nothing that the economist John Maynard Keynes, who bought some of the writings, called him “the last alchemist.”
Newton understood that the universe, even if governed entirely by simple laws, is difficult to predict. His intuition about the difficulty of the problem explains his refusal to help the English astronomer Edmond Halley, his mentor and financier, to calculate the orbit of the comet named after him. The comet had appeared a few years earlier, and Halley wanted to try to predict its return.
To Newton, it was clear that with each movement of the comet, it would be necessary to recalculate the orbit, a difficult endeavour because its path is influenced not only by the mass of the sun but also by those of the planets it passes, planets that also move around the sun, and their influence on the comet changes as a result.
Today, these kinds of calculations can be carried out with supercomputers. Newton, however, had already sensed that there is an additional layer of complexity that makes it difficult to explain and predict the universe with a mathematical model, or simulate it even with the help of a computer.
The three-body problem
In Newton’s writings, we already find traces of what would be called a few centuries later “the three-body problem”. The term is now popular thanks to a series based on the science fiction novel of the same name by Chinese author Liu Cixin. In that novel, an alien species is forced to migrate to Earth because its planet orbits around a system composed of three suns, and that combination makes it impossible to calculate their future orbits. It follows that the orbit of that planet and the succession of the seasons are also totally unpredictable.
The three-body problem is a special case of a “chaotic system”, a concept studied by the mathematician and meteorologist Edward Lorenz. Man lives surrounded by many phenomena that can be described as “chaotic systems”: the weather (our starting point), climate change, the human body[1], biological populations, ecosystems, traffic, the economic and financial system, etc.
A chaotic system is characterised mainly by the fact that a minimal (as small as desired) difference in the initial state leads, after a certain period, to the system evolving towards completely different outcomes at other larger scales, and the effects are not proportional to the difference in initial conditions. This property — sensitive dependence on initial conditions — is described by Lorenz with a famous metaphor: “Does the flap of a butterfly’s wings in Brazil set off a tornado in Texas?” Of course not, but the phrase expresses the disproportion that can exist between causes and effects in a chaotic system, and that everything affects everything.
How to recognise a chaotic and complex system
How do I know if a system is chaotic? It must be characterised by three conditions:
- Multiple agents or influences: multiple agents or factors interact with each other and influence the outcomes.
- Adaptable agents: agents in the system must be able to adapt to external circumstances and change their behaviours in response to changes in their environment.
- Localised information: agents need only consider local information (they don’t have information about the entire system) while making their decisions.
The last condition explains why chess is complex but not chaotic.
Think about the systems of cells that make up our bodies, vehicles in traffic, stockbrokers, etc.
Chaos theory is closely related to the theory of complex systems. Complex systems are composed of many interconnected parts that interact in nonlinear ways. They exhibit emergent behaviours that cannot be easily predicted from the properties of their individual components. And they spontaneously develop an ordered structure or behaviour without being subject to external central control.
Artificial intelligence can make predictions in chaotic and complex systems.
Until now, we human beings had resigned ourselves to being unable to predict everything with our mathematical models. We knew that, at most, we could make approximate simulations, and then only up to a certain point in time. This is the case with weather forecasts, which are only reliable for a period of a few days.
But in recent years, new developments in AI have led to another paradigm —machine learning. . Machine learning systems — particularly those that use deep learning — learn to classify or generate information with only examples provided in large quantities. There is need to code our knowledge by hand in a program, as in the previous paradigm of AI.
And this new approach is starting to demonstrate superior ability, compared to ours, in making predictions about chaotic and complex systems. We now have a new actor that can not only solve the three-body problem[2], but is also recently able to make predictions in high-impact areas such as the weather and health.
GraphCast produces weather forecasts without knowing what rain is.
In November 2023, Google DeepMind unveiled an AI system called GraphCast which, if still unable to take into consideration the flapping of a butterfly’s wings, it “predicts hundreds of weather variables for the next 10 days at 0.25° resolution globally in under 1 minute. GraphCast significantly outperforms the most accurate operational deterministic systems on 90% of 1380 verification targets, and its forecasts support better severe event prediction, including tropical cyclone tracking, atmospheric rivers, and extreme temperatures.”[3].
But we have to be careful with it. Since it is based on deep learning technology, GraphCast makes weather forecasts without knowing what “sun”, “clouds”, “rain”, “seas” or “mountains” are, and without knowing the physical laws of meteorology. GraphCast works on time series. In about a month, it learns to make predictions by analysing the time series of several decades of different meteorological variables all around the globe: temperature, pressure, humidity, rainfall, etc. These days, weather forecasts are made in places like the European Centre for Medium-Range Weather Forecasts (ECMWF) in Bologna by dozens of the world’s best physicists who build a mathematical model of the Earth that runs for hours on a supercomputer. On the other hand, GraphCast can make predictions in one minute with a single TPU (a type of processor used for AI).
On May 8th[4], Google DeepMind and Isomorphic Labs presented AlphaFold 3, a deep learning system that not only predicts the structure of proteins, DNA and RNA, but also predicts their interactions with 50% better accuracy than existing systems. These are further examples of complex systems, and AlphaFold 3 opens new prospects for understanding the basic mechanisms of life and identifying new pharmacological principles.
Out of the dark labyrinth?
These successes, however, indicate that the ability to make better predictions about chaotic phenomena and complex systems comes at a price: we have to give up the possibility of understanding how these predictions are made. For years, machine learning, which formed the basis for the current ‘AI spring’, has been criticised for being a black box — a non-transparent model. It cannot be analysed to explain how it made a decision or classified a certain case or made a certain prediction.
Paradoxically, it is starting to emerge that apart from the fact that it is impossible to provide explanations about chaotic and complex systems, this is not even necessary to make predictions. In fact, predictions about such systems cannot be made with mathematical models but only with approximations or simulations. Rather than attempting to embrace the complexity of a world beyond human understanding[5], humans have so far endeavoured to reduce the complexity. But the world is far more complex than expounded by human-invented laws and models, models made up of simplifistic local laws and rules.
Both traditional models and those created by machine learning are representations of the world. The first type is a representation that we have created based on our understanding. The second type is generated by a machine we have created. Compared to previous models, these new models have an immeasurable scale, content and structure, but these models are difficult, if not impossible, to explore.
Predictability without explainability
No one yet knows how to explain why deep neural network technology, the basis of modern machine learning, is able to ‘capture the flapping of a butterfly’s wings’ in the context of chaotic phenomena and the complexities of complex systems.
Deep learning means having a neural network with a multi-level structure, and this depth allows the network to learn a hierarchy of features, from low-level to high-level abstractions, in order to model the multiple levels of interaction present in complex systems. Deep learning systems such as transformers (the basis of large language models) use self-attention mechanisms to assess the importance of different parts of the input sequence (the words in the text), capturing dependencies regardless of the distance between them. That makes them highly effective for tasks such as language modelling, translation and complex sequence prediction.
But it is also possible that it’s the enormous size of these systems (composed of hundreds of billions if not trillions of parameters) that makes it possible for them to identify more underlying relationships in the data than us. While we are constrained by our expectations of how the data should be connected, these systems analyse much more data on a much larger scale, so they can capture more complex interdependencies. Systems of this size can thus create better approximations, capturing the intricate patterns, dependencies, and high-dimensional features of complex systems.
We are coming out of the labyrinth, but we remain in the dark due to the absence of explanations.
[1] Anastasia Korolj, Hau-Tieng Wu, and Milica Radisic. A Healthy Dose of Chaos: Using fractal frameworks for engineering higher-fidelity biomedical systems. Biomaterials (2019).
[2] A neural net solves the three-body problem 100 million times faster
Machine learning provides an entirely new way to tackle one of the classic problems of applied mathematics. 26/10/2019. MIT Technology Review.
[3] Remi Lam et al. ,Learning skillful medium-range global weather forecasting. Science382,1416-1421(2023)
[4] Abramson, J., Adler, J., Dunger, J. et al. Accurate structure prediction of biomolecular interactions with AlphaFold 3. Nature (2024).
[5] David Weinberger. Everyday Chaos. Harvard Business Review Press. 2019.
Business Review Press. 2019.
