On the 8th of October, 2024, two scientists named Geoffrey Hinton and John J Hopfield were awarded the Nobel Prize in Physics at a ceremony in Stockholm, Sweden; on the surface, there wasn’t anything unusual, except that Hinton and J. Hopfield were not physicists, but computer scientists, who were awarded the prize because of their foundational work which is the basis of modern artificial intelligence: artificial neural networks.
A neural network, in the broadest sense of the term, is an ensemble of interconnected neurons relaying signals to one another to perform some function. The human brain is made of such neural networks, making them the basic element of cognition. For this reason, scientists interested in developing artificial intelligence have been trying to replicate them.
Artificial neural networks are a model of neural networks simulated on a computer characterized by a series of “nodes” (representing neurons) that interact through signals. Each one of these “artificial neurons” has as many parameters as connections toward other neurons. These parameters can be represented by numbers, which are also present in some sense in biological brains. They regulate the signals exchanged between neurons. While initially set randomly, the parameters are adjusted through training. The development of an adjustment method or algorithm was indeed the major achievement of the two scientists. During this “training” for example, the artificial network may be fed with images in an attempt to make it identify the colors or objects depicted; when the neural network fails (as it usually does at the beginning), the parameters are re-assigned until the network eventually succeeds.
The work of John Hopfield consisted in finding a way to employ neural network qualities and use them in physics such as the energy level attributed to objects in an image. Geoffrey Hinton employed this method to create his own type of neural network: the Boltzmann machine, named after physicist Ludwig Boltzmann. Today, this process is that the basis of every featured artificial intelligence such as ChatGPT. Originally, the two computer scientists wanted to employ these systems to categorize physical phenomena. Now, they are used for everything else (though preferably not school homework).
Despite these incredible advancements, artificial intelligence based on neural networks has several drawbacks: the process of changing the parameters of each neuron in such a way that the network gives the desired answer is incredibly long, as millions of mathematical operations need to be carried out. It is also energy-costly, as these operations are carried out by electricity-consuming computers that mostly derive their energy from fossil fuels. Many of these issues sprout from the inability of artificial neural networks to retain the “teachings” given to them by training, which the human brain instead can do much more efficiently. For example, a child who is first pointed out a plant will be able, from that moment on, to recognize almost every plant species. An artificial neural network needs to be provided with images of different plant species, with different views and illuminations for hundreds of thousands of times.
This inability of artificial intelligence to quickly and efficiently learn is connected to another mystery within human nature: the neurons and synapses of the nervous system are vastly more numerous (~80 trillion) than the DNA bases (12 Billion bases). DNA theoretically includes all the information needed for the body to develop, including the brain. In other words, there is not enough information in the DNA to fully define all the brain “parameters”.
Yesterday I interviewed my father Andrea, who has co-founded a few companies in AI, on the issues faced by modern neural networks. He thinks that the human DNA does not have instructions for each specific neuron within the nervous system. Instead, it contains a natural algorithm. While most people think that algorithms are solely employed by computers, algorithms are simply an interesting term to indicate a series of steps to follow. According to my father’s theory, human DNA contains an algorithm that allows growing nerve cells to organize themselves in a way that favors as efficient a neural network as possible. Finding such an algorithm and replicating it on a computer may be the key to developing true artificial “general “intelligence, the ability of artificial intelligence to carry out any task human beings can.
