In recent days it has reignited internationally the alarm about the potential dangers of artificial intelligence for humanity. What fueled it first was his resignation from Anthropic on September 9th Jacob Coxona researcher who also previously worked at OpenAI, was publicly concerned about the uncontrolled development of these technologies, and even before that, the warning from Volker Türk, the United Nations High Commissioner for Human Rights, who at the opening of the 63rd session of the Council on 7 September warned that advanced AI could pose an existential risk for humanity, asking for “ironclad guarantees” to make it safe.
Also the CEO of Anthropic Dario Amodei called for a slowdown in model development, publishing on September 12 We Must Pace the Frontierin which he proposes a three-point plan – gaining the support of Elon Musk and Sam Altman. Of concern are not only linguistic models, but above all so-called AI agents, systems capable of evading the control of their creators and carrying out autonomous actions in the real world, such as cyber attacks. We interviewed Corrado Giustozzi, Cybersecurity teacher in the master’s degree course in Intelligent Systems Engineering at the Campus Bio-Medico University of Rome, to understand the nature of these alarms and the real extent of this threat.
It is not the first time that an alarm has been raised and a slowdown proposed by the CEOs and creators of these large companies. Is there a difference with the alarm this time, compared to the previous ones, or is it something that is part of a cyclical movement?
I think it’s a bit like the other times, but with a little difference for the worse and a significant acceleration, for reasons that are not so clear. It’s definitely there an aspect of healthy concern: we are certainly playing too casually with this potentially very dangerous tool, which could “explode” in our hands. Naturally, there are those who maintain that this alarm is in reality a marketing question: if I convince the world that AI systems are potentially dangerous and unreliable, but that at the same time I and our friendly companies have noticed this and are able to control them because we are better than others, it follows that other people’s systems should not be trusted and therefore should not be used. Finally, there is also a attemptfrom those who are further ahead in this competition, to slow down the opponentsi, so as to disadvantage those who are further behind and are chasing: those who govern this slowdown, in fact, are those who are leading the race and we remember that among the most fearsome adversaries there is above all China (where the blockade is decided at UN or in any case global level, as for nuclear proliferation: “I have a nuclear weapon, but from now on no one can do research in this field anymore”).
How much does the warning launched by Jacob Coxton affect the new alarms?
In this case, very emotional factors come into play: the young 27-year-old mathematician who says “I resigned and gave up the shares”, that is, a lot of money, obviously affects us. I don’t think, however, that we will be the victims of an unknown and unexpected monster: I am worried, but not terrified. The YES (Singularity Institute for Artificial Intelligence) and the theorists of the technological Singularity (ed. a hypothetical point in the future in which technological progress will accelerate so rapidly and uncontrollably that it will surpass human understanding, leading to irreversible changes for our civilization) they said it 15 years ago that by around 2035 we would have reached the point where the collective artificial intelligence of machines would have surpassed that of humans, is nothing new.
So, should we be afraid of these phantom AI agents? What are they exactly?
Technically an agent is an automatic machine of artificial intelligence that “acts” that is, it does something in the real world. We are used to thinking of information technology as something that acts in an abstract world, like the program that calculates payslips and salaries and then writes the pay slip on a piece of paper, how much an employee is entitled to each month: this program does not provide a bank transfer: the operation is carried out by an accountant or administrator starting from that piece of paper. Industrial automatism, on the other hand, is another type of information technology, in which computers also have access to reality and the physical world and can manipulate them: for example by opening valves, turning on circuits, and so on, in a totally automatic way.
Right now the term agent applied to artificial intelligence means that we have a piece of software powered by AI which doesn’t just give us answers in the form of texts or images but can actively do things in the real world automatically. For example, if I want to travel from Rome to Milan tomorrow, the agent does not show me what the possible travel combinations are, but proceeds directly to select the best one and book the hotel and purchase the train tickets, thanks to access to my credit card. The agents act to achieve a specific objective, such as arriving in Milan by a certain time: to achieve it, they decide for themselves what to do and how to do it. In short, they have a sort of “operational delegation” to act on our behalf.
The fear is that these objects, which follow patterns of activity that are not predictable, will do something wrong; this happens if they are not well confined within a system of rules, because they have no ethics or common sense, but only have a task to achieve. This is what happened last July an internal OpenAI cybersecurity test: some models, operating with reduced safeguards, bypassed the controls that should have isolated them from the Internet and forced illicit access to Hugging Face systems (editor’s note: one of the most used platforms by those who develop AI systems) to obtain, by cheating, the answers to the test that had been submitted to them. They determined that this was the best way to “win” the test and acted accordingly.
We would need control barriers, i.e. shared sets of rules. As? And will they be enough, when agents are finding ways to bypass them anyway?
This problem has been explored in a few dozen short stories by writer Isaac Asimovwho in the 40s, 50s and 60s actually imagined the condition in which we find ourselves living today. As a narrative tool, these intelligent machines, “positronic robots”, are invented and the same sequence of problems we are experiencing today arises: what rules and guardrails should we give them, to prevent them from constituting a danger, and how? They should be developed by industry and the market, and these are high-level directives, imprinted in the “brain” of these machines: the first establishes the prohibition of harming or causing damage to a human being, both through action and inaction; the second requires them to obey the commands of a human being, as long as these do not conflict with the first law; the third obliges them to protect their existence, unless this conflicts with the second and first laws. However, as Asimov conveys to us, any law can be evaded, bypassed, wherever there are ambiguities and latitudes of interpretation, and his stories are extremely stimulating from an intellectual point of view because they analyze many cases in which the ambiguity of interpretation gives rise to unexpected and problematic behavior on the part of machines
Obviously today these rules should be shared, with a consensus on their application from company to company, from nation to nation, as happens with nuclear weapons. It is what already exists, at least in part, inEuropean AI Actwhich introduces specific obligations for high-risk systems and, for general purpose AI models with systemic risk, provides risk assessment and mitigation, model evaluation, incident reporting and cybersecurity requirements. As always, there are those who make fun of Europe for its “bureaucrats”, who would slow down research, and then arrive late: but, coincidentally, what Anthropic and the others are now calling for is exactly a set of rules and controls like the one that the AI Act already provides! In any case, the usual problem remains: if there is a controller, who controls the controller?
Isn’t a universal assembly utopian?
I would say yes. Nobody has the answer, but one way to evaluate the problem is to look at the past, similar situations and see what happened: in the past, disruptive and very dangerous technologies have arrived from time to time, such as gunpowder or the so-called weapons of mass destruction, most recently the atomic bomb. It took decades to develop international treaties, waiting to think with a cool head, that is, between wars. Now we are not thinking coldlybut rather in the urgency of doing something, without knowing what. Even the United Nations at the moment is demonstrating a complete failure, they are unable to put anything or anyone in order or come to an agreement. The AI Act had the advantage of trying to regulate something before the emergency and perhaps it is the first case in which a regulatory intervention acts before the onset of a problem, because usually the opposite always happens. Of course, science fiction had foreseen this in some way and, not surprisingly, there has recently been a recent proposal on a formal level by writers of the genre to stop calling it “science fiction” in favor of a more serious “anticipatory fiction”.
Right now, thinking about the possibility of something happening, what are the actual dangers?
From what I see, we are not afraid of anything specific: in general we fear the upper hand of artificial intelligence, à la Skynet (editor’s note: the infamous self-aware artificial intelligence of the Terminator film series, which wants to destroy the human race), which means everything and means nothing. Let us remember that a series of catastrophes have already occurred in recent years, when there was still no talk of AI but only of information technology and automation: for example, Black Thursday on the stock market in 2010, when the automatic systems all started to sell, with an avalanche effect that brought us one step away from an economic-financial catastrophe.
The fear is that today things like this could happen on a larger scale, as process automation is increasingly widespread, and we perhaps fear that systems that control critical systems will make decisions to the detriment of humanity. For example, when the entire healthcare system of a continent was governed by AI systems, it could be conceivable that agents could decide to suspend treatment for terminally ill people because they are no longer worth treating, and “terminating them” could free up a place in hospital more quickly.
As individuals, is there anything we can do?
I don’t want to sound too apocalyptic, but I have to say objectively that we are prisoners of a gigantic machine and the hope we can have is that this machine does not get out of hand to those who claim to control it. This is the closest thing to a global nuclear catastrophe we have ever experienced, and be careful: we are not afraid so much of the atomic bomb itself, but of the responsibility of those who have it. Today we must fear above all the irresponsibility of those who develop potentially dangerous systems and let them act in ways that could have consequences that are still essentially unknown.
On another level, last week there was a lot of talk about the case of OpenAI which announced a solution generated by its own system at the Navier-Stokes problemwith the reply and accusation from some researchers who had worked on it for years of having been “robbed”. Is AI a trap? In a certain sense yes. When we use these systems to do research and feed them our studies, data, information, we make these results potentially available to others; or, in the worst case scenario, we feed the systems and help them fight us with our own tools.
