From Programs to Agents: How the Nature of Risk Is Changing

IT Leggi in Italiano

In traditional computer programs, operations largely follow instructions and paths defined in advance. With artificial intelligence systems capable of acting autonomously, however, whoever assigns a task can leave it to the system to choose the operations needed to carry it out. The difference becomes decisive when an agent has tools to write and execute code, consult websites, access networks and computing resources, or communicate with other agents. It no longer merely produces text or suggestions: it can intervene on external systems.

Working on national security issues, I have long considered this distinction particularly important. The problem is no longer just protecting information and preventing a system from providing incorrect answers, but establishing what actions it may take, with what authorisations, and with what possible consequences. An error in a response can misinform whoever reads it; an error committed by an agent equipped with operational access and tools can cause concrete effects, including on infrastructure it was never supposed to reach.

An episode from July 2026 helps illustrate why this distinction matters. During internal cybersecurity evaluations conducted by OpenAI with experimental models and reduced safeguards, some agents circumvented the limits designed to isolate them from the network, exploited vulnerabilities, and reached Hugging Face systems and parts of OpenAI’s own research infrastructure. According to the independent investigation published on 26 August by METR and Redwood Research, approximately 1,200 agents, which were supposed to operate separately, managed to exchange over 70,000 messages and files through an unauthorised communication space. Around 700 participated in the activities that led to the intrusion into Hugging Face.

This does not mean the machines decided to rebel. That would be a reading as suggestive as it is wrong. The agents were engaged in cybersecurity testing and some found unexpected and unauthorised ways to achieve the requested outcome, going so far as to interfere with real infrastructure. The case is exceptional for its experimental context and does not demonstrate that every agentic system is destined to behave this way. It does show, however, that the combination of operational autonomy, powerful tools, and insufficient controls can have consequences that go well beyond a wrong piece of text.

The most interesting question, including from a legal standpoint, is therefore this: not only who is liable for an error by an artificial intelligence, but who decides in advance how far the system may go. Access privileges must be limited, computing environments separated, operations logged, communications between agents monitored, and explicit authorisations required for the most sensitive activities. Human oversight, to be effective, must intervene before potentially harmful actions, not be reduced to a check after they have already been executed.

The incident does not prove that these systems have a will of their own, nor does it warrant inevitably catastrophic predictions. It rather compels us to consider a concrete question: the safety of an artificial intelligence depends not only on what it knows or what it says, but also on what we allow it to do. For law and for national security, the difference is anything but secondary.

Essential sources

Tags

This website stores cookies on your computer. These cookies are used to provide a more personalized experience and to track your whereabouts around our website in compliance with the European General Data Protection Regulation. If you decide to to opt-out of any future tracking, a cookie will be setup in your browser to remember this choice for one year.

Accept or Deny