Leggi in ItalianoYesterday afternoon I was invited to take part in a seminar at the University of Milano-Bicocca, one of the meetings in the series Beyond the Mirror of Technology: Social Scoring, War and Control, organised as part of the activities of TecSEDU – Emerging Technologies, Society, Ethics and Human Rights. The discussion centred on an episode of Black Mirror I remembered well, Men Against Fire, and bore a title that already hinted at the problem: Augmented Soldiers: Enhancement or Dehumanisation?
Watching the episode again to prepare for the seminar, I realised the question I wanted to start from was very simple: who really decides to kill?
At first sight, the answer is straightforward. The soldier decides. Stripe holds the rifle, identifies the target and pulls the trigger. We are not dealing, at least not in the strict sense, with an autonomous weapon that selects and strikes its own target. A human being performs the decisive act. Yet after watching the episode it becomes hard to maintain that Stripe exercises effective control over the conditions in which he makes that decision.
The MASS system, the neural implant used by the military (called “mask” in the Italian version), does not simply order him to fire. It does something far more sophisticated: it constructs the reality in which the decision is taken. The people he must kill appear to him as monstrous creatures, the roaches; their voices are altered and his perception of smells is manipulated. When the system begins to malfunction, Stripe gradually discovers that the enemy has not changed. What has changed is the way he has been allowed to perceive it.
At this point I found myself returning to Paul Watzlawick, and to one of the books I value most and keep coming back to: How Real Is Real? Confusion, Disinformation, Communication. The introduction opens with a sentence that, re-read with Stripe in mind, seems written for this episode: “This book deals with the way communication creates what we call reality.”
Watzlawick was of course talking about something different. He did not envisage a neural implant capable of transforming a person’s face before our eyes. His reasoning concerned the way in which, through communication, language, context and relationships, we assign meaning to what we perceive. We are accustomed to thinking that reality is simply there, in front of us, and that to communicate means to describe it. Yet the categories we use help to define the world in which we act.
This does not imply that facts do not exist or that all interpretations are equivalent. It means acknowledging a possible distance between what happens and the representation through which we understand it. Men Against Fire pushes this intuition to an extreme. In front of Stripe there is a person, but that person has first been classified as a roach, associated with disease, genetic degeneration and danger. MASS completes the process, turning her in his eyes into a monster. Material reality does not disappear: it becomes inaccessible to Stripe precisely when he must decide.
Here Watzlawick’s question, “how real is real?”, acquires a concrete meaning for the debate on artificial intelligence applied to warfare. There is much talk of human in the loop, of meaningful human control, of the need for the final decision on the use of force to remain in the hands of a human being. These are important expressions, but they risk becoming reassuring procedural formulas when we do not ask what information that person receives and how well they are truly able to assess it. To decide, one must have sufficiently reliable data, understand the limitations of the tools used, have the time to compare conflicting information and, above all, be able to doubt the machine’s answer. If the system has already determined what the operator can see, the fact that he is still the one pulling the trigger tells us little about the substance of his judgement. In the case of MASS, I would call this an automation of perception: the soldier remains in the decision-making process, but the cognitive and moral conditions in which he decides have been altered.
The episode’s title takes us to a story that precedes artificial intelligence. Men Against Fire: The Problem of Battle Command is the volume published in 1947 by Samuel Lyman Atwood Marshall, an American military historian who, studying the behaviour of US soldiers during the Second World War, claimed that only 15–20 per cent of infantrymen actually used their weapons against the enemy in combat. Marshall attributed this reluctance, at least in part, to a psychological resistance to killing another human being. Those percentages and the methodology behind them were later subjected to very serious challenges: it would therefore be wrong to treat them as scientifically established fact. Nevertheless, the influence his thesis exerted on the way military training and combat behaviour are conceived has endured.
Almost fifty years later, Dave Grossman, a former US officer, took up that reflection in On Killing: The Psychological Cost of Learning to Kill in War and Society (1995). For Grossman, modern training, through repetition, conditioning and the use of human-shaped targets, can reduce the reluctance to fire on another person. Some of his quantitative reconstructions also remain controversial, but the connection with Black Mirror is not a conjecture: Charlie Brooker has expressly named On Killing as one of the sources for Men Against Fire, referring in particular to the relationship between distance from the target and the psychological consequences of killing. In essence, the episode imagines an even more radical transformation. Training seeks to modify the soldier’s response when confronted with a human being; MASS ensures the soldier no longer recognises the target as such.
A passage from Marshall is particularly striking when read today. He wrote that adapting the machine to man constitutes only half the problem, and that the other half consists in “conditioning man to the machine”. The phrase had a precise meaning in the context of the relationship between fighter, weapon and training. Re-read in the light of Men Against Fire, however, it takes on a more disquieting meaning: the machine is no longer merely a tool to which the soldier’s body and mind must become accustomed; it intervenes in the very perceptual mechanisms that might restrain him from killing. The other’s face, voice, cries, the possibility of recognising them as a person: all of it can be removed or transformed.
The attempt to alter the psychophysical conditions in which a combatant operates, moreover, long predates Black Mirror. In the twentieth century, pharmacology too was pressed into service. During the Second World War psychostimulants were used by armed forces: Pervitin, based on methamphetamine, by the Germans; amphetamines, including Benzedrine, by the British and Americans. It would be inaccurate to present this as a programme designed simply to turn soldiers into killers. The stimulants served above all to combat fatigue, maintain alertness and extend operational capacity; they could also affect mood, confidence and behaviour. What strikes me is the continuity of the question, despite the radical difference in instruments: how far can one reshape the human being to make them more effective in combat? Training acts on learned responses, pharmacology on the psychophysical state, MASS on the very perception of what stands before one’s eyes.
Artificial intelligence today introduces a different possibility, and in some respects a more subtle one: altering the informational environment in which a decision is taken. One need not imagine an implant capable of materially transforming the enemy’s face. A decision-support system can select the information shown to a commander, combine data from different sources, classify people and objects, assign probabilities, set priorities and flag an individual as a potential target. The machine can formally leave the final decision to the human being and already have contributed decisively to delimiting what that person will take into account.
The International Committee of the Red Cross has also drawn attention to this, highlighting the risks of AI-based decision-support systems. These tools can facilitate the analysis of large quantities of data and, if properly designed and used, contribute to more informed decisions. They can, however, also produce automation bias: the tendency to attribute excessive authority to the machine’s indications, especially when decisions must be taken quickly, to the point of merely rubber-stamping its conclusions. The risk concerns not only the error of an algorithm, but also the loss of the human capacity to recognise it as such.
This is where I return to Watzlawick. An algorithmic system contributes to constructing what we might call our operational reality, determining what becomes visible, relevant or suspicious. Suppose a system analyses ten thousand people and shows me ten because it considers them potentially dangerous. I can still decide what to do about those ten, but my field of attention has already been structured by a selection I did not make and whose criteria I may not know. Contemporary artificial intelligence is not MASS, and the analogy should not be forced. The episode does, however, allow us to see with particular clarity that control over a decision also depends on control over the representation that precedes it.
There is another aspect of the episode that makes this story still more disturbing. Technology does not create the enemy on its own. When Katarina finally manages to speak to Stripe, she tells him about the screenings, the genetic controls, the registries and the emergency measures through which certain people were progressively isolated and represented as carriers of diseases and undesirable hereditary traits. In the end, Katarina and Alec are no longer individuals with a name and a history: they belong to the category of roaches. Before the implant visually transforms a person into a monster, language and social practices have already turned them into a being to whom protection can be denied.
One detail is particularly telling: the villagers do not have MASS. They see the roaches for what they are — human beings — and yet consider them just as dangerous and undesirable. They do not need a neural implant to dehumanise them: propaganda, classifications and prejudice suffice. MASS makes technologically more efficient, for those who must actually carry out the violence, a process that has already occurred in society. One can come to deny the other’s humanity while continuing to see them perfectly well.
The history of persecution shows how often this process also passes through administrative procedures of neutral appearance: classifying, registering, separating, attributing biological or hereditary characteristics, replacing the concrete person with a category deemed dangerous. In the episode, the technological device arrives at the end of this chain. And perhaps that is precisely why it works so well: its representation of the enemy finds ground already prepared by years of discriminatory discourse and practice.
There remains the question that, as a lawyer, I cannot avoid: who is responsible? Stripe killed people believing he was seeing monstrous creatures. It is hard not to regard him, at least in part, as a victim of the manipulation. This, however, is not enough either to affirm or to rule out his individual responsibility. One would need to inquire into what he knew, what he could have known, the nature of his error and the actual possibilities of refusing the order or questioning what he saw. The legal consequences would also depend on the context and the applicable rules; the episode does not provide enough to reach a definitive characterisation.
Stopping at the soldier alone, on the other hand, would mean ignoring the workings of the system that made the killing possible. Who decided which people to classify as roaches? Who authorised the operations? Who designed and approved a technology capable of manipulating perception and memory? Who knew the effects of the device, and who had the power to prevent its use? The intervention of technology may make it harder to reconstruct the knowledge, powers and conduct of the various actors involved. It does not, however, create a zone beyond the reach of law. The different forms of responsibility must be distinguished, without assuming that the person who wrote the code, the one who commanded the operation and the one who fired answer automatically on the same grounds. The point is to prevent the distribution of functions from turning into a distribution of excuses.
One scene is particularly effective: the consent scene. Stripe protests that he never agreed to what was done to him. They show him a recording of the moment of his enlistment: he gave his consent, his thumbprint is there, the conditions are there. Part of the agreement, however, concerns precisely the manipulation of memory and perception, and therefore the possibility that he will no longer remember what he had accepted. It is hard not to think of the terms of use of digital services, which we often approve without reading. Here the question is far more radical: what is the value of consent when the technology one consents to is designed to affect the conditions in which the individual will subsequently be able to understand his own experience and his own actions? And, above all, could even genuinely given consent ever authorise any treatment whatsoever, or relieve the organisation of its legal obligations? The answer to this last question is no: individual consent does not make lawful conduct that is otherwise prohibited.
I come back, then, to the question I began with. Marshall saw in the soldier’s reluctance to fire an obstacle to military effectiveness; Grossman described the attempt to reduce it through training; Black Mirror imagines circumventing the problem by intervening in what the soldier is able to perceive. Artificial intelligence systems today compel us to pose a related question: how much can the informational environment of the person who decides be selected, ordered or transformed before their intervention becomes a mere rubber stamp?
Perhaps we should reassess what, from the standpoint of efficiency, appears as friction: doubt, hesitation, empathy, the need to check once more, the willingness to challenge what a machine shows us. All of this can slow a decision. In war, time may be short and hesitation too may carry risks. Not every slowing down, however, is a defect to be eliminated. Distinguishing between a civilian and a combatant, verifying whether a person may be lawfully attacked, assessing the risk of incidental harm to civilians and taking all feasible precautions require information and judgements tied to the circumstances. None of these operations is exhausted by reading a score or an automated classification.
From the standpoint of international humanitarian law, the rules of distinction, proportionality and precaution in attack continue to apply even when artificial intelligence tools are used. Proportionality, in particular, requires an assessment of whether the expected incidental loss of civilian life and damage to civilian objects would be excessive in relation to the concrete and direct military advantage anticipated. Computer systems can assist those who must make such assessments, without replacing the legal judgement required. If a technology presents a probabilistic conclusion as certain, conceals relevant information or pre-emptively classifies individuals as threats, formally assigning a human being the last word does not ensure that judgement has truly been exercised.
If I had to condense Men Against Fire into a single idea, it would be this: the problem can begin well before the moment at which the machine takes a decision in the human’s place. It begins already when someone else, through the machine, determines what reality the human will be able to see before deciding. This is where, for me, Watzlawick and Black Mirror end up meeting.
“How real is real?” seemed like a question about communication. Today it can also be a question about the power to select and construct the representation of the world on which others will be called upon to act. And in war it becomes, inevitably, a question about life and death: how real is the reality on the basis of which we are deciding to pull the trigger?
P.S. On reflection, the question is much older than Watzlawick. The myth of the cave that Plato tells in Book VII of the Republic comes to mind. The prisoners see only the shadows cast on the wall and take them for reality, because they have never known anything else. Stripe finds himself in a strikingly similar condition: what he sees is perfectly real to him, even though someone has decided in advance what he should perceive. There is, however, an important difference. For Plato it is possible to leave the cave and know a more authentic reality. Watzlawick confronts us with a further problem: even when we believe we have freed ourselves from deceptive representations, we continue to interpret the world through categories, languages and meanings that help to construct what we regard as real. And so one wonders whether Stripe’s problem is merely that of being a prisoner of an artificial reality, or also the far more general one of lacking the tools to recognise how far his own perception has been shaped by others.
Sources and references
- University of Milano-Bicocca, Beyond the Mirror of Technology: Social Scoring, War and Control, seminar series programme, 2026.
- Paul Watzlawick, How Real Is Real? Confusion, Disinformation, Communication, 1976.
- S. L. A. Marshall, Men Against Fire: The Problem of Battle Command, 1947; repr. University of Oklahoma Press, 2000, esp. p. 22 (on conditioning man to the machine).
- Dave Grossman, On Killing: The Psychological Cost of Learning to Kill in War and Society, 1995.
- Charlie Brooker, interview with The Guardian, 19 October 2016, “Charlie Brooker: ‘Someone threatened to smuggle a rifle through customs and kill me’”.
- Nicolas Rasmussen, “Medical Science and the Military: The Allies’ Use of Amphetamine during World War II”, Journal of Interdisciplinary History, 42 (2011), pp. 205–233, DOI: 10.1162/JINH_a_00212.
- Netta Goussac, Human control and AI-enabled military systems: Human rights risks, accountability and policy options for the European Union, European Parliament, 2026 (reference to be confirmed in the final bibliographic version).
- International Committee of the Red Cross, Frequently asked questions: AI decision-support systems in armed conflict, 8 October 2026.
- International Committee of the Red Cross, Frequently asked questions: Artificial Intelligence (AI) in the military domain, 11 June 2026.