AIthropology Lab
ESEN

Notes

Digital violence is not a design flaw

From synthetic images created in a school to the automatic reuse of photographs on Instagram. An anthropological account of how platforms and institutions distribute harm and the work of repair.

·Martín González Senosiain
Portrait of Olympe de Gouges seated in an armchair and holding a book, used to illustrate an analysis of the digital appropriation and manipulation of images.

Olympe de Gouges, seated with a book, unknown artist, 1793. Watercolour and graphite on paper. Edmond de Rothschild Collection, Musée du Louvre. Public domain, via Wikimedia Commons.

At the Carlos Pellegrini Higher School of Commerce in Buenos Aires, students aged thirteen and fourteen were accused of using artificial intelligence to alter photographs of female classmates and create false images in which they appeared naked. According to the complaints that were filed, the files did not merely circulate among peers. They were allegedly classified and offered in exchange for money. Institutional intervention came later, after the girls spoke out, their families demanded action, and the case reached the media and the courts.

The case is often explained as the misuse of a technology by a group of teenagers. The formulation is reassuring. It isolates responsibility. There is a neutral tool, and there are a few people who choose to use it harmfully. Yet producing and commercialising those images required more than that. It required accessible photographs, easy-to-use software, private channels of circulation, an audience willing to consume the material, and a peer community in which girls’ bodies could become objects of exchange.

It also required an institution that did not become fully visible until those affected managed to turn a private assault into a public problem.

To understand how girls’ bodies could become commodities inside the classroom, it is not enough to identify those who took part. We need to ask which gender relations organised the group, which forms of masculinity were being produced among peers, and how status could be built through the display and exchange of images.

For decades, feminist anthropology has shown that gender is not an individual property that simply distinguishes women from men. It is a social relation that organises differences, distributes positions, and legitimises inequalities. Lourdes Méndez places the critique of androcentrism and ethnocentrism at the centre of feminist anthropology, while reminding us that the category ‘woman’ must be examined in all its heterogeneity rather than presented as a universal experience.

Joan W. Scott proposes using gender to investigate how meanings of difference are constructed and which relations of power they produce. Its value does not lie in adding women to a history that has already been written. It lies in questioning the categories through which that history is organised. From this perspective, the Pellegrini case is not only about attacks on female bodies. It is also about a masculinity performed before other boys, the transformation of consent into an object of ridicule, and the status that can be gained through accessing, manipulating, and distributing images.

Raewyn Connell warns that hegemonic masculinity should not be understood as a fixed personality type. It is a configuration of practices generated in particular situations and within a structure of relations. Alongside it are complicit, subordinated, and marginalised masculinities. This distinction prevents us from presenting the students involved as natural bearers of male violence. Attention should instead turn towards the practices a group rewards, tolerates, or normalises. Who makes the image, who asks for it, who pays, who laughs, who forwards it, and who remains silent.

From the school group to the technical ecosystem

Generative artificial intelligence did not create these relations, but it has reduced the knowledge and time required to turn them into images. What once demanded a degree of technical skill can now be done with an app, a photograph, and a short instruction. Digital sexual violence is therefore no longer confined to specialist communities. It becomes embedded in tools used every day.

The Grok episode showed what happens when this possibility is integrated into a global platform. According to a sample-based estimate released by the Center for Countering Digital Hate in January 2026, around three million sexualised images may have been generated with X’s tool over eleven days. Approximately 23,000 appeared to depict children. This was not an exhaustive count, but the estimate makes the difference in scale visible. A non-consensual synthetic image can move from a school group to a planetary infrastructure within seconds.

Research by Michelle L. Ding, Harini Suresh, and Suresh Venkatasubramanian challenges the idea that the problem can be solved by blocking one application after each scandal. Their mapping identifies eleven technological categories involved in the creation, distribution, discovery, hosting, and monetisation of non-consensual synthetic intimate images. Fragmented responses therefore produce a policy of whack-a-mole. One access point is closed while the rest of the ecosystem remains intact.

The unit of analysis should not be the image alone, nor the person who enters a prompt. It must also include models, interfaces, accounts, messaging groups, storage services, recommendation systems, and payment mechanisms.

Violence is individual in the way it is experienced, but collective and infrastructural in the way it is produced.

Gender does not erase men

Recognising the misogynistic structure of much synthetic sexual violence does not require excluding men from the analysis. A survey of more than 16,000 people across ten countries found that 2.2 per cent had experienced some form of non-consensual synthetic intimate imagery. The study does not present this percentage as an exclusively female experience.

Another study by the same team widened the focus to image-based sexual abuse more broadly. Some 22.6 per cent of respondents reported an experience involving the non-consensual creation, capture, threat, or dissemination of intimate material. Victimisation rates were similar among women and men, although women reported more intense harm and more serious negative consequences. Younger and LGBTQI+ people appeared particularly exposed.

The forms of abuse also change according to the gendered expectations used to inflict harm. Women account for much of the pornographic falsification and for markets that sexualise real faces. Men, especially younger men, feature prominently in financially motivated sextortion. Australia’s eSafety Commissioner received 2,206 reports of sexual extortion between July and December 2025. The largest reporting group was men aged eighteen to twenty-four, with close to eight hundred cases.

The point is not to dilute inequality by claiming that everything affects everyone in the same way. It is to examine which body is fabricated, which form of shame is mobilised, and which threat proves socially effective. Gender does not determine in advance who can experience violence. It helps us analyse the form that violence takes and the conditions that intensify its consequences.

Switch it on first, apologise later

Muse Image, the tool introduced by Meta, reveals the same problem from the perspective of corporate design. The feature allowed photographs posted by public Instagram profiles to be used as references for generating images with AI. Public accounts were initially included unless each person located and changed the relevant setting. No notification was sent when their content was reused.

Meta withdrew that possibility after public criticism and acknowledged that the feature had failed to provide the control it was supposed to offer. The reaction came after the product had already been deployed and once the potential non-consensual use of images had become a reputational cost for the company.

Enabling the feature is not the same as deliberately designing every possible abuse. It does, however, distribute risk in a particular way. The company gains experimentation, attention, and adoption. Users must discover that their photographs are available, anticipate possible uses, locate the setting, and opt out.

Consent appears as a subsequent individual task, rather than as a precondition of the product.

‘Switch it on first, apologise later’ describes something deeper than one poor business decision. It describes a technological culture in which rapid launch is treated as innovation, while harm prevention can be postponed until reputational, legal, or economic pressure arises. Design is not only code. It also consists of defaults, friction, warnings, response times, and the person on whom the work of self-protection is placed.

The companies’ financial capacity makes it harder to present prevention as a technically unreachable aspiration. Meta ended 2025 with revenues of US$200.966 billion and net income of US$60.458 billion. It spent US$57.372 billion on research and development. Its capacity to anticipate the risks of a feature that reused other people’s faces must be considered alongside the priorities that guided its deployment.

There is no comparable public figure for X or xAI. Both are private companies, and the information they release does not offer the same level of detail or auditability as Meta’s accounts. This economic opacity is part of the problem. Society can observe the harm, but has fewer tools for knowing how much is genuinely invested in preventing it.

OnlyFans as an imperfect counterweight

OnlyFans introduces a useful contrast. Not because it is free from exploitation, leaks, or inequality, but because its architecture begins from a different decision. The person creating the content initiates publication, sets a price, and receives a share of the value generated. The difference is not that one image is sexual and another is not. It lies in who authorises circulation, under which conditions, and who captures the income.

The platform retains 20 per cent of payments. During the financial year ending in November 2024, it processed US$7.2 billion, paid US$5.8 billion to creator accounts, generated US$1.4 billion in revenue, and reported US$684 million in pre-tax profit. It had 4.6 million creator accounts and 377.5 million fan accounts. Its owner received US$701 million in dividends during the financial year and the months that followed.

These figures show a different distribution of value, but not automatic emancipation. Creators bear the exposure, constant self-promotion, leaks, content copying, and much of the burden of protecting their identity. The platform recognises authorisation and remunerates publication while extracting one fifth of every transaction and concentrating extraordinary profits.

OnlyFans demonstrates that consent can occupy a visible place within design. It also shows that consenting to publication is not the same as consenting to every future reuse. Permission to access an image within one service does not authorise its download, sale outside the agreed environment, or transformation through artificial intelligence.

The contrast makes the thesis more precise. The decisive issue is who authorises the circulation of sexual images, which limits accompany that decision, how consent can be withdrawn, and who bears the losses when content escapes that framework.

The image does not end with the file

The Western conception of a photograph as a detachable, available, and transferable copy is not universal. Anthropology has shown that images can act as presences, mediators of relations, or extensions of the people represented. Alfred Gell proposed studying visual objects not only for what they mean, but for what they do within a social network. An image can distribute agency, activate obligations, and affect those who engage with it.

This idea helps us avoid an old colonial cliché. It is not rigorous to claim that particular peoples uniformly believe a camera ‘steals the soul’. Practices are more diverse and more complex. What we do find are contexts in which an image is not treated as a neutral object available for anyone to use.

Across different Aboriginal and Torres Strait Islander communities, protocols govern access to, reproduction of, and display of images or voices of people who have died. Practices vary among families and peoples. The Australian Institute of Aboriginal and Torres Strait Islander Studies preserves visual material under cultural restrictions and requires community requirements to be checked before reproduction or digitisation. Some images are not shown without the authorisation of the communities connected to them.

Similar limits have been documented among Hopi photographers around ceremonies, collective secrets, and portraits intended to circulate only within particular relationships. Victor Masayesva Jr described the camera as a weapon capable of violating silences essential to the group’s survival. Other Hopi photographers have declined to sell portraits outside the community when they considered those images bound to collective intimacy.

In Aotearoa New Zealand, Māori data sovereignty frameworks connect information with whakapapa, genealogy, relationships, and collective authority. Data are not conceived merely as resources available to whoever can copy them. Their use carries responsibilities towards the people and communities from which they come. Recent proposals for Māori algorithmic sovereignty extend these principles into the design and governance of automated systems.

These cases are not equivalent and cannot be reduced to a universal theory of the soul. They do, however, remind us of something that digital design tends to erase. Representation does not always mean duplication. An image may remain connected to the person, their ancestors, their community, and the relationships that constitute them.

Manipulating a photograph without permission does not merely alter a file. It intervenes in a social presence. One need not believe that an image literally contains a soul to understand that identity does not end at the physical skin or with ownership of the camera that took the photograph.

The Valencia DANA made this relationship visible in a nearby and material form. In March 2026, the anthropologist Beatriz Santamarina Campos gave a session entitled Inhabiting the emergency, inhabiting the catastrophe. Valencia’s DANA from the perspective of public anthropology. The title itself placed the catastrophe beyond the meteorological event. To inhabit an emergency also means rebuilding the relations, objects, and memories that allow a life to become recognisable as one’s own again.

Following the floods, Valencia’s public universities and several heritage institutions launched Salvem les fotos. The initiative recovered family photographs and albums covered in water and mud. In January 2026, the Universitat de València reported that nearly two and a half million images belonging to more than 1,500 families had been recovered. The aim was not to return decorative objects after homes and streets had been repaired. The images held weddings, childhoods, people who had died, migrations, celebrations, and everyday ways of belonging to a community.

The project also revealed that the content of a photograph is not exhausted by what can still be seen. Marisa Vázquez de Ágredos, coordinator of Salvem les fotos, explained that a family continues to recognise what deterioration has erased for an outside observer. A nearly empty surface can still hold relationships, stories, and presences that only shared memory can reconstruct.

Restoration did not seek to pretend that the catastrophe had never happened. Mud, lost pigments, and watermarks became part of the material biography of the images. Recovering them meant sustaining continuity between before and after, rather than returning them to an imagined original purity.

This example clarifies what is violated when an image is deliberately manipulated. A deepfake does not merely add false information to a file. It inserts a non-consensual scene into a person’s public memory and forces them to contest which version of their own body will be seen, shared, or believed. If restoring a photograph can be part of care, protecting it from appropriation and falsification must also be understood as a collective responsibility.

The circulation of punishment

The harassment directed at the footballer Alexander Sørloth and his partner, Lena Selnes, reveals another form of the same pattern. After Norway’s elimination from the 2026 World Cup, Selnes made public some of the messages sent to them. They included insults and comments urging the footballer to harm himself. Norway’s manager condemned the abuse, but his practical recommendation was that the team avoid social media during periods of heightened tension.

There were no synthetic sexual images or children in this case. The harms should not be conflated. Yet a familiar sequence reappears. A platform enables the immediate accumulation of messages, the targeted person documents what has happened, and the institutional solution ultimately becomes temporary withdrawal from the space.

The displacement of the attack towards Selnes is also significant. She had not played in the match, yet she was drawn into the punishment directed at her partner. Digital violence therefore widens the perimeter of the person deemed responsible and turns affective relationships into additional surfaces of attack.

Sara Ahmed proposes studying emotions not as contents enclosed within an individual, but by examining what they do and how they circulate among bodies, discourses, and objects. Repetition causes certain affects to adhere to particular figures and produce social effects. In sporting spectacle, frustration can circulate as shared indignation, attach itself to a footballer, and then extend to the woman associated with him.

The platform is not a passive container. Its metrics, trends, and visibility mechanisms turn emotional intensity into quantifiable interaction. Insult, outrage, and humiliation circulate because they generate responses, and those responses become data, attention, and time spent within the service.

Waiting for harm to become visible

The anthropologist Hlengiwe Ndlovu analyses the experience of fragile state institutions in post-apartheid South Africa through four dimensions. Waiting, silence, refusal, and care. Although her ethnography does not examine digital platforms, and the two contexts should not be forced into equivalence, her categories can be used as a conceptual loan for observing institutions that appear in fragments. They are present to administer procedures, but absent when protection is needed.

These four dimensions are not successive stages. They are overlapping layers whose meanings change according to the position occupied by those affected, institutions, and platforms within the conflict.

Waiting lies in the distance between the speed of the assault and the slowness of the response. An image can be copied and distributed in seconds. Securing its removal requires screenshots, forms, complaints, and repeated explanations. Harm travels at the speed of infrastructure, while protection moves at the pace of the case file.

Silence appears in automated responses, opaque moderation decisions, and the hesitation of educational institutions. It can also form part of the experience of those affected. Not reporting, not showing the images, or not participating in particular procedures does not necessarily mean passivity. It may be a way of preserving privacy, avoiding revictimisation, or refusing a system that requires the harm to be displayed again before it will acknowledge that harm exists.

Yet silence may also be produced by fear, exhaustion, or the conviction that no one will respond. Not every withdrawal is resistance. Sometimes it is expulsion.

Refusal reveals the limits of what an institution considers knowable. Platforms may refuse to recognise content as harmful if it does not fit a predetermined category. Those affected may, in turn, refuse to surrender every detail of their intimate life to a police, school, or media procedure. That refusal does not erase the harm, but it protects a zone of autonomy from systems that often confuse investigation with extraction.

This is why the specialists consulted about the Pellegrini case insist on first asking what the girls need. A criminal complaint should not be imposed as the only response, but nor should repair be confused with an absence of consequences. A restorative intervention requires safety, voluntary participation, acknowledgement of harm, and accountability from those who caused it. It cannot compel those affected to enter dialogue, forgive, or reconcile.

Irene Comins Mingol argues that an ethics of care brings a different priority to the transformation of conflict. Needs should be addressed before the response is reduced to punishment. The point is not to replace justice with benevolence, but to ask what can stop the harm, restore safety, allow an educational trajectory to continue, and rebuild a community.

Who sustains life afterwards

An analysis of care also reveals who performs the work required to sustain life after the assault. Preserving evidence, requesting removals, accompanying a teenager, speaking with a school, seeking psychological support, and following each case file are all forms of labour. They have temporal, emotional, and material costs.

Amaia Pérez Orozco shows how, when institutions and markets privatise risk, households become the system’s final buffer. Her proposal to place the sustainability of life at the centre makes visible what conventional economies leave out.

Something similar happens in digital violence. The platform enables circulation, while families, friends, teaching staff, and specialist organisations absorb the labour of repair. They accompany, document, insist, care, and sustain while the company manages forms and the institution decides which procedure applies.

The platform enables circulation; families, friends, and teaching staff absorb the labour of repair.

The Ema Guide and proposed Ema Law, Argentina’s Olimpia Law, and the Inter-American Model Law to Prevent, Punish and Eradicate Digital Violence against Women are attempts to anticipate part of this work. Their approaches are not identical, and their effectiveness depends on implementation, available resources, and the ability to hold companies accountable when they operate across jurisdictions.

No legal text eliminates violence on its own or automatically resolves educational, administrative, and judicial delays. These instruments shift the centre of the response. Attention therefore extends to the institutions, protocols, and obligations that must exist before someone has to turn a violation of their intimacy into a public scandal, as well as to punishment after the event.

What support is available in Spain

The first need is not always to report the offence. It may be to recover a minimal sense of safety, halt circulation, ask another person to accompany the process, and take time to decide which routes to pursue. Support should not be reduced to a generic recommendation. It requires technical, emotional, legal, educational, and social responses that can be combined without forcing the affected person through a single pathway.

The Priority Channel of the Spanish Data Protection Agency allows people to request the urgent removal of unlawfully disseminated sexual or violent photographs, videos, or audio when their circulation places the rights or physical or mental health of those affected at serious risk. The request may be submitted by the person directly affected or by someone who knows of the publication.

INCIBE’s 017 service offers free, confidential, and personalised help. A multidisciplinary team provides technical, legal, and psychosocial guidance every day of the year between 8 am and 11 pm. The service assists adults, children, families, professionals, and businesses.

When the assault forms part of violence against women, the 016 service provides general information and immediate psychosocial support twenty-four hours a day. Legal advice is available daily between 8 am and 10 pm. The service is free and confidential, and can refer emergencies to 112 or cases involving children to the ANAR helpline.

Crime Victim Support Offices provide free public assistance in legal, psychological, and social matters. Their model includes individualised plans and seeks to minimise initial victimisation and prevent secondary victimisation. A person does not need to wait for a judgment before seeking information and support.

A reasonable intervention may begin by preserving only the essential evidence without continuing to download or forward the material. Removal can then be requested, account security reviewed, protection sought within the school or workplace, and legal options considered without imposing an immediate complaint. Continuing psychological care may remain necessary even after the material has been removed. The disappearance of a file does not end uncertainty about its copies, nor does it by itself repair the breakdown of trust.

Who pays for repair

Public services assume part of the cost through data protection, cybersecurity, healthcare, education, victim support, and the justice system. Platforms finance moderation and removal systems, and may face penalties or corrective measures. The person who committed the violence may bear civil or criminal responsibility when identified and when a ruling is reached.

A considerable part of the cost nevertheless remains outside official accounts. The affected person and those around them spend hours searching for copies, saving screenshots, completing forms, speaking with the educational institution, taking time off work, changing phone numbers, or closing accounts. They may pay for private therapy, legal advice, expert reports, or digital reputation services because the public response does not arrive quickly enough.

A fine imposed on a platform does not automatically repair each person’s harm. Removal does not return the time invested or guarantee that the file has disappeared from other servers. Establishing the legal responsibility of the person who created or distributed the content may take years. In the meantime, care operates as an emergency infrastructure sustained by families, friends, and professionals who rarely receive resources proportionate to the work they perform.

The real economy of harm can be summarised differently. The platform gains interaction. Public institutions pay part of the response. Close networks provide care. The assaulted person contributes the time, exposure, and often the money required to try to recover an image that never ceased to belong to them.

Europe investigates after the harm

In January 2026, the European Union opened a formal investigation into X to determine whether the platform had adequately assessed and mitigated the systemic risks associated with Grok. The proceedings rely on the Digital Services Act and examine the dissemination of manipulated sexual images, including non-consensual content and material that could involve children.

European scrutiny encompasses the lawfulness of each image, the company’s assessment of risk before integrating the tool, the adequacy of its safeguards, and the possible amplification of harm by its recommendation systems. This represents an important change in scale. Responsibility no longer rests only with the person who entered a prompt. It reaches the infrastructure that enabled mass production and circulation.

EU Member States and representatives of the European Parliament have also supported proposals to prohibit AI applications that generate explicit sexual images without authorisation. The reform was still under negotiation in mid-2026 and should not yet be presented as a final prohibition already enforced throughout the European Union.

In Spain, the response combines existing tools with reforms still in progress. The Spanish Data Protection Agency can intervene to seek urgent removal. On 7 July 2026, the Council of Ministers approved a draft organic law updating civil protection of honour, privacy, and one’s own image in relation to unauthorised recreations made with artificial intelligence. The bill also raises to sixteen the age for consenting to certain uses of one’s image and strengthens the assessment of non-material harm.

Spain is also processing legislation intended to protect children in digital environments. Its aims include strengthening the response to synthetic sexual content, harassment, and other forms of technology-enabled violence. These are significant developments, but they continue to reveal the same temporal divide that runs through this article. Innovation is deployed first. Investigation, removal, and legal reform arrive after those affected have been required to prove the harm.

Europe is trying to catch up with an infrastructure that has already turned society into its testing ground. The difficulty is not simply to legislate more quickly. It is to prevent corporate speed from continuing to operate as a temporary exemption from responsibility.

Digital violence is not a deviation that platforms correct once they eventually manage to detect it. It is the foreseeable result of earlier decisions about what is prioritised, who is exposed, and who must carry the work of self-protection.

It is not a flaw the system discovers too late. It is what the system allows for as long as preventing it does not become too costly.

Sources and further reading

Verification note

The factual information in this version was reviewed up to 16 July 2026. Figures attributed to the Center for Countering Digital Hate are sample-based estimates, not an exhaustive count. OnlyFans figures are drawn from Fenix International’s accounts for the financial year ending 30 November 2024 and from the information on subsequent dividends included in those accounts. Meta’s results come from its annual financial statement for 2025.

The cases discussed are not presented as equivalent harms. Digital sexual violence against teenagers, non-consensual reuse of photographs, commercial exploitation of intimate content, and harassment directed at public figures share an institutional sequence of exposure and delayed response, but differ in their characteristics, asymmetries, and duties of protection.

References to Aboriginal Australian peoples, Hopi communities, and Māori frameworks do not support the existence of a universal belief that a photograph literally contains the soul. They are used to show that the Western separation between person, image, and data is not universal, and that access to a representation may be governed by collective authority, kinship, memory, and cultural protocols.

The reference to Beatriz Santamarina is limited to the existence, title, and public framing of her academic session on the DANA. No specific claim about photographic restoration is attributed to her without a complete publication or transcript. Figures for Salvem les fotos refer to different moments and teams within the Valencia programme. The most recent source cited attributes nearly two and a half million recovered images from more than 1,500 families to the Universitat de València.

The categories of waiting, silence, refusal, and care are used as a conceptual loan from Hlengiwe Ndlovu’s ethnography, without equating digital platforms with the South African state context studied by the author. The Spanish and European reforms cited were at different stages of investigation or legislative processing on the review date and should not be interpreted as fully implemented law in all respects.