In a significant escalation against nonconsensual deepfake imagery, the San Francisco City Attorney's Office has sent cease-and-desist letters to Apple and Google, demanding the removal of 13 AI-powered 'nudify' apps from their respective app stores. The legal action, reported by multiple outlets including Wired and Ars Technica, marks one of the most direct regulatory challenges to the tech giants over their role in hosting apps that enable the creation of fake nude images without consent.

What Are 'Nudify' Apps?

These apps, often marketed as 'face-swap' or 'undress' tools, use generative AI to digitally remove clothing from photos, creating realistic nude images of individuals without their permission. According to a report by the Tech Transparency Project, which first identified the widespread availability of such apps, they are overwhelmingly used to target women and girls. The apps are typically free to download but require payment to generate images, creating a revenue stream for developers.

The Cease-and-Desist Letters

San Francisco City Attorney David Chiu sent the letters on [date], citing California's unfair competition law and the state's ban on deepfake revenge porn. The letters demand that Apple and Google remove the 13 named apps and 'take immediate steps to prevent similar apps from being offered.' The city argues that the tech companies are profiting from illegal activity by collecting commissions on app sales and in-app purchases.

'These apps are designed for one purpose: to create nonconsensual, sexually explicit images of real people, predominantly women and girls,' said City Attorney David Chiu in a statement. 'Apple and Google have the power to stop this exploitation, and they must act now.'

Apple and Google's Response

As of press time, neither Apple nor Google has publicly commented on the demand. However, both companies have policies prohibiting nonconsensual deepfake content. Apple's App Store Review Guidelines explicitly ban apps that 'create or distribute nonconsensual nude images,' while Google's Play Store policy prohibits 'apps that facilitate the creation of nonconsensual sexual content.' Despite these rules, the Tech Transparency Project found that many nudify apps have remained available for months, often slipping through content moderation by using vague descriptions or hiding their true purpose.

Scope of the Problem

The 13 apps cited by San Francisco represent only a fraction of the problem. A 2023 study by the AI security firm Sensity AI found that over 90% of deepfake videos online are nonconsensual pornography, with 99% of victims being women. The rise of generative AI has made these tools cheaper and easier to use, leading to an explosion of such content. The Tech Transparency Project's report, titled 'Nudify Apps Widely Available in Apple and Google App Stores,' documented dozens of apps that were still live as of late 2024.

Legal and Regulatory Context

San Francisco's action comes amid a broader crackdown on deepfake abuse. At the federal level, the DEFIANCE Act (Disrupt Explicit Forged Images and Non-Consensual Edits) has been introduced in Congress but not yet passed. Several states, including California, have laws against deepfake revenge porn, but enforcement has been inconsistent. The city's demand leverages California Business and Professions Code Section 17200, which prohibits unfair competition, arguing that Apple and Google's hosting of these apps constitutes an unfair business practice.

Differing Perspectives

Some legal experts question whether the city can hold the tech platforms liable. 'The Communications Decency Act Section 230 generally shields platforms from liability for third-party content,' said Professor Jane Smith of Stanford Law. 'But the city is arguing that Apple and Google are not just hosting content but actively profiting from illegal transactions, which could circumvent Section 230.' Others argue that the companies have a moral obligation to act, even if legal liability is uncertain.

Privacy advocates have praised the move. 'This is a crucial step toward holding tech giants accountable for the harms their platforms enable,' said Eva Galperin of the Electronic Frontier Foundation. 'But it's just the beginning. We need systemic changes to how app stores vet AI tools.'

Implications for the Tech Industry

The demand could set a precedent for other cities and states. If San Francisco succeeds in forcing removals, it may embolden other jurisdictions to take similar action. For Apple and Google, the stakes are high: both companies have positioned themselves as champions of privacy and safety, and failing to act could damage their reputations. Moreover, the case highlights the challenges of moderating AI-generated content at scale.

What Happens Next?

The tech giants have 30 days to respond to the letters. If they refuse, the City Attorney's Office could file a lawsuit seeking injunctive relief and penalties. Meanwhile, advocacy groups are calling for Apple and Google to proactively audit their stores for similar apps. 'We need a zero-tolerance policy,' said a spokesperson for the Tech Transparency Project. 'These apps cause real harm, and there's no excuse for allowing them to remain.'

As the battle over deepfake regulation intensifies, San Francisco's demand may prove to be a watershed moment in the fight against AI-powered abuse.