Nudify Apps Drag Apple and Google Into Legal Crosshairs

If Apple and Google take a cut from nudify apps, how long can they argue they’re just running app stores?

San Francisco City Attorney David Chiu has ordered the companies to remove AI apps that can digitally alter images to make people appear nude, according to TechCrunch. The legal theory is blunt: if the platforms host the apps, process the payments, and collect fees, they may be more than passive distributors.

That is the real fight under the headline. The city isn’t just asking for cleaner app review. It is testing whether App Store and Google Play monetization can create legal exposure when AI tools are built around nonconsensual sexual imagery.

Can Apple and Google still call nudify apps a moderation problem?

Not convincingly.

Nudify apps use AI to generate or simulate nude images of real people, often from ordinary photos. The source material describes them as software that can digitally alter pictures to unclothe the people in them. Additional reporting reviewed by Ars Technica and Wired says the targeted apps were often presented as face-swapping tools, while their sexual deepfake functions appeared once users engaged with them.

That matters because app stores are not neutral shelves. They rank, distribute, approve, reject, and monetize software. Chiu’s office says both companies have continued hosting and making money from programs that violate California law.

“Apple and Google are profiting off apps that exploit women and girls by generating nonconsensual intimate deepfakes,” Chiu said in an emailed statement to TechCrunch. “While the companies cut ties with some problematic apps, Apple and Google have a responsibility to be proactive and vigilant to prevent sexual abuse.”

XOOMAR analysis: this turns app review into a product-safety question. If a category of AI apps predictably enables illegal or abusive outputs, removing a few names after outside reports won’t satisfy regulators for long. The harder demand is category-level enforcement, especially when apps can relabel themselves as generic face-swapping or image-editing products.

That pressure lands as Apple and Google are already under scrutiny for their control over mobile distribution. We’ve covered that tension in other app-store fights, including Rival Android App Stores Invade Google Play Next Week and Apple’s expanding services push in Plumbers Get Shut Out as Apple Maps Ads Take On Google. This case is different because the dispute isn’t about competition or placement. It’s about whether the stores helped monetize sexual abuse tools.

Where does California law put the pressure point?

California law criminalizes activity that “knowingly facilitates” or “recklessly aids or abets” the creation of nonconsensual deepfake pornography, according to the source material. In 2025, California also passed a law allowing victims to bring civil actions against third-party facilitators of that material.

Chiu’s letters, viewed by TechCrunch, say Apple and Google have “been on notice” for their role in “processing payments for illegal purchases for almost a year” and continued doing so. The letters also warn that the companies could face civil penalties and ask them to contact the city within 28 days.

A city attorney’s letter is not the same thing as a court ruling. It does not by itself prove liability. But it does something legally important: it creates a public record that the companies were warned.

That notice issue is central. The Tech Transparency Project sent reports and letters in January and April, saying there were “dozens of apps” in the stores that “sold deepfake NCII [non-consensual intimate images] in exchange for payments” processed by the firms. TTP’s April report also said Apple and Google had intentionally “steered” users toward such apps and called both companies “key participants in the spread of AI tools that can turn real people into sexualized images.”

XOOMAR analysis: the city is trying to move the debate from “bad apps slipped through” to “the platforms were repeatedly warned, still processed payments, and still profited.” That is a much more dangerous frame for Apple and Google.

Why did 13 apps become enough to force a broader reckoning?

Because the number is small enough to be actionable and large enough to expose a system problem.

Ars Technica, citing Wired’s report, said Chiu demanded removal of 13 nudification apps: five from Google and eight from Apple. Wired reported that one app had more than 1 million downloads and advertised features to sexualize images of women or create “free and uncensored” videos. TechCrunch’s source material also says Chiu told Wired that both companies had likely made “millions of dollars in fees” from apps offering such services.

Google has already responded publicly. A Google spokesperson told Ars that the five flagged apps were suspended from Google Play for violating policies against harmful content.

“Google Play does not allow apps that contain sexual content, and we continually take proactive steps to detect and remove apps with harmful content,” Google spokesperson Dan Jackson said. “When violations are reported to us, we investigate and take swift action, which in the case of these apps has included suspending hundreds of violating apps and restricting related search terms like ‘nudify’ on our store.”

Apple told Wired that it removed three of the apps flagged by San Francisco and was “in the process of terminating their developer accounts from our program.” Apple also said the other four developers must address policy violations or risk removal.

Here is the enforcement problem in plain view:

A May preprint paper cited by Ars found 420 apps marketed as generic face-swapping tools. Researchers tested 155 and found nudification was possible in 70 percent of those tested. That doesn’t prove every such app violates the law. It does show why keyword bans alone are weak.

Who carries the burden when a fake nude image spreads?

Victims do.

The source material says deepfake pornography has largely affected female celebrities, but nudify apps make anyone with a publicly available photo a potential target. Wired quoted Chiu saying the images are used to “bully, humiliate, and threaten women and girls” and that the impact can include reputational harm, mental health damage, loss of autonomy, and suicidal victims.

That is why the usual platform defense sounds thin here. Apple and Google can point to policies banning sexual content, pornography, abuse, and harassment. They can also say they remove violating apps when flagged. Those statements are relevant. They are not enough if regulators can show the companies repeatedly received warnings and still processed in-app payments.

Developers of legitimate AI image tools have a real interest in not being swept into a blunt ban. But nudify apps sit in a narrower category. They are not merely image editors with a risky edge case. The apps Chiu targeted allegedly allow users to generate nonconsensual intimate images, often through paid functions.

XOOMAR analysis: the innovation argument loses force when the core monetized behavior is the sexualization of real people without consent. The better question for developers is whether they can prove safeguards work before distribution, not after reporters or watchdogs find abuse.

Why does this fight not end with deleting 13 apps?

Because nudify apps are a moving target.

The city’s demand can force Apple and Google to purge named apps. That is the easy part. The difficult part is proving that similar tools don’t return under new branding, softer descriptions, or face-swapping labels.

The next phase will likely turn on evidence, not press statements:

• Review logs: Did Apple and Google detect these apps before outside reports?
• Payment records: How much revenue flowed through in-app purchases tied to the flagged apps?
• Search controls: Do restrictions on terms like “nudify” reduce discovery or merely push users to adjacent terms?
• Developer tracking: Can repeat operators relaunch under new accounts?
• Victim reporting: Can targeted people get rapid review without carrying the full burden themselves?

The strongest signal that Apple and Google have absorbed the lesson would be durable enforcement against the category, not a one-time purge after legal letters. The weakest signal would be another watchdog report in a few months finding the same functions under cleaner names.

This is now a test case for AI platform liability in practice. If app stores profit from distribution, regulators will ask whether they also own part of the risk.

Impact Analysis

• The order tests whether app stores can face legal exposure when they host and monetize AI tools used for nonconsensual sexual imagery.
• Apple and Google may need stronger review systems for apps that disguise or enable abusive deepfake functions.
• The case could reshape how platforms are treated when they profit from software linked to illegal or harmful content.