Deepfake Laws and AI Content Regulation

Deepfakes have moved from technological curiosity to a regulated category in three years. The legal landscape that exists in April 2026 is substantively different from the one most general explainers describe, and the differences matter. The first federal US law specifically targeting AI deepfakes is now in force. The EU AI Act's deepfake disclosure regime is months away from full effect. China has enforceable mandatory labelling rules. State-level US deepfake laws have proliferated to over 30 jurisdictions, with active enforcement actions producing real convictions. This article maps the deepfake legal regime as it actually operates in 2026, the categories of deepfake harm regulators target, and the practical implications for individuals, creators, and platforms.

A "deepfake" in legal usage is generally an AI-generated or AI-manipulated audiovisual depiction that creates a realistic but synthetic representation of a real person's face, voice, or actions. The technology underlying deepfakes is not new (rudimentary photo manipulation has existed for decades), but two characteristics drive the current regulatory response:

  • Accessibility: tools that previously required graphics expertise are now available as consumer apps.
  • Verisimilitude: outputs are increasingly difficult to distinguish from authentic content.

Most deepfake regulation does not prohibit synthetic media outright. It targets specific harm categories where deepfake technology amplifies pre-existing harms:

  • Non-consensual intimate imagery (NCII), particularly AI-generated NCII of identifiable persons
  • Election interference, particularly synthetic political speech
  • Financial fraud, particularly voice-cloning and CEO impersonation scams
  • Reputational harm and defamation
  • Right of publicity violations, particularly commercial use of likeness without consent

Different jurisdictions address these categories through different combinations of criminal law, civil rights of action, platform-side notice-and-removal regimes, and disclosure or labelling obligations.

United States: federal and state deepfake law

The TAKE IT DOWN Act (federal, signed 19 May 2025)

The Tools to Address Known Exploitation by Immobilizing Technological Deepfakes on Websites and Networks Act (TAKE IT DOWN Act, S. 146) is the first federal US statute specifically targeting AI-generated harmful content. It was passed by the House on a 409-2 vote on 28 April 2025, signed by President Trump on 19 May 2025, and amends Section 223 of the Communications Act of 1934.

The Act establishes seven criminal offenses covering combinations of authentic intimate visual depictions, "digital forgeries" (the Act's term for AI-generated NCII), and threats to publish, with separate offenses for adults and minors. Penalties include:

  • Up to 2 years imprisonment for sharing or threatening to share digital forgeries or authentic NCII of adults
  • Up to 3 years imprisonment for sharing digital forgeries or authentic NCII of minors
  • Up to 30 months imprisonment for threats involving deepfakes of minors

Criminal prohibitions took effect immediately on 19 May 2025. The first conviction under the Act was announced in April 2026 against an Ohio defendant who used AI to create NCII of adults and children in his neighbourhood.

Beyond criminal prohibitions, the Act requires "covered platforms" (essentially user-generated content platforms) to establish a notice-and-removal process by 19 May 2026. Upon receiving a valid removal request from a depicted individual, platforms must remove the depiction within 48 hours and make reasonable efforts to remove identical copies. Failure is treated as an unfair or deceptive practice under the FTC Act, with FTC enforcement.

The Act provides safe harbour for platforms acting in good faith on takedown requests even where the content is later determined not to violate the Act. It does not preempt state law. It does not create a private right of action.

The DEFIANCE Act (federal, pending)

The Disrupt Explicit Forged Images and Non-Consensual Edits (DEFIANCE) Act, reintroduced in 2025 by Representative Ocasio-Cortez and others, would create a federal civil right of action for victims of digital forgeries. As of April 2026, it has not been enacted. The TAKE IT DOWN Act's lack of an express private right of action increases the importance of state-law civil claims, including state versions of revenge porn statutes and right of publicity claims.

State deepfake laws

Per Ballotpedia and the National Conference of State Legislatures, more than 30 US states have enacted deepfake-specific laws covering some combination of:

  • Election deepfakes: California (AB 2655, AB 2839, AB 2355), Texas, Minnesota, Michigan, Washington, and others have laws restricting synthetic media in political advertising or election communications, with varying disclosure and prohibition windows. Many face First Amendment challenges, with mixed judicial outcomes.
  • Sexual deepfakes: most states with revenge porn statutes have extended them to AI-generated NCII; specific deepfake laws exist in Minnesota, Virginia, Texas, and others.
  • Synthetic performers in commercial advertising: New York's SB 8420-A, signed 11 December 2025 and effective 9 June 2026, requires conspicuous disclosure when synthetic performers appear in commercial advertising, with penalties of $1,000 first violation and $5,000 subsequent.
  • Right of publicity: Tennessee's ELVIS Act (effective 1 July 2024) and similar laws specifically address voice replication.
  • Election deepfakes in 2024: state authorities in Minnesota, Texas, California, and other states acted against AI-generated political content during the 2024 election cycle, though enforcement was uneven and several state laws faced First Amendment challenges.

European Union: GDPR plus the AI Act

GDPR foundation

Under GDPR Article 9, biometric data (face, voice) processed for unique identification is special category data requiring an Article 9(2) condition. Deepfakes that incorporate identifiable faces or voices typically involve processing of biometric data. The GDPR's right to erasure (Article 17), right to rectification (Article 16), and right to object (Article 21) provide the data protection framework that operates in parallel with criminal law remedies.

EU AI Act Article 50(4) deepfake disclosure (from 2 August 2026)

Article 50(4) of the EU AI Act requires deployers of AI systems generating or manipulating image, audio, or video content that constitutes a "deep fake" to disclose that the content has been artificially generated or manipulated. Limited exceptions apply for criminal-offence-related uses authorised by law and for evidently artistic, creative, satirical, or fictional content (with minimal non-intrusive disclosure required even in those cases).

Article 50 applies from 2 August 2026. Article 50(2) separately requires providers of AI systems generating synthetic audio, image, video, or text to ensure outputs are marked in machine-readable format and detectable as artificially generated. The draft Code of Practice on Transparency of AI-Generated Content (first draft published 17 December 2025, final expected May or June 2026) sets out a multi-layered marking approach combining metadata, imperceptible watermarks, and logging or fingerprinting. Penalties under Article 99(4) reach up to €15 million or 3% of worldwide turnover.

Article 5(1)(e) prohibition on untargeted scraping

EU AI Act Article 5(1)(e), in force since 2 February 2025, prohibits the placing on the market, putting into service, or use of AI systems that create or expand facial recognition databases through untargeted scraping of facial images from the internet or CCTV footage. This provision targets the Clearview AI business model and is also relevant for deepfake-creation pipelines that depend on scraped image databases. Penalties for breach reach up to €35 million or 7% of worldwide turnover.

Digital Services Act

The Digital Services Act (DSA) imposes content moderation obligations on platforms, including specific obligations for Very Large Online Platforms (VLOPs) and Very Large Online Search Engines (VLOSEs) to assess systemic risks including disinformation amplified by AI-generated content. The DSA operates in parallel with the AI Act, not instead of it.

Asia-Pacific: China leads on synthetic media regulation

China's deep synthesis and labelling regime

China has the most active AI-specific synthetic media rule-making outside the EU, and several rules are directly relevant to deepfakes:

  • Provisions on the Administration of Deep Synthesis Internet Information Services (effective 10 January 2023): the original rule, requiring service providers to ensure deep-synthesis content is appropriately labelled and to register algorithm providers.
  • Interim Measures for the Management of Generative AI Services (effective 15 August 2023): apply to generative AI services accessible to the Chinese public, with obligations including content quality, training data, security assessment, and labelling.
  • AI Labelling Measures (effective 1 September 2025): require both explicit labels (visible or audible disclosures attached to outputs) and implicit labels (machine-readable metadata) for AI-generated synthetic content. The Measures were jointly issued by CAC, MIIT, MPS, and NRTA.
  • Provisions on Security Management of Facial Recognition Technology (effective 1 June 2025): impose specific obligations including alternatives provision and necessity assessment.

CAC enforcement has produced platform-level penalties for failures to label synthetic media or to register algorithm providers. The combination of provider registration and mandatory labelling makes China's regime substantively more prescriptive than most jurisdictions.

South Korea

Korea's AI Framework Act took effect on 22 January 2026, the second comprehensive AI law in the world. It includes specific labelling obligations for generative AI outputs alongside its high-impact AI risk management framework. Korea's Information and Communications Network Act and related criminal statutes already addressed synthetic NCII before the AI Framework Act, with specific 2024-2025 amendments expanding penalties for AI-generated sexual deepfakes.

Japan

Japan's AI Promotion Act (effective from June and September 2025) is a framework law without direct deepfake-specific penalties, but the Diet's accompanying supplementary resolution explicitly urged stronger measures against deepfakes. Generative AI deployments in Japan are also covered by the AI Guidelines for Business v1.1 (March 2025) and existing criminal statutes covering fraud, defamation, and obscenity.

Singapore

Singapore's Model AI Governance Framework for Generative AI (May 2024) addresses content provenance and synthetic media transparency through nine governance dimensions. The PDPA continues to govern personal data processed by deepfake creation systems. Specific election-period synthetic media rules exist under the Online Criminal Harms Act 2023 and election-specific legislation.

UK, Canada, Australia

United Kingdom: the Online Safety Act 2023, fully in force in stages through 2024-2025, imposes platform obligations covering illegal content including criminal deepfakes. The Sexual Offences (Amendment) Act 2025 and earlier amendments criminalise the creation of intimate deepfakes without consent. Right of publicity claims are limited under English law, but the Data Protection Act 2018 and UK GDPR provide privacy and biometric protections.

Canada: revenge porn statutes have been extended to AI-generated NCII in several provinces; Bill C-26 and provincial laws address platform obligations. Federal AIDA (the Artificial Intelligence and Data Act) provisions of Bill C-27 have not been enacted in their original form.

Australia: the Criminal Code Amendment (Deepfake Sexual Material) Act 2024 criminalises non-consensual sexual deepfakes federally. The eSafety Commissioner has active enforcement powers against deepfake-related online harms.

Documented enforcement and case examples

Some real cases that illustrate how the legal regime operates in practice:

  • Arup Hong Kong CFO deepfake fraud (February 2024): a finance employee at engineering firm Arup transferred approximately HK$200 million (about US$25 million) after attending a video conference with what appeared to be the company's CFO and other senior staff, all of whom were AI-generated deepfakes. The case is a leading example of AI-driven business email compromise.
  • First TAKE IT DOWN Act conviction (April 2026): an Ohio defendant was convicted of using AI to create NCII of adults and children in his neighbourhood and sharing them on a website promoting child sexual abuse.
  • Aledo, Texas, school deepfake incident (2023-2024): high school students generated AI-manipulated nude images of classmates, the precipitating case that motivated Senator Cruz's drafting of the TAKE IT DOWN Act.
  • Taylor Swift NCII deepfakes (January 2024): AI-generated explicit imagery of the singer spread across X (formerly Twitter) before being removed, contributing to political momentum behind federal NCII legislation.
  • Italian Garante action against Replika (February 2023): an early GDPR-grounded enforcement action against an AI service that processed personal data including in synthetic content generation contexts.
  • OpenAI consent and sender voice cases: actions including the Scarlett Johansson voice dispute and ongoing right of publicity claims against AI providers signal the importance of voice and likeness consent in commercial AI products.

Right of publicity and likeness

Most jurisdictions provide a right to control commercial use of one's likeness, voice, or image. US states have varied right of publicity statutes with different post-mortem terms. Tennessee's ELVIS Act extends specifically to voice. The EU treats this through GDPR plus member-state personality rights. UK lacks a general right of publicity but provides protection through passing-off, defamation, and data protection.

Defamation and false light

Deepfakes that depict identifiable persons engaging in conduct they did not engage in can be defamatory regardless of explicit "fake" disclaimers, particularly when the depiction harms reputation. Defamation analysis varies materially by jurisdiction; "false light" claims exist in some US states but not others.

Disclosure and labelling

EU AI Act Article 50(4), China's AI Labelling Measures, NY synthetic performer law, and Korea AI Framework Act all impose specific disclosure obligations. The exact technical requirements vary (visible labels, watermarks, machine-readable metadata, content provenance manifests), but the direction is consistent: AI-generated synthetic media affecting real persons or audiences should be marked as such.

Platform liability and notice-and-removal

The TAKE IT DOWN Act introduces a US federal notice-and-removal regime for NCII. The DSA requires risk assessment and content moderation by VLOPs. China's Deep Synthesis Provisions impose direct platform obligations. Section 230 of the US Communications Decency Act continues to provide broad immunity for general user-generated content, but the TAKE IT DOWN Act's FTC enforcement framework operates as an exception specific to NCII.

Election integrity

More than half of US states now have election-specific deepfake disclosure or prohibition laws, with mixed First Amendment outcomes. The EU operates election protections through the DSA, the AI Act's transparency provisions, and the European Democracy Action Plan. Most jurisdictions globally have moved to address election-period synthetic media in some form.

What the laws mean for different actors

Individuals

If you are a US resident victimised by deepfake NCII, the TAKE IT DOWN Act's criminal prohibitions are now law and the platform notice-and-removal regime takes effect on 19 May 2026. State revenge porn and deepfake statutes provide additional civil and criminal options. For international cases, GDPR rights, platform terms of service, and cross-border legal mechanisms are typically faster than court litigation.

For non-NCII deepfake harms (defamation, fraud impersonation, election interference), the available remedies depend heavily on jurisdiction and the type of harm.

Creators and marketers

Treat the rules below as cumulative and overlapping rather than alternative:

  • Obtain written consent for any commercial use of identifiable likeness, voice, or image.
  • Label AI-generated content visibly where the audience or platform requires it.
  • Implement C2PA Content Credentials or equivalent provenance metadata where feasible.
  • For synthetic performers in advertising reaching New York audiences, prepare for the 9 June 2026 conspicuous disclosure requirement.
  • For deepfake content distributed in the EU, plan for Article 50(4) disclosure compliance from 2 August 2026.
  • For Chinese audiences, comply with explicit and implicit labelling under the AI Labelling Measures.
  • Avoid creating deepfakes of identifiable persons depicting them in sexual contexts. Period. The criminal exposure is substantial in most major jurisdictions.

Businesses and platforms

Platforms operating user-generated content services in the US must implement TAKE IT DOWN Act notice-and-removal procedures by 19 May 2026. Platforms operating in the EU must address Article 50 obligations and, where designated as VLOPs, DSA risk assessment for AI-generated content. Platforms operating in China are subject to direct labelling, registration, and content tracking obligations. Cross-jurisdictional platforms should treat compliance as cumulative.

Beyond compliance, business email compromise via deepfake voice or video (the Arup-style scenario) is a significant cybersecurity exposure that warrants:

  • Out-of-band verification for high-value financial transactions
  • Authenticated communication channels for executive instructions
  • Training programmes covering AI-driven impersonation
  • Incident response procedures specific to AI-driven fraud

Detecting deepfakes: practical reality

The visual and audio "tells" that featured in early deepfake detection guidance (unnatural blinking, inconsistent lighting, mouth artefacts) are increasingly unreliable as AI quality improves. Practical defences in 2026 include:

  • Content provenance verification: tools that check C2PA Content Credentials and similar manifests where present.
  • Reverse image and video search: tracing media to original sources to verify authenticity.
  • Cross-source verification: corroborating high-stakes content through independent reporting or direct contact.
  • Authenticated channels: relying on signed or otherwise authenticated communications for sensitive matters rather than unverified video or audio.
  • Platform-side detection tools: increasingly built into major social platforms, though imperfect.

No detection method is perfect. Critical thinking remains the primary defence, but the legal regime is increasingly designed to support it.

Compliance FAQ

Is creating a deepfake illegal in itself?

No, not in most jurisdictions. The legality depends on the content (sexual, defamatory, electoral), the consent of any depicted person, the use (commercial, satirical, election-related), the disclosure (labelled or unlabelled), and the jurisdiction. Creating a clearly labelled satirical deepfake of a political figure is generally lawful. Creating a non-consensual sexual deepfake of an identifiable person is criminal in most major jurisdictions. The middle ground is wide and jurisdiction-specific.

Does the TAKE IT DOWN Act apply to me if I am outside the US?

The criminal prohibitions apply to publication on US-accessible interactive computer services. The platform notice-and-removal obligations apply to covered platforms regardless of formal headquarters location, where the platform serves US users. International deepfake creators publishing on US-accessible services face exposure under the criminal provisions, though enforcement and extradition outcomes depend on jurisdictional cooperation.

Are deepfakes prohibited in EU election advertising?

Not categorically across the EU, but most member states impose disclosure or restriction rules during election periods. The EU AI Act's Article 50 transparency obligations apply to deepfakes generally; the DSA imposes platform risk assessment for VLOPs covering disinformation including AI-generated content; and member-state election laws apply. The Florence-style overlapping framework requires case-by-case analysis.

What happens if a platform fails to remove a deepfake within the required timeframe?

Under the TAKE IT DOWN Act, FTC enforcement applies, treating non-compliance as an unfair or deceptive practice. Under the DSA, member-state Digital Services Coordinators can impose fines up to 6% of global annual turnover. Under China's Deep Synthesis Provisions, CAC enforcement can produce platform-level fines. Under the EU AI Act, deployer non-compliance with Article 50(4) faces fines up to €15M or 3% of turnover from August 2026.

What about voice cloning specifically?

Voice cloning of identifiable persons without consent raises right of publicity claims (Tennessee ELVIS Act and similar), defamation risk where the synthetic voice depicts conduct or statements that did not occur, and fraud risk where used for impersonation. The Arup case demonstrates that voice cloning has become a primary vector for business email compromise. Treat voice as biometric data subject to GDPR, PIPL, BIPA, and equivalent privacy frameworks.

What is C2PA and is it mandatory?

C2PA Content Credentials are the leading content provenance standard, supported by Adobe, Microsoft, Google, OpenAI, Meta, BBC, and others. It is not mandatory under any major jurisdiction, but it is the most widely implemented technical approach to satisfying machine-readable marking obligations under the EU AI Act, China's AI Labelling Measures, and similar regimes. The EU draft Code of Practice on transparency rejects single-standard reliance, recommending a multi-layered approach combining metadata, watermarks, and fingerprinting.

The bottom line

Deepfake regulation is no longer "emerging" or "evolving." Multiple regimes are operational in April 2026: the TAKE IT DOWN Act's criminal prohibitions, China's labelling rules, state-level deepfake statutes, EU AI Act prohibitions on untargeted scraping. More obligations come online in the next four months: TAKE IT DOWN Act platform notice-and-removal (19 May 2026), NY synthetic performer disclosure (9 June 2026), EU AI Act Article 50 deepfake disclosure (2 August 2026). The direction across regimes is consistent: transparency where deepfakes are lawful, prohibition where they are not, and platform-side notice-and-removal as the operational backstop. Treat compliance as cumulative across jurisdictions, not alternative. Document consent, disclosure, and provenance rigorously. Build audited workflows for synthetic content. The technology will continue to improve. The legal infrastructure is now keeping pace.

Last updated: April 2026. This article is educational content and is not legal advice. Deepfake regulation is rapidly evolving across jurisdictions, including the imminent TAKE IT DOWN Act platform obligations (19 May 2026), NY synthetic performer law (9 June 2026), and EU AI Act Article 50 (2 August 2026). Consult qualified counsel before making compliance decisions.