Regulations (e.g., the EU AI Act) and major platforms now require labels disclosing that an image was generated by AI. A pre-registered survey with 1,354 US and EU participants found that such labels deliver a security benefit — reduced belief in false claims supported by AI-generated images — but also two side effects: an implied-truth effect, where participants became more susceptible to false claims accompanied by unlabeled human-made images, and reduced belief in true claims illustrated with labeled AI-generated images. The open problem is to determine when and why this trust-discernment dissociation arises and under what deployment conditions the benefits of labeling outweigh its side effects: whether the implied-truth effect persists in realistic mixed-content feeds and prolonged real-world use (concurrent studies disagree about its sign and size), which causal mechanisms drive over-reliance on labels, how users' judgments are affected by the performance of the labeling system — including whether the risk of mislabeling is directly quantified — and how label design choices jointly modulate benefits and side effects.

Background

Generative image models can produce photorealistic 'AI-generated images' (AIGIs), raising the fear that synthetic imagery will be used for visual misinformation. A common policy response is the 'AI label' or provenance label: a marker attached to content stating that it was AI-generated, implemented via platform labels, watermarking, or embedded metadata, and mandated for high-risk content by Article 50 of the EU AI Act and similar rules. The relevant psychological backdrop is the 'implied truth effect': in a classic experiment, Pennycook, Bear, Collins and Rand (2020, Management Science) showed that attaching fact-check warnings to a subset of false news headlines increased the perceived accuracy of the untagged headlines, because the absence of a warning was read as implicit verification. A parallel literature documents 'AI aversion': audiences discount content disclosed as AI-made — Altay and Gilardi (2024, PNAS Nexus) found that labeling headlines as AI-generated lowered their perceived accuracy whether they were true or false, and Toff and Simon (2025) found AI disclosure reduces trust in news articles. Hoes et al. (2024, Nature Human Behaviour) showed more broadly that prominent misinformation interventions increase skepticism toward accurate information. Within this landscape, the source study (Höltervennhoff et al., presented at CHI 2026, the ACM Conference on Human Factors in Computing Systems; arXiv:2505.22845, v1 titled 'Security Benefits and Side Effects of Labeling AI-Generated Images') combined five focus groups with a pre-registered online survey of 1,354 US and EU participants who judged news-like posts pairing true or false claims with labeled or unlabeled AI-generated or human-made images. Labels reduced belief in false claims supported by AI-generated images, but participants with labels present believed false claims with human-made images more (an implied-truth effect for provenance labels), and believed true claims less when these were illustrated with labeled AI-generated images; focus groups additionally showed that trust in labeling systems is fragile under mislabeling. Related experiments had already shown that label wording is understood differently across countries (Epstein et al., 2023), that label design dimensions change trust in the label itself (Gamage et al., CHI 2025), and that both process-based ('AI-generated') and harm-based ('misleading') labels reduce belief in labeled AI-image posts (Wittenberg et al., 2025, PNAS Nexus) — but none of these tested what happens to unlabeled content in a labeled feed. The source study's own design was artificial in exactly the respects that matter for interpreting these side effects: a balanced stimulus set (equal numbers of true/false and human-made/AI items, unlike real feeds) and a short single-session experiment, so habituation, ecosystem-level, and in-the-wild effects are unknown.

Problem Statement

The common objective is to characterize the trust-discernment dissociation produced by provenance labels on AI-generated images — labels shift what users believe without improving their ability to discern true from false — and to determine the deployment conditions under which the labels' security benefit (reduced belief in AI-generated misinformation) outweighs their side effects (increased credulity toward unlabeled human-made misinformation and discounted belief in true claims illustrated with labeled AI images). Progress toward an empirically grounded answer spans five connected components, each traceable to the source study's open questions: (1) Generalization: whether the implied-truth effect of image provenance labels persists in realistic mixed-content social feeds (where AI-generated and human-made, true and false content are unbalanced) and over prolonged or in-the-wild usage, settling the currently conflicting estimates of the net spillover's sign and size; (2) Mechanisms: which causal mechanism drives label over-reliance — users interpreting labels (and label absence) as veracity endorsements or warnings, labels distracting attention from content cues, or heuristic substitution of the label for content evaluation; (3) Error tolerance: how perceived and actual operational error rates of the labeling pipeline (detector false positives and false negatives, watermark/metadata stripping, self-disclosure non-compliance) affect user trust and reliance, including whether directly quantifying the risk of mislabeling changes users' judgments; (4) Design trade-off: how label design choices (wording, prominence, binary labels versus probabilistic or quantified-risk presentation, contextual explanation of what the label does and does not mean) modulate the security benefit and the side effects jointly, so that designs can be ranked on the benefit-to-side-effect ratio rather than on belief reduction alone; and (5) External validity: whether these conclusions hold beyond US/EU convenience samples, across cultures and regulatory contexts, and in deployed platform settings where labels interact with source credibility, reputation signals, and mixed media. Each component feeds the same deployment decision — whether, how, and under what conditions platforms and regulators should label AI-generated images. Because (1)–(3) ask how audiences actually respond, settling them requires empirical evidence about that behavior. Because the security benefit and the two side effects are heterogeneous belief changes on separate claims, no single aggregate 'net effect' is fixed in advance.

References

  1. Sandra Höltervennhoff, Jonas Ricker, Maike M. Raphael, Charlotte Schwedes, Rebecca Weil, Asja Fischer, Thorsten Holz, Lea Schönherr, Sascha Fahl (2026). 'That's another doom I haven't thought about': A User Study on AI Labels as a Safeguard Against Image-Based Misinformation. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. DOI: 10.1145/3772318.3791006. https://doi.org/10.1145/3772318.3791006 (preprint arXiv:2505.22845, v1 titled 'Security Benefits and Side Effects of Labeling AI-Generated Images')
  2. Gordon Pennycook, Adam Bear, Evan T. Collins, David G. Rand (2020). The Implied Truth Effect: Attaching Warnings to a Subset of Fake News Headlines Increases Perceived Accuracy of Headlines Without Warnings. Management Science 66(11):4944–4957. DOI: 10.1287/mnsc.2019.3478. https://doi.org/10.1287/mnsc.2019.3478
  3. Sacha Altay, Fabrizio Gilardi (2024). People are skeptical of headlines labeled as AI-generated, even if true or human-made, because they assume full AI automation. PNAS Nexus 3(10). DOI: 10.1093/pnasnexus/pgae403. https://doi.org/10.1093/pnasnexus/pgae403
  4. Chloe Wittenberg, Ziv Epstein, Gabrielle Péloquin-Skulski, Adam J. Berinsky, David G. Rand (2025). Labeling AI-generated media online. PNAS Nexus 4(5). DOI: 10.1093/pnasnexus/pgaf170. https://doi.org/10.1093/pnasnexus/pgaf170
  5. Ziv Epstein, Mengying Cathy Fang, Antonio Alonso Arechar, David Gertler Rand (2023). What label should be applied to content produced by generative AI? PsyArXiv preprint. DOI: 10.31234/osf.io/v4mfz. https://doi.org/10.31234/osf.io/v4mfz
  6. Dilrukshi Gamage, Dilki Sewwandi, Min Zhang, Arosha K Bandara (2025). Labeling Synthetic Content: User Perceptions of Label Designs for AI-Generated Content on Social Media. Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems. DOI: 10.1145/3706598.3713171. https://doi.org/10.1145/3706598.3713171
  7. Benjamin Toff, Felix M. Simon (2025). 'Or They Could Just Not Use It?': The Dilemma of AI Disclosure for Audience Trust in News. The International Journal of Press/Politics 30:881–903. DOI: 10.1177/19401612241308697. https://doi.org/10.1177/19401612241308697
  8. Emma Hoes, Brian Aitken, Jingwen Zhang, Tomasz Gackowski, Magdalena Wojcieszak (2024). Prominent misinformation interventions reduce misperceptions but increase scepticism. Nature Human Behaviour 8:1545–1553. DOI: 10.1038/s41562-024-01884-x. https://doi.org/10.1038/s41562-024-01884-x
  9. Fabian Pawelczyk, Drew Dimmery, Pu Yan (2026). Implied Authenticity Effect? The Impact of Explicit Labels on AI-Generated Content. Proceedings of the International AAAI Conference on Web and Social Media 20:1738–1766. DOI: 10.1609/icwsm.v20i1.42721. https://doi.org/10.1609/icwsm.v20i1.42721
  10. Zeve Sanderson, Joshua A. Tucker, Wei Zhong (2025). It Works When It Works: Measuring the Direct and Indirect Effects of AI Labels on Political Images. SocArXiv preprint. DOI: 10.31235/osf.io/nf785_v1. https://doi.org/10.31235/osf.io/nf785_v1
  11. Chuyao Wang, Patrick Sturgis, Daniel de Kadt (2025). AI labeling reduces the perceived accuracy of online content but has limited broader effects. arXiv:2506.16202. DOI: 10.48550/arXiv.2506.16202. https://arxiv.org/abs/2506.16202
  12. Darian Harff (2026). Detection and spill-over effects of AI-generated images in political messages: Evidence from two pre-registered experiments. Computers in Human Behavior. DOI: 10.1016/j.chb.2026.108989. https://doi.org/10.1016/j.chb.2026.108989
  13. Eitan Wolf, Yasith Samaradivakara, Om Gokhale, Sally Ahmed, Yuhan Wang, Pat Pataranutaporn, Pattie Maes (2026). Seeing Is Not Believing: Realistic AI Videos Disrupt Confidence in Authentic Videos and Perceived Reality. CHI '26 Extended Abstracts. DOI: 10.1145/3772363.3798358. https://doi.org/10.1145/3772363.3798358
  14. Teresa Elena Weikmann, Marina Tulin, Michael Hameleers, Claes de Vreese (2025). No easy fix to countering AI-generated visual disinformation: The (in)effectiveness of AI-labels, fact-check labels and community notes. PsyArXiv preprint. DOI: 10.31219/osf.io/8237p_v1. https://doi.org/10.31219/osf.io/8237p_v1
  15. Jingruo Chen, Tung-Yen Wang, Marie Williams, Natalia Andrea Jordan, Mingyi Shao, Linda Zhang, Susan R. Fussell (2025). Examining the Impact of Label Detail and Content Stakes on User Perceptions of AI-Generated Images on Social Media. Proceedings of the ACM on Human-Computer Interaction (CSCW Companion). DOI: 10.1145/3715070.3749237. https://doi.org/10.1145/3715070.3749237
  16. Zhixuan Gong, Danling Peng, Jinwei Cui, Zhuoru Lv (2026). The paradox of AI content labeling: how clarity influences information avoidance via cognitive dissonance on social platforms. Frontiers in Psychology. DOI: 10.3389/fpsyg.2026.1751670. https://doi.org/10.3389/fpsyg.2026.1751670
  17. Magdalena Wischnewski, Alisa Scharmann, Annika Ridder, Nicole Krämer (2026). Certified But Imperfect: Investigating The Role of AI Certifications And System Performance on Trust in And Reliance on AI Systems. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. DOI: 10.1145/3772318.3790983. https://doi.org/10.1145/3772318.3790983
  18. Simon Clark, Stephan Lewandowsky (2026). The continued influence of AI-generated deepfake videos despite transparency warnings. Communications Psychology. DOI: 10.1038/s44271-025-00381-9. https://doi.org/10.1038/s44271-025-00381-9
  19. Lorena Licenji, Julian Hoxha (2026). When news is 'written by artificial intelligence': a systematic review of provenance and disclosure cues in journalism and their effects on credibility and trust. Frontiers in Artificial Intelligence. DOI: 10.3389/frai.2026.1815243. https://doi.org/10.3389/frai.2026.1815243
  20. Maja Adena, Eleonora Alabrese, Francesco Capozza, Isabelle Leader (2025). AI Images, Labels and News Demand. SSRN working paper. DOI: 10.2139/ssrn.5780187. https://doi.org/10.2139/ssrn.5780187
  21. Yichen Xiao, Juan Du, Yidan Ding, Minyang Zhang, Yumei Jiang, Yilin Yang, Jie Liu (2026). AI-Generated Content Disclosure and Prolonged Short-Video Engagement: A Heuristic-Systematic Risk-Trust Model Among Late-Adolescent and Emerging-Adult TikTok Users. Behavioral Sciences 16(7):1179. DOI: 10.3390/bs16071179. https://doi.org/10.3390/bs16071179
  22. Han Sol Lim, Bong Gyou Lee, Yoon Hi Sung, Chang Won Jung (2026). Human-made vs. AI-generated: how provenance labels drive strategic curation via perceived effort. Frontiers in Psychology. DOI: 10.3389/fpsyg.2026.1840483. https://doi.org/10.3389/fpsyg.2026.1840483
  23. Christina Yeung, Galen Weld, Jaron Mink, Franziska Roesner (2026). Is This AI? Longitudinal Analysis of Strategies Used for AI Detection on Two Subreddits. arXiv:2606.22689. DOI: 10.48550/arXiv.2606.22689. https://arxiv.org/abs/2606.22689
  24. Shuning Zhang, Hui Wang, Rongjun Ma, Xin Yi, Kanye Ye Wang, Robert Xiao, Hewu Li (2026). Understanding Security and Privacy Perceptions of Content Creators Regarding AI Labels of AI-Generated Content. arXiv:2608.08129. DOI: 10.48550/arXiv.2608.08129. https://arxiv.org/abs/2608.08129