A 2018 study of active Twitter users (Glenski, Weninger, and Volkova) found systematic differences between automated accounts (bots) and human accounts in how they react to news sources of varying credibility: bots produced 9–15% of reactions while comprising only 7–10% of reacting accounts, bot and human accounts differed in reaction speed for several reaction and source-type combinations, and the two groups showed significantly different inequality in reaction volume. The authors explicitly left open whether these patterns hold (1) over longer timescales, as the composition of reaction types and reacting actors evolves after a source posts; (2) when users who react infrequently (fewer than 5 reactions in the collection period) are included in the analysis; and (3) on platforms other than Twitter, specifically Facebook and Reddit. Later literature covers adjacent ground — aggregate engagement dynamics on Facebook and Twitter, population-level propagation, and all-user reaction typing on Reddit — but no published work replicates the bot-versus-human comparison (prevalence, reaction-type variety, reaction speed, and reaction-volume inequality) along any of the three named axes.

Background

As social media became a primary news channel, researchers sought to quantify how audiences engage with news sources of differing credibility, independently of the veracity of any single story. Glenski, Weninger, and Volkova (IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining, ASONAM 2018; arXiv:1807.05327) studied this question on Twitter with an explicit bot-versus-human comparison. Their setup, which defines the terms of the open questions, is as follows. News sources (Twitter accounts of outlets) were partitioned into five credibility classes using previously compiled, third-party-annotated lists: trusted (factual, no intent to deceive), clickbait (attention-grabbing or vague headlines), conspiracy (uncorroborated explanations), propaganda (intentionally misleading content advancing an agenda), and disinformation (fabricated content meant to deceive). Reactions were tweets that directly retweeted or @-mentioned one of these source accounts, collected from January 2016 to January 2017. Each reaction was automatically labeled with a fine-grained reaction typology of eight types (answer, appreciation, agreement, disagreement, elaboration, humor, negative reaction, question) plus a residual "other" label for reactions the model did not place in one of the eight ("other" is among the four most frequent reaction types and appears in the paper's inequality results), adapted from Zhang, Culbertson, and Paritosh's coarse discourse classification of Reddit comments (International AAAI Conference on Web and Social Media, ICWSM 2017). Accounts were classified as bots or humans using Botometer, a supervised machine-learning classifier that scores how likely a Twitter account is automated (here, bot score at or above 0.5 meant bot). To focus on reliably observable behavior, the study analyzed only active users, defined as accounts with at least 5 reactions in the collection period. Four families of measures were computed for bots versus humans across the five source classes: prevalence (the share of reactions and of reacting accounts attributable to bots), reaction variety (the distribution over the eight reaction types), reaction speed (the delay between the source's post and the reaction, compared via cumulative distribution functions and Mann–Whitney U tests, a nonparametric test of whether two samples come from the same distribution), and reaction inequality (how unevenly reaction volume is spread across accounts, measured with Lorenz curves and Gini coefficients, the standard income-inequality tools). Key findings: bots accounted for 9–15% of reactions but only 7–10% of reacting accounts; humans elaborated on and questioned clickbait faster than bots; bots posted answer-reactions to propaganda significantly faster than humans; and bots and humans showed significantly different inequality in reaction volume. The paper's conclusion states verbatim: "Future work will focus on further exploration of the differences in evolution of the response to deceptive sources, an expanded analysis that incorporates both frequent and infrequently reacting users, and comparisons across multiple platforms e.g., Facebook and Reddit." A companion paper by the same authors (Conference on Empirical Methods in Natural Language Processing, EMNLP 2018) extended the reaction-type and speed analysis (without bots or inequality) to Reddit, finding far smaller trusted-versus-deceptive differences there than on Twitter, and itself deferred bot-level analysis to future work.

Problem Statement

Determine whether the bot-versus-human reaction patterns and reaction-volume inequality that Glenski, Weninger, and Volkova (ASONAM 2018, the IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining) documented for active Twitter users reacting to deceptive versus trusted news sources generalize beyond the original observational window, along the three axes the authors themselves named as future work: (1) Temporal evolution — track how responses to deceptive versus trusted news sources (the five classes: trusted, clickbait, conspiracy, propaganda, disinformation) evolve over longer timescales than the few days covered in the original study, on the order of weeks to months after a source posts, including how the composition of reaction types (answer, appreciation, agreement, disagreement, elaboration, humor, negative reaction, question, and other) and the composition of reacting actors (bots versus humans) change over time after a source posts. (2) Population coverage — recompute the four measure families (prevalence of bots among reactions and reacting accounts, reaction-type variety, reaction speed, and reaction-volume inequality measured with Lorenz curves and Gini coefficients) including infrequently reacting users (those with fewer than 5 reactions in the collection period), and establish how the original estimates and conclusions change, i.e., whether patterns observed among frequent reactors hold for the broader user base. (3) Platform generality — test whether the bot-versus-human reaction patterns and inequality measures replicate on at least one platform other than Twitter, chosen from Facebook or Reddit, where reaction-type taxonomies and account-automation classifiers must be adapted and validated for the platform. The common objective uniting the three parts is to establish the robustness and scope of the claim that bots and humans react differently to news sources of varying credibility; all three parts are jointly in scope, and the question asks, for each part and using the four measure families of the original study, which of the original findings persist. A documented bot-versus-human difference counts as persisting in a part only if it keeps its original direction and remains statistically significant under that measure family's own test — cumulative distribution functions and Mann–Whitney U tests for reaction speed, and the study's significance tests for reaction-type variety and for reaction-volume inequality — at the significance level used by that test. Each part rests on empirical resources valid for the platform it addresses — the reaction corpus, the source-credibility assignments, the account-automation classifier, and the reaction-type taxonomy — not on estimated ones.

References

  1. Glenski, Maria; Weninger, Tim; Volkova, Svitlana (2018). How Humans versus Bots React to Deceptive and Trusted News Sources: A Case Study of Active Users. ASONAM 2018. DOI: 10.1109/ASONAM.2018.8508700; arXiv:1807.05327. https://arxiv.org/abs/1807.05327
  2. Glenski, Maria; Weninger, Tim; Volkova, Svitlana (2018). Identifying and Understanding User Reactions to Deceptive and Trusted Social News Sources. EMNLP 2018. DOI: 10.18653/v1/p18-2029; arXiv:1805.12032. https://arxiv.org/abs/1805.12032
  3. Glenski, Maria; Weninger, Tim; Volkova, Svitlana (2018). Propagation From Deceptive News Sources: Who Shares, How Much, How Evenly, and How Quickly? IEEE Transactions on Computational Social Systems 5(4), 1071-1082. DOI: 10.1109/TCSS.2018.2881071; arXiv:1812.03533. https://arxiv.org/abs/1812.03533
  4. Glenski, Maria; Volkova, Svitlana; Kumar, Srijan (2020). User Engagement with Digital Deception. In: Disinformation, Misinformation, and Fake News in Social Media, Springer. DOI: 10.1007/978-3-030-42699-6_3. https://link.springer.com/chapter/10.1007/978-3-030-42699-6_3
  5. Zhang, Amy; Culbertson, Bryan; Paritosh, Praveen (2017). Characterizing Online Discussion Using Coarse Discourse Sequences. ICWSM 2017. DOI: 10.1609/icwsm.v11i1.14886. https://doi.org/10.1609/icwsm.v11i1.14886
  6. Edelson, Laura; Nguyen, Minh-Kha; Goldstein, Ian; Goga, Oana; McCoy, Damon; Lauinger, Tobias (2021). Understanding engagement with U.S. (mis)information news sources on Facebook. IMC 2021. DOI: 10.1145/3487552.3487859. https://doi.org/10.1145/3487552.3487859
  7. Weld, Galen; Glenski, Maria; Althoff, Tim (2021). Political Bias and Factualness in News Sharing across more than 100,000 Online Communities. ICWSM 2021. DOI: 10.1609/icwsm.v15i1.18104; arXiv:2102.08537. https://arxiv.org/abs/2102.08537
  8. Mohammadinodooshan, Alireza; Carlsson, Niklas (2022). Temporal Dynamics of User Engagement with U.S. News Sources on Facebook. SNAMS 2022. DOI: 10.1109/SNAMS58071.2022.10062675. https://doi.org/10.1109/SNAMS58071.2022.10062675
  9. Mohammadinodooshan, Alireza; Carlsson, Niklas (2023). Effects of Political Bias and Reliability on Temporal User Engagement with News Articles Shared on Facebook. PAM 2023, LNCS, Springer. DOI: 10.1007/978-3-031-28486-1_8. https://doi.org/10.1007/978-3-031-28486-1_8
  10. Mohammadinodooshan, Alireza; Carlsson, Niklas (2025). Understanding Engagement Dynamics with (Un)Reliable News Publishers on Twitter. ASONAM 2024, LNCS 15213, Springer. DOI: 10.1007/978-3-031-78548-1_4. https://link.springer.com/chapter/10.1007/978-3-031-78548-1_4
  11. Amini, Arash; Bayiz, Yigit Ege; Ram, Ashwin; Marculescu, Radu; Topcu, Ufuk (2024). News Source Credibility Assessment: A Reddit Case Study. ICWSM 2024. arXiv:2402.10938. https://arxiv.org/abs/2402.10938
  12. Akhtar, Mohammad Majid; Masood, Rahat; Ikram, Muhammad; Kanhere, Salil S. (2024). SoK: False Information, Bots and Malicious Campaigns: Demystifying Elements of Social Media Manipulations. Proceedings of the 19th ACM Asia Conference on Computer and Communications Security (AsiaCCS 2024). DOI: 10.1145/3634737.3644998; arXiv:2308.12497. https://doi.org/10.1145/3634737.3644998