In the classic Srull and Wyer (1979) experiments, descrambing mostly hostile sentences led participants to rate an ambiguously behaving person as substantially more hostile. A 26-lab Registered Replication Report (McCarthy et al., 2018) estimated this assimilative priming effect at roughly 0.08 points on the 0–10 hostility rating scale, roughly 40 times smaller than the original study's approximately 3 points, and a 29-lab follow-up (McCarthy et al., 2021) again found null effects for both close and conceptual replications under deliberately favorable conditions. The open problem is the one the replication report itself left open: determine whether any experimental operationalization of hostile priming — in particular the within-participants repeated-measures designs with a brief delay between prime and outcome measure that the report itself pointed to — produces assimilative effects on person impressions that are reliably detectable and theoretically informative, which procedural or population moderators, if any, are critical, and account for why the historical published literature reports effects vastly larger than the multilab estimates. The sub-question for Srull-and-Wyer-style between-participants designs is already resolved negatively; the surviving core concerns alternative designs, moderators, and the meta-scientific account of the discrepancy.

Background

Social priming refers to the claim that briefly presented stimuli ("primes") can unconsciously activate mental representations that then influence subsequent judgments or behavior. A foundational example is the hostile priming effect of Srull and Wyer (1979): participants completed 30 trials of a sentence-descrambling task (rearranging scrambled word sets into grammatical sentences), where either 80% or 20% of the solved sentences described hostile behaviors, and then immediately read a vignette about a man named Donald who behaved in an ambiguously hostile way and rated his personality. Participants exposed to mostly hostile sentences rated Donald about 3 points more hostile (on 0–10 scales) than those exposed to mostly neutral sentences, which the authors interpreted through category accessibility — the idea that activating a knowledge category makes it more likely to be used when interpreting later ambiguous information (Srull & Wyer, 1979, 1980). A priming effect of this kind, in which judgment is pulled toward the primed category, is called assimilative priming, in contrast to contrast effects, in which an activated category pushes judgments away from it. This became one of the most influential results in social cognition, cited well over a thousand times, and the paradigm anchored decades of person-perception and behavioral priming research.

During psychology's replication crisis, behavioral priming came under scrutiny after several failures to reproduce related famous effects (e.g., Doyen et al., 2012, on elderly-related priming of walking speed) and after a number of priming publications were retracted following the Diederik Stapel data-fabrication investigation. The DeCoster and Claypool (2004) meta-analysis of priming effects on impression formation had estimated sizeable effects, but showed a pattern (smaller published effects in more precise studies) consistent with publication bias, the selective reporting of significant results. To test the hostile priming effect directly, McCarthy et al. (2018) organized a Registered Replication Report (RRR) — a pre-approved, preregistered, multi-lab direct replication with pooled analysis — covering 26 labs (N=7,373N = 7{,}373 total). The random-effects meta-analysis (pooling lab-level estimates while allowing the true effect size to vary from lab to lab) yielded an effect of about 0.08 scale points on protagonist ratings (95% CI [0.004, 0.16]) and about −0.08-0.08 on ratings of ambiguously hostile behaviors; detecting an effect this small would require roughly 4,362 participants per condition for 80% power.

The RRR's discussion identified what remains unresolved. It tested one specific between-participants protocol (group-administered packets, immediate judgment) and could not determine whether other operationalizations — it explicitly named within-participants repeated-measures designs with a brief delay between prime and outcome measure, citing Payne, Brown-Iannuzzi, and Loersch (2016) — would show larger, more robust effects. Payne et al. (2016) demonstrated that primes can reliably affect behavior in a within-subjects, trial-by-trial gambling paradigm (six replications, N=988N = 988), suggesting priming may be real but fleeting, detectable only with immediate and repeated measurement. The RRR also could not isolate which unmeasured procedural or cultural moderators (testing context, attentiveness, task-completion mind-sets, cover stories) matter, and it provided no definitive causal account of how publication bias and researcher degrees of freedom (flexible procedural and analytic choices that make supportive findings easier to produce) inflated the historical literature. Notably, a p-curve analysis (a test of whether the distribution of significant p-values in the published literature indicates evidentiary value) of the published hostile-priming literature by McCarthy (2016, PsyArXiv preprint) concluded that this literature does contain evidentiary value, and Kvarven et al. (2020) showed across 15 psychology topics that meta-analyses overestimate effects roughly threefold relative to multilab replications — so the tension between the historical record and the replication evidence is itself an open meta-scientific question.

Problem Statement

The 2018 Registered Replication Report on Srull and Wyer (1979) estimated the hostile priming effect on person impressions at roughly 0.08 points on the 0–10 hostility rating scale, versus the roughly 3 points reported in the original study, and asked what remains undetermined. The problem is to resolve this discrepancy, with one part already settled: Srull-and-Weyer-style between-participants protocols — sentence descrambling followed by a single-shot impression rating — have been shown by the 2021 29-lab follow-up not to produce detectable effects even under deliberately favorable conditions, so that sub-question is closed negatively. The surviving open core has jointly linked parts: (1) determine whether any alternative experimental operationalization of hostile priming produces assimilative effects on person impressions that are reliably detectable and theoretically informative — in particular the direction the RRR itself identified, within-participants repeated-measures designs with a brief delay between the priming stimuli and the outcome measure, and related immediate trial-level measurement variants (cf. Payne et al., 2016), but also other prime dosages, timings, modalities, and cover stories — and under which procedural or population moderators (e.g., testing context, attentiveness, culture, sex, trauma history) any such effect emerges; and (2) account for why the historical published literature reports effects vastly larger than the multilab estimates, weighing the candidate explanations discussed in the literature — publication bias (which the RRR itself offered as one possible explanation), researcher degrees of freedom, retraction-contaminated reports, and genuine moderator differences between the original studies and the replications. Both parts are needed to settle the true status of the hostile priming effect; the enumerated designs, variants, moderators, and explanations above describe the scope of the open question. For part (1), an effect is reliably detectable only if it is reproduced by a test whose effect-size estimate has a confidence interval excluding zero and is corroborated by independent data under the same operationalization — not a single significant result in one sample — and theoretically informative only if it is specific to the construct-accessibility account, dissociable from demand, attentional, and measurement artifacts, and large enough to constrain that account; a single well-supported demonstration of such an effect, or a well-supported argument that no alternative operationalization yields one, resolves part (1). For part (2), explain why the historical published literature reports effects vastly larger than the multilab estimates.

References

  1. Srull, T. K., & Wyer, R. S., Jr. (1979). The role of category accessibility in the interpretation of information about persons: Some determinants and implications. Journal of Personality and Social Psychology, 37(10), 1660–1672. DOI: 10.1037/0022-3514.37.10.1660. https://doi.org/10.1037/0022-3514.37.10.1660
  2. Srull, T. K., & Wyer, R. S., Jr. (1980). Category accessibility and social perception: Some implications for the study of person memory and interpersonal judgments. Journal of Personality and Social Psychology, 38(6), 841–856. DOI: 10.1037/0022-3514.38.6.841. https://doi.org/10.1037/0022-3514.38.6.841
  3. McCarthy, R. J., Skowronski, J. J., Verschuere, B., Meijer, E. H., et al. (2018). Registered Replication Report on Srull and Wyer (1979). Advances in Methods and Practices in Psychological Science, 1(3), 321–336. DOI: 10.1177/2515245918777487. https://doi.org/10.1177/2515245918777487 (70 authors; complete published list: McCarthy, Randy J., Skowronski, John J., Verschuere, Bruno, Meijer, Ewout H., Jim, Ariane, Hoogesteyn, Katherine, Orthey, Robin, Acar, Oguz A., Aczel, Balazs, Bakos, Bence E., Barbosa, Fernando, Baskin, Ernest, Bègue, Laurent, Ben-Shakhar, Gershon, Birt, Angie R., Blatz, Lisa, Charman, Steve D., Claesen, Aline, Clay, Samuel L., Coary, Sean P., Crusius, Jan, Evans, Jacqueline R., Feldman, Noa, Ferreira-Santos, Fernando, Gamer, Matthias, Gerlsma, Coby, Gomes, Sara, González-Iraizoz, Marta, Holzmeister, Felix, Huber, Juergen, Huntjens, Rafaele J. C., Isoni, Andrea, Jessup, Ryan K., Kirchler, Michael, klein Selle, Nathalie, Koppel, Lina, Kovacs, Marton, Laine, Tei, Lentz, Frank, Loschelder, David D., Ludvig, Elliot A., Lynn, Monty L., Martin, Scott D., McLatchie, Neil M., Mechtel, Mario, Nahari, Galit, Özdoğru, Asil Ali, Pasion, Rita, Pennington, Charlotte R., Roets, Arne, Rozmann, Nir, Scopelliti, Irene, Spiegelman, Eli, Suchotzki, Kristina, Sutan, Angela, Szecsi, Peter, Tinghög, Gustav, Tisserand, Jean-Christian, Tran, Ulrich S., Van Hiel, Alain, Vanpaemel, Wolf, Västfjäll, Daniel, Verliefde, Thomas, Vezirian, Kévin, Voracek, Martin, Warmelink, Lara, Wick, Katherine, Wiggins, Bradford J., Wylie, Keith, & Yıldız, Ezgi)
  4. McCarthy, R., Gervais, W., Aczel, B., et al. (2021). A Multi-Site Collaborative Study of the Hostile Priming Effect. Collabra: Psychology, 7(1), article 18738. DOI: 10.1525/collabra.18738. https://doi.org/10.1525/collabra.18738 (60 authors; complete published list: McCarthy, Randy, Gervais, Will, Aczel, Balazs, Al-Kire, Rosemary L., Aveyard, Mark, Baraldo, Silvia Marcella, Baruh, Lemi, Basch, Charlotte, Baumert, Anna, Behler, Anna, Bettencourt, Ann, Bitar, Adam, Bouxom, Hugo, Buck, Ashley, Cemalcilar, Zeynep, Chekroun, Peggy, Chen, Jacqueline M., del Fresno-Díaz, Ángel, Ducham, Alec, Edlund, John E., ElBassiouny, Amanda, Evans, Thomas Rhys, Ewell, Patrick J., Forscher, Patrick S., Fuglestad, Paul T., Hauck, Lauren, Hawk, Christopher E., Hermann, Anthony D., Hines, Bryon, Irumva, Mukunzi, Jordan, Lauren N., Joy-Gaba, Jennifer A., Haley, Catherine, Kačmár, Pavol, Kezer, Murat, Körner, Robert, Kosaka, Muriel, Kovacs, Marton, Lair, Elicia C., Légal, Jean-Baptiste, Leighton, Dana C., Magee, Michael W., Markman, Keith, Martončik, Marcel, Müller, Martin, Norman, Jasmine B., Olsen, Jerome, Oyler, Danielle, Phills, Curtis E., Ribeiro, Gianni, Rohain, Alia, Sakaluk, John, Schütz, Astrid, Toribio-Flórez, Daniel, Tsang, Jo-Ann, Vezzoli, Michela, Williams, Caitlin, Willis, Guillermo B., Young, Jason, & Zogmaister, Cristina)
  5. Payne, B. K., Brown-Iannuzzi, J. L., & Loersch, C. (2016). Replicable effects of primes on human behavior. Journal of Experimental Psychology: General, 145(10), 1269–1279. DOI: 10.1037/xge0000201. https://doi.org/10.1037/xge0000201
  6. McCarthy, R. J. (2016). Assessing the Evidence for the Effect of “Hostile Priming” on Impression Formation Tasks [Preprint]. PsyArXiv. DOI: 10.31234/osf.io/zjych. https://doi.org/10.31234/osf.io/zjych
  7. DeCoster, J., & Claypool, H. M. (2004). A meta-analysis of priming effects on impression formation supporting a general model of informational biases. Personality and Social Psychology Review, 8(1), 2–27. DOI: 10.1207/s15327957pspr0801_1. https://doi.org/10.1207/s15327957pspr0801_1
  8. Doyen, S., Klein, O., Pichon, C.-L., & Cleeremans, A. (2012). Behavioral priming: It's all in the mind, but whose mind? PLoS ONE, 7(1), e29081. DOI: 10.1371/journal.pone.0029081. https://doi.org/10.1371/journal.pone.0029081
  9. Kvarven, A., Strømland, E., & Johannesson, M. (2020). Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature Human Behaviour, 4(4), 423–434. DOI: 10.1038/s41562-019-0787-z. https://doi.org/10.1038/s41562-019-0787-z
  10. Shanks, D. R., & Vadillo, M. A. (2021). Publication bias and low power in field studies on goal priming. Royal Society Open Science, 8(10), 210544. DOI: 10.1098/rsos.210544. https://doi.org/10.1098/rsos.210544
  11. Ripley, A. J., Clapp, J. D., & Wilkowski, B. M. (2022). Trauma and aggression: Evaluating the influence of primed hostility and survivor sex. British Journal of Clinical Psychology, 61(4), 1154–1168. DOI: 10.1111/bjc.12386. https://doi.org/10.1111/bjc.12386
  12. Baumeister, R. F., Tice, D. M., & Bushman, B. J. (2023). A review of multisite replication projects in social psychology: Is it viable to sustain any confidence in social psychology's knowledge base? Perspectives on Psychological Science, 18(4), 912–935. DOI: 10.1177/17456916221121815. https://doi.org/10.1177/17456916221121815
  13. Olsson-Collentine, A., Wicherts, J. M., & van Assen, M. A. L. M. (2020). Heterogeneity in direct replications in psychology and its association with effect size. Psychological Bulletin, 146(10), 922–940. DOI: 10.1037/bul0000294. https://doi.org/10.1037/bul0000294