Research on the demand side of populism measures citizens' populist attitudes with survey batteries of agree–disagree items. A head-to-head comparison of seven such batteries across nine countries (Castanho Silva, Jungkunz, Helbling and Littvay, 2020) found that no battery achieves scalar invariance — the level of measurement invariance required to compare latent mean populism across countries — that item response theory (IRT) information is concentrated near moderate trait levels, and that external validity varies by country and party system. This record states the still-open measurement problem: develop and validate populist-attitude batteries, together with a generalizable construction and adaptation procedure, that achieve cross-nationally scalar-invariant measurement supporting latent-mean comparison, adequate information across the trait continuum including its extremes, and external/predictive validity robust across contexts in which populists are in opposition and in power, validated in at least one nationally representative probability sample. A 2025–2026 validation in European Social Survey Round 10 representative data again attained metric but not scalar invariance, so the core of the problem remains open.

Background

Populism is widely theorized in political science as a 'thin' ideology — a set of ideas with little programmatic content of its own — consisting of three elements: people-centrism (politics should express the will of a homogeneous 'pure people'), anti-elitism (a morally antagonistic view of corrupt elites), and a Manichean or black-and-white outlook on politics (good people versus evil elite). The individual-level disposition to endorse these ideas is called a populist attitude, and studying it is referred to as the demand side of populism (citizens' demand), as opposed to the supply side (populist parties and leaders). Since Akkerman, Mudde and Zaslove (2014) introduced a six-item survey scale, several competing batteries have been proposed, including three-item (Elchardus and Spruyt, 2016), three-dimensional (Schulz, Müller, Schemer, Wirz, Wettstein and Wirth, 2018), and nine-item (the battery of Castanho Silva, Vegetti and Littvay, 2017) instruments, plus items fielded in the Comparative Study of Electoral Systems (CSES), a cross-national post-election survey programme. All batteries ask respondents to agree or disagree with statements such as 'politicians in Congress need to follow the will of the people'.

Comparing scores across countries requires measurement invariance, tested typically with multiple-group confirmatory factor analysis (MGCFA, i.e., factor-analysis models estimated simultaneously in all country groups) or with item response theory (IRT, models that locate each item and respondent on a latent trait from graded responses). Configural invariance means the same factor structure holds everywhere; metric invariance means factor loadings are equal, so correlations and regression slopes can be compared; scalar invariance additionally requires equal item intercepts, and only then can latent means — average levels of populist attitudes — be compared across countries. IRT analysis also yields test information functions showing how precisely a battery measures at each point of the trait continuum; agree–disagree Likert items (response scales from strongly disagree to strongly agree) are further vulnerable to acquiescence bias, the tendency of some respondents to agree with statements regardless of content, which varies across countries.

Castanho Silva, Jungkunz, Helbling and Littvay (2020) administered all seven batteries to the same roughly 2,250 respondents in online convenience samples of about 250 people in each of nine countries (Brazil, France, Greece, Ireland, Italy, Mexico, Spain, the United Kingdom, and the United States). Only the Elchardus–Spruyt and Castanho Silva et al. batteries achieved metric invariance; none achieved scalar invariance, so cross-national comparisons of latent means were unsupported for every battery. IRT information concentrated near moderate trait levels, leaving extreme populist and anti-populist positions poorly measured. Correlations with external criteria (political trust and conspiracy mentality) and predictive performance for populist voting varied by country, notably weakening for countries such as Greece. The authors framed the unresolved issues as: whether other country selections, larger or representative samples, or better item translations could produce scalar-invariant batteries; which item-selection or adaptation procedures yield generalizable, invariant short-form batteries balancing coherence, invariance, and breadth; how to design or augment item sets to capture extreme attitudes without sacrificing invariance; whether the country-varying validity patterns reflect substantive differences in populism across party systems, instrument misspecification, or convenience-sample composition; and whether the psychometric findings replicate in nationally representative probability samples with within-country subgroup invariance (by age, education, gender).

Problem Statement

Develop and empirically validate survey measurement of populist attitudes that supports valid cross-national comparison. The overarching objective, taken from the unresolved issues of Castanho Silva, Jungkunz, Helbling and Littvay (2020), comprises the following components, each of which is an independent open problem: (1) construct, translate, or adapt item batteries — including short forms — that achieve cross-national scalar invariance in multi-country data spanning diverse national and party-system contexts, so that latent mean levels of populist attitudes can be compared across countries rather than only correlations and slopes; (2) characterize and, where deficient, redesign or augment the item sets so that measurement information is adequate across the whole trait continuum, including extreme low and extreme high populist attitudes, without sacrificing the invariance achieved; (3) demonstrate external and predictive validity (e.g., against political trust, conspiracy mentality, and populist voting) that holds across contexts in which populist parties are in opposition and in power, distinguishing substantive cross-country differences from instrument misspecification; and (4) accompany the instrument with a prescriptive, generalizable procedure for item selection, translation, and cultural adaptation that others can apply to produce equivalent batteries, with the key psychometric properties validated in at least one large, nationally representative probability sample and within-country subgroup invariance (by age, education, and gender) established where subgroup sizes permit.

References

  1. Bruno Castanho Silva, Sebastian Jungkunz, Marc Helbling & Levente Littvay (2020), "An Empirical Comparison of Seven Populist Attitudes Scales", Political Research Quarterly 73(2):409–424, doi:10.1177/1065912919833176, https://doi.org/10.1177/1065912919833176
  2. Agnes Akkerman, Cas Mudde & Andrej Zaslove (2014), "How Populist Are the People? Measuring Populist Attitudes in Voters", Comparative Political Studies 47(9):1324–1353, doi:10.1177/0010414013512600, https://doi.org/10.1177/0010414013512600
  3. Mark Elchardus & Bram Spruyt (2016), "Populism, Persistent Republicanism and Declinism: An Empirical Analysis of Populism as a Thin Ideology", Government and Opposition 51(1):111–133, doi:10.1017/gov.2014.27, https://doi.org/10.1017/gov.2014.27
  4. Anne Schulz, Philipp Müller, Christian Schemer, Dominique Stefanie Wirz, Martin Wettstein & Werner Wirth (2018), "Measuring Populist Attitudes on Three Dimensions", International Journal of Public Opinion Research 30(2):316–326, doi:10.1093/ijpor/edw037, https://doi.org/10.1093/ijpor/edw037
  5. Bruno Castanho Silva, Federico Vegetti & Levente Littvay (2017), "The Elite Is Up to Something: Exploring the Relation Between Populism and Belief in Conspiracy Theories", Swiss Political Science Review 23(4):423–443, doi:10.1111/spsr.12270, https://doi.org/10.1111/spsr.12270
  6. Steven M. Van Hauwaert & Stijn Van Kessel (2018), "Beyond protest and discontent: A cross-national analysis of the effect of populist attitudes and issue positions on populist party support", European Journal of Political Research 57(1):68–92, doi:10.1111/1475-6765.12216, https://doi.org/10.1111/1475-6765.12216
  7. Steven M. Van Hauwaert, Christian H. Schimpf & Flavio Azevedo (2020), "The measurement of populist attitudes: Testing cross-national scales using item response theory", Politics 40(1):3–21, doi:10.1177/0263395719859306, https://doi.org/10.1177/0263395719859306
  8. Martin Wettstein, Anne Schulz, Marco Steenbergen, Christian Schemer, Philipp Müller, Dominique S. Wirz & Werner Wirth (2020), "Measuring Populism across Nations: Testing for Measurement Invariance of an Inventory of Populist Attitudes", International Journal of Public Opinion Research 32(2):284–305, doi:10.1093/ijpor/edz018, https://doi.org/10.1093/ijpor/edz018
  9. Bruno Castanho Silva & Christopher Wratil (2023, first published online 18 October 2021), "Do parties' representation failures affect populist attitudes? Evidence from a multinational survey experiment", Political Science Research and Methods 11(2):347–362, doi:10.1017/psrm.2021.63, https://doi.org/10.1017/psrm.2021.63
  10. Sebastian Jungkunz, Robert A. Fahey & Airo Hino (2021), "How populist attitudes scales fail to capture support for populists in power", PLOS ONE 16(12):e0261658, doi:10.1371/journal.pone.0261658, https://doi.org/10.1371/journal.pone.0261658
  11. José Javier Olivas Osuna & José Rama (2022), "Recalibrating populism measurement tools: Methodological inconsistencies and challenges to our understanding of the relationship between the supply- and demand-side of populism", Frontiers in Sociology 7:970043, doi:10.3389/fsoc.2022.970043, https://doi.org/10.3389/fsoc.2022.970043
  12. Hugo Marcos-Marne, Homero Gil de Zúñiga & Porismita Borah (2023), "What do we (not) know about demand-side populism? A systematic literature review on populist attitudes", European Political Science 22(3):293–307, doi:10.1057/s41304-022-00397-3, https://doi.org/10.1057/s41304-022-00397-3
  13. Filipe Falcão, Carlos Jalali & Patrício Costa (2023), "Empirical validation study and psychometric evaluation of the properties of the populist attitudes scale for the portuguese population", BMC Psychology 11(1):93, doi:10.1186/s40359-023-01118-1, https://doi.org/10.1186/s40359-023-01118-1
  14. Giorgio Dolci & Giacomo Melli (2026, online 2025), "Measuring populism in Europe. Comparison and validation in the European Social Survey", Quality & Quantity 60(1):837–855, doi:10.1007/s11135-025-02301-9, https://doi.org/10.1007/s11135-025-02301-9
  15. Scott Clifford & Andrew M. Engelhardt (2026), "Acquiescence Bias and Criterion Validity: Problems and Potential Solutions for Agree-Disagree Scales", Political Behavior (published online 25 February 2026), doi:10.1007/s11109-026-10124-z, https://doi.org/10.1007/s11109-026-10124-z
  16. Xue-Fei Yan & Zhihao Ma (2025), "Assessing the SciPop Scale across 68 Countries/Regions: Testing Measurement Invariance and Alignment in Science-Related Populist Attitudes", PsyArXiv preprint (version 1, 11 September 2025; later 2026 versions retitled "Revisiting the SciPop Scale: Measurement Alignment, Scoring Rules, and Cross-Context Comparability"), doi:10.31234/osf.io/u3ygh_v1, https://doi.org/10.31234/osf.io/u3ygh_v1