World Health Organization (WHO) guidelines increasingly draw on qualitative research, but a 2024 document analysis of WHO guidelines published in 2020-2022 found that its use is ad hoc: qualitative evidence frequently informed recommendations in maternal and newborn health but rarely in infectious disease, appraisals of primary qualitative studies were often absent, the reasoning behind categorical evidence-to-decision judgements was undocumented, and qualitative findings rarely informed guideline scope, implementation considerations, or contextualisation of recommendations. The open problem, reaffirmed by the same authors' 2026 interview study and by independent 2025-2026 studies of guideline processes, is to develop the methods and standards needed to make this integration systematic: procedures for searching, commissioning, and appraising qualitative evidence (including when to require qualitative evidence syntheses versus appraising primary studies); transparent documentation of how qualitative findings are interpreted and weighed against quantitative evidence and how panels justify categorical judgements; approaches for using explanatory, transferable qualitative findings to produce context-tailored recommendations and implementation considerations; consistent application of certainty appraisal (GRADE-CERQual) across topic areas; and standards for qualitative methodological expertise on guideline panels and for reporting who produced qualitative summaries.

Background

Clinical practice and public health guidelines are developed by expert panels who weigh systematic reviews of research evidence against criteria such as benefits and harms, acceptability, feasibility, equity, and resource use, typically using an evidence-to-decision (EtD) framework — a structured table recording the panel's judgement on each criterion and the rationale for the recommendation. Qualitative research (interview, focus-group, and ethnographic studies, and syntheses of these) provides evidence about how people experience interventions, why they accept or reject them, and what barriers and facilitators operate in real settings — questions that randomised trials cannot answer. A qualitative evidence synthesis (QES) is a systematic review of primary qualitative studies; GRADE-CERQual (Confidence in the Evidence from Reviews of Qualitative Research) is the companion approach for rating how much confidence a decision-maker can place in each finding from such a synthesis.

A WHO/Cochrane methods series in 2019 (the 'QES for Guidelines' papers 1-3) set out how QES findings could inform guideline scope, EtD frameworks, and implementation considerations. Subsequent audits showed this potential was not being realised: Wang et al. (2020) found that WHO and national guidelines used qualitative evidence to inform decisions in 86% of cases but to set clinical questions in only 20% and implementation considerations in 19%; Carmona et al. (2021, 2022) found the methodological literature thin and the conduct and reporting of QES in guidelines incomplete; Wainwright et al. (2023) found that 41.6% of 233 reviews applying GRADE-CERQual seriously misapplied it (for example treating it as a quality-appraisal tool for primary studies), with fidelity concerns in over half of the remainder. Taylor, Garner, Oliver and Desmond (2024) then performed a document analysis of WHO guidelines published in 2020-2022 in maternal and newborn health (MANH) and infectious disease, coding how qualitative research was used. They found: all MANH guidelines applied GRADE-CERQual to QES findings but none formally appraised stand-alone primary qualitative studies, while no infectious-disease guideline made any certainty judgement; rationales for categorical judgements such as 'probably yes' or 'varied' acceptability, feasibility, and equity were often not provided; qualitative research rarely informed guideline scope or implementation considerations; explanatory findings were under-used relative to descriptive ones; summaries were often reduced to categorical judgements at the expense of the original complexity; and only some MANH guidelines — and no infectious-disease guidelines — included qualitative expertise on the panel. Their conclusion states: 'We identified unmet potential in informing implementation considerations and contextualisation of the recommendations. Use in these areas needs further methods development.' The context is the recognised efficacy-to-effectiveness gap — interventions that work in trials often fail to deliver intended outcomes in real settings because context is not understood, which is precisely what explanatory qualitative evidence addresses.

Problem Statement

How can qualitative research be integrated into WHO-type guideline development systematically and transparently? Concretely, methods and standards are needed for: (1) standard procedures for searching for, commissioning, and appraising qualitative evidence for guideline use, including when guideline panels should require a qualitative evidence synthesis versus appraising primary qualitative studies directly; (2) transparently documenting how qualitative findings are interpreted, how conflicting qualitative and quantitative evidence are weighted within evidence-to-decision frameworks, and how panels justify categorical judgements (such as 'probably yes' or 'varied') drawn from complex qualitative syntheses without reducing them prematurely; (3) using explanatory, transferable qualitative findings — findings that account for how or why an intervention works and that can reasonably be held to apply beyond the settings in which they were produced — to inform contextualised, population-specific recommendations and explicit implementation considerations during guideline development, so that qualitative evidence addresses the efficacy-to-effectiveness gap; (4) applying and reporting certainty appraisal of qualitative findings (GRADE-CERQual) consistently and with fidelity across guideline topic areas beyond maternal and newborn health; and (5) defining the qualitative methodological expertise required within guideline author teams and panels, and reporting who produced the qualitative summaries on which recommendations rest. These five components jointly serve one objective — replacing the currently ad hoc use of qualitative research in guideline development with a systematic and transparent methodology. The objective is the integrated methodology that the five components jointly constitute, and for each of the five components a concrete, usable procedure or standard that a WHO-type guideline process could adopt and evaluate is needed.

References

  1. Taylor M, Garner P, Oliver S, Desmond N. Use of qualitative research in World Health Organisation guidelines: a document analysis. Health Research Policy and Systems 2024;22(1):44. https://doi.org/10.1186/s12961-024-01120-y
  2. Taylor M, Garner P, Oliver S, Desmond N. Expert stakeholders on the role of qualitative research in World Health Organisation guidelines. Health Policy and Planning 2026;41(3):433-441. https://doi.org/10.1093/heapol/czaf105
  3. Carmona C, Carroll C, Baxter S. Understanding the role of qualitative evidence synthesis in health and social care guideline development: perspectives from guideline committee members and technical staff in a leading UK producer of health and social care guidelines. Systematic Reviews 2026. https://doi.org/10.1186/s13643-026-03268-w
  4. Wainwright M, Zahroh RI, Tunçalp Ö, Booth A, Bohren MA, Noyes J, Cheng W, Munthe-Kaas H, Lewin S. The use of GRADE-CERQual in qualitative evidence synthesis: an evaluation of fidelity and reporting. Health Research Policy and Systems 2023;21(1):77. https://doi.org/10.1186/s12961-023-00999-3
  5. Meneses-Echavez JF, Alonso-Coello P, Vist G, Bracchiglione J, Song Y, Flottorp S, Rosenbaum SE. Evidence-to-Decision Frameworks: Enhancing the Quality and Rigour of Guidelines and Recommendations. Clinical and Public Health Guidelines 2026;3(3):e70078. https://doi.org/10.1002/gin2.70078
  6. Movsisyan A, Asres Ioab K, Himmels JW, Bantle GL, Dobrescu A, Flottorp S, Forland F, Gadinger A, Koscher-Kien C, Klerings I, Meerpohl JJ, Nussbaumer-Streit B, Strahwald B, Rehfuess EA. Conducting evidence synthesis and developing evidence-based advice in public health and beyond: A scoping review and map of methods guidance. Research Synthesis Methods 2026;17(2):240-264. https://doi.org/10.1017/rsm.2025.10051
  7. Ayine B, Suom-Kogle CF. Assessing the Quality of World Health Organisation Guidelines during Health Emergencies: A Domain-Based Analysis. Journal of Epidemiology and Global Health 2025;15(1):117. https://doi.org/10.1007/s44197-025-00461-3
  8. Downe S, Finlayson KW, Lawrie TA, Lewin SA, Glenton C, Rosenbaum S, Barreix M, Tunçalp Ö. Qualitative Evidence Synthesis (QES) for Guidelines: Paper 1 – Using qualitative evidence synthesis to inform guideline scope and develop qualitative findings statements. Health Research Policy and Systems 2019;17(1):76. https://doi.org/10.1186/s12961-019-0467-5
  9. Lewin S, Glenton C, Lawrie TA, Downe S, Finlayson KW, Rosenbaum S, Barreix M, Tunçalp Ö. Qualitative Evidence Synthesis (QES) for Guidelines: Paper 2 – Using qualitative evidence synthesis findings to inform evidence-to-decision frameworks and recommendations. Health Research Policy and Systems 2019;17(1):75. https://doi.org/10.1186/s12961-019-0468-4
  10. Glenton C, Lewin S, Lawrie TA, Barreix M, Downe S, Finlayson KW, Tamrat T, Rosenbaum S, Tunçalp Ö. Qualitative Evidence Synthesis (QES) for Guidelines: Paper 3 – Using qualitative evidence syntheses to develop implementation considerations and inform implementation processes. Health Research Policy and Systems 2019;17(1):74. https://doi.org/10.1186/s12961-019-0450-1
  11. Wang Y-Y, Liang D-D, Lu C, Shi Y-X, Zhang J, Cao Y, Fang C, Huang D, Jin Y-H. An exploration of how developers use qualitative evidence: content analysis and critical appraisal of guidelines. BMC Medical Research Methodology 2020;20(1):160. https://doi.org/10.1186/s12874-020-01041-8
  12. Carmona C, Baxter S, Carroll C. Systematic review of the methodological literature for integrating qualitative evidence syntheses into health guideline development. Research Synthesis Methods 2021;12(4):491-505. https://doi.org/10.1002/jrsm.1483
  13. Carmona C, Baxter S, Carroll C. The conduct and reporting of qualitative evidence syntheses in health and social care guidelines: a content analysis. BMC Medical Research Methodology 2022;22(1):267. https://doi.org/10.1186/s12874-022-01743-1
  14. Lewin S, Booth A, Glenton C, Munthe-Kaas H, Rashidian A, Wainwright M, Bohren MA, Tunçalp Ö, Colvin CJ, Garside R, Carlsen B, Langlois EV, Noyes J. Applying GRADE-CERQual to qualitative evidence synthesis findings: introduction to the series. Implementation Science 2018;13(Suppl 1):2. https://doi.org/10.1186/s13012-017-0688-3
  15. Ames HMR, France EF, Cooper S, Bianchim MS, Lewin S, Schmidt B-M, Uny I, Noyes J. Assessing qualitative data richness and thickness: Development of an evidence-based tool for use in qualitative evidence synthesis. Cochrane Evidence Synthesis and Methods 2024;2(7):e12059. https://doi.org/10.1002/cesm.12059