intervention case study report

• Open access
• Published: 10 November 2020

Case study research for better evaluations of complex interventions: rationale and challenges

• Sara Paparini   ORCID: orcid.org/0000-0002-1909-2481 1 ,
• Judith Green 2 ,
• Chrysanthi Papoutsi 1 ,
• Jamie Murdoch 3 ,
• Mark Petticrew 4 ,
• Trish Greenhalgh 1 ,
• Benjamin Hanckel 5 &
• Sara Shaw 1  

BMC Medicine volume  18 , Article number:  301 ( 2020 ) Cite this article

17k Accesses

42 Citations

35 Altmetric

Metrics details

The need for better methods for evaluation in health research has been widely recognised. The ‘complexity turn’ has drawn attention to the limitations of relying on causal inference from randomised controlled trials alone for understanding whether, and under which conditions, interventions in complex systems improve health services or the public health, and what mechanisms might link interventions and outcomes. We argue that case study research—currently denigrated as poor evidence—is an under-utilised resource for not only providing evidence about context and transferability, but also for helping strengthen causal inferences when pathways between intervention and effects are likely to be non-linear.

Case study research, as an overall approach, is based on in-depth explorations of complex phenomena in their natural, or real-life, settings. Empirical case studies typically enable dynamic understanding of complex challenges and provide evidence about causal mechanisms and the necessary and sufficient conditions (contexts) for intervention implementation and effects. This is essential evidence not just for researchers concerned about internal and external validity, but also research users in policy and practice who need to know what the likely effects of complex programmes or interventions will be in their settings. The health sciences have much to learn from scholarship on case study methodology in the social sciences. However, there are multiple challenges in fully exploiting the potential learning from case study research. First are misconceptions that case study research can only provide exploratory or descriptive evidence. Second, there is little consensus about what a case study is, and considerable diversity in how empirical case studies are conducted and reported. Finally, as case study researchers typically (and appropriately) focus on thick description (that captures contextual detail), it can be challenging to identify the key messages related to intervention evaluation from case study reports.

Whilst the diversity of published case studies in health services and public health research is rich and productive, we recommend further clarity and specific methodological guidance for those reporting case study research for evaluation audiences.

Peer Review reports

The need for methodological development to address the most urgent challenges in health research has been well-documented. Many of the most pressing questions for public health research, where the focus is on system-level determinants [ 1 , 2 ], and for health services research, where provisions typically vary across sites and are provided through interlocking networks of services [ 3 ], require methodological approaches that can attend to complexity. The need for methodological advance has arisen, in part, as a result of the diminishing returns from randomised controlled trials (RCTs) where they have been used to answer questions about the effects of interventions in complex systems [ 4 , 5 , 6 ]. In conditions of complexity, there is limited value in maintaining the current orientation to experimental trial designs in the health sciences as providing ‘gold standard’ evidence of effect.

There are increasing calls for methodological pluralism [ 7 , 8 ], with the recognition that complex intervention and context are not easily or usefully separated (as is often the situation when using trial design), and that system interruptions may have effects that are not reducible to linear causal pathways between intervention and outcome. These calls are reflected in a shifting and contested discourse of trial design, seen with the emergence of realist [ 9 ], adaptive and hybrid (types 1, 2 and 3) [ 10 , 11 ] trials that blend studies of effectiveness with a close consideration of the contexts of implementation. Similarly, process evaluation has now become a core component of complex healthcare intervention trials, reflected in MRC guidance on how to explore implementation, causal mechanisms and context [ 12 ].

Evidence about the context of an intervention is crucial for questions of external validity. As Woolcock [ 4 ] notes, even if RCT designs are accepted as robust for maximising internal validity, questions of transferability (how well the intervention works in different contexts) and generalisability (how well the intervention can be scaled up) remain unanswered [ 5 , 13 ]. For research evidence to have impact on policy and systems organisation, and thus to improve population and patient health, there is an urgent need for better methods for strengthening external validity, including a better understanding of the relationship between intervention and context [ 14 ].

Policymakers, healthcare commissioners and other research users require credible evidence of relevance to their settings and populations [ 15 ], to perform what Rosengarten and Savransky [ 16 ] call ‘careful abstraction’ to the locales that matter for them. They also require robust evidence for understanding complex causal pathways. Case study research, currently under-utilised in public health and health services evaluation, can offer considerable potential for strengthening faith in both external and internal validity. For example, in an empirical case study of how the policy of free bus travel had specific health effects in London, UK, a quasi-experimental evaluation (led by JG) identified how important aspects of context (a good public transport system) and intervention (that it was universal) were necessary conditions for the observed effects, thus providing useful, actionable evidence for decision-makers in other contexts [ 17 ].

The overall approach of case study research is based on the in-depth exploration of complex phenomena in their natural, or ‘real-life’, settings. Empirical case studies typically enable dynamic understanding of complex challenges rather than restricting the focus on narrow problem delineations and simple fixes. Case study research is a diverse and somewhat contested field, with multiple definitions and perspectives grounded in different ways of viewing the world, and involving different combinations of methods. In this paper, we raise awareness of such plurality and highlight the contribution that case study research can make to the evaluation of complex system-level interventions. We review some of the challenges in exploiting the current evidence base from empirical case studies and conclude by recommending that further guidance and minimum reporting criteria for evaluation using case studies, appropriate for audiences in the health sciences, can enhance the take-up of evidence from case study research.

Case study research offers evidence about context, causal inference in complex systems and implementation

Well-conducted and described empirical case studies provide evidence on context, complexity and mechanisms for understanding how, where and why interventions have their observed effects. Recognition of the importance of context for understanding the relationships between interventions and outcomes is hardly new. In 1943, Canguilhem berated an over-reliance on experimental designs for determining universal physiological laws: ‘As if one could determine a phenomenon’s essence apart from its conditions! As if conditions were a mask or frame which changed neither the face nor the picture!’ ([ 18 ] p126). More recently, a concern with context has been expressed in health systems and public health research as part of what has been called the ‘complexity turn’ [ 1 ]: a recognition that many of the most enduring challenges for developing an evidence base require a consideration of system-level effects [ 1 ] and the conceptualisation of interventions as interruptions in systems [ 19 ].

The case study approach is widely recognised as offering an invaluable resource for understanding the dynamic and evolving influence of context on complex, system-level interventions [ 20 , 21 , 22 , 23 ]. Empirically, case studies can directly inform assessments of where, when, how and for whom interventions might be successfully implemented, by helping to specify the necessary and sufficient conditions under which interventions might have effects and to consolidate learning on how interdependencies, emergence and unpredictability can be managed to achieve and sustain desired effects. Case study research has the potential to address four objectives for improving research and reporting of context recently set out by guidance on taking account of context in population health research [ 24 ], that is to (1) improve the appropriateness of intervention development for specific contexts, (2) improve understanding of ‘how’ interventions work, (3) better understand how and why impacts vary across contexts and (4) ensure reports of intervention studies are most useful for decision-makers and researchers.

However, evaluations of complex healthcare interventions have arguably not exploited the full potential of case study research and can learn much from other disciplines. For evaluative research, exploratory case studies have had a traditional role of providing data on ‘process’, or initial ‘hypothesis-generating’ scoping, but might also have an increasing salience for explanatory aims. Across the social and political sciences, different kinds of case studies are undertaken to meet diverse aims (description, exploration or explanation) and across different scales (from small N qualitative studies that aim to elucidate processes, or provide thick description, to more systematic techniques designed for medium-to-large N cases).

Case studies with explanatory aims vary in terms of their positioning within mixed-methods projects, with designs including (but not restricted to) (1) single N of 1 studies of interventions in specific contexts, where the overall design is a case study that may incorporate one or more (randomised or not) comparisons over time and between variables within the case; (2) a series of cases conducted or synthesised to provide explanation from variations between cases; and (3) case studies of particular settings within RCT or quasi-experimental designs to explore variation in effects or implementation.

Detailed qualitative research (typically done as ‘case studies’ within process evaluations) provides evidence for the plausibility of mechanisms [ 25 ], offering theoretical generalisations for how interventions may function under different conditions. Although RCT designs reduce many threats to internal validity, the mechanisms of effect remain opaque, particularly when the causal pathways between ‘intervention’ and ‘effect’ are long and potentially non-linear: case study research has a more fundamental role here, in providing detailed observational evidence for causal claims [ 26 ] as well as producing a rich, nuanced picture of tensions and multiple perspectives [ 8 ].

Longitudinal or cross-case analysis may be best suited for evidence generation in system-level evaluative research. Turner [ 27 ], for instance, reflecting on the complex processes in major system change, has argued for the need for methods that integrate learning across cases, to develop theoretical knowledge that would enable inferences beyond the single case, and to develop generalisable theory about organisational and structural change in health systems. Qualitative Comparative Analysis (QCA) [ 28 ] is one such formal method for deriving causal claims, using set theory mathematics to integrate data from empirical case studies to answer questions about the configurations of causal pathways linking conditions to outcomes [ 29 , 30 ].

Nonetheless, the single N case study, too, provides opportunities for theoretical development [ 31 ], and theoretical generalisation or analytical refinement [ 32 ]. How ‘the case’ and ‘context’ are conceptualised is crucial here. Findings from the single case may seem to be confined to its intrinsic particularities in a specific and distinct context [ 33 ]. However, if such context is viewed as exemplifying wider social and political forces, the single case can be ‘telling’, rather than ‘typical’, and offer insight into a wider issue [ 34 ]. Internal comparisons within the case can offer rich possibilities for logical inferences about causation [ 17 ]. Further, case studies of any size can be used for theory testing through refutation [ 22 ]. The potential lies, then, in utilising the strengths and plurality of case study to support theory-driven research within different methodological paradigms.

Evaluation research in health has much to learn from a range of social sciences where case study methodology has been used to develop various kinds of causal inference. For instance, Gerring [ 35 ] expands on the within-case variations utilised to make causal claims. For Gerring [ 35 ], case studies come into their own with regard to invariant or strong causal claims (such as X is a necessary and/or sufficient condition for Y) rather than for probabilistic causal claims. For the latter (where experimental methods might have an advantage in estimating effect sizes), case studies offer evidence on mechanisms: from observations of X affecting Y, from process tracing or from pattern matching. Case studies also support the study of emergent causation, that is, the multiple interacting properties that account for particular and unexpected outcomes in complex systems, such as in healthcare [ 8 ].

Finally, efficacy (or beliefs about efficacy) is not the only contributor to intervention uptake, with a range of organisational and policy contingencies affecting whether an intervention is likely to be rolled out in practice. Case study research is, therefore, invaluable for learning about contextual contingencies and identifying the conditions necessary for interventions to become normalised (i.e. implemented routinely) in practice [ 36 ].

The challenges in exploiting evidence from case study research

At present, there are significant challenges in exploiting the benefits of case study research in evaluative health research, which relate to status, definition and reporting. Case study research has been marginalised at the bottom of an evidence hierarchy, seen to offer little by way of explanatory power, if nonetheless useful for adding descriptive data on process or providing useful illustrations for policymakers [ 37 ]. This is an opportune moment to revisit this low status. As health researchers are increasingly charged with evaluating ‘natural experiments’—the use of face masks in the response to the COVID-19 pandemic being a recent example [ 38 ]—rather than interventions that take place in settings that can be controlled, research approaches using methods to strengthen causal inference that does not require randomisation become more relevant.

A second challenge for improving the use of case study evidence in evaluative health research is that, as we have seen, what is meant by ‘case study’ varies widely, not only across but also within disciplines. There is indeed little consensus amongst methodologists as to how to define ‘a case study’. Definitions focus, variously, on small sample size or lack of control over the intervention (e.g. [ 39 ] p194), on in-depth study and context [ 40 , 41 ], on the logic of inference used [ 35 ] or on distinct research strategies which incorporate a number of methods to address questions of ‘how’ and ‘why’ [ 42 ]. Moreover, definitions developed for specific disciplines do not capture the range of ways in which case study research is carried out across disciplines. Multiple definitions of case study reflect the richness and diversity of the approach. However, evidence suggests that a lack of consensus across methodologists results in some of the limitations of published reports of empirical case studies [ 43 , 44 ]. Hyett and colleagues [ 43 ], for instance, reviewing reports in qualitative journals, found little match between methodological definitions of case study research and how authors used the term.

This raises the third challenge we identify that case study reports are typically not written in ways that are accessible or useful for the evaluation research community and policymakers. Case studies may not appear in journals widely read by those in the health sciences, either because space constraints preclude the reporting of rich, thick descriptions, or because of the reported lack of willingness of some biomedical journals to publish research that uses qualitative methods [ 45 ], signalling the persistence of the aforementioned evidence hierarchy. Where they do, however, the term ‘case study’ is used to indicate, interchangeably, a qualitative study, an N of 1 sample, or a multi-method, in-depth analysis of one example from a population of phenomena. Definitions of what constitutes the ‘case’ are frequently lacking and appear to be used as a synonym for the settings in which the research is conducted. Despite offering insights for evaluation, the primary aims may not have been evaluative, so the implications may not be explicitly drawn out. Indeed, some case study reports might properly be aiming for thick description without necessarily seeking to inform about context or causality.

Acknowledging plurality and developing guidance

We recognise that definitional and methodological plurality is not only inevitable, but also a necessary and creative reflection of the very different epistemological and disciplinary origins of health researchers, and the aims they have in doing and reporting case study research. Indeed, to provide some clarity, Thomas [ 46 ] has suggested a typology of subject/purpose/approach/process for classifying aims (e.g. evaluative or exploratory), sample rationale and selection and methods for data generation of case studies. We also recognise that the diversity of methods used in case study research, and the necessary focus on narrative reporting, does not lend itself to straightforward development of formal quality or reporting criteria.

Existing checklists for reporting case study research from the social sciences—for example Lincoln and Guba’s [ 47 ] and Stake’s [ 33 ]—are primarily orientated to the quality of narrative produced, and the extent to which they encapsulate thick description, rather than the more pragmatic issues of implications for intervention effects. Those designed for clinical settings, such as the CARE (CAse REports) guidelines, provide specific reporting guidelines for medical case reports about single, or small groups of patients [ 48 ], not for case study research.

The Design of Case Study Research in Health Care (DESCARTE) model [ 44 ] suggests a series of questions to be asked of a case study researcher (including clarity about the philosophy underpinning their research), study design (with a focus on case definition) and analysis (to improve process). The model resembles toolkits for enhancing the quality and robustness of qualitative and mixed-methods research reporting, and it is usefully open-ended and non-prescriptive. However, even if it does include some reflections on context, the model does not fully address aspects of context, logic and causal inference that are perhaps most relevant for evaluative research in health.

Hence, for evaluative research where the aim is to report empirical findings in ways that are intended to be pragmatically useful for health policy and practice, this may be an opportune time to consider how to best navigate plurality around what is (minimally) important to report when publishing empirical case studies, especially with regards to the complex relationships between context and interventions, information that case study research is well placed to provide.

The conventional scientific quest for certainty, predictability and linear causality (maximised in RCT designs) has to be augmented by the study of uncertainty, unpredictability and emergent causality [ 8 ] in complex systems. This will require methodological pluralism, and openness to broadening the evidence base to better understand both causality in and the transferability of system change intervention [ 14 , 20 , 23 , 25 ]. Case study research evidence is essential, yet is currently under exploited in the health sciences. If evaluative health research is to move beyond the current impasse on methods for understanding interventions as interruptions in complex systems, we need to consider in more detail how researchers can conduct and report empirical case studies which do aim to elucidate the contextual factors which interact with interventions to produce particular effects. To this end, supported by the UK’s Medical Research Council, we are embracing the challenge to develop guidance for case study researchers studying complex interventions. Following a meta-narrative review of the literature, we are planning a Delphi study to inform guidance that will, at minimum, cover the value of case study research for evaluating the interrelationship between context and complex system-level interventions; for situating and defining ‘the case’, and generalising from case studies; as well as provide specific guidance on conducting, analysing and reporting case study research. Our hope is that such guidance can support researchers evaluating interventions in complex systems to better exploit the diversity and richness of case study research.

Availability of data and materials

Not applicable (article based on existing available academic publications)

Abbreviations

Qualitative comparative analysis

Quasi-experimental design

Randomised controlled trial

Diez Roux AV. Complex systems thinking and current impasses in health disparities research. Am J Public Health. 2011;101(9):1627–34.

Article   Google Scholar  

Ogilvie D, Mitchell R, Mutrie N, M P, Platt S. Evaluating health effects of transport interventions: methodologic case study. Am J Prev Med 2006;31:118–126.

Walshe C. The evaluation of complex interventions in palliative care: an exploration of the potential of case study research strategies. Palliat Med. 2011;25(8):774–81.

Woolcock M. Using case studies to explore the external validity of ‘complex’ development interventions. Evaluation. 2013;19:229–48.

Cartwright N. Are RCTs the gold standard? BioSocieties. 2007;2(1):11–20.

Deaton A, Cartwright N. Understanding and misunderstanding randomized controlled trials. Soc Sci Med. 2018;210:2–21.

Salway S, Green J. Towards a critical complex systems approach to public health. Crit Public Health. 2017;27(5):523–4.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Bonell C, Warren E, Fletcher A. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials. 2016;17:478.

Pallmann P, Bedding AW, Choodari-Oskooei B. Adaptive designs in clinical trials: why use them, and how to run and report them. BMC Med. 2018;16:29.

Curran G, Bauer M, Mittman B, Pyne J, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217–26. https://doi.org/10.1097/MLR.0b013e3182408812 .

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015 [cited 2020 Jun 27];350. Available from: https://www.bmj.com/content/350/bmj.h1258 .

Evans RE, Craig P, Hoddinott P, Littlecott H, Moore L, Murphy S, et al. When and how do ‘effective’ interventions need to be adapted and/or re-evaluated in new contexts? The need for guidance. J Epidemiol Community Health. 2019;73(6):481–2.

Shoveller J. A critical examination of representations of context within research on population health interventions. Crit Public Health. 2016;26(5):487–500.

Treweek S, Zwarenstein M. Making trials matter: pragmatic and explanatory trials and the problem of applicability. Trials. 2009;10(1):37.

Rosengarten M, Savransky M. A careful biomedicine? Generalization and abstraction in RCTs. Crit Public Health. 2019;29(2):181–91.

Green J, Roberts H, Petticrew M, Steinbach R, Goodman A, Jones A, et al. Integrating quasi-experimental and inductive designs in evaluation: a case study of the impact of free bus travel on public health. Evaluation. 2015;21(4):391–406.

Canguilhem G. The normal and the pathological. New York: Zone Books; 1991. (1949).

Google Scholar  

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

King G, Keohane RO, Verba S. Designing social inquiry: scientific inference in qualitative research: Princeton University Press; 1994.

Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q. 2004;82(4):581–629.

Yin R. Enhancing the quality of case studies in health services research. Health Serv Res. 1999;34(5 Pt 2):1209.

CAS   PubMed   PubMed Central   Google Scholar  

Raine R, Fitzpatrick R, Barratt H, Bevan G, Black N, Boaden R, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016 [cited 2020 Jun 30];4(16). Available from: https://www.journalslibrary.nihr.ac.uk/hsdr/hsdr04160#/abstract .

Craig P, Di Ruggiero E, Frohlich KL, E M, White M, Group CCGA. Taking account of context in population health intervention research: guidance for producers, users and funders of research. NIHR Evaluation, Trials and Studies Coordinating Centre; 2018.

Grant RL, Hood R. Complex systems, explanation and policy: implications of the crisis of replication for public health research. Crit Public Health. 2017;27(5):525–32.

Mahoney J. Strategies of causal inference in small-N analysis. Sociol Methods Res. 2000;4:387–424.

Turner S. Major system change: a management and organisational research perspective. In: Rosalind Raine, Ray Fitzpatrick, Helen Barratt, Gywn Bevan, Nick Black, Ruth Boaden, et al. Challenges, solutions and future directions in the evaluation of service innovations in health care and public health. Health Serv Deliv Res. 2016;4(16) 2016. https://doi.org/10.3310/hsdr04160.

Ragin CC. Using qualitative comparative analysis to study causal complexity. Health Serv Res. 1999;34(5 Pt 2):1225.

Hanckel B, Petticrew M, Thomas J, Green J. Protocol for a systematic review of the use of qualitative comparative analysis for evaluative questions in public health research. Syst Rev. 2019;8(1):252.

Schneider CQ, Wagemann C. Set-theoretic methods for the social sciences: a guide to qualitative comparative analysis: Cambridge University Press; 2012. 369 p.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12:219–45.

Tsoukas H. Craving for generality and small-N studies: a Wittgensteinian approach towards the epistemology of the particular in organization and management studies. Sage Handb Organ Res Methods. 2009:285–301.

Stake RE. The art of case study research. London: Sage Publications Ltd; 1995.

Mitchell JC. Typicality and the case study. Ethnographic research: A guide to general conduct. Vol. 238241. 1984.

Gerring J. What is a case study and what is it good for? Am Polit Sci Rev. 2004;98(2):341–54.

May C, Mort M, Williams T, F M, Gask L. Health technology assessment in its local contexts: studies of telehealthcare. Soc Sci Med 2003;57:697–710.

McGill E. Trading quality for relevance: non-health decision-makers’ use of evidence on the social determinants of health. BMJ Open. 2015;5(4):007053.

Greenhalgh T. We can’t be 100% sure face masks work – but that shouldn’t stop us wearing them | Trish Greenhalgh. The Guardian. 2020 [cited 2020 Jun 27]; Available from: https://www.theguardian.com/commentisfree/2020/jun/05/face-masks-coronavirus .

Hammersley M. So, what are case studies? In: What’s wrong with ethnography? New York: Routledge; 1992.

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A. The case study approach. BMC Med Res Methodol. 2011;11(1):100.

Luck L, Jackson D, Usher K. Case study: a bridge across the paradigms. Nurs Inq. 2006;13(2):103–9.

Yin RK. Case study research and applications: design and methods: Sage; 2017.

Hyett N, A K, Dickson-Swift V. Methodology or method? A critical review of qualitative case study reports. Int J Qual Stud Health Well-Being. 2014;9:23606.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Greenhalgh T, Annandale E, Ashcroft R, Barlow J, Black N, Bleakley A, et al. An open letter to the BMJ editors on qualitative research. Bmj. 2016;352.

Thomas G. A typology for the case study in social science following a review of definition, discourse, and structure. Qual Inq. 2011;17(6):511–21.

Lincoln YS, Guba EG. Judging the quality of case study reports. Int J Qual Stud Educ. 1990;3(1):53–9.

Riley DS, Barber MS, Kienle GS, Aronson JK, Schoen-Angerer T, Tugwell P, et al. CARE guidelines for case reports: explanation and elaboration document. J Clin Epidemiol. 2017;89:218–35.

Download references

Acknowledgements

Not applicable

This work was funded by the Medical Research Council - MRC Award MR/S014632/1 HCS: Case study, Context and Complex interventions (TRIPLE C). SP was additionally funded by the University of Oxford's Higher Education Innovation Fund (HEIF).

Author information

Authors and affiliations.

Nuffield Department of Primary Care Health Sciences, University of Oxford, Oxford, UK

Sara Paparini, Chrysanthi Papoutsi, Trish Greenhalgh & Sara Shaw

Wellcome Centre for Cultures & Environments of Health, University of Exeter, Exeter, UK

Judith Green

School of Health Sciences, University of East Anglia, Norwich, UK

Jamie Murdoch

Public Health, Environments and Society, London School of Hygiene & Tropical Medicin, London, UK

Mark Petticrew

Institute for Culture and Society, Western Sydney University, Penrith, Australia

Benjamin Hanckel

You can also search for this author in PubMed   Google Scholar

Contributions

JG, MP, SP, JM, TG, CP and SS drafted the initial paper; all authors contributed to the drafting of the final version, and read and approved the final manuscript.

Corresponding author

Correspondence to Sara Paparini .

Ethics declarations

Ethics approval and consent to participate, consent for publication, competing interests.

The authors declare that they have no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Paparini, S., Green, J., Papoutsi, C. et al. Case study research for better evaluations of complex interventions: rationale and challenges. BMC Med 18 , 301 (2020). https://doi.org/10.1186/s12916-020-01777-6

Download citation

Received : 03 July 2020

Accepted : 07 September 2020

Published : 10 November 2020

DOI : https://doi.org/10.1186/s12916-020-01777-6

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Qualitative
• Case studies
• Mixed-method
• Public health
• Health services research
• Interventions

BMC Medicine

ISSN: 1741-7015

• Submission enquiries: [email protected]
• General enquiries: [email protected]

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Better reporting of...

Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide

• Related content
• Peer review
• Tammy C Hoffmann , associate professor of clinical epidemiology 1 ,
• Paul P Glasziou , director and professor of evidence based medicine 1 ,
• Isabelle Boutron , professor of epidemiology 2 ,
• Ruairidh Milne , professorial fellow in public health and director 3 ,
• Rafael Perera , university lecturer in medical statistics 4 ,
• David Moher , senior scientist 5 ,
• Douglas G Altman , professor of statistics in medicine 6 ,
• Virginia Barbour , medicine editorial director, PLOS 7 ,
• Helen Macdonald , assistant editor 8 ,
• Marie Johnston , emeritus professor of health psychology 9 ,
• Sarah E Lamb , Kadoorie professor of trauma rehabilitation and co-director of Oxford clinical trials research unit 10 ,
• Mary Dixon-Woods , professor of medical sociology 11 ,
• Peter McCulloch , clinical reader in surgery 12 ,
• Jeremy C Wyatt , leadership chair of ehealth research 13 ,
• An-Wen Chan , Phelan scientist 14 ,
• Susan Michie , professor 15
• 1 Centre for Research in Evidence Based Practice, Faculty of Health Sciences and Medicine, Bond University, Queensland, Australia, 4229
• 2 INSERM U738, Université Paris Descartes-Sorbonne Paris Cité, Paris, France
• 3 Wessex Institute, University of Southampton, Southampton, UK
• 4 Department of Primary Care Health Sciences, University of Oxford, UK
• 5 Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
• 6 Centre for Statistics in Medicine, University of Oxford, UK
• 7 PLOS, Brisbane, Australia
• 8 BMJ, London, UK
• 9 Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
• 10 Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, Botnar Research Centre, University of Oxford, Oxford, UK
• 11 Department of Health Sciences, University of Leicester, Leicester, UK
• 12 Nuffield Department of Surgical Science, University of Oxford, Oxford, UK
• 13 Leeds Institute of Health Sciences, University of Leeds, Leeds, UK
• 14 Women’s College Research Institute, University of Toronto, Toronto, Canada
• 15 Centre for Outcomes Research and Effectiveness, Department of Clinical, Educational and Health Psychology, University College London, London, UK
• Correspondence to: T C Hoffmann thoffmann{at}bond.edu.au
• Accepted 4 February 2014

Without a complete published description of interventions, clinicians and patients cannot reliably implement interventions that are shown to be useful, and other researchers cannot replicate or build on research findings. The quality of description of interventions in publications, however, is remarkably poor. To improve the completeness of reporting, and ultimately the replicability, of interventions, an international group of experts and stakeholders developed the Template for Intervention Description and Replication (TIDieR) checklist and guide. The process involved a literature review for relevant checklists and research, a Delphi survey of an international panel of experts to guide item selection, and a face to face panel meeting. The resultant 12 item TIDieR checklist (brief name, why, what (materials), what (procedure), who provided, how, where, when and how much, tailoring, modifications, how well (planned), how well (actual)) is an extension of the CONSORT 2010 statement (item 5) and the SPIRIT 2013 statement (item 11). While the emphasis of the checklist is on trials, the guidance is intended to apply across all evaluative study designs. This paper presents the TIDieR checklist and guide, with an explanation and elaboration for each item, and examples of good reporting. The TIDieR checklist and guide should improve the reporting of interventions and make it easier for authors to structure accounts of their interventions, reviewers and editors to assess the descriptions, and readers to use the information.

Introduction

The evaluation of interventions is a major research activity, yet the quality of descriptions of interventions in publications remains remarkably poor. Without a complete published description of the intervention, other researchers cannot replicate or build on research findings. For effective interventions, clinicians, patients, and other decision makers are left unclear about how to reliably implement the intervention. Intervention description involves more than providing a label or the ingredients list. Key features—including duration, dose or intensity, mode of delivery, essential processes, and monitoring—can all influence efficacy and replicability but are often missing or poorly described. For complex interventions, this detail is needed for each component of the intervention. For example, a recent analysis found that only 11% of 262 trials of cancer chemotherapy provided complete details of the trial treatments. 1 The most frequently missing elements were dose adjustment and “premedications,” but 16% of trials omitted even the route of drug administration. The completeness of intervention description is often worse for non-pharmacological interventions: one analysis of trials and reviews found that 67% of descriptions of drug interventions were adequate compared with only 29% of non-pharmacological interventions. 2 A recent study of 137 interventions, from 133 trials of non-drug interventions, found that only 39% of interventions were described adequately in the primary paper or any references, appendices, or websites. 3 This increased, albeit to only 59%, by contacting authors for additional information—a task almost no clinicians and few researchers have time to undertake.

The Consolidated Standards of Reporting Trials (CONSORT) 2010 statement 4 currently suggests in item 5 that authors should report on “The interventions for each group with sufficient details to allow replication, including how and when they were actually administered.” This is appropriate advice, but further guidance seems to be needed: despite endorsement of the CONSORT statement by many journals, reporting of interventions is deficient. The problem arises partly from lack of awareness among authors about what comprises a good description and partly from lack of attention by peer reviewers and editors. 5

A small number of CONSORT extension statements contain expanded guidance about describing interventions, such as non-pharmacological interventions, 6 and specific categories of interventions, such as acupuncture and herbal interventions. 7 8 The guidance for content of trial protocols, SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials), provides some recommendations for describing interventions in protocols. 9 More generic and comprehensive guidance is needed along with robust ways to implement such guidance. We developed an extension of item 5 of the CONSORT 2010 statement and item 11 of the SPIRIT 2013 statement in the form of a checklist and guidance entitled TIDieR (Template for Intervention Description and Replication), with the objective of improving the completeness of reporting, and ultimately the replicability, of interventions. This article describes the methods used to develop and obtain consensus for this checklist and, for each item, provides an explanation, elaboration, and examples of good reporting. While the emphasis of the checklist is on trials, the guidance is intended to apply across all evaluative study designs, such as trials, case-control studies, and cohort studies.

Methods for development of the TIDieR checklist and guide

Development of the checklist followed the methodological framework for developing reporting guidelines suggested by the EQUATOR Network. 10 In collaboration with the CONSORT steering group, we established a TIDieR steering committee (PPG, TCH, IB, RM, RP). The committee generated a list of 34 potential items from relevant CONSORT checklists and checklists for reporting discipline-specific or particular categories of interventions. The group also reviewed other sources of guidance on intervention reporting identified from a thorough search of the literature, followed by a forward and backward citation search (see appendix 1).

We then used a two round modified Delphi consensus survey method 11 involving a broad range of expertise and stakeholders. In the first round, each of the 34 items generated by the steering committee was rated by survey participants as “omit,” “possible,” “desirable,” or “essential” to include in the final checklist. From the first round, some items were reworded and combined, and then the ranked items were divided into three groups for the second round. The first group contained 13 items with the highest rankings (rated as “essential” by ≥70% participants or “essential or desirable” by ≥85%), and participants were advised that these would be included in the checklist unless strong objection to their inclusion was received in the second round. The second group contained 13 items with moderate rankings (“essential or desirable” by ≥65%); participants were asked to rate each of these again as “omit,” “possible,” “desirable,” or “essential.” The third group contained three items with low rankings, and participants were advised that these items would be removed unless strong objection to their omission was received in the second round. In both rounds, participants could also suggest additional items, comment on item wording, or provide general comments.

Delphi participants (n=125) were authors of research on describing interventions, clinicians, authors of existing reporting guidelines, clinical trialists, methodologists or statisticians with expertise in clinical trials, and journal editors (see appendix 2). They were invited by email to complete the two rounds of the web based survey. The response rate was 72% (n=90) for the first round. Only those who completed round one and were willing to participate in round two were invited to participate in round two. The response rate for round two was 86% (74 of 86 invited).

After the two Delphi rounds, 13 items were included in the draft checklist, and 13 moderately rated items were retained for further discussion at the in person meeting. The results of the Delphi survey were reported at a two day consensus meeting on 27-28 March 2013, in Oxford, UK. Thirteen invited experts, representing a range of health disciplines (see author list) and with expertise in the development of trial, methodological, and/or reporting guidelines, attended and are all authors of this paper. The meeting began with a review of the literature on intervention reporting, followed by a report of the Delphi process, the draft checklist of 13 items, and rankings of and comments about the additional 13 moderately rated items. Meeting participants discussed the proposed items and agreed which should be included and the wording of each item.

After the meeting, the checklist was distributed to the participants to ensure it reflected the decisions made, and this explanation and elaboration document was drafted. This was then piloted with 26 researchers who were authoring papers of intervention studies and minor clarifications were made in the elaboration of some items.

Scope of the TIDieR checklist and guide for describing interventions

The overarching purpose of the TIDieR checklist is to prompt authors to describe interventions in sufficient detail to allow their replication. The checklist contains the minimum recommended items for describing an intervention. Authors should provide additional information where they consider it necessary for the replication of an intervention.

Most TIDieR items are relevant for most interventions and applicable to even apparently simple drug interventions, which are sometimes poorly described. 2 If we consider the elements of an evaluation of an intervention—the population, intervention, comparison, outcome (“PICO”)—TIDieR can be seen as a guide for reporting the intervention and comparison (and co-interventions, when relevant) elements of a study. Other elements (such as population, outcomes) and methodological features are covered by CONSORT 2010 or SPIRIT 2013 items for randomised trials and by other checklists (such as the STROBE statement 12 ) for alternate study designs. They have not been duplicated as part of the TIDieR checklist.

The order in which items are presented in the checklist does not necessarily reflect the order in which information should be presented. It might also be possible to combine a number of items from the checklist into one sentence. For example, information about what materials (item 3) and what processes (item 4) can be combined (example 3c).

We emphasise that our definition of “intervention” extends to describing the intervention received by the comparison group/s in a study. Control interventions and co-interventions are often particularly poorly described; “usual care” is not a sufficient description. When a controlled study is reported, authors should describe what participants in the control group received with the same level of detail used to describe the intervention group, within the limits of feasibility. Full understanding of the comparison group care can help to explain the observed efficacy of an intervention, with greater apparent effect sizes being potentially found when control group care is minimal. 13 Describing the care that each group received will usually require the replication of the checklist for each group in a study.

As well as describing which interventions (or control conditions) were delivered to different groups, authors should also explain legitimate variants of the intervention. Authors might find it helpful to locate their trial on the pragmatic explanatory continuum. 14 If, for example in a pragmatic trial, authors expect there to be variants in aspects of the intervention (for instance, in the “usual care” group across various centres), those variants should be described under the appropriate checklist items.

We recognise that limitations (such as format and length) for journals that are only paper based can sometimes preclude inclusion of all intervention information in the primary paper (that is, the paper that is reporting the main results of the intervention evaluation). The information that is prompted by the TIDieR checklist might therefore be reported in locations beyond the primary paper itself, including online supplementary material linked to the primary paper, a published protocol and/or other published papers, or a website. Authors should specify the location of additional detail in the primary paper (for example, “online appendix 2 for the training manual,” “available at www ...,” or “details are in our published protocol”). When websites provide further details, URLs that are designed to remain stable over time are essential.

The TIDieR checklist explanation and elaboration

The items included in the checklist are shown in table 1 ⇓ . The complete checklist is available in appendix 3 and a Word version, which authors and reviewers can fill out, is available on the EQUATOR Network website ( www.equator-network.org/reporting-guidelines/tidier/ ). An explanation for each item is given below, along with examples of good reporting. Citations for the examples are in table 2. ⇓

Items included in the Template for Intervention Description and Replication (TIDieR) checklist: information to include when describing an intervention. Full version of checklist provides space for authors and reviewers to give location of the information (see appendix 3)

• View inline

List of references for the examples used

Item 1. Brief name: Provide the name or a phrase that describes the intervention

1a. Single . . . dose of dexamethasone

1b. TREAD (TREAtment of Depression with physical activity) study

1c. Internet based, nurse led vascular risk factor management programme promoting self management

Explanation —Precision in the name, or brief description, of an intervention enables easy identification of the type of intervention and facilitates linkage to other reports on the same intervention. Give the intervention name (examples 1a, 1b), explaining any abbreviations or acronyms in full (example 1b), or a short (one or two line) statement of the intervention without elaboration (example 1c).

Item 2. Why: Describe any rationale, theory, or goal of the elements essential to the intervention

2a. Dexamethasone (10 mg) or placebo was administered 15 to 20 minutes before or with the first dose of antibiotic. . . Studies in animals have shown that bacterial lysis, induced by treatment with antibiotics, leads to inflammation in the subarachnoid space, which may contribute to an unfavourable outcome [references]. These studies also show that adjuvant treatment with anti-inflammatory agents, such as dexamethasone, reduces both cerebrospinal fluid inflammation and neurologic sequelae [references]

2b. Self management of oral anticoagulant therapy may result in a more individualised approach, increased patient responsibility, and enhanced compliance, which may lead to improvement in the regulation of anticoagulation

2c. The TPB [Theory of Planned Behaviour] informed the hypothesised mediators of intention and physical activity that were targeted in the intervention program: instrumental and affective attitude, subjective norm and perceived behavioural control

2d. We chose a 5° wedge because greater wedging is less likely to be tolerated by the wearer [reference] and is difficult to accommodate within a normal shoe

Explanation —Inclusion of the rationale, theory, or goals that underpin an intervention, or the components of a complex intervention, 15 can help others to know which elements are essential, rather than optional or incidental. For example, the colour of capsules used in a pharmacological intervention is likely to be an incidental, not essential, contributor to the intervention’s efficacy and hence reporting of this is not necessary. In some reports, the term “active ingredient” is used and refers to the components within an intervention that can be specifically linked to its effect on outcomes such that, if they were omitted, the intervention would be ineffective. 16 The known or supposed mechanism of action of the active component/s of the intervention should be described.

Example 2a illustrates the rationale for treating bacterial meningitis with dexamethasone in addition to an antibiotic. Behaviour change and implementation interventions might require different forms of description, but the basic principles are the same. It might, alongside an account of the components of the intervention, also be appropriate to describe the intervention in terms of its theoretical basis, including its hypothesised mechanisms of action (examples 2b, 2c). 17 18 19 The rationale behind an important element of an intervention can sometimes be pragmatic and relate to acceptability of the intervention by participants (example 2d).

Item 3. What (materials): Describe any physical or informational materials used in the intervention, including those provided to participants or used in intervention delivery or in training of intervention providers. Provide information on where the materials can be accessed (for example, online appendix, URL)

3a. The educational package included a 12-minute cartoon . . . The presentation of the cartoon was complemented by classroom discussions, display of the same poster that was used for the control group [see figure in appendix 4], dissemination of a pamphlet summarising the key messages delivered in the cartoon, and drawing and essay writing competitions to reinforce the messages . . . The cartoon can be accessed at NEJM.org or at [URL provided]. A specific teacher training workshop was held before commencement of the trial (for details, see the protocol, available at NEJM.org)

3b. The intervention group received a behaviour change counselling training programme called the Talking Lifestyle learning programme that took practitioners through a portfolio-driven set of learning activities. Precise details of both intervention content and the training programme can be found in [URL, login and password provided]. . . Box 1 provides a more detailed description of the components of the training programme

3c. The “local” group received a sonographically guided injection of 2 mL (10 mg/mL) triamcinolone (Kenacort-T, Bristol-Myers Squibb) and 5 mL (10 mg/mL) lidocaine hydrochloride (Xylocaine, AstraZeneca) to the subacromial bursa and an intramuscular injection of 4 mL (10 mg/mL) lidocaine hydrochloride to the upper gluteal region

Explanation —A full description of an intervention should describe what different physical and information materials were used as part of the intervention (this typically will not extend to study consent forms unless they provide written instructions about the intervention that are not provided elsewhere). Intervention materials are the most commonly missing element of intervention descriptions. 3 This list of materials can be regarded as comparable with the “ingredients” required for a recipe. It can include materials provided to participants (example 3a), training materials used with the intervention providers (examples 3a, 3b), or the surgical device or pharmaceutical drug used and its manufacturer (example 3c). For some interventions, it might be possible to describe the materials and the procedures (item 4) together (examples 3c, 4c). If the information is too long or complex to describe in the primary paper, alternative options and formats for providing the materials should be used (see appendix 4 for some examples) and details of where they can be obtained (examples 3a, 3b) should be provided in the primary paper.

Item 4. What (procedures): Describe each of the procedures, activities, and/or processes used in the intervention, including any enabling or support activities

4a. The TREPP [transrectus sheath preperitoneal] technique can be performed under spinal anaesthesia. To reach the PPS [preperitoneal space], a 5 cm straight incision is made about 1 cm above the pubic bone. The anterior rectus sheath is opened, as is the underlying fascia transversalis [figure]. After retraction of the muscle fibres medially, the inferior epigastric vein and artery are identified and retracted medially as well

4b. . . . identified a suitable vein for cannulation. The overlying skin was wiped with an alcohol swab and allowed to dry, as per standard operating procedures. The principal investigator then administered the allocated spray from a distance of about 12 cm for two seconds. This technique avoided “frosting up” of vapocoolant on the skin. Liquid spray on the skin was allowed to evaporate for up to 10 seconds. The area was again wiped with an alcohol swab and cannulation proceeded immediately. Cannulation had to be carried out within 15 seconds of administration of the spray

4c. . . . three periods of exercise each lasting 5 min, supervised by a physiotherapist. The first period consisted of 2 min of indoor jogging, 1 min of stair climbing (three floors), and 2 min of cycling on an ergometer. Resistance on the ergometer was adjusted to ensure that the participant’s respiratory rate was elevated during the 2 min of cycling. At the end of the first period, the patient performed several prolonged and brief expiratory flow accelerations with open glottis, the forced expiratory technique, and finally cough and sputum expectoration. These clearance manoeuvres were performed over 1.5 min. The second period consisted of 1 min of stretching repeated five times, followed by the same expiratory manoeuvres for 1.5 min, as described above. The third period consisted of continuous jumping on a small trampoline. It included 2 min of jumping, 2 min of jumping while throwing and catching a ball, and 1 min of jumping while hitting a tossed ball. This was again followed by expiratory manoeuvres for 1.5 min. The entire regimen was followed by 40 min rest

4d. All health workers doing outpatient consultations in the intervention group received text messages about malaria case management for 6 months . . . The key messages addressed recommendations from the Kenyan national malaria guidelines and training manuals [references]

4e. Onsite activities were implemented by hospital personnel responsible for quality improvement initiatives . . . Standard communication channels were used, including group specific computer based training modules and daily electronic documentation by nursing staff for all groups. On-site training in bathing with chlorhexidine-impregnated cloths was provided to hospitals assigned to a decolonisation regimen . . . Nursing directors performed at least three quarterly observations of bathing, including questioning staff about protocol details. Investigators hosted group specific coaching teleconferences at least monthly to discuss implementation, compliance, and any new potentially conflicting initiatives

Explanation— Describe what processes, activities, or procedures the intervention provider/s carried out. Continuing the recipe metaphor used above, this item refers to the “methods” section of a recipe and where intervention materials (“ingredients”) are involved, describes what is to be done with them. “Procedure” can refer to the sequence of steps to be followed (examples 3c, 4b) and is a term used by some disciplines, particularly surgery, and includes, for example, preoperative assessment, optimisation, type of anaesthesia, and perioperative and postoperative care, along with details of the actual surgical procedure used (example 4a). Examples of processes or activities include referral, screening, case finding, assessment, education, treatment sessions (example 4c), telephone contacts (example 4d), etc. Some interventions, particularly complex ones, might require additional activities to enable or support the intervention to occur (in some disciplines these are known as implementation activities), and these should also be described (example 4e). Elaboration about how to report interventions where the procedure is not the same for all participants is provided at item 9 (tailoring).

Item 5. Who provided: For each category of intervention provider (for example, psychologist, nursing assistant), describe their expertise, background and any specific training given

5a. Only female counsellors were included in this rural area, after consultation with the village chiefs, because it would not have been deemed culturally appropriate for men to counsel women without their husband present . . . Selection criteria for lay counsellors included completion of 12 years of schooling, residence in the intervention area, and a history of community work

5b. The procedure is simple, uses existing surgical skills, and has a short learning curve, with the manufacturers recommending at least five mentored cases before independently practising. All surgeons involved in the study will have completed this training and will have carried out over five procedures prior to recruiting to the study

5c. Therapists received at least one day of training specific to the trial from an experienced CBT [cognitive behaviour therapy] therapist and trainer and weekly supervision from skilled CBT supervisors at each centre. . . The intervention was delivered by 11 part time therapists in the three sites who were representative of those working within NHS psychological services [reference]. Ten of the 11 therapists were female, their mean age was 39.2 years (SD 8.1), and they had practised as a therapist for a mean of 9.7 years (8.1) . . . Nine of the 11 therapists delivered 97% of the intervention and, for these nine, the number of patients per therapist ranged from 13 (6%) to 41 (18%)

5d. . . . brief lifestyle counselling was practised with trained actors and tape recorded. The competency of counselling was checked using the behaviour change counselling index [reference]. Only practitioners who reached a required standard (agreed by inter-rater consensus between three independent clinical assessors) were approved to deliver brief lifestyle counselling in the trial

Explanation —The term “intervention provider” refers to who was involved in providing the intervention (for example, by delivering it to recipients or undertaking specific tasks). This is important in circumstances where the providers’ expertise and other characteristics (example 5a) could affect the outcomes of the intervention. Important issues to address in the description might include the number of providers involved in delivering or undertaking the intervention; their disciplinary background (for example, nurse, occupational therapist, colorectal surgeon, expert patient); what pre-existing specific skills, expertise, and experience providers required and if and how these were verified; details of any additional training specific to the intervention that needed to be given to providers before (example 3b) and/or during the study (example 5c); and if competence in delivering the intervention was assessed before (example 5d) or monitored throughout the study and whether those deemed lacking in competence were excluded (example 5d) or retrained. Other information about providers could include whether the providers were doing the intervention as part of their normal role (example 3b) or were specially recruited as providers for purposes of the study (example 5c); whether providers were reimbursed for their time or provided with other incentives (if so, what) to deliver the intervention as part of the study, and whether such time or incentives might be needed to replicate the intervention.

Item 6. How: Describe the modes of delivery (such as face to face or by some other mechanism, such as internet or telephone) of the intervention and whether it was provided individually or in a group

6a. . . . sessions . . . held weekly and facilitated in groups of 6-12 by . . .

6b. Drugs were delivered by . . . members of the [Reproductive and Child Health] trekking teams . . . teams visited each of the study villages . . .

6c. The text messaging intervention, SMS Turkey, provided six weeks of daily messages aimed at giving participants skills to help them quit smoking. Messages were sent in an automated fashion, except two days and seven days after the initial quit day

6d. . . . made their own appointments online . . . Participants and therapists typed free text into the computer, with messages sent instantaneously; no other media or means of communication were used

6e. . . . three 1 hour home visits (televisits) by a trained assistant . . . ; participants’ daily use of an in-home messaging device . . .… that was monitored weekly by the teletherapist; and five telephone intervention calls between the teletherapist and the participant . . .

Explanation —Specify whether the intervention was provided to one participant at a time (such as a surgical intervention) or to a group of participants and, if so, the group size (example 6a). Also describe whether it was delivered face to face (example 6b), by distance (such as by telephone, surface mail, email, internet, DVD, mass media campaign, etc) as in examples 6c, 6d, or a combination of modes (example 6e). When relevant, describe who initiated the contact (example 6c), and whether the session was interactive (example 6d) or not (example 6c), and any other delivery features considered essential or likely to influence outcome.

Item 7. Where: Describe the type(s) of location(s) where the intervention occurred, including any necessary infrastructure or relevant features

7a. . . . medication . . . and a spacer (as appropriate) were delivered to the school nurse for directly observed therapy on the days on which the child attended school. . . An additional canister of preventive medication was delivered to the child’s home to use on weekends and other days the child did not attend school, and the child’s caregiver was shown proper administration technique

7b. Women were recruited from three rural and one peri-urban antenatal clinic in Southern Malawi . . . tablets were taken under supervision at the clinic

7c. . . . participants for the . . telehealth trial, across three sociodemographically distinct regions in England (rural Cornwall, rural and urban Kent, and urban Newham in London) comprising four primary care trusts. . . Control participants had no telehealth or telecare equipment installed their homes for the duration of the study. A Lifeline pendant (a personal alarm) plus a smoke alarm linked to a monitoring centre were not, on their own, sufficient to classify as telecare for current purposes

7d. Most births in African countries occur at home, especially in rural areas . . . They identified pregnant women and made five home visits during and after pregnancy . . . Peer counsellors lived in the same communities, so informal contacts to make arrangements for visits were common. . . counsellors were . . . given a bicycle, T shirt. . .

7e. This paper contains a box, titled “Key features of healthcare systems in Northern Ireland and Republic of Ireland,” which summarises relevant aspects of general practices such as funding, registration, and access to free prescriptions

Explanation —In some studies the intervention can be delivered in the same location where participants were recruited and/or data were collected and details might therefore already be included in the primary paper (for example, as in item 4b of CONSORT 2010 statement if reporting a trial). If, however, the intervention occurred in different locations, this should be specified. At its simplest level, the location might be, for example, in the participants’ home (example 7a), residential aged care facility, school (example 7a), outpatient clinic (example 7b), inpatient hospital room, or a combination of locations (example 7a). Features or circumstances about the location can be relevant to the delivery of the intervention and should be described (examples 7e). For example, they might include the country (example 7b), type of hospital or primary care (example 7c), publicly or privately funded care, volume of activity, details of the healthcare system, or the availability of certain facilities or equipment (examples 7c, 7d, 7e). These features can impact on various aspects of the intervention such as its feasibility (example 7d) or provider or participant adherence and are important for those considering replicating the intervention.

Item 8. When and how much: Describe the number of times the intervention was delivered and over what period of time including the number of sessions, their schedule, and their duration, intensity or dose

8a. . . . a loading dose of 1 g of tranexamic acid infused over 10 min, followed by an intravenous infusion of 1 g over 8 h

8b. They received five text messages a day for the first five weeks and then three a week for the next 26 weeks

8c. . . . . exercise three times a week for 24 weeks. . . Participants began with 15 minutes of exercise and increased to 40 minutes by week eight . . . Between weeks eight and 24, attempts to increase exercise intensity were made at least weekly either by increasing treadmill speed or by increasing the treadmill grade. Participants with leg symptoms were encouraged to exercise to near maximal leg symptoms. Asymptomatic participants were encouraged to exercise to a level of 12 to 14 . . . . on the Borg rating of perceived exertion scale [reference]

8d. . . . delivered weekly one hour sessions in the woman’s home, for up to eight weeks . . . starting at around eight weeks postnatally

Explanation —The type of information needed about the “when and how much” of the intervention will differ according to the type of intervention. For some interventions some aspects will be more important than others. For example, for pharmacological interventions, the dose and scheduling is often important (example 8a); for many non-pharmacological interventions, the “how much” of the intervention is instead described by the duration and number of sessions (examples 8b, 8c). For multiple session interventions, the schedule of the sessions is also needed (example 8b) and if the number of sessions, their schedule, and/or intensity was fixed (examples 8b, 4c, 6a) or if it could be varied according to rules and if so, what they were (example 8c). Tailoring of the intervention to individuals or groups of individuals is elaborated on in item 9 (tailoring). For some interventions, as part of the “when” information, detail about the timing of the intervention in relation to relevant events might also be important (for example, how long after diagnosis, first symptoms, or a crucial event did the intervention start) (example 8d). As described below in item 12, the “amount” or dose of intervention that participants actually received might differ from the amount intended. This detail should be described, usually in the results section (examples 12a-c).

Item 9. Tailoring: If the intervention was planned to be personalised, titrated or adapted, then describe what, why, when, and how

9a. Those allocated to the intervention arm followed an intensive stepped programme of management, with mandatory visits to their doctor at weeks 6, 10, 14, and 18 after randomisation to review their blood pressure and to adjust their treatment if needed according to prespecified algorithms [provided in supplementary appendix]

9b. All patients received laparoscopic mini-gastric bypass surgery. . . The bypass limb was adjusted according to the preoperative BMI of the patient. A 150 cm limb was used for BMI 35, with a 10 cm increase in the bypass limb with every BMI category increase, instead of using a fixed limb for all patients

9c. Participants began exercising at 50% of their 1 rm [repetition maximum]. Weights were increased over the first five weeks until participants were lifting 80% of their 1 rm. Weights were adjusted after each monthly 1 rm and as needed to achieve an exercise intensity of a rating of perceived exertion of 12 to 14

9d. Stepped-care decisions for patients . . . were guided by responses to the nine item patient health questionnaire [reference], administered at each treatment visit and formally evaluated at eight week intervals. Patients who did not show prespecified improvement were offered the choice of switching treatments (for example, from problem solving therapy to medication), adding the other treatment, or intensifying the original treatment choice, based on the treatment team’s recommendation (for details, see [reference])

Explanation —In tailored interventions, not all participants receive an identical intervention. Interventions can be tailored for several reasons, such as titration to obtain an appropriate “dose” (example 9a); participant’s preference, skills, or situation (example 9b); or it may be an intrinsic element of the intervention as with increasing intensity of an exercise (example 9c). Hence, a brief rationale and guide for tailoring should be provided, including any variables/constructs used for participant assessment (examples 9b, 9c) and subsequent tailoring. Tailoring can occur at several stages and authors should describe any decision points and rules used at each point (example 9d). If any decisional or instructional materials are used, such as flowcharts, algorithms or dosing nomograms, these should be included, referenced (example 9d), or their location provided (example 9a).

Item 10. Modifications: If the intervention was modified during the course of the study, describe the changes (what, why, when, and how)

10a. A mixture of general practitioners and practice care nurses delivered 95% of screening and brief intervention activity in this trial. . . Owing to this slow recruitment, research staff who had delivered training in study procedures supported screening and brief intervention delivery in 10 practices and recruited 152 patients, which was 5% of the total number of trial participants

10b. Computers with slow processing units and poor internet connections meant that seven general practitioners never got functional software; they used a structured paper version that was faxed between the research team and general practitioner after each appointment

Explanation — This item refers to modifications that occur at the study level, not individual tailoring as described in item 9. Unforeseen modifications to the intervention can occur during the course of the study, particularly in early studies. If this happens, it is important to explain what was modified, why and when modifications occurred, and how the modified intervention differed from the original (example 10a—modification to who provided the intervention; example 10b— modification in the materials). Modifications sometimes reflect changing circumstances. In other studies, they can show learning about the intervention, which is important to transmit to the reader and others to prevent unnecessary repetition of errors during attempts to replicate the intervention. If changes to the intervention occurred between the published protocol or published pilot study and the primary paper, these changes should also be described.

Item 11. How well (planned): If intervention adherence or fidelity was assessed, describe how and by whom, and if any strategies were used to maintain or improve fidelity, describe them

11a. Pathologists were trained to identify lateral spread of tumour according to the protocol [reference]. The results of histopathological examination of the specimens were reviewed by a panel of supervising pathologists and a quality manager

11b. Staff in the study sites were trained initially, and therapy supervision was provided by weekly meetings between therapists and investigators. Cognitive therapy sessions were taped with the participant’s consent so that participants could be asked to listen to the tapes as part of their homework and to assist supervision. During the course of the trial a sample of 80 tapes was rated according to the cognitive therapy scale-revised [reference] and the cognitive therapy for at risk populations adherence scale [reference] to ensure rigorous adherence to the protocol throughout the duration of the trial. These tapes were drawn from both early and late phases of therapy and included participants from each year of recruitment

11c. Adherence to trial medication was assessed by means of self reported pill counts collected during follow-up telephone calls. These data were categorised as no pills taken, hardly any taken (1-24% of prescribed doses), some taken (25-49%), most taken (50-74%), or all taken (75-100%)

11d. Training will be delivered independently in each of the three regional study centres. All trainers will adhere to a single training protocol to ensure standardised delivery of the training across centres. Training delivery will be planned and rehearsed jointly by all trainers using role play and peer review techniques. In addition, the project manager will act as an observer during the first two training sessions in each centre and will provide feedback to trainers with a view to further standardising the training [note, this example is from a protocol]

Explanation —Fidelity refers to the degree to which an intervention happened in the way the investigators intended it to 20 and can affect the success of an intervention. 21 The terms used to describe this concept vary among disciplines and include treatment integrity, provider or participant adherence, and implementation fidelity. This item—and item 12—extends beyond simple receipt of the intervention (such as how many participants were issued with the intervention drug or exercises) and refers to “how well” the intervention was received or delivered (such as how many participants took the drug/did the exercises, how much they took/did, and for how long). Depending on the intervention, fidelity can apply to one or more parts of the intervention, such as training of providers (examples 11a, 11b, 11d), delivery of the intervention (example 11b), and receipt of the intervention (example 11c). The types of measures used to determine intervention fidelity will also vary according to the type of intervention. For example, in simple pharmacological interventions, assessing fidelity often focuses on recipients’ adherence to taking the drug (example 11b). In complex interventions, such as rehabilitation, psychological, or behaviour change interventions, however, assessment of fidelity is also more complex (example 11b). There are various preplanned strategies and tools that can be used to maintain fidelity before delivery of the intervention (example 11d) or during the study (example 11b). If any strategies or tools were used to maintain fidelity, they should be clearly described. Any materials used as part of assessing or maintaining fidelity should be included, referenced, or their location provided.

Item 12: How well (actual): If intervention adherence or fidelity was assessed, describe the extent to which the intervention was delivered as planned

12a. The mean (SD) number of physiotherapy sessions attended was 7.5 (1.9). Seven patients (9%) completed less than four physiotherapy sessions; the reasons included non-attendance, moving interstate, or recovery from pain. Of patients in the physiotherapy groups, 70% were compliant with their home exercise program during at least five of seven weeks

12b. The EE [early exercise] group reported an adherence rate of 73% at [time] T2 and 75.7% at [time] T3, and the CE [delayed exercise] group reported 86.7% adherence at T3 . . . with the early exercise EE group reporting disease and treatment related barriers to exercise during their cancer treatment (“week of chemotherapy” 14%; “fatigue” 10%); or life related barriers (“illness eg, colds or flu” 16%; “family obligations” 13%)”

12c. A total of 214 participants (78%) reported taking at least 75% of the study tablets; the proportion of patients who reported taking at least 75% of the tablets was similar in the two groups

12d. The integrity of the psychological therapy was assessed with the cognitive therapy rating scale [reference] to score transcripts of 40 online sessions for patients who had completed at least five sessions of therapy. With use of computer generated random numbers, at least one such patient was selected for each therapist. For these patients, either session six or the penultimate session was rated by two independent CBT [cognitive behaviour therapy]-trained psychologists, who gave mean ratings of 31 (SD between therapists 9) and 32 (13) of 72

Explanation — For various reasons, an intervention, or parts of it, might not be delivered as intended, thus affecting the fidelity of the intervention. If this is assessed, authors should describe the extent to which the delivered intervention varied from the intended intervention. This information can help to explain study findings, minimise errors in interpreting study outcomes, inform future modifications to the intervention, and, when fidelity is poor, can point to the need for further studies or strategies to improve fidelity or adherence. 22 23 For example, there might be some aspects of the intervention that participants do not like and this could influence their adherence. The way in which the intervention fidelity is reported will reflect the measures used to assess it (examples 12a-d), as described in item 11.

Who should use TIDieR?

We describe a short list of items that we believe can be used to improve the reporting of interventions and make it easier for authors to structure accounts of their interventions, reviewers and editors to assess the descriptions, and readers to use the information. Consistent with the CONSORT 2010 and SPIRIT 2013 statements, we recommend that interventions are described in enough detail to enable replication, and recommend that authors use the TIDieR checklist to achieve this. As inclusion of all intervention details is not always possible in the primary paper of a study, the TIDieR checklist encourages authors to indicate that they have reported each of the items and to state where this information is located (see appendix 3).

The number of checklist items reported is improved when journals require checklist completion as part of the submission process. 24 We encourage journals to endorse the use of the TIDieR checklist, in a similar way to CONSORT and related statements. This can be done by modifying their author instructions, publishing an editorial about intervention reporting, and including a link to the checklist on their website. Few journals currently provide specific guidance about how to report interventions. 25 A small number have editorial policies stating that they will not publish trials unless intervention protocols or full details are available. 26 We encourage other journals to consider adopting similar policies. Any links provided by journals and authors should be reliable and enduring. Stable depositories for descriptions of interventions are also required, and their development needs the contribution and collaboration of all stakeholders in the research community (such as researchers, journal editors, publishers, research funding bodies).

Authors might also want to be guided by the TIDieR items when describing interventions in systematic reviews so that readers of reviews have access to full details of any intervention (or at least details about where to obtain further information) that they want to replicate after reading the review.

Using TIDieR in conjunction with the CONSORT and SPIRIT Statements

For authors submitting reports of randomised trials, we suggest using the TIDieR in conjunction with the CONSORT checklist: when authors complete item 5 of the CONSORT checklist, they should insert “refer to TIDieR checklist” and provide a separate completed TIDieR checklist. For journals that adopt this recommendation, their instructions to authors will need to be modified accordingly and their editors and reviewers made aware of the change. Similarly, for authors submitting protocols of trials, the TIDieR checklist can be referred to when dealing with item 11 of the SPIRIT 2013 checklist. One point of difference is that two TIDieR items (items 10 and 12) are not applicable to intervention reporting in protocols because they cannot be completed until the study is complete. This is noted on the TIDieR checklist. Published protocols are likely to grow in importance as a source of information about the intervention and use of TIDieR in conjunction with the SPIRIT 2013 statement can facilitate this. For authors of study designs other than randomised trials, TIDieR can be used alone as a standalone checklist or in conjunction with the relevant statement for that study design (such as the STROBE statement 12 ). We acknowledge that describing complex interventions well can be challenging and that for some particularly complex interventions, a checklist, such as TIDieR, could go some way towards assisting with intervention reporting but might not be able to capture the full complexity of these interventions.

We recognise that adhering to the TIDieR checklist might increase the word count of a paper, particular if the study protocol is not publicly available. We believe this might be necessary to help improve the reporting of studies generally and interventions specifically. As journals recognise the importance of well reported studies and fully described methods, and many move to a model of online only, or a hybrid of printed and online with posting of the full study protocol, this might become less of a barrier to quality reporting. For example, the Nature Publishing Group recently removed word limits on the methods section of submitted papers and advises that: “If more space is required to describe the methods completely, the author should include the 300-word section ‘Methods Summary’ and provide an additional ‘Methods’ section at the end of the text, following the figure legends. This Methods section will appear in the online . . . version of the paper, but will not appear in the printed issue. The Methods section should be written as concisely as possible but should contain all elements necessary to allow interpretation and replication of the results.” 27

The TIDieR checklist and guide should assist authors, editors, peer reviewers, and readers. Some authors might perceive this checklist as another time consuming hurdle and elect to seek publication in a journal that does not endorse reporting guidelines. There is a large evidence base indicating that the quality of reporting of health research is unacceptably poor. Properly endorsed and implemented reporting guidelines offer a way for publishers, editors, peer reviewers, and authors to do a better job of completely and transparently describing what was done and found. 28 Doing so will help reduce wasteful research 29 30 and increase the potential impact of research on health.

Summary points

Without a complete published description of interventions, clinicians and patients cannot reliably implement effective interventions

The quality of description of interventions in publications, regardless of type of intervention, is remarkably poor

The Template for Intervention Description and Replication (TIDieR) checklist and guide has been developed to improve the completeness of reporting, and ultimately the replicability, of interventions

TIDieR can be used by authors to structure reports of their interventions, by reviewers and editors to assess completeness of descriptions, and by readers who want to use the information

Cite this as: BMJ 2014;348:g1687

We are grateful to everyone who responded to the Delphi survey and for their thoughtful comments. We also thank Nicola Pidduck (Department of Primary Care Health Sciences, Oxford University) for her assistance in organising the consensus meeting in Oxford.

Contributors: PPG and TCH initiated the TIDieR group and led the organising of the Delphi survey and consensus meeting, in conjunction with the other members of the steering group (IB, RM, and RP). TCH led the writing of the paper. All authors contributed to the drafting and revision of the paper and approved the final version. TCH and PPG are guarantors.

Funding: There was no explicit funding for the development of this checklist and guide. The consensus meeting in March 2013 was partially funded by a NIHR Senior Investigator Award held by PPG. TCH is supported by a National Health and Medical Research Council of Australia (NHMRC)/Primary Health Care Research Evaluation and Development Career Development Fellowship (1033038) with funding provided by the Australian Department of Health and Ageing. PPG is supported by a NHMRC Australia Fellowship (527500). DGA is supported by a programme grant from Cancer Research UK (C5529). MDW is supported by a Wellcome Trust Senior Investigator award (WT097899MA).

Competing interests: All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: RM is employed by NETSCC, part of the National Institute for Health Research (NIHR) in England. NETSCC manages on behalf of NIHR the NIHR Journals Library, “a suite of five open access journals providing an important and permanent archive of research funded by the National Institute for Health Research.” The NIHR Journals Library places great value on reporting the full results of funded research and so is likely to be a user of TIDieR, as it is of other reporting guidelines. VB was the Chief Editor of PLOS Medicine at the time of the consensus meeting and initial drafting of this paper. HM is an assistant editor at BMJ but was not involved in any decision making regarding this paper.

Provenance and peer review: Not commissioned; externally peer reviewed.

• ↵ Duff J, Leather H, Walden E, LaPlant K, George T. Adequacy of published oncology randomised controlled trials to provide therapeutic details needed for clinical application. J Natl Cancer Inst 2010 ; 102 : 702 -5. OpenUrl Abstract / FREE Full Text
• ↵ Glasziou P, Meats E, Heneghan C, Shepperd S. What is missing from descriptions of treatment in trials and reviews? BMJ 2008 ; 336 : 1472 -4. OpenUrl FREE Full Text
• ↵ Hoffmann T, Erueti C, Glasziou P. Poor description of non-pharmacological interventions: analysis of consecutive sample of randomised trials. BMJ 2013 ; 347 : f3755 . OpenUrl Abstract / FREE Full Text
• ↵ Schulz K, Altman D, Moher D, CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010 ; 340 : c332 . OpenUrl FREE Full Text
• ↵ Schroter S, Glasziou P, Heneghan C. Quality of descriptions of treatments: a review of published randomised controlled trials. BMJ Open 2012 ; 2 : e001978 . OpenUrl Abstract / FREE Full Text
• ↵ Boutron I, Moher D, Altman D, Schulz K, Ravaud P. Extending the CONSORT statement to randomised trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med 2008 ; 148 : 295 -310. OpenUrl CrossRef PubMed Web of Science
• ↵ MacPherson H, Altman DG, Hammerschlag R, Youping L, Taixiang W, White A, et al. Revised standards for reporting interventions in clinical trials of acupuncture (STRICTA): extending the CONSORT statement. PLoS Med 2010 ; 7 : e1000261 . OpenUrl CrossRef PubMed
• ↵ Gagnier J, Boon H, Rochon P, Moher D, Barnes J, Bombardier C, et al. Reporting randomised, controlled trials of herbal interventions: an elaborated CONSORT statement. Ann Intern Med 2006 ; 144 : 364 -7. OpenUrl CrossRef PubMed Web of Science
• ↵ Chan A, Tetzlaff J, Gøtzsche P, Altman D, Mann H, Berlin J, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013 ; 346 : e7586 . OpenUrl Abstract / FREE Full Text
• ↵ Moher D, Schulz K, Simera I, Altman D. Guidance for developers of health research reporting guidelines. PLoS Med 2010 ; 7 : e1000217 . OpenUrl CrossRef PubMed
• ↵ Murphy M, Black N, Lamping D, McKee C, Sanderson C, Askham J, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess 1998 ; 2 : 1 -88. OpenUrl PubMed
• ↵ Von Elm E, Altman D, Egger M, Pocock S, Gøtzsche P, Vandenbroucke J, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007 ; 335 : 806 -8. OpenUrl FREE Full Text
• ↵ De Bruin M, Viechtbauer W, Hospers H, Schaalma H, Kok G. Standard care quality determines treatment outcomes in control groups of HAART-adherence intervention studies: implications for the interpretation and comparison of intervention effects. Health Psychol 2009 ; 28 : 668 -74. OpenUrl CrossRef PubMed Web of Science
• ↵ Thorpe K, Zwarenstein M, Oxman AD, Treweek S, Furberg C, Altman D, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 2009 ; 62 : 464 -75. OpenUrl CrossRef PubMed Web of Science
• ↵ Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: the new Medical Research Council guidance. BMJ 2008 ; 337 : a1655 . OpenUrl FREE Full Text
• ↵ McCleary N, Duncan E, Stewart F, Francis J. Active ingredients are reported more often for pharmacologic than non-pharmacologic interventions: an illustrative review of reporting practices in titles and abstracts. Trials 2013 ; 14 : 146 . OpenUrl CrossRef PubMed
• ↵ Michie S, West R. Behaviour change theory and evidence: a presentation to Government. Health Psychol Rev 2013 ; 7 : 1 -22. OpenUrl CrossRef Web of Science
• ↵ Dixon-Woods M, Leslie M, Tarrant C, Bion J. Explaining Matching Michigan: an ethnographic study of a patient safety program. Implement Sci 2013 ; 8 : 70 . OpenUrl CrossRef PubMed
• ↵ Dixon-Woods M, Bosk C, Aveling E, Goeschel C, Pronovost P. Explaining Michigan: developing an ex post theory of a quality improvement program. Milbank Q 2011 ; 89 : 167 -205. OpenUrl CrossRef PubMed Web of Science
• ↵ Carroll C, Patterson M, Wood S, Booth A, Rick J, Balain S. A conceptual framework for implementation fidelity. Implement Sci 2007 ; 2 : 40 . OpenUrl CrossRef PubMed
• ↵ Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci D, Ory M, et al. Enhancing treatment fidelity in health behaviour change studies: best practices and recommendations from the NIH Behaviour Change Consortium. Health Psychol 2004 ; 23 : 443 -51. OpenUrl CrossRef PubMed Web of Science
• ↵ Hardeman W, Michie S, Fanshawe T, Prevost T, McLoughlin K, Kinmonth AL. Fidelity of delivery of a physical activity intervention: predictors and consequences. Psychol Health 2008 ; 23 : 11 -24. OpenUrl CrossRef Web of Science
• ↵ Spillane V, Byrne M, Byrne M, Leathem C, O’Malley M, Cupples M. Monitoring treatment fidelity in a randomised controlled trial of a complex intervention. J Adv Nursing 2007 ; 60 : 343 -52. OpenUrl CrossRef PubMed Web of Science
• ↵ Hopewell S, Ravaud P, Baron G, Boutron I. Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012 ; 344 : e4178 . OpenUrl Abstract / FREE Full Text
• ↵ Hoffmann T, English T, Glasziou P. Reporting of interventions in randomised trials: an audit of journal Instructions to Authors. Trials 2014 ; 15 : 20 . OpenUrl CrossRef PubMed
• ↵ Michie S, Fixsen D, Grimshaw JM, Eccles MP. Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci 2009 ; 4 : 40 . OpenUrl CrossRef PubMed
• ↵ Nature. For authors: manuscript formatting guide [Internet]. www.nature.com/nature/authors/gta/index.html#a5.3 .
• ↵ Turner L, Shamseer L, Altman DG, Schulz KF, Moher D. Does use of the CONSORT Statement impact the completeness of reporting of randomised controlled trials published in medical journals? A Cochrane review. Syst Rev 2012 ; 1 : 60 . OpenUrl CrossRef PubMed
• ↵ Chalmers I, Glasziou P. Avoidable waste in the production and reporting of research evidence. Lancet 2009 ; 374 : 86 -9. OpenUrl CrossRef PubMed Web of Science
• ↵ Glasziou P, Altman D, Bossuyt P, Boutron I, Clarke M, Julious S, et al. Reducing waste from incomplete or unusable reports of biomedical research. Lancet 2014 ; 383 : 267 -76. OpenUrl CrossRef PubMed Web of Science

Want to create or adapt books like this? Learn more about how Pressbooks supports open publishing practices.

Module 3 Chapter 1: Overview of Intervention/Evaluation Research Approaches

In our prior course, you learned how the nature of an investigator’s research question dictates the type of study approach and design that might be applied to achieve the study aims. Intervention research typically asks questions related to the outcomes of an intervention effort or approach. However, questions also arise concerning implementation of interventions, separate from understanding their outcomes. Practical, philosophical, and scientific factors contribute to investigators’ intervention study approach and design decisions.

In this chapter you learn:

• how content from our earlier course about study approaches and designs relate to intervention research;
• additional approaches to intervention research (participatory research; formative, process, outcome, and cost-related evaluation research)
• intervention research strategies for addressing intervention fidelity and internal validity concerns.

Review and Expansion: Study Approaches

In our earlier course you became familiar with the ways that research questions lead to research approach and methods. Intervention and evaluation research are not different: the question dictates the approach. In the earlier course, you also became familiar with the philosophical, conceptual and practical aspects of different approaches to social work research: qualitative, quantitative, and mixed methods. These methods are used in research for evaluating practice and understanding interventions, as well. The primary emphasis in this module revolves around quantitative research designs for practice evaluation and understanding interventions. However, taking a few moments to examine qualitative and mixed methods in these applications is worthwhile. Additionally, we introduce forms of participatory research—something we did not discuss regarding efforts to understand social work problems and diverse populations. Participatory research is an approach rich in social work tradition.

Qualitative methods in intervention & evaluation research.

The research questions asked by social workers about interventions often lend themselves to qualitative study approaches. Here are 5 examples.

• Early in the process of developing an intervention, social workers might simply wish to create a rich description of the intervention, the contexts in which it is being delivered, or the clients’ experience with the intervention. This type of information is going to be critically important in developing a standardized protocol which others can use in delivering the intervention, too. Remember that qualitative methods are ideally suited for answering exploratory and descriptive questions.
• Qualitative methods are well-suited to exploring different experiences related to diversity—the results retain individuality arising from heterogeneity rather than homogenizing across individuals to achieve a “normative” picture.
• Qualitative methods are often used to assess the degree to which the delivery of an intervention adheres to the procedures and protocol originally designed and empirically tested. This is known as an intervention fidelity issue (see the section below on the topic of process evaluation).
• Intervention outcomes are sometimes evaluated using qualitative approaches. For example, investigators wanted to learn from adult day service participants what they viewed as the impact of the program on their own lives (Dabelko-Schoeny & King, 2010). The value of such information is not limited to evaluating this one program. Evaluators are informed about important evaluation variables to consider in their own efforts to study interventions delivered to older adults—variables beyond the typical administrative criteria of concern. The study participants identified social connections, empowering relationships with staff, and enjoyment of activities as important evaluation criteria.
• Assessing the need for intervention (needs assessment) is often performed with qualitative approaches, especially focus groups, open-ended surveys, and GIS mapping.
• Qualitative approaches are an integral aspect of mixed-methods approaches.

Qualitative approaches often involve in-depth data from relatively few individuals, seeking to understand their individual experiences with an intervention. As such, these study approaches are relatively sensitive to nuanced individual differences—differences in experience that might be attributed to cultural, clinical, or other demographic diversity. This is true, however, only to the extent that diversity is represented among study participants, and individuals cannot be presumed to represent groups or populations.

Quantitative methods in intervention & evaluation research.

Many intervention and evaluation research questions are quantitative in nature, leading investigators to adopt quantitative approaches or to integrate quantitative approaches in mixed methods research. In these instances, “how much” or “how many” questions are being asked, questions such as:

• how much change was associated with intervention;
• how many individuals experienced change/achieved change goals;
• how much change was achieved in relation to the resources applied;
• what trends in numbers were observed.

Many study designs detailed in Chapter 2 reflect the philosophical roots of quantitative research, particularly those designed to zero in on causal inferences about intervention—the explanatory research designs. Quantitative approaches are also used in descriptive and exploratory intervention and evaluation studies. By nature, quantitative studies tend to aggregate data provided by individuals, and in this way are very different from qualitative studies. Quantitative studies seek to describe what happens “on average” rather than describing individual experiences with the intervention—you learned about central tendency and variation in our earlier course (Module 4). Differences in experience related to demographic, cultural, or clinical diversity might be quantitatively assessed by comparing how the intervention was experienced by different groups (e.g., those who differ on certain demographic or clinical variables). However, data for the groups are treated in the aggregate (across individuals) with quantitative approaches.

Mixed methods in intervention & evaluation research.

Qualitative and quantitative approaches are very helpful in evaluation and intervention research as part of a mixed-methods strategy for investigating the research questions. In addition to the examples previously discussed, integrating qualitative and quantitative approaches in intervention and evaluation research is often done as means of enriching the results derived from one or the other approach. Here are 3 scenarios to consider.

• Investigators wish to use a two-phase approach in studying or evaluating an intervention. First, they adopt a qualitative approach to inform the design of a quantitative study, then they implement the quantitative study as a second phase. The qualitative phase might help inform any aspect of the quantitative study design, including participant recruitment and retention, measurement and data collection, and presenting study results.
• Investigators use a two-phase approach in studying or evaluating an intervention. First, they implement a quantitative study. Then, they use a qualitative approach to explore the appropriateness and adequacy of how they interpret their quantitative study results.
• Investigators combine qualitative and quantitative approaches in a single intervention or evaluation study, allowing them to answer different kinds of questions about the intervention.

For example, a team of investigators applied a mixed methods approach in evaluating outcomes of an intensive experiential learning experience designed to prepare BSW and MSW students to engage effectively in clinical supervision (Fisher, Simmons, & Allen, 2016). BSW students provided quantitative data in response to an online survey, and MSW students provided qualitative self-assessment data. The quantitative data answered a research question about how students felt about supervision, whereas the qualitative data were analyzed for demonstrated development in critical thinking about clinical issues. The investigators concluded that their experiential learning intervention contributed to the outcomes of forming stronger supervisory alliance, BSW student satisfaction with their supervisor, and MSW students thinking about supervision as being more than an administrative task.

Cross-Sectional & Longitudinal Study Designs.

You are familiar with the distinction between cross-sectional and longitudinal study designs from our earlier course. In that course, we looked at these designs in terms of understanding diverse populations, social work problems, and social phenomena. Here we address how the distinction relates to the conduct of research to understand social work interventions.

• A cross-sectional study involves data collection at just one point in time. In a program evaluation, for example, the agency might look at some outcome variable at the point when participants complete an intervention or program. Or, perhaps an agency surveys all clients at a single point in time to assess their level of need for a potential new service the agency might offer. Because the data are collected from each person at only one point in time, these are both cross-sectional studies. In terms of intervention studies, one measurement point obviously needs to be after the intervention for investigators to draw inferences about the intervention. As you will see in the discussion of intervention study designs, there exist considerable limitations to using only one single measurement to evaluate an intervention (see post-only designs in Chapter 2).
• A longitudinal study involves data collection at two or more points in time. A great deal of intervention and evaluation research is conducted using longitudinal designs—answering questions about what changes might be associated with the intervention being delivered. For example, in program evaluation, an agency might compare how clients were functioning on certain variables at the time of discharge compared to their level of functioning at intake to the program. Because the same information is collected from each individual at two points in time (pre-intervention and post-intervention), this is a longitudinal design.
• Distinguishing cross-section and longitudinal in studies of systems beyond the individual person can become confusing. When social workers intervene with individuals or families or small groups, that longitudinal study involves the same individuals or members at different points in time is evident—perhaps measuring individuals before, immediately after, and months after intervention (this is called follow-up ). However, if an intervention is conducted in a community, a state, or across the nation, the data might not be collected from the same individual persons at each point in time—the unit of analysis is what matters here. For example, if the longitudinal study’s unit of analysis is the 50 states, District of Columbia, and 5 inhabited territories of the United States, data are repeatedly collected at that level (states, DC, and territories), perhaps not from the same individual persons in each of those communities.

Formative, Process, and Outcome Evaluation

Practice and program evaluation are important aspects of social work practice. It would be nice if we could simply rely on our own sense of what works and what does not. However, social workers are only human and, as we learned in our earlier course, human memory and decisions are vulnerable to bias. Sources of bias include recency, confirmation, and social desirability biases.

• Recency bias occurs when we place higher emphasis on what has just happened (recently) than on what might have happened in the more distant past. In other words, a social worker might make a casual practice evaluation based on one or two exceptionally good or exceptionally bad recent outcomes rather than a longer, larger history of outcomes and systematic evidence.
• Confirmation bias occurs when we focus on outcomes that reinforce what we believed, feared, or hoped would happen and de-emphasize alternative events or interpretations that might contradict those beliefs, fears, or hopes.
• Social desirability bias by practitioners occurs when practice decisions are influenced by a desire to be viewed favorably by others—that could be clients, colleagues, supervisors, or others. In other words, a practice decision might be based on “popular” rather than “best” practices, and casual evaluation of those practices might be skewed to create a favorable impression.

In all three of these forms of bias, the problem is not necessarily intentional, but does result in a lack of sufficient attention to evidence in monitoring one’s practices. For example, relying solely on qualitative comments volunteered by consumers (anecdotal evidence) is subject to a selection bias —individuals with strong opinions or a desire to support the social workers who helped them are more likely to volunteer than the general population of those served.

Thus, it is incumbent on social work professionals to engage in practice evaluation that is as free of bias as possible. The choice of systematic evaluation approach is dictated by the evaluation research question being asked. According to the Centers for Disease Control and Prevention (CDC), there are four most common types of intervention or program evaluation: formative, process, outcome, and impact evaluation ( https://www.cdc.gov/std/Program/pupestd/Types%20of%20Evaluation.pdf ). Here, we consider these as three types, combining impact and outcome evaluation into a single category, and we consider an additional category, as well: cost evaluation.

Formative Evaluation.

Formative evaluation is emphasized during the early stages of developing or implementing a social work intervention, as well as following process or outcome evaluation as changes to a program or intervention strategy are considered. The aim of formative evaluation is to understand the context of an intervention, define the intervention, and evaluate feasibility of adopting a proposed intervention or change in the intervention (Trochim & Donnelly, 2007). For example, a needs assessment might be conducted to determine whether the intervention or program is needed, calculate how large the unmet need is, and/or specify where/for whom the unmet need exists. Needs assessment might also include conducting an inventory of services that exist to meet the identified need and where/why a gap exists (Engel & Schutt, 2013). Formative evaluation is used to help shape an intervention, program, or policy.

Process Evaluation.

Investigating how an intervention is delivered or a program operates is the purpose behind process evaluation (Engel & Schutt, 2013). The concept of intervention fidelity was previously introduced. Fidelity is a major point of process evaluation but is not the only point. We know that the greater the degree of fidelity in delivery of an intervention, the more applicable the previous evidence about that intervention becomes in reliably predicting intervention outcomes. As fidelity in the intervention’s delivery drifts or wanes, previous evidence becomes less reliable and less useful in making practice decisions. Addressing this important issue is why many interventions with an evidence base supporting their adoption are manualized , providing detailed manuals for how to implement the intervention with fidelity and integrity. For example, the Parent-Child Interaction Therapy for Traumatized Children (PCIT-TC) treatment protocol is manualized and training certification is available for practitioners to learn the evidence-based skills involved ( https://pcit.ucdavis.edu/ ). This strategy increases practitioners’ adherence to the protocol.

Process evaluation, sometimes called implementation evaluation and sometimes referred to as program monitoring, helps investigators determine the extent to which fidelity has been preserved. But, process evaluation serves other purposes, as well. For example, according to King, Morris and Fitz-Gibbon (1987), process evaluation helps:

• document details about the intervention that might help explain outcome evaluation results,
• keep programs accountable (delivering what they claim to deliver),
• inform planned modifications and changes to the intervention based on evidence.

Process evaluation also helps investigators determine where the facilitators and barriers to implementing an intervention might operate and can help interpret outcomes/results from the intervention, as well. Process evaluation efforts addresses the following:

• Who delivered the intervention
• Who received the intervention
• What was (or was not) done during the intervention
• When intervention activities occurred
• Where intervention activities occurred
• How the intervention was delivered
• What facilitated implementation with fidelity/integrity
• What presented as barriers to implementation with fidelity/integrity

For these reasons, many authors consider process evaluation to be a type of formative evaluation.

Outcome and Impact Evaluation.

The aim of outcome or impact evaluation is to determine effects of the intervention. Many authors refer to this as a type of summative evaluation , distinguishing it from formative evaluation: its purpose is to understand the effects of an intervention once it has been delivered. The effects of interest usually include the extent to which intervention goals or objectives were achieved. An important factor to evaluate concerns positive and negative “side effects”—those unintended outcomes associated with the intervention. These might include unintended impact of the intervention participants or impacts on significant others, those delivering the intervention, the program/agency/institutions involved, and others. While impact evaluation, as described by the CDC, is about policy and funding decisions and longer-term changes, we can include it as a form of outcome evaluation since the questions answered are about achieving intervention objectives. Outcome evaluation is based on the elements presented in the logic model created at the outset of intervention planning.

Cost-Related Evaluation.

Social workers are frequently faced with efficiency questions related to the interventions we deliver—thus, cost-related evaluation is part of our professional accountability responsibilities. For example, once an agency has applied the evidence-based practice (EBP) process to select the best-fitting program options for addressing an identified practice concern, program planning is enhanced by information concerning which of the options is most cost-effective.  Here are some types of questions addressed in cost-related evaluation.

cost analysis: How much does it cost to deliver/implement the intervention with fidelity and integrity? This type of analysis typically analyzes monetary costs, converting inputs into their financial impact (e.g., space resources would be converted into cost per square foot, staffing costs would include salary, training, and benefits costs, materials and technology costs might include depreciation).

• cost-benefit: What are the inputs and outputs associated with the intervention? This type of analysis involves placing a monetary value on each element of input (resources) and each of the outputs. For example, preventing incarceration would be converted to the dollars saved on jail/prison costs; and, perhaps, including the individuals’ ability to keep their jobs and homes which could be lost with incarceration, as well as preventing family members needing public assistance and/or children being placed in foster care if their family member is incarcerated.
• cost-effectiveness: What is the ratio of cost units (numerator) to outcome units (denominator) associated with delivering an intervention. Outcomes are tied to the intervention goals rather than monetary units. For example, medical interventions are often analyzed in terms of DALYs (disability-adjusted life years)—units designed to indicate “disease burden,” calculated to represent the number of years lost to illness, disability, or premature death (morbidity and mortality). Outcomes might also be numbers of “cases,” such as deaths or hospitalizations related to suicide attempts, drug overdose events, students dropping out from high school, children reunited with their families (family reunification), reports of child maltreatment, persons un- or under-employed, and many more examples. Costs are typically presented as monetary units estimated from a costs analysis. (See http://www.who.int/heli/economics/costeffanalysis/en/ ).
• cost-utility: A comparison of cost-effectiveness for two or more intervention options, designed to help decision-makers make informed choices between the options.

Two of the greatest challenges with these kinds of evaluation are (1) ensuring that all relevant inputs and outputs are included in the analysis, and (2) realistically converting non-monetary costs and benefits into monetary units to standardize comparisons. An additional challenge has to do with budget structures: the gains might be realized in a different budget than where the costs are borne. For example, implementing a mental health or substance misuse treatment program in jails and prisons costs those facilities; the benefits are realized in budgets outside those facilities—schools, workplaces, medical facilities, family services, and mental health programs in the community. Thus, it is challenging to make decisions based on these analyses when constituents are situated in different systems operating with “siloed” budgets where there is little or no sharing across systems.

An Additional Point.

An intervention or evaluation effort does not necessarily need to be limited to one types. As in the case of mixed-methods approaches, it is sometimes helpful to engage in multiple evaluation efforts with a single intervention or program. A team of investigators described how they used formative, process, and outcome evaluation all in the pursuit of understanding a single preventive public health intervention called VERB, designed to increase physical activity among youth (Berkowitz et al., 2008). Their formative evaluation efforts allowed the team to assess the intervention’s appropriateness for the target audience and to test different messages. The process evaluation addressed fidelity of the intervention during implementation. And, the outcome evaluation led the team to draw conclusions concerning the intervention’s effects on the target audience. The various forms of evaluation utilized qualitative and quantitative approaches.

Participatory Research Approaches

One contrasts previously noted between qualitative and quantitative research is the nature of the investigator’s role. Every effort is made to minimize investigator influence on the data collection and analysis processes in quantitative research. Qualitative research, on the other hand, recognizes the investigator as an integral part of the research process. Participatory research fits into this latter category.

“Participant observation is a method in which natural social processes are studied as they happen (in the field, rather than in the laboratory) and left relatively undisturbed. It is a means of seeing the social world as the research subjects see it, in its totality, and of understanding subjects’ interpretations of that world” (Engel & Schutt, 2013, p. 276).

This quote describes naturalistic observation very well. The difference with participatory observation is that the investigator is embedded in the group, neighborhood, community, institution, or other entity under study. Participatory observation is one approach used by anthropologists to understand cultures from an embedded rather than outsider perspective. For example, this is how Jane Goodall learned about chimpanzee culture in Tanzania: she became accepted as part of the group she observed, allowing her to describe the members’ behaviors and social relationships, her own experiences as a member of the group, and the theories she derived from 55 years of this work. In social work, the participant approach may be used to answer the research questions of the type we explored in our earlier course: understanding diverse populations, social work problems, or social phenomena. The investigator might be a natural member of the group, where the role as group member precedes the role as observer. This is where the term indigenous membership applies: naturally belonging to the group. (The term “indigenous people” describes the native, naturally occurring inhabitants of a place or region.) It is sometimes difficult to determine how the indigenous member’s observations and conclusions might be influenced by his or her position within the group—for example, the experience might be different for men and women, members of different ages, or leaders. Thus, the conclusions need to be confirmed by a diverse membership.

Participant observers are sometimes “adopted” members of the group, where the role of observer precedes their role as group member. It is somewhat more difficult to determine if evidence collected under these circumstances reflects a fully accurate description of the members’ experience unless the evidence and conclusions have been cross-checked by the group’s indigenous members. Turning back to our example with Jane Goodall, she was accepted into the chimpanzee troop in many ways, but not in others—she could not experience being a birth mother to members of the group, for example.

Sometimes investigators are more actively engaged in the life of the group being observed. As previously noted, participant observation is about the processes being left relatively undisturbed (Engel & Schutt, 2013, p. 276).  However, participant observers might be more actively engaged in change efforts, documenting the change process from “inside” the group promoting change. These instances are called participatory action research (PAR) , where the investigator is an embedded member of the group, joining them in making a concerted effort to influence change. PAR involves three intersecting roles: participation in the group, engaging with the action process (planning and implementing interventions), and conducting research about the group’s action process (see Figure 2-1, adapted from Chevalier & Buckles, 2013, p. 10).

Figure 2-1. Venn diagram of participatory action research roles.

For example, Pyles (2015) described the experience of engaging in participatory action research with rural organizations and rural disaster survivors in Haiti following the January 12, 2010 earthquake. The PAR aimed to promote local organizations’ capacity to engage in education and advocacy and to secure much-needed resources for their rural communities (Pyles, 2015, p. 630). According to the author, rural Haitian communities have a history of experience with exploitative research where outsiders conduct investigations without the input or participation of community members, and where little or no capacity-building action occurs based on study results and recommendations. Pyles also raised the point that, “there are multiple barriers impeding the participation of marginalized people” in community building efforts, making PAR approaches even more important for these groups (2015, p. 634).

The term community-based participatory research (CBPR) refers to collaborative partnerships between members of a community (e.g., a group, neighborhood, or organization) and researchers throughout the entire research process. CBPR partners (internal and external members) all contribute their expertise to the process, throughout the process, and share in all steps of decision-making. Stakeholder members of the community (or organization) are involved as active, equal partners in the research process, co-learning by all members of the collaboration is emphasized, and it represents a strengths-focused approach (Harris, 2010; Holkup, Tripp-Reier, Salois, & Weinert, 2004). CBPR is relevant in our efforts to understand social work interventions since the process can result in interventions that are culturally appropriate, feasible, acceptable, and applicable for the community since they emerged from within that community. Furthermore, it is a community empowerment approach whereby self-determination plays a key role and the community is left with new skills for self-study, evaluation, and understanding the change process (Harris, 2010). These characteristics of CBPR help define the approach.

(a) recognizing the community as a unit of identity,

(b) building on the strengths and resources of the community,

(c) promoting colearning among research partners,

(d) achieving a balance between research and action that mutually benefits both science and the community,

(e) emphasizing the relevance of community-defined problems,

(f) employing a cyclical and iterative process to develop and maintain community/ research partnerships,

(g) disseminating knowledge gained from the CBPR project to and by all involved partners, and

(h) requiring long-term commitment on the part of all partners ( Holkup, Tripp-Reier, Salois, & Weinert, 2004, p. 2).

Quinn et al (2017) published a case study of CBPR practices being employed with youth at risk of homelessness and exposure to violence. The authors cited a “paucity of evidence-based, developmentally appropriate interventions” to address the mental health needs of youth exposed to violence (p. 3). The CBPR process helped determine the acceptability of a person-centered trauma therapy approach called narrative exposure therapy (NET). The results of three pilot projects combined to inform the design of a randomized controlled trial (RCT) to study the impact of the NET intervention. The three pilot projects engaged researchers and members of the population to be served (youth at risk of homelessness and exposure to violence). The authors of the case study article discussed some of the challenges of working with youth in the CBPR process and research process. Adapted from Quinn et al (2017), these included:

• Compliance with federal regulations for research involving minors (defined as “children” in the policies). Compounding this challenge was the vulnerable status of the youth due to their homeless status, and the frequency with which many of the youth were not engaged with any adults who had legal authority to provide consent for them to participate.
• The team was interdisciplinary, which brings many advantages. However, it also presented challenges regarding different perspectives about how to engage in the varied research processes of participant recruitment and retention, measurement, and intervention.
• Logistics of conducting focus groups with this vulnerable population. Youth encounter difficulties with participating predictably, and for this vulnerable population the practical difficulties are compounded. They experience complex and often competing demands on their schedules, “including school obligations, court, group or other agency appointments, or childcare,” as well as managing public transportation schedules and other barriers (p. 11). Furthermore, members of the group may have pre-existing relationships and social network ties that can impinge on their comfort with openly sharing their experiences or perspectives in the group setting. They may also have skepticism and reservations about sharing with the adults leading the focus group sessions.

Awareness of these challenges can help CBPR teams develop solutions to overcome the barriers. The CBPR process, while time and resource intensive, can result in appropriate intervention designs for under-served populations where existing evidence is not available to guide intervention planning.

A somewhat different approach engages members of the community as consultants regarding interventions with which they may be engaged, rather than a fully CBPR approach. This adapted consultation approach presents an important option for ensuring that interventions are appropriate and acceptable for serving the community. However, community members are less integrally involved in the action-related aspects of defining and implementing the intervention, or in the conduct of the implementation research. An example of this important community-as-consultant approach involved a series of six focus group sessions conducted with parents, teachers, and school stakeholders discussing teen pregnancy prevention among high-school aged Latino youth (Johnson-Motoyama et al., 2016). The investigating team reported recommendations and requests from these community members concerning the important role played by parents and potential impact of parent education efforts in preventing teen pregnancy within this population. The community members also identified the importance of comprehensive, empowering, tailored programming that addresses self-respect, responsibility, and “realities,” and incorporates peer role models. They concluded that local school communities have an important role to play in planning for interventions that are “responsive to the community’s cultural values, beliefs, and preferences, as well as the school’s capacity and teacher preferences” (p. 513). Thus, the constituencies involved in this project served as consultants rather than CBPR collaborators. However, the resulting intervention plans could be more culturally appropriate and relevant than intervention plans developed by “outsiders” alone.

One main limitation to conducting CBPR work is the immense amount of time and effort involved in developing strong working collaborative relationships—relationships that can stand the test of time. Collaborative relationships are often built from a series of “quick wins” or small successes over time, where the partners learn about each other, learn to trust each other, and learn to work together effectively.

Chapter Summary

This chapter began with a review of concepts from our earlier course: qualitative, quantitative, mixed-methods, cross-sectional and longitudinal approaches. Expanded content about approach came next: formative, process, outcome, and cost evaluation approaches were connected to the kinds of intervention questions social workers might ask, and participatory research approaches were introduced. Issues of cultural relevance were explored, as well. This discussion of approach leads to an expanded discussion of quantitative study design strategies, which is the topic of our next chapter.

Stop and Think

Social Work 3402 Coursebook Copyright © by Dr. Audrey Begun is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License , except where otherwise noted.

Share This Book

• Open access
• Published: 27 November 2020

Designing process evaluations using case study to explore the context of complex interventions evaluated in trials

• Aileen Grant 1 ,
• Carol Bugge 2 &
• Mary Wells 3  

Trials volume  21 , Article number:  982 ( 2020 ) Cite this article

11k Accesses

10 Citations

5 Altmetric

Metrics details

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail, and whether they can be transferred to other settings and populations. However, historically, context has not been sufficiently explored and reported resulting in the poor uptake of trial results. Therefore, suitable methodologies are needed to guide the investigation of context. Case study is one appropriate methodology, but there is little guidance about what case study design can offer the study of context in trials. We address this gap in the literature by presenting a number of important considerations for process evaluation using a case study design.

In this paper, we define context, the relationship between complex interventions and context, and describe case study design methodology. A well-designed process evaluation using case study should consider the following core components: the purpose; definition of the intervention; the trial design, the case, the theories or logic models underpinning the intervention, the sampling approach and the conceptual or theoretical framework. We describe each of these in detail and highlight with examples from recently published process evaluations.

Conclusions

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation. We provide a comprehensive overview of the issues for process evaluation design to consider when using a case study design.

Trial registration

DQIP - ClinicalTrials.gov number, NCT01425502 - OPAL - ISRCTN57746448

Peer Review reports

Contribution to the literature

We illustrate how case study methodology can explore the complex, dynamic and uncertain relationship between context and interventions within trials.

We depict different case study designs and illustrate there is not one formula and that design needs to be tailored to the context and trial design.

Case study can support comparisons between intervention and control arms and between cases within arms to uncover and explain differences in detail.

We argue that case study can illustrate how components have evolved and been redefined through implementation.

Key issues for consideration in case study design within process evaluations are presented and illustrated with examples.

Process evaluations are an important component of an effectiveness evaluation as they focus on understanding the relationship between interventions and context to explain how and why interventions work or fail and whether they can be transferred to other settings and populations. However, historically, not all trials have had a process evaluation component, nor have they sufficiently reported aspects of context, resulting in poor uptake of trial findings [ 1 ]. Considerations of context are often absent from published process evaluations, with few studies acknowledging, taking account of or describing context during implementation, or assessing the impact of context on implementation [ 2 , 3 ]. At present, evidence from trials is not being used in a timely manner [ 4 , 5 ], and this can negatively impact on patient benefit and experience [ 6 ]. It takes on average 17 years for knowledge from research to be implemented into practice [ 7 ]. Suitable methodologies are therefore needed that allow for context to be exposed; one appropriate methodological approach is case study [ 8 , 9 ].

In 2015, the Medical Research Council (MRC) published guidance for process evaluations [ 10 ]. This was a key milestone in legitimising as well as providing tools, methods and a framework for conducting process evaluations. Nevertheless, as with all guidance, there is a need for reflection, challenge and refinement. There have been a number of critiques of the MRC guidance, including that interventions should be considered as events in systems [ 11 , 12 , 13 , 14 ]; a need for better use, critique and development of theories [ 15 , 16 , 17 ]; and a need for more guidance on integrating qualitative and quantitative data [ 18 , 19 ]. Although the MRC process evaluation guidance does consider appropriate qualitative and quantitative methods, it does not mention case study design and what it can offer the study of context in trials.

The case study methodology is ideally suited to real-world, sustainable intervention development and evaluation because it can explore and examine contemporary complex phenomena, in depth, in numerous contexts and using multiple sources of data [ 8 ]. Case study design can capture the complexity of the case, the relationship between the intervention and the context and how the intervention worked (or not) [ 8 ]. There are a number of textbooks on a case study within the social science fields [ 8 , 9 , 20 ], but there are no case study textbooks and a paucity of useful texts on how to design, conduct and report case study within the health arena. Few examples exist within the trial design and evaluation literature [ 3 , 21 ]. Therefore, guidance to enable well-designed process evaluations using case study methodology is required.

We aim to address the gap in the literature by presenting a number of important considerations for process evaluation using a case study design. First, we define the context and describe the relationship between complex health interventions and context.

What is context?

While there is growing recognition that context interacts with the intervention to impact on the intervention’s effectiveness [ 22 ], context is still poorly defined and conceptualised. There are a number of different definitions in the literature, but as Bate et al. explained ‘almost universally, we find context to be an overworked word in everyday dialogue but a massively understudied and misunderstood concept’ [ 23 ]. Ovretveit defines context as ‘everything the intervention is not’ [ 24 ]. This last definition is used by the MRC framework for process evaluations [ 25 ]; however; the problem with this definition is that it is highly dependent on how the intervention is defined. We have found Pfadenhauer et al.’s definition useful:

Context is conceptualised as a set of characteristics and circumstances that consist of active and unique factors that surround the implementation. As such it is not a backdrop for implementation but interacts, influences, modifies and facilitates or constrains the intervention and its implementation. Context is usually considered in relation to an intervention or object, with which it actively interacts. A boundary between the concepts of context and setting is discernible: setting refers to the physical, specific location in which the intervention is put into practice. Context is much more versatile, embracing not only the setting but also roles, interactions and relationships [ 22 ].

Traditionally, context has been conceptualised in terms of barriers and facilitators, but what is a barrier in one context may be a facilitator in another, so it is the relationship and dynamics between the intervention and context which are the most important [ 26 ]. There is a need for empirical research to really understand how different contextual factors relate to each other and to the intervention. At present, research studies often list common contextual factors, but without a depth of meaning and understanding, such as government or health board policies, organisational structures, professional and patient attitudes, behaviours and beliefs [ 27 ]. The case study methodology is well placed to understand the relationship between context and intervention where these boundaries may not be clearly evident. It offers a means of unpicking the contextual conditions which are pertinent to effective implementation.

The relationship between complex health interventions and context

Health interventions are generally made up of a number of different components and are considered complex due to the influence of context on their implementation and outcomes [ 3 , 28 ]. Complex interventions are often reliant on the engagement of practitioners and patients, so their attitudes, behaviours, beliefs and cultures influence whether and how an intervention is effective or not. Interventions are context-sensitive; they interact with the environment in which they are implemented. In fact, many argue that interventions are a product of their context, and indeed, outcomes are likely to be a product of the intervention and its context [ 3 , 29 ]. Within a trial, there is also the influence of the research context too—so the observed outcome could be due to the intervention alone, elements of the context within which the intervention is being delivered, elements of the research process or a combination of all three. Therefore, it can be difficult and unhelpful to separate the intervention from the context within which it was evaluated because the intervention and context are likely to have evolved together over time. As a result, the same intervention can look and behave differently in different contexts, so it is important this is known, understood and reported [ 3 ]. Finally, the intervention context is dynamic; the people, organisations and systems change over time, [ 3 ] which requires practitioners and patients to respond, and they may do this by adapting the intervention or contextual factors. So, to enable researchers to replicate successful interventions, or to explain why the intervention was not successful, it is not enough to describe the components of the intervention, they need to be described by their relationship to their context and resources [ 3 , 28 ].

What is a case study?

Case study methodology aims to provide an in-depth, holistic, balanced, detailed and complete picture of complex contemporary phenomena in its natural context [ 8 , 9 , 20 ]. In this case, the phenomena are the implementation of complex interventions in a trial. Case study methodology takes the view that the phenomena can be more than the sum of their parts and have to be understood as a whole [ 30 ]. It is differentiated from a clinical case study by its analytical focus [ 20 ].

The methodology is particularly useful when linked to trials because some of the features of the design naturally fill the gaps in knowledge generated by trials. Given the methodological focus on understanding phenomena in the round, case study methodology is typified by the use of multiple sources of data, which are more commonly qualitatively guided [ 31 ]. The case study methodology is not epistemologically specific, like realist evaluation, and can be used with different epistemologies [ 32 ], and with different theories, such as Normalisation Process Theory (which explores how staff work together to implement a new intervention) or the Consolidated Framework for Implementation Research (which provides a menu of constructs associated with effective implementation) [ 33 , 34 , 35 ]. Realist evaluation can be used to explore the relationship between context, mechanism and outcome, but case study differs from realist evaluation by its focus on a holistic and in-depth understanding of the relationship between an intervention and the contemporary context in which it was implemented [ 36 ]. Case study enables researchers to choose epistemologies and theories which suit the nature of the enquiry and their theoretical preferences.

Designing a process evaluation using case study

An important part of any study is the research design. Due to their varied philosophical positions, the seminal authors in the field of case study have different epistemic views as to how a case study should be conducted [ 8 , 9 ]. Stake takes an interpretative approach (interested in how people make sense of their world), and Yin has more positivistic leanings, arguing for objectivity, validity and generalisability [ 8 , 9 ].

Regardless of the philosophical background, a well-designed process evaluation using case study should consider the following core components: the purpose; the definition of the intervention, the trial design, the case, and the theories or logic models underpinning the intervention; the sampling approach; and the conceptual or theoretical framework [ 8 , 9 , 20 , 31 , 33 ]. We now discuss these critical components in turn, with reference to two process evaluations that used case study design, the DQIP and OPAL studies [ 21 , 37 , 38 , 39 , 40 , 41 ].

The purpose of a process evaluation is to evaluate and explain the relationship between the intervention and its components, to context and outcome. It can help inform judgements about validity (by exploring the intervention components and their relationship with one another (construct validity), the connections between intervention and outcomes (internal validity) and the relationship between intervention and context (external validity)). It can also distinguish between implementation failure (where the intervention is poorly delivered) and intervention failure (intervention design is flawed) [ 42 , 43 ]. By using a case study to explicitly understand the relationship between context and the intervention during implementation, the process evaluation can explain the intervention effects and the potential generalisability and optimisation into routine practice [ 44 ].

The DQIP process evaluation aimed to qualitatively explore how patients and GP practices responded to an intervention designed to reduce high-risk prescribing of nonsteroidal anti-inflammatory drugs (NSAIDs) and/or antiplatelet agents (see Table  1 ) and quantitatively examine how change in high-risk prescribing was associated with practice characteristics and implementation processes. The OPAL process evaluation (see Table  2 ) aimed to quantitatively understand the factors which influenced the effectiveness of a pelvic floor muscle training intervention for women with urinary incontinence and qualitatively explore the participants’ experiences of treatment and adherence.

Defining the intervention and exploring the theories or assumptions underpinning the intervention design

Process evaluations should also explore the utility of the theories or assumptions underpinning intervention design [ 49 ]. Not all theories underpinning interventions are based on a formal theory, but they based on assumptions as to how the intervention is expected to work. These can be depicted as a logic model or theory of change [ 25 ]. To capture how the intervention and context evolve requires the intervention and its expected mechanisms to be clearly defined at the outset [ 50 ]. Hawe and colleagues recommend defining interventions by function (what processes make the intervention work) rather than form (what is delivered) [ 51 ]. However, in some cases, it may be useful to know if some of the components are redundant in certain contexts or if there is a synergistic effect between all the intervention components.

The DQIP trial delivered two interventions, one intervention was delivered to professionals with high fidelity and then professionals delivered the other intervention to patients by form rather than function allowing adaptations to the local context as appropriate. The assumptions underpinning intervention delivery were prespecified in a logic model published in the process evaluation protocol [ 52 ].

Case study is well placed to challenge or reinforce the theoretical assumptions or redefine these based on the relationship between the intervention and context. Yin advocates the use of theoretical propositions; these direct attention to specific aspects of the study for investigation [ 8 ] can be based on the underlying assumptions and tested during the course of the process evaluation. In case studies, using an epistemic position more aligned with Yin can enable research questions to be designed, which seek to expose patterns of unanticipated as well as expected relationships [ 9 ]. The OPAL trial was more closely aligned with Yin, where the research team predefined some of their theoretical assumptions, based on how the intervention was expected to work. The relevant parts of the data analysis then drew on data to support or refute the theoretical propositions. This was particularly useful for the trial as the prespecified theoretical propositions linked to the mechanisms of action on which the intervention was anticipated to have an effect (or not).

Tailoring to the trial design

Process evaluations need to be tailored to the trial, the intervention and the outcomes being measured [ 45 ]. For example, in a stepped wedge design (where the intervention is delivered in a phased manner), researchers should try to ensure process data are captured at relevant time points or in a two-arm or multiple arm trial, ensure data is collected from the control group(s) as well as the intervention group(s). In the DQIP trial, a stepped wedge trial, at least one process evaluation case, was sampled per cohort. Trials often continue to measure outcomes after delivery of the intervention has ceased, so researchers should also consider capturing ‘follow-up’ data on contextual factors, which may continue to influence the outcome measure. The OPAL trial had two active treatment arms so collected process data from both arms. In addition, as the trial was interested in long-term adherence, the trial and the process evaluation collected data from participants for 2 years after the intervention was initially delivered, providing 24 months follow-up data, in line with the primary outcome for the trial.

Defining the case

Case studies can include single or multiple cases in their design. Single case studies usually sample typical or unique cases, their advantage being the depth and richness that can be achieved over a long period of time. The advantages of multiple case study design are that cases can be compared to generate a greater depth of analysis. Multiple case study sampling may be carried out in order to test for replication or contradiction [ 8 ]. Given that trials are often conducted over a number of sites, a multiple case study design is more sensible for process evaluations, as there is likely to be variation in implementation between sites. Case definition may occur at a variety of levels but is most appropriate if it reflects the trial design. For example, a case in an individual patient level trial is likely to be defined as a person/patient (e.g. a woman with urinary incontinence—OPAL trial) whereas in a cluster trial, a case is like to be a cluster, such as an organisation (e.g. a general practice—DQIP trial). Of course, the process evaluation could explore cases with less distinct boundaries, such as communities or relationships; however, the clarity with which these cases are defined is important, in order to scope the nature of the data that will be generated.

Carefully sampled cases are critical to a good case study as sampling helps inform the quality of the inferences that can be made from the data [ 53 ]. In both qualitative and quantitative research, how and how many participants to sample must be decided when planning the study. Quantitative sampling techniques generally aim to achieve a random sample. Qualitative research generally uses purposive samples to achieve data saturation, occurring when the incoming data produces little or no new information to address the research questions. The term data saturation has evolved from theoretical saturation in conventional grounded theory studies; however, its relevance to other types of studies is contentious as the term saturation seems to be widely used but poorly justified [ 54 ]. Empirical evidence suggests that for in-depth interview studies, saturation occurs at 12 interviews for thematic saturation, but typically more would be needed for a heterogenous sample higher degrees of saturation [ 55 , 56 ]. Both DQIP and OPAL case studies were huge with OPAL designed to interview each of the 40 individual cases four times and DQIP designed to interview the lead DQIP general practitioner (GP) twice (to capture change over time), another GP and the practice manager from each of the 10 organisational cases. Despite the plethora of mixed methods research textbooks, there is very little about sampling as discussions typically link to method (e.g. interviews) rather than paradigm (e.g. case study).

Purposive sampling can improve the generalisability of the process evaluation by sampling for greater contextual diversity. The typical or average case is often not the richest source of information. Outliers can often reveal more important insights, because they may reflect the implementation of the intervention using different processes. Cases can be selected from a number of criteria, which are not mutually exclusive, to enable a rich and detailed picture to be built across sites [ 53 ]. To avoid the Hawthorne effect, it is recommended that process evaluations sample from both intervention and control sites, which enables comparison and explanation. There is always a trade-off between breadth and depth in sampling, so it is important to note that often quantity does not mean quality and that carefully sampled cases can provide powerful illustrative examples of how the intervention worked in practice, the relationship between the intervention and context and how and why they evolved together. The qualitative components of both DQIP and OPAL process evaluations aimed for maximum variation sampling. Please see Table  1 for further information on how DQIP’s sampling frame was important for providing contextual information on processes influencing effective implementation of the intervention.

Conceptual and theoretical framework

A conceptual or theoretical framework helps to frame data collection and analysis [ 57 ]. Theories can also underpin propositions, which can be tested in the process evaluation. Process evaluations produce intervention-dependent knowledge, and theories help make the research findings more generalizable by providing a common language [ 16 ]. There are a number of mid-range theories which have been designed to be used with process evaluation [ 34 , 35 , 58 ]. The choice of the appropriate conceptual or theoretical framework is, however, dependent on the philosophical and professional background of the research. The two examples within this paper used our own framework for the design of process evaluations, which proposes a number of candidate processes which can be explored, for example, recruitment, delivery, response, maintenance and context [ 45 ]. This framework was published before the MRC guidance on process evaluations, and both the DQIP and OPAL process evaluations were designed before the MRC guidance was published. The DQIP process evaluation explored all candidates in the framework whereas the OPAL process evaluation selected four candidates, illustrating that process evaluations can be selective in what they explore based on the purpose, research questions and resources. Furthermore, as Kislov and colleagues argue, we also have a responsibility to critique the theoretical framework underpinning the evaluation and refine theories to advance knowledge [ 59 ].

Data collection

An important consideration is what data to collect or measure and when. Case study methodology supports a range of data collection methods, both qualitative and quantitative, to best answer the research questions. As the aim of the case study is to gain an in-depth understanding of phenomena in context, methods are more commonly qualitative or mixed method in nature. Qualitative methods such as interviews, focus groups and observation offer rich descriptions of the setting, delivery of the intervention in each site and arm, how the intervention was perceived by the professionals delivering the intervention and the patients receiving the intervention. Quantitative methods can measure recruitment, fidelity and dose and establish which characteristics are associated with adoption, delivery and effectiveness. To ensure an understanding of the complexity of the relationship between the intervention and context, the case study should rely on multiple sources of data and triangulate these to confirm and corroborate the findings [ 8 ]. Process evaluations might consider using routine data collected in the trial across all sites and additional qualitative data across carefully sampled sites for a more nuanced picture within reasonable resource constraints. Mixed methods allow researchers to ask more complex questions and collect richer data than can be collected by one method alone [ 60 ]. The use of multiple sources of data allows data triangulation, which increases a study’s internal validity but also provides a more in-depth and holistic depiction of the case [ 20 ]. For example, in the DQIP process evaluation, the quantitative component used routinely collected data from all sites participating in the trial and purposively sampled cases for a more in-depth qualitative exploration [ 21 , 38 , 39 ].

The timing of data collection is crucial to study design, especially within a process evaluation where data collection can potentially influence the trial outcome. Process evaluations are generally in parallel or retrospective to the trial. The advantage of a retrospective design is that the evaluation itself is less likely to influence the trial outcome. However, the disadvantages include recall bias, lack of sensitivity to nuances and an inability to iteratively explore the relationship between intervention and outcome as it develops. To capture the dynamic relationship between intervention and context, the process evaluation needs to be parallel and longitudinal to the trial. Longitudinal methodological design is rare, but it is needed to capture the dynamic nature of implementation [ 40 ]. How the intervention is delivered is likely to change over time as it interacts with context. For example, as professionals deliver the intervention, they become more familiar with it, and it becomes more embedded into systems. The OPAL process evaluation was a longitudinal, mixed methods process evaluation where the quantitative component had been predefined and built into trial data collection systems. Data collection in both the qualitative and quantitative components mirrored the trial data collection points, which were longitudinal to capture adherence and contextual changes over time.

There is a lot of attention in the recent literature towards a systems approach to understanding interventions in context, which suggests interventions are ‘events within systems’ [ 61 , 62 ]. This framing highlights the dynamic nature of context, suggesting that interventions are an attempt to change systems dynamics. This conceptualisation would suggest that the study design should collect contextual data before and after implementation to assess the effect of the intervention on the context and vice versa.

Data analysis

Designing a rigorous analysis plan is particularly important for multiple case studies, where researchers must decide whether their approach to analysis is case or variable based. Case-based analysis is the most common, and analytic strategies must be clearly articulated for within and across case analysis. A multiple case study design can consist of multiple cases, where each case is analysed at the case level, or of multiple embedded cases, where data from all the cases are pulled together for analysis at some level. For example, OPAL analysis was at the case level, but all the cases for the intervention and control arms were pulled together at the arm level for more in-depth analysis and comparison. For Yin, analytical strategies rely on theoretical propositions, but for Stake, analysis works from the data to develop theory. In OPAL and DQIP, case summaries were written to summarise the cases and detail within-case analysis. Each of the studies structured these differently based on the phenomena of interest and the analytic technique. DQIP applied an approach more akin to Stake [ 9 ], with the cases summarised around inductive themes whereas OPAL applied a Yin [ 8 ] type approach using theoretical propositions around which the case summaries were structured. As the data for each case had been collected through longitudinal interviews, the case summaries were able to capture changes over time. It is beyond the scope of this paper to discuss different analytic techniques; however, to ensure the holistic examination of the intervention(s) in context, it is important to clearly articulate and demonstrate how data is integrated and synthesised [ 31 ].

There are a number of approaches to process evaluation design in the literature; however, there is a paucity of research on what case study design can offer process evaluations. We argue that case study is one of the best research designs to underpin process evaluations, to capture the dynamic and complex relationship between intervention and context during implementation [ 38 ]. Case study can enable comparisons within and across intervention and control arms and enable the evolving relationship between intervention and context to be captured holistically rather than considering processes in isolation. Utilising a longitudinal design can enable the dynamic relationship between context and intervention to be captured in real time. This information is fundamental to holistically explaining what intervention was implemented, understanding how and why the intervention worked or not and informing the transferability of the intervention into routine clinical practice.

Case study designs are not prescriptive, but process evaluations using case study should consider the purpose, trial design, the theories or assumptions underpinning the intervention, and the conceptual and theoretical frameworks informing the evaluation. We have discussed each of these considerations in turn, providing a comprehensive overview of issues for process evaluations using a case study design. There is no single or best way to conduct a process evaluation or a case study, but researchers need to make informed choices about the process evaluation design. Although this paper focuses on process evaluations, we recognise that case study design could also be useful during intervention development and feasibility trials. Elements of this paper are also applicable to other study designs involving trials.

Availability of data and materials

No data and materials were used.

Abbreviations

Data-driven Quality Improvement in Primary Care

Medical Research Council

Nonsteroidal anti-inflammatory drugs

Optimizing Pelvic Floor Muscle Exercises to Achieve Long-term benefits

Blencowe NB. Systematic review of intervention design and delivery in pragmatic and explanatory surgical randomized clinical trials. Br J Surg. 2015;102:1037–47.

Article   CAS   PubMed   Google Scholar  

Dixon-Woods M. The problem of context in quality improvement. In: Foundation TH, editor. Perspectives on context: The Health Foundation; 2014.

Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):95.

Article   PubMed   PubMed Central   Google Scholar  

Grant A, Sullivan F, Dowell J. An ethnographic exploration of influences on prescribing in general practice: why is there variation in prescribing practices? Implement Sci. 2013;8(1):72.

Lang ES, Wyer PC, Haynes RB. Knowledge translation: closing the evidence-to-practice gap. Ann Emerg Med. 2007;49(3):355–63.

Article   PubMed   Google Scholar  

Ward V, House AF, Hamer S. Developing a framework for transferring knowledge into action: a thematic analysis of the literature. J Health Serv Res Policy. 2009;14(3):156–64.

Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20.

Yin R. Case study research and applications: design and methods. Los Angeles: Sage Publications Inc; 2018.

Google Scholar  

Stake R. The art of case study research. Thousand Oaks, California: Sage Publications Ltd; 1995.

Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, et al. Process evaluation of complex interventions: Medical Research Council guidance. Br Med J. 2015;350.

Hawe P. Minimal, negligible and negligent interventions. Soc Sci Med. 2015;138:265–8.

Moore GF, Evans RE, Hawkins J, Littlecott H, Melendez-Torres GJ, Bonell C, Murphy S. From complex social interventions to interventions in complex social systems: future directions and unresolved questions for intervention development and evaluation. Evaluation. 2018;25(1):23–45.

Greenhalgh T, Papoutsi C. Studying complexity in health services research: desperately seeking an overdue paradigm shift. BMC Med. 2018;16(1):95.

Rutter H, Savona N, Glonti K, Bibby J, Cummins S, Finegood DT, Greaves F, Harper L, Hawe P, Moore L, et al. The need for a complex systems model of evidence for public health. Lancet. 2017;390(10112):2602–4.

Moore G, Cambon L, Michie S, Arwidson P, Ninot G, Ferron C, Potvin L, Kellou N, Charlesworth J, Alla F, et al. Population health intervention research: the place of theories. Trials. 2019;20(1):285.

Kislov R. Engaging with theory: from theoretically informed to theoretically informative improvement research. BMJ Qual Saf. 2019;28(3):177–9.

Boulton R, Sandall J, Sevdalis N. The cultural politics of ‘Implementation Science’. J Med Human. 2020;41(3):379-94. h https://doi.org/10.1007/s10912-020-09607-9 .

Cheng KKF, Metcalfe A. Qualitative methods and process evaluation in clinical trials context: where to head to? Int J Qual Methods. 2018;17(1):1609406918774212.

Article   Google Scholar  

Richards DA, Bazeley P, Borglin G, Craig P, Emsley R, Frost J, Hill J, Horwood J, Hutchings HA, Jinks C, et al. Integrating quantitative and qualitative data and findings when undertaking randomised controlled trials. BMJ Open. 2019;9(11):e032081.

Thomas G. How to do your case study, 2nd edition edn. London: Sage Publications Ltd; 2016.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: case study evaluation of adoption and maintenance of a complex intervention to reduce high-risk primary care prescribing. BMJ Open. 2017;7(3).

Pfadenhauer L, Rohwer A, Burns J, Booth A, Lysdahl KB, Hofmann B, Gerhardus A, Mozygemba K, Tummers M, Wahlster P, et al. Guidance for the assessment of context and implementation in health technology assessments (HTA) and systematic reviews of complex interventions: the Context and Implementation of Complex Interventions (CICI) framework: Integrate-HTA; 2016.

Bate P, Robert G, Fulop N, Ovretveit J, Dixon-Woods M. Perspectives on context. London: The Health Foundation; 2014.

Ovretveit J. Understanding the conditions for improvement: research to discover which context influences affect improvement success. BMJ Qual Saf. 2011;20.

Medical Research Council: Process evaluation of complex interventions: UK Medical Research Council (MRC) guidance. 2015.

May CR, Johnson M, Finch T. Implementation, context and complexity. Implement Sci. 2016;11(1):141.

Bate P. Context is everything. In: Perpesctives on Context. The Health Foundation 2014.

Horton TJ, Illingworth JH, Warburton WHP. Overcoming challenges in codifying and replicating complex health care interventions. Health Aff. 2018;37(2):191–7.

O'Connor AM, Tugwell P, Wells GA, Elmslie T, Jolly E, Hollingworth G, McPherson R, Bunn H, Graham I, Drake E. A decision aid for women considering hormone therapy after menopause: decision support framework and evaluation. Patient Educ Couns. 1998;33:267–79.

Creswell J, Poth C. Qualiative inquiry and research design, fourth edition edn. Thousan Oaks, California: Sage Publications; 2018.

Carolan CM, Forbat L, Smith A. Developing the DESCARTE model: the design of case study research in health care. Qual Health Res. 2016;26(5):626–39.

Takahashi ARW, Araujo L. Case study research: opening up research opportunities. RAUSP Manage J. 2020;55(1):100–11.

Tight M. Understanding case study research, small-scale research with meaning. London: Sage Publications; 2017.

May C, Finch T. Implementing, embedding, and integrating practices: an outline of normalisation process theory. Sociology. 2009;43:535.

Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice. A consolidated framework for advancing implementation science. Implement Sci. 2009;4.

Pawson R, Tilley N. Realist evaluation. London: Sage; 1997.

Dreischulte T, Donnan P, Grant A, Hapca A, McCowan C, Guthrie B. Safer prescribing - a trial of education, informatics & financial incentives. N Engl J Med. 2016;374:1053–64.

Grant A, Dreischulte T, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: active and less active ingredients of a multi-component complex intervention to reduce high-risk primary care prescribing. Implement Sci. 2017;12(1):4.

Dreischulte T, Grant A, Hapca A, Guthrie B. Process evaluation of the Data-driven Quality Improvement in Primary Care (DQIP) trial: quantitative examination of variation between practices in recruitment, implementation and effectiveness. BMJ Open. 2018;8(1):e017133.

Grant A, Dean S, Hay-Smith J, Hagen S, McClurg D, Taylor A, Kovandzic M, Bugge C. Effectiveness and cost-effectiveness randomised controlled trial of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL (Optimising Pelvic Floor Exercises to Achieve Long-term benefits) trial mixed methods longitudinal qualitative case study and process evaluation. BMJ Open. 2019;9(2):e024152.

Hagen S, McClurg D, Bugge C, Hay-Smith J, Dean SG, Elders A, Glazener C, Abdel-fattah M, Agur WI, Booth J, et al. Effectiveness and cost-effectiveness of basic versus biofeedback-mediated intensive pelvic floor muscle training for female stress or mixed urinary incontinence: protocol for the OPAL randomised trial. BMJ Open. 2019;9(2):e024153.

Steckler A, Linnan L. Process evaluation for public health interventions and research; 2002.

Durlak JA. Why programme implementation is so important. J Prev Intervent Commun. 1998;17(2):5–18.

Bonell C, Oakley A, Hargreaves J, VS, Rees R. Assessment of generalisability in trials of health interventions: suggested framework and systematic review. Br Med J. 2006;333(7563):346–9.

Article   CAS   Google Scholar  

Grant A, Treweek S, Dreischulte T, Foy R, Guthrie B. Process evaluations for cluster-randomised trials of complex interventions: a proposed framework for design and reporting. Trials. 2013;14(1):15.

Yin R. Case study research: design and methods. London: Sage Publications; 2003.

Bugge C, Hay-Smith J, Grant A, Taylor A, Hagen S, McClurg D, Dean S: A 24 month longitudinal qualitative study of women’s experience of electromyography biofeedback pelvic floor muscle training (PFMT) and PFMT alone for urinary incontinence: adherence, outcome and context. ICS Gothenburg 2019 2019. https://www.ics.org/2019/abstract/473 . Access 10.9.2020.

Suzanne Hagen, Andrew Elders, Susan Stratton, Nicole Sergenson, Carol Bugge, Sarah Dean, Jean Hay-Smith, Mary Kilonzo, Maria Dimitrova, Mohamed Abdel-Fattah, Wael Agur, Jo Booth, Cathryn Glazener, Karen Guerrero, Alison McDonald, John Norrie, Louise R Williams, Doreen McClurg. Effectiveness of pelvic floor muscle training with and without electromyographic biofeedback for urinary incontinence in women: multicentre randomised controlled trial BMJ 2020;371. https://doi.org/10.1136/bmj.m3719 .

Cook TD. Emergent principles for the design, implementation, and analysis of cluster-based experiments in social science. Ann Am Acad Pol Soc Sci. 2005;599(1):176–98.

Hoffmann T, Glasziou P, Boutron I, Milne R, Perera R, Moher D. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348.

Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? Br Med J. 2004;328(7455):1561–3.

Grant A, Dreischulte T, Treweek S, Guthrie B. Study protocol of a mixed-methods evaluation of a cluster randomised trial to improve the safety of NSAID and antiplatelet prescribing: Data-driven Quality Improvement in Primary Care. Trials. 2012;13:154.

Flyvbjerg B. Five misunderstandings about case-study research. Qual Inq. 2006;12(2):219–45.

Thorne S. The great saturation debate: what the “S word” means and doesn’t mean in qualitative research reporting. Can J Nurs Res. 2020;52(1):3–5.

Guest G, Bunce A, Johnson L. How many interviews are enough?: an experiment with data saturation and variability. Field Methods. 2006;18(1):59–82.

Guest G, Namey E, Chen M. A simple method to assess and report thematic saturation in qualitative research. PLoS One. 2020;15(5):e0232076.

Article   CAS   PubMed   PubMed Central   Google Scholar  

Davidoff F, Dixon-Woods M, Leviton L, Michie S. Demystifying theory and its use in improvement. BMJ Qual Saf. 2015;24(3):228–38.

Rycroft-Malone J. The PARIHS framework: a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;4:297-304.

Kislov R, Pope C, Martin GP, Wilson PM. Harnessing the power of theorising in implementation science. Implement Sci. 2019;14(1):103.

Cresswell JW, Plano Clark VL. Designing and conducting mixed methods research. Thousand Oaks: Sage Publications Ltd; 2007.

Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43:267–76.

Craig P, Ruggiero E, Frohlich KL, Mykhalovskiy E, White M. Taking account of context in population health intervention research: guidance for producers, users and funders of research: National Institute for Health Research; 2018. https://www.ncbi.nlm.nih.gov/books/NBK498645/pdf/Bookshelf_NBK498645.pdf .

Download references

Acknowledgements

We would like to thank Professor Shaun Treweek for the discussions about context in trials.

No funding was received for this work.

Author information

Authors and affiliations.

School of Nursing, Midwifery and Paramedic Practice, Robert Gordon University, Garthdee Road, Aberdeen, AB10 7QB, UK

Aileen Grant

Faculty of Health Sciences and Sport, University of Stirling, Pathfoot Building, Stirling, FK9 4LA, UK

Carol Bugge

Department of Surgery and Cancer, Imperial College London, Charing Cross Campus, London, W6 8RP, UK

You can also search for this author in PubMed   Google Scholar

Contributions

AG, CB and MW conceptualised the study. AG wrote the paper. CB and MW commented on the drafts. All authors have approved the final manuscript.

Corresponding author

Correspondence to Aileen Grant .

Ethics declarations

Ethics approval and consent to participate.

Ethics approval and consent to participate is not appropriate as no participants were included.

Consent for publication

Consent for publication is not required as no participants were included.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Grant, A., Bugge, C. & Wells, M. Designing process evaluations using case study to explore the context of complex interventions evaluated in trials. Trials 21 , 982 (2020). https://doi.org/10.1186/s13063-020-04880-4

Download citation

Received : 09 April 2020

Accepted : 06 November 2020

Published : 27 November 2020

DOI : https://doi.org/10.1186/s13063-020-04880-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Process evaluation
• Case study design

ISSN: 1745-6215

• Submission enquiries: Access here and click Contact Us
• General enquiries: [email protected]

Study Design 101

• Helpful formulas
• Finding specific study types

Case Report

• Meta- Analysis
• Systematic Review
• Practice Guideline
• Randomized Controlled Trial
• Cohort Study
• Case Control Study
• Case Reports

An article that describes and interprets an individual case, often written in the form of a detailed story. Case reports often describe:

• Unique cases that cannot be explained by known diseases or syndromes
• Cases that show an important variation of a disease or condition
• Cases that show unexpected events that may yield new or useful information
• Cases in which one patient has two or more unexpected diseases or disorders

Case reports are considered the lowest level of evidence, but they are also the first line of evidence, because they are where new issues and ideas emerge. This is why they form the base of our pyramid. A good case report will be clear about the importance of the observation being reported.

If multiple case reports show something similar, the next step might be a case-control study to determine if there is a relationship between the relevant variables.

• Can help in the identification of new trends or diseases
• Can help detect new drug side effects and potential uses (adverse or beneficial)
• Educational – a way of sharing lessons learned
• Identifies rare manifestations of a disease

Disadvantages

• Cases may not be generalizable
• Not based on systematic studies
• Causes or associations may have other explanations
• Can be seen as emphasizing the bizarre or focusing on misleading elements

Design pitfalls to look out for

The patient should be described in detail, allowing others to identify patients with similar characteristics.

Does the case report provide information about the patient's age, sex, ethnicity, race, employment status, social situation, medical history, diagnosis, prognosis, previous treatments, past and current diagnostic test results, medications, psychological tests, clinical and functional assessments, and current intervention?

Case reports should include carefully recorded, unbiased observations.

Does the case report include measurements and/or recorded observations of the case? Does it show a bias?

Case reports should explore and infer, not confirm, deduce, or prove. They cannot demonstrate causality or argue for the adoption of a new treatment approach.

Does the case report present a hypothesis that can be confirmed by another type of study?

Fictitious Example

A physician treated a young and otherwise healthy patient who came to her office reporting numbness all over her body. The physician could not determine any reason for this numbness and had never seen anything like it. After taking an extensive history the physician discovered that the patient had recently been to the beach for a vacation and had used a very new type of spray sunscreen. The patient had stored the sunscreen in her cooler at the beach because she liked the feel of the cool spray in the hot sun. The physician suspected that the spray sunscreen had undergone a chemical reaction from the coldness which caused the numbness. She also suspected that because this is a new type of sunscreen other physicians may soon be seeing patients with this numbness.

The physician wrote up a case report describing how the numbness presented, how and why she concluded it was the spray sunscreen, and how she treated the patient. Later, when other doctors began seeing patients with this numbness, they found this case report helpful as a starting point in treating their patients.

Real-life Examples

Hymes KB. Cheung T. Greene JB. Prose NS. Marcus A. Ballard H. William DC. Laubenstein LJ. (1981). Kaposi's sarcoma in homosexual men-a report of eight cases. Lancet. 2(8247), 598-600.

This case report was published by eight physicians in New York city who had unexpectedly seen eight male patients with Kaposi’s sarcoma (KS). Prior to this, KS was very rare in the U.S. and occurred primarily in the lower extremities of older patients. These cases were decades younger, had generalized KS, and a much lower rate of survival. This was before the discovery of HIV or the use of the term AIDS and this case report was one of the first published items about AIDS patients.

Wu, E. B., & Sung, J. J. Y. (2003). Haemorrhagic-fever-like changes and normal chest radiograph in a doctor with SARS. Lancet, 361(9368), 1520-1521.

This case report is written by the patient, a physician who contracted SARS, and his colleague who treated him, during the 2003 outbreak of SARS in Hong Kong. They describe how the disease progressed in Dr. Wu and based on Dr. Wu’s case, advised that a chest CT showed hidden pneumonic changes and facilitate a rapid diagnosis.

Related Terms

Case Series

A report about a small group of similar cases.

Preplanned Case-Observation

A case in which symptoms are elicited to study disease mechanisms. (Ex. Having a patient sleep in a lab to do brain imaging for a sleep disorder).

Now test yourself!

1. Case studies are not considered evidence-based even though the authors have studied the case in great depth.

a) True b) False

2. When are Case reports most useful?

a) When you encounter common cases and need more information b) When new symptoms or outcomes are unidentified c) When developing practice guidelines d) When the population being studied is very large

← Previous Next →

© 2011-2019, The Himmelfarb Health Sciences Library Questions? Ask us .

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .

Transforming early intervention for families: a map of EIF case studies

25 Aug 2022

Our case study map provides quick and easy access to examples of innovation and good practice in early intervention for families by local authorities and partners across England and Wales. 

Case studies include the practical application of tools and research developed by EIF, and are intended to support local areas to use these in their own work.

The map covers three key areas of early intervention for families – early childhood services, relationships & parental conflict, and speech & language support – and you can filter results to focus on eight strands of local planning and delivery: strategy, workforce, partnerships, co-production, services & interventions, coordinated working, experience & outcomes, and evaluation.

Launch the map in a new window

EIF works with local areas and partners across England and Wales, including 85 different places in 2021/22.  

This map provides case studies on key issues in supporting families, drawn from work we've published over the past three years, and doesn't represent all the areas we've worked with or all the early intervention issues we focus on through our research and local support. 

See all EIF case studies

Share this page

Related content.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 10 May 2024

On-farm crop diversity, conservation, importance and value: a case study of landraces from Western Ghats of Karnataka, India

• G. M. Puneeth 1 ,
• Ravi Gowthami 2 ,
• Ashvinkumar Katral 3 ,
• Kerekoppa Manjunatha Laxmisha 1 ,
• Ramesh Vasudeva 4 ,
• Gyanendra Pratap Singh 2 &
• Sunil Archak 2  

Scientific Reports volume  14 , Article number:  10712 ( 2024 ) Cite this article

119 Accesses

1 Altmetric

Metrics details

• Natural variation in plants
• Plant breeding

Landraces are important genetic resources that have a significant role in maintaining the long-term sustainability of traditional agro-ecosystems, food, nutrition, and livelihood security. In an effort to document landraces in the on-farm conservation context, Central Western Ghat region in India was surveyed. A total of 671 landraces belonging to 60 crops were recorded from 24 sites. The custodian farmers were found to conserve a variety of crops including vegetables, cereals and pulses, perennial fruits, spices, tuber and plantation crops. The survey indicated a difference in the prevalence of landraces across the sites. A significant difference with respect to the Shannon-diversity index, Gini-Simpson index, evenness, species richness, and abundance was observed among the different survey sites. Computation of a prevalence index indicated the need for immediate intervention in the form of collecting and ex situ conservation of landraces of some crops as a back-up to on-farm conservation. The study also identified the critical determinants of on-farm conservation, including (i) suitability to regional conditions, (ii) relevance in regional cuisine and local medicinal practices, (iii) cultural and traditional significance, and (iv) economic advantage. The information documented in this study is expected to promote the collection and conservation of landraces ex situ. The National Genebank housed at ICAR-NBPGR, New Delhi conserves around 550 accessions of landraces collected from the Central Western Ghats region surveyed in this report. Information collected from custodian farmers on specific uses will be helpful to enhance the utilization of these accessions.

Similar content being viewed by others

State of ex situ conservation of landrace groups of 25 major crops

Diversity, trait preferences, management and utilization of yams landraces (Dioscorea species): an orphan crop in DR Congo

An increase in food production in Europe could dramatically affect farmland biodiversity

Introduction.

Plant genetic resources (PGR) are the foundation for crop improvement and global food security 1 . The genetic diversity of crop plants has been maintained by farming communities by cultivating landraces. Landraces are variously known as heritage, heirloom or primitive cultivars or folk and farmers’ varieties 2 . Villa et al. 3 defined landrace as “a dynamic population of a cultivated plant that has historical origin, distinct identity and lacks formal crop improvement, as well as often being genetically diverse, locally adapted and associated with traditional farming systems.” Due to the adaptive evolution, landraces constitute a reservoir of genes for nutritive value and tolerance to biotic and abiotic stresses 4 . Landraces play a significant role in maintaining the long-term stability of traditional agro-ecosystems.

Since landraces are lost due to genetic erosion 5 , 6 , genebanks around the world have captured their diversity in the form of 7.4 million germplasm accessions 7 . Although these ex situ collections have been providing the base material for crop improvement programs around the world, the material is no longer continuously adapting to changes in the environment, such as new races of pest or diseases, or major climatic changes. On the other hand, the population conserved on-farm continues to be dynamic in response to changes in local biotic and abiotic interactions as well as selection by custodian farmers thereby retaining its adaptation to the local environment and its distinguishing characteristics. In fact, landraces continue to exist “on farm” resulting in a traditional set up of “conservation by cultivation”. On-farm conservation has been defined as “the sustainable management of genetic diversity of locally developed traditional crop varieties, with associated wild and weedy species or forms, by farmers within traditional agricultural, horticultural or agri-silvicultural cultivation systems” 8 . Advantage of adaptive evolution offered by landraces, such as locally adapted alleles and allele complexes, exist only under on-farm conservation 9 .

Researchers have been endeavoring to document the on-farm conservation activities in many parts of the world 10 , 11 , 12 , 13 , 14 . However, insufficient documentation, inadequate transfer of ethnobotanical relevance from generation to generation, lack of interest among younger generations and inefficient policy intervention have led to and poor conservation and inadequate exploitation of landraces in plant breeding.

Western Ghats are a chain of mountains lying along the western coast of peninsular India for a length of 1600 km with an average elevation of 1500 m above mean sea level covering Gujarat, Maharashtra, Goa, Karnataka, Kerala and Tamil Nadu 15 . Western Ghats in India form one of the 34 biodiversity hotspots in the world 16 . As one of the four biodiversity hotspots in India, Western Ghats are home to 5000 angiosperm species, of which 34% are endemic 17 . The region is a primary center of origin for many crop species and houses a vast diversity of cultivated and wild crop plants. In addition to diverse flora and fauna, the Western Ghats are also native to diverse social, religious, cultural and linguistic groups. The crop biodiversity of the Western Ghats region has been documented previously by Asha et al. 18 ; Gajanana et al. 19 and Ramachandra et al. 20 .

Landraces should not be perceived merely as farmers’ cultivars that are reservoirs of useful traits. Landraces in an on-farm setup also include components of cultural landscapes and conservation agriculture and vistas to new market opportunities. Therefore, standalone inventorization of material becomes an inadequate exercise of documentation. On the other hand, documentation of the landraces in the on-farm conservation context provides insights into food-systems and sustainability. The present study was conducted in the context of the Central Western Ghats region of Karnataka state of India (i) to document the on-farm landrace diversity and conservation practices and (ii) to determine factors affecting on-farm conservation practices.

Methodology

Sampling methods and data collection.

The survey to document the on-farm conservation was conducted in Central Western Ghats covering four districts viz ., Uttara Kannada, Shivamogga, Dakshina Kannada and Belagavi of Karnataka state in the southern part of India (Fig.  1 ). Initially, basic information regarding the distribution and type of crops, landraces and custodian farmers were collected by communicating with the resource-rich persons, local Krishi Vigyan Kendras (KVKs), agriculture colleges, non-government organizations and Protection of Plant Varieties and Farmers' Rights Authority, (PPV&FRA) New Delhi.

Map showing the survey area ( a ) India map highlighted with the survey state ( b ) Karnataka map with four survey districts (highlighted with different colours).

With the gradual replacement of landraces and changes in cropping pattern, there is a significant reduction in the on-farm conservation sites where landraces are cultivated. In such a scenario, where participants are difficult to locate, surveyors have used snowball sampling techniques 21 . We also used this nonprobability technique to identify potential on-farm conservation sites. Snowball sampling or chain referral sampling is a useful tool for analyzing rare instances or in our case, unexposed conservation sites. Resource persons from local KVKs and/or other organizations provided information about farmers conserving landraces in the respective jurisdiction.

The survey was conducted from November 2020 to November 2021 to record the landrace diversity of different crops and their status. Based on the initial information obtained, farm households/farmers engaged in landrace conservation or mainly practicing low-input organic agriculture were pooled for information. The data was collected using socio-metric survey with snowball sampling technique 22 where, custodian farmers were identified through data provided by other fellow farmers. A simple questionnaire was used to conduct the survey, which was based on an interview method and field observations. Farmers’ fields were visited to record the details of landraces and factors determining their conservation and use. Farmsteads were mapped based on geo-coordinates. Each landrace was recorded with botanical name, common name and the local name. Further, the farmers were also interviewed to obtain personal information, their farm details (landrace cultivation area, total farm area, cropping pattern), cultivation constraints, reasons for not growing landraces, community knowledge of biodiversity conservation and on-farm conservation, present and past use of landraces, traditional and cultural uses associated with the landraces, etc. Efforts were made to document the purpose of cultivating each crop species based on the total economic value (Fig.  2 ). In the entire exercise, we made sure to comply with relevant institutional, national, and international guidelines and legislations while documenting of on-farm conservation of landraces.

The nature of total economic value of landraces.

Features of the study area

The Western Ghats Region in Karnataka, locally known as Malenadu lies between 12° 2′ 7″ N and 15° 44′ 46″ N latitude and 74° 14′ 3″ E and 75° 76′ 17″ E longitude. The Central Western Ghats Region extends through an area of 37,000 km 2 covering areas of Chamarajanagara, Mysuru, Kodagu, Dakshina Kannada, Udupi, Hassan, Chikkamagaluru, Shivamogga, Uttara Kannada and Belagavi districts. The climate in the Western Ghats varies with the altitudinal gradation and distance from the Arabian Sea coast. The climate is humid and tropical in the lower reaches tempered by the proximity to the sea. Mean temperatures range from 20 °C (68°F) to 24 °C (75°F). The area is plain and the climate is humid along the west coast; thick forest and hilly area in the center; and tropical monsoon climate with undulated area towards the east side. Agricultural land in this region comprises of a variety of soil types including red soils, laterite soil, black soils and humid soils. Rice, spices, areca nut, jackfruit, cashew nut and sugarcane crops cover majority of the agricultural land area 23 , 24 . Weather parameters, land utilization and cropping pattern of the surveyed area are given in Table 1 .

Statistical analysis

The data on the status of on-farm conservation and management in the region was gathered from multiple survey visits over a period of one-year. The data was analyzed by considering the cultivation of landraces across different districts in order to determine diversity and distribution. The following indices were computed to interpret the collected data:

where H = Shannon diversity index; P i  = Proportion of individuals of ith species in a whole community 25 .

where n = Individuals of a given type/species; N = Total number of individuals in a community.

The Gini-Simpson index (or Simpson's index of diversity) measures the probability that the two randomly selected individuals belong to different species 26 .

where n i  = Number of individuals in the i th species; and N = Total number of individuals in the community.

Informed consent

The authors have obtained the consent from the custodian farmers and they are aware of the intended publication of information and images of the same.

Results and discussion

Landrace diversity, inventory of landraces and prevalence.

Landraces were documented from the Central Western Ghats spanning four districts of Karnataka. Snowball sampling allowed us to reach 24 farmsteads during the study. A total of 671 unique landraces belonging to 60 different crops were documented. These landraces belonged to fruits (8 crops, 181 landraces), vegetables (9 crops, 54 landraces), spices (8 crops, 40 landraces), pulse crops (4 crops, 15 landraces), plantation crops (3 crops, 11 landraces), tuber crops (4 crops, 32 landraces) and few miscellaneous plants (23 crops, 24 landraces). Rice was the only cereal crop documented in the region. However, out of the 671 unique landraces documented, 314 belonged to rice. Rice and fruits together accounted for three fourths of the total landraces documented.

The survey indicated that farmsteads with on-farm conservation and custodian farmers involved in cultivation of landraces were few and far between. It was clearly observed that even farmers immediately neighboring the on-farm farmsteads were not cultivating landraces. If a given landrace is cultivated at more than one on-farm conservation site, there is greater probability of continuity of conservation and availability. In other words, it is important to understand the prevalence of a landrace during the documentation process.

Based on the nomenclature synonymy of landraces, a Prevalence Index (PI) for on-farm conserved landraces was assessed as:

where O is the occurrence frequency of landraces in the surveyed area; N is the number of landraces recorded; n is the number sites surveyed.

Out of 671 landraces documented during the survey, it was found that as many as 243 landraces were cultivated by two or more custodian farmers. It was observed that frequency of occurrence ranged from 1 to 7 (Table 2 ). The prevalence index was highest for banana (1.95). The PI of rice was 1.53 followed by chili (1.5), jackfruit (1.35), mango (1.33), ridge gourd (1.28), pepper (1.26), lab-lab bean (1.25) and brinjal (1.06) (Table 2 ). Meanwhile, for remaining 28 crops the value of PI was 1, showing that only one occurrence per landrace was recorded across the survey area.

A solitary farmstead practicing on-farm conservation of local landraces in a given area is matter of pride as well as concern. A family practicing on-farm cultivation of traditional cultivars singlehandedly contributes, as the custodians, towards conservation and perpetuity of landrace diversity. On the other hand, discontinuation of this practice by the custodian family due to any reason could possibly lead to irreversible loss of the landraces. Germplasm explorations by collectors may need to urgently focus on such landraces to ensure that these are conserved ex situ. Stakeholders may also look at promoting seed exchange among the farming community in order to increase their chance of “conservation by use”.

Importance of landraces

The landraces across farmsteads were documented to be conserved for their specific uses such as culinary purposes in raw, cooked, pickled and processed form, medicinal value or multiple uses (Table 3 ).

Fruit landraces were found growing on the bunds, near the roadside, on marginal lands, in kitchen gardens and in between plantation crops in the fields mainly for personal use and not as commercial crops. Mango landraces have been conserved for their use in making pickles 27 . One of the unique landraces of mango to this region, Appemidi (used for pickle preparation only) has got Geographical Indication (GI) for Shivamogga and Uttara Kannada region. Neeru Kukku, a special landrace of Dakshina Kannada, can be soaked in salt water for 1–2 years without damaging the quality (named for this quality) and is also used in making huli for sambar (adds aroma and sour taste to sambar instead of tamarind) (Fig.  3 a). This area also accommodates huge diversity of jackfruit, kokum and banana landraces than any region, owing for their adaptation and good growing conditions in the region. In jackfruit, soft fruited ( Biluva or Ambli types) varieties are used in preparation of local cuisines ( idli, kadubu ) and hard fruited ( bakke type) landraces are used in the production of processed food products such as paapad and chips (matured fruit before ripening); and is also consumed as a fruit. Mankale Red , a landrace of jackfruit from Mankale, a place in Sagara (Shivamogga) has red fruits in which both rags and tendrils have a pleasant flavor when consumed (Fig.  3 b). Banana landraces like Elakki Baale , Mitli; pineapple landrace like Ananus Local and jamun landraces are used as dessert fruit; Hoo Baale (flowers), Kadhali/Deva Baale (flowers), Sakkare Baale are used as vegetable (Fig.  3 c). Landraces like Betta Baale (pseudo-stem is used for kidney stones) and Kallu Baale (used to treat kidney stones and grows on stones in the hilly regions) have medicinal properties. Meanwhile, some kokum landraces like Uppage Local (making huli for sambar, extract oil from seeds, juice preparation), Muruga Huli and Punarpuli (Fig.  3 d) are well-maintained. Vasugi et al. 27 assessed the diversity and morphological variations of Appemidi mango varieties in the same region. In the same way, Pradeep et al. 28 observed cultivation of many landraces (> 20 each) in native crops such as mango and banana in Kerala, which is adjacent to our study area.

( a ) Neeru Kukku, a local mango landrace, ( b ) farmer with original tree of ‘ Mankale Red’ a jackfruit variety, ( c ) Elakki Baale, grown in a farmer field, ( d ) A farmer with landrace, Bili Murugalu , of kokum.

Spices are high-value crops with large-scale export potential 29 . As a result, collecting and preserving spice crop germplasm is critical. In this regard, the Central Western Ghats holds a vast diversity of spice landraces for yield traits, resistance to disease and pests than improved varieties. Karimunda (spicier, good keeping quality i.e., 50–100 years), Vakkalu , Gejje Hipli , Malligesara (good yield potential) (Fig.  4 a) and Thekkam Bunch Pepper (Fig.  4 b) are few landraces of pepper 30 . Maavina Kaayi Arishina and Kukku Shunti were fragrant type landraces in turmeric and ginger respectively. While Jawari Arishina (turmeric) (Fig.  4 c) contains more curcumin (7–8%) content and Jawari Shunti (ginger) is spicier and more pungent than the released varieties as per the farmers’ knowledge. Nutmeg used for preparation of ayurvedic formulations. Highly fragrant landrace of clove and some cardamom landraces like Naati Yelakki (Fig.  4 d) , Kilara, Lambodi Thali and Gundu Kaalu were recorded during the survey. Saji et al. 31 reviewed the conservation aspects and cultivar diversity of different spices of Western Ghats in particular and India in general.

Representative photos of the spice landraces observed in the survey area ( a ) Malligesara, landrace of pepper, ( b ) Thekkam Bunch Pepper , landrace of pepper, ( c ) Jawari Shunti, landrace of turmeric ( d ) Naati Yelakki, landrace of cardamom.

Plantation crops

Plantation crops like areca nut, coffee, coconut and betel vine are the major crops grown in the region, hence their landraces too. Betel wine landraces namely, Kasaravalli, Lakkavalli, Naagavalli and Panchavalli are recorded during the study with limited information of the same regarding their importance and characteristics 31 . Areca nut landraces like Sonda and Dodda Adike are reported from the locality. Tiptur Local is a famous variety of coconut grown all over the Karnataka state.

Though the climate and weather of the Central Western Ghats region restrict the commercial cultivation of vegetables, many farmers grow landraces of different vegetables which are adapted to the locality’s soil and climate (Fig.  5 ). They grow, maintain, promote and preserve these landraces by harvesting the matured fruits for seeds to sow in the next generation. These landraces develop special traits over the years for its climate and soil conditions. Brinjal landrace Udupi Mattu Gulla has very thin skin and small spines on the fruit surface 32 . It has a unique taste and virtually gets dissolved while cooking and is also less astringent and less bitter when compared to other varieties of brinjal, and has got GI tag in the Udupi region, and Marabadane/Kudane (bacterial wilt resistance/used grafting) are among other brinjal landraces. Landraces of okra conserved are Bahuvarshika Bende (perennial), Aane Kombu Bende (very long fruits), Sunkada Bende (fruit contain protective hairs) and Entugere Bende (8 ridged fruit, large size) (Table 2 ). Cucumber is one of the important vegetables and many dual-purpose cucumber landraces are conserved. Aane Mottu/Hegge Southe (red pulp, pumpkin size, sweet taste), Aati Southe (in rainy season), Ibbudla (juice making), Neeru Southe (waterier content, grown near canals), Oddu Southe (for summer season, bitter pulp) are some of the landraces conserved on-farm. Landraces of other crops includes Sihi Haagala (bitter gourd) vegetable with no bitterness used by diabetes patients, Sooji Menasu (chili pepper) with spicy richness used for preparation of dishes and also act as a pain killer and a coolant. Likewise, Latha et al. 32 recorded and documented different vegetables and their landrace diversity in the Western Ghats region of India. In a comparable manner the landrace variety of vegetables in the Italian Pugglia region was documented by Conversa et al. 11 .

Representation of different landraces of vegetables ( a ) Chili, ( b ) Brinjal, ( c ) Ridge gourd, ( d ) Pumpkin.

Tubers are important for food and nutrition security, as well as adaptation to climate change. Among the tuber crops, Kunabi Mudli (taro) (3ft long, big size, soft after boiling) is used in the preparation of patrode (a local dish). Most of the tuber landraces viz ., Bili Genasu, Thuppada Genasu, Kempu Genasu, Nagar Cone and Taambde Cone (yam) are used as vegetable and making sambar. Some tubers like Chirike (highest vitamins) (yam) and Taikilo (taro) (immunity booster and healing of wounds as antiseptic) are also used for their medicinal properties. In a special case of Kunabi tribes in the Joida area of Uttara Kannada, different tuber crops and their wild relatives were documented. Some of the documented landraces are also mentioned by Asha et al. 18 . Similar documentation work in tuber crops was also conducted by Alwis et al. 10 in Sri Lanka.

Cereals (Rice)

Rice is the staple crop of the region. Despite having many adversaries, many farmers are indulging themselves in cultivating, maintaining, promoting hundreds of rice landraces, which are very well adapted to the region’s climate and other agro-ecological factors. Some of the interviewed farmers are preserving hundreds of landraces because of their passion for conservation, market value, and in order to maintain the legacy of their ancestors. Some of the famous landraces maintained even now in the region are Nereguruli Batta (thrives in submergence for 40 days), Rajamudi (high tillering ability, organic cultivation, kernels are red and white rice type with soft rice, good for diabetes and was once patronized by kings of Mysuru Wodeyars), Kayime and Kutti Kayime (red seed kernel, high fodder yield, rabi season variety) and Kempu Hasudi (higher yield; resistant to diseases; red grains with good taste). Puffed rice ( Adnen Kelti , Bili Hegge ), medicinal value for humans and livestock diseases ( Athikaraya, Chitaga ), for making sweets ( Bile Aloorsanna ), for dose and idli ( Mallige Sanna ), aromatic rice and for making sweet dishes ( Gandasaale, Gulvadi Sanna, Indrani, Kaagi Saale ), red rice landraces to increase blood hemoglobin ( Hasudi, Hejje Batta ), can be grown in saline water ( Kagga Batta ), increases milk in lactating women, good for pregnant, more iron content ( Kare Gajuli ), good for snacks ( Mullare, Bili Halaga ), good for diabetes ( Rajamudi, Sorata ). Importance of rice landrace conservation and their characteristics are also highlighted by Rathi et al. 33 in Chhattisgarh region and by Agnihotri et al. 34 in Kumaon region of Uttarakhand.

People consume pulses as their side dish along with staple food. There were six type of pulses were documented during the survey. Only few farmers are growing the pulses though not as main crop but as intercrop or in bunds. Lab–lab bean, a crop mainly grown in southern India, has different morphological variation in each landrace ( Chapparada Avare, Matti Avare, Katti Avare, Chaturbuja Avare ). Same for Bengal gram ( Kempu Kadale, Hasiru Kadale, Kappu Kadale ) and cowpea ( Kappu Halasande, Kempu Halasande, Bannada Halasande ) has variation in colour of the seeds and pods. Immature pods and leaves of some pulses use as vegetable.

Miscellaneous crops

India is known for traditional medicine system since ancient times. Thus, significance of the medicinal plants is known as part of Indian codified medicinal systems like Ayurveda as well as indigenous traditional knowledge about the medicinal uses by the community. In our survey, some plants were recorded for their multipurpose utility including medicine. The people in the region were found to treat various ailments since generations using local plants including Kalmegh ( Jeerad Kaddi ) for fever, Basella’s ( Basale Soppu ) leaves as coolant, wild purslane ( Golisoppu ) as leafy vegetable to increase hemoglobin, etc. Other popular plants included Malabar tamarind, Wild coriander, Indian coffee plum, Indian sorrel, Curry leaf, etc.

Diversity indices

Based on the landrace nomenclature, Shannon-diversity index (H), Gini-Simpson index (1-D), Evenness (E), Species richness (R), and Abundance (A) were assessed between the crop groups of different districts. Significant differences in these parameters among the four study areas were observed. Shannon diversity index (H) dictates how diverse the species in a given area. Higher the index, more diverse the nature of species in that habitat 35 . Among the study areas, Shannon diversity (H) of Uttara Kannada (H = 2.01) was highest, followed by Shivamogga (H = 1.85), Dakshina Kannada (H = 1.61), and Belagavi (H = 1.3) had limited landrace diversity (Fig.  6 ). The value of Gini-Simpson's index (1-D) reflects how many different types of species are in a community and how evenly each species is distributed. Similarly, Uttara Kannada showed more diversity in landraces with a Gini-Simpson index value of 0.77 and Belagavi showed very less diversity with a value of 0.56. In terms Gini-Simpson index, Belagavi and Dakshina Kannada exhibited comparable diversity levels with values of 0.56 and 0.59, respectively (Fig.  6 ). Ocimati et al. 15 assessed the same for Musa cultivars in Rwanda and found a lower diversity index, which was prone to genetic erosion. Species evenness (E) is the measurement of the relative abundance of different species. The species evenness ranges from zero to one, with zero signifying no evenness and one signifying complete evenness. Shivamogga had more diverse landraces than other districts in terms of Evenness (E) with a value of 0.5 followed by Dakshina Kannada (E = 0.556), Uttara Kannada (E = 0.591) and Belagavi (E = 0.625) (Fig.  6 ). The highest species richness (R) was observed in Shivamogga (40) followed by Uttara Kannada (29), Dakshina Kannada (18) and Belagavi (7) (Fig.  6 ). The current study revealed that Uttara Kannada and Shivamogga had more landrace diversity for their practice of sustenance farming in remote areas, use of landraces in local food systems, traditional and cultural links. While, Belagavi had less diversity in all the terms due to various probable reasons like commercialization of agriculture, use of more improved and hybrid varieties and, so on.

Diversity and distribution assessment of landraces using SDI = Shannon Diversity Index, SiDi = Gini-Simpson Index, E = Evenness. A = Abundance, R = Richness for four surveyed districts.

On-farm conservation and management

Socio-economic characteristics of custodian farmers.

Among the 24 sites of on-farm conservation, three farmers possessed more than 10 ha farm land, 12 had medium sized farms (2–10 ha), whereas nine were found to have small farms (< 2 ha). Landraces of field crops (rice and pulses) along with vegetables and tuber crops were found conserved mainly on the small farms. On the other hand, perennial species including fruits, spices and plantation crops were in medium to large farms. Among the 24 on-farm conservation sites, the age of the custodian farmers ranged from 35 to 75 years; majority (12) were in the age bracket of 40–60 years. The two young farmers (< 40 years) actively engaged in on-farm conservation were exclusively involved in commercially attractive activity of maintaining the rootstocks of traditional cultivars of perennial crops (pepper, mango and jackfruit) popular for their adaptation and resistance to pests and diseases 36 .

However, it was starkly clear in our survey that younger generation of farmers is not inclined to engage in on-farm conservation. It was evident from the interaction with farmers that traditional knowledge accumulated over the years with the experienced farmers could be in danger of not finding next-generation custodians.

From the current study, it was also found that majority of farmers are conserving many landraces dedicated to few crops instead of single landrace covering entire farming area. Few farmers conserve rice landraces, by growing most of the landraces in a 10 m 2 area in order to maintain and preserve their self-interest and passion for conservation; and only a few landraces are grown in a large area because of their potential use. For example, Nereguruli Batta (rice) tolerates submergence condition up to 40 days in Shivamogga district. Fruits were grown majorly as an intercrop with spice and plantation crops. Some vegetables, fruits (mango and jackfruit) and miscellaneous crops were well-looked-after and maintained in home gardens for their use in preparing traditional dishes (mango-pickle, jackfruit-chips, idli ) and traditional medicine. The marketing of the farm produce is distinct for different farmers based on the reason for cultivation. Many farmers grow landraces mainly for their personal use and are part of local food systems (pickling varieties of mango and vegetable landraces). Few landraces have cultural and traditional importance along with some quality traits that enjoy demand in local markets and weekly fairs (jackfruit varieties, local tuber crops). Very few landraces have demand in the countrywide market for their nutritional and medicinal importance ( Navara and Ambe Mohar in rice).

Custodian farmers in the study area are majorly residing in remote villages and villages located in the vicinity of the forest. It was found that, the nearest proper road was 20–25 km away from the on-farm conservation sites. In the absence of market attraction, these farmers were found to cultivate landraces mainly for home consumption. At best, grains are sold at the local weekly bazaars and seeds are exchanged with fellow farmers and relatives. In exceptional cases, farmers who reside nearer to markets of nearby towns (< 10 km) were observed to get good prize for their produce.

Farmers generally designate local landraces names after specific characteristics. The appearance of seed and kernel, crop plants, taste, aroma, maturity, plant size, use and growing conditions are all crucial factors in determining a landrace name 33 , 37 . The same pattern of use/characteristics and other features are used to name traditional landraces/cultivars in the Central Western Ghats. It was observed that number of farmsteads having on-farm conservation activity was very less. Farmers belonging to post-green revolution era (born after 1970’s), tend to cultivate high yielding modern varieties with a focus on enhanced income generation. In the absence of formal documentation, the only source of information about names of landraces and their specialty uses is the senior farmers belonging to age bracket of 60 and above.

Determinants of on-farm conservation

Farmers have indulged in selecting, growing and maintaining landrace biodiversity within and among the crops in their fields and community seed banks from generation to generation. Farmers were well aware of the benefits of local cultivars, which includes high market value 10 , adaptation to adverse weather conditions, good eating quality, lodging resistance, resistance to pests and diseases, low production costs, and a consistent yield 14 . Scientific investigations in rice provided some insights into the utility of the landraces in crop improvement programs 38 , 39 . The Central Western Ghats is a partial forest area and few farms under this study are located in the vicinity of the forest area (Joida, Sagara, Sirsi, and Thirthahalli). The region receives heavy rainfall during monsoon (June–September) and a good amount of groundwater facility enhances agriculture in the region. Agriculture practiced in the region is mainly rainfed with few exceptions. This has led farmers to follow organic farming with fewer inputs, which indirectly chooses the local traditional varieties for their adaptation to the local environment for generations 14 .

From the present survey, it was observed that one set of farmers conserved landraces on-farm with sound knowledge on importance of landrace and conservation (direct conservation), while another set of farmers conserved landraces for their food and other needs without any knowledge on importance of landrace and awareness of conservation (indirect conservation). Special mention for the Kunabi tribes from the Joida area of Uttara Kannada, for their cultivation of unique rice and tuber crops’ landraces in marginal land, forest land and kitchen gardens for the sake of family sustenance and tradition without knowing the actual importance of these landraces in national plant genetic resources system. The distribution of tuber crop landraces in the area follows the ideal environment for their growth and development, such as soils, precipitation, elevations and drainage are in line with the results reported by Alwis et al. 10 in Sri Lanka. This shows that many farmers were not aware of the concept of biodiversity conservation and on-farm germplasm conservation. However, they have contributed to on-farm germplasm conservation without their theoretical knowledge and awareness of germplasm conservation.

Though the economic benefit is the major driver of conservation 12 , 40 , non-economic factors like prestige for being the owner of diversity 12 , exchange of specific landraces and their products with neighbors, relatives and family friends 22 are among the others which motivate farmers to engage in on-farm conservation. Landraces/traditional folk varieties are also conserved because of their adaptability to agro-climatic conditions viz., higher rainfall in the western side of the Western Ghats (adaptation of rice landraces for rainfed condition ( Kayime ), Rabi season ( Kutti Kayime ) in Dakshina Kannada, low fertility of forest soils in Uttara Kannada (tuber crops in Joida) 18 and Shivamogga. Socio-economic conditions including fragmented land, limited availability of inputs, poor financial condition of farmers and tolerance to biotic and abiotic stresses (e.g., Anthara Saale for drought and weed tolerance in rice; Marabadane/Kudane for bacterial wilt resistance in brinjal) have motivated farmers to cultivate landraces. Similarly, unique biological traits such as—size in Mituga banana, colour 41 in red rice Kempu Sanna , flavor 14 in Adderi Jeerige (mango) and Kothambri Saale (rice) and/or specific use viz., pickles in case of mango 27 and Ibbudla (cucumber) for making juice; preparation of traditional meals viz., sweet dish, puffed rice, kaayi kadubu , subzi, patrode 18 , 42 also motivate farmers to conserve and promote conservation of landraces.

Many of the landraces are associated with the traditions and cultural practices of the communities. As a result, ethnic traditional cultural practices and customs play an important role in the preservation of traditional variations and crop genetic diversity on farms. Hence, conserving traditions indirectly helps to conserve landraces 43 . These motivations are in line with the results of Gajanana et al. 19 in India and Alwis et al. 10 in Sri Lanka. The present study is coherent with the results regarding the determinants/factors highlighted by Sthapit et al. 13 . Considering the high level of diversity among custodian farmers and improving their ties with other members of the community can result in on-farm agro-biodiversity conservation in situ 13 , 19 .

Exchange of conserved material

The seed exchange takes place between individuals or families inside the community or between close communities 12 . Seed flow occurs through purchase of seeds from inside or beyond the community mainly in bio-diversity fairs and seed melas , as well as seed borrowing from relatives and fellow farmers. These exchanges and borrowings occur in the study region for a variable number of reasons, including—lack of seed of a particular variety or landraces in the market; a desire to replace poor-quality seeds from old lots and seasons which may have poor germination; an interest in growing better cultivars by seeing other farmers' fields; a desire to test a different landrace/folk variety in search of suitable landrace to replace the existing one for specific land suitability; and exchanging seeds of one landrace for the seeds of different landrace. From the interaction, it was observed that, seed exchange among the farming communities is in practice for several years, which in turn increases the diversity in the farmers’ field and indirectly conserves the specific landrace. Normally, the custodian farmers have a practice of collecting and storing the seeds for the next growing season, contributing to the maintenance of the crop diversity. Custodian farmers are farmers (men and women) who actively maintain, adapt and promote agricultural biodiversity and related knowledge at farm and community levels over an extended period of time, and are recognized by community members for doing so 13 . Often, custodian farmers do not act alone, but rather are actively supported in their efforts by family or household members. These features of seed conservation and exchange were discussed by Conversa et al. 11 in vegetable landraces conservation in the Puglia region of Italy. Similar kind of exchange was also found in our study area, where a collective exchange happens during events such as local markets or traditional ceremonies where a group of farmers or communities from different parts of the state exchange seeds through purchase or barter system. Furthermore, certain farmers in the study area had a great knowledge on the importance of conservation of landraces and they played an integrated role in motivating/encouraging/involving other fellow farmers in conservation. While interacting with a custodian farmer from Belagavi, who maintains a large number of landraces, we found that he distributed two to five landraces to interested farmers with a motive to increase area under landrace cultivation and to help the fellow farmers to sustain during difficult times. Thus, custodian farmers play a significant role in the seed flow and they are the main source of seeds in the region. Few farmers from Shivamogga develop nursery for sale of landraces, mainly pepper, jackfruit and pickling varieties of mango 36 . Through germplasm movement, these farmers are developing a dynamic process of diversity on their farms 22 . Increased cultivation of landraces achieved through seed exchange within and between communities, diversity fairs, and public awareness of the importance of landraces improves their use and conservation 34 .

Total economic value

Landraces are an essential component of agro-biodiversity conservation due to their direct and indirect benefits to farmers. The farmers conserve landraces to improve the sustainability of food, fuel, medical care and for future. Farmers value the landraces based on the importance. Poudel and Johnsen 44 summarized the total economic value of crop landraces as inclusion of both use value (direct use value, ecological function value, and option use value) and non-use value (existence value and bequest value). Economic valuation of landrace diversity is essential to generate information and knowledge for resource allocation to identify least cost strategies to conserve landraces diversity 45 . Among several species of landraces conserved on-farm, 76.67% (46 species) landraces are conserved for both use and non-use values (Fig.  7 ). The landraces of the majority of the species (46) are conserved on-farm for either existence or/and bequest value in addition to direct use value, ecological function value and option value, highlights the farmers involvement in conservation of landraces for benefit of others in current and future generations.

Total economic value of landraces of different species conserved on-farm by the farmers of Western Ghats.

Constraints for conservation

The custodian farmers are doing their best to maintain, manage and promote the local varieties and landraces through seed exchange. In addition, they are passing traditional knowledge about these cultivars and disseminating their importance among their fellow farmers within and outside their community 13 . Unfortunately, the genetic diversity of landraces is rapidly diminishing in various parts of the world for a variety of reasons. This fact has also been supported by Hammer and Teklu 46 citing the introduction of high yielding varieties (HYVs) leading to replacement of landraces/traditional folk varieties.

Conservation constraints are broadly divided into agro-ecological, socio-economic and technical aspects. Flooding, drought, rainfall during harvest, landslides, poor soil quality and abnormal weather are the main agro-ecological constraints. Whereas socio-economic constraints include inadequate input, lack of availability of seed material, poor yield, lack of marketing facilities, deterioration of culture and traditions, lack of awareness of conservation, adoption of HYVs, lack of interest among young people and their migration to urban areas, the non-multiplication of seeds by the family and poor knowledge transfer. Constraints of technical cultivation comprises of pest and disease infestation, labor scarcity, improper storage conditions and poor germination rate 14 . The Western Ghats encompass hilly area covered with forests that receive heavy rainfall deteriorating soil conditions, by erosion, flooding, submergence of fields, landslides that are common intimidations for cultivar conservation. Based on the information obtained from custodian farmers, the loss of many landraces over the years due to the frequent occurrence of natural calamities in the study area was highlighted. They also emphasized that, lack of prevalence with different farmers in many landraces, they are unable to protect those landraces. Socio-economic conditions make a huge impact on conservation and are the core threatening factors for the conservation of landraces. The practice of adoption of HYVs since the green revolution replaces several landraces, especially in Belagavi and Dakshina Kannada, due to the lack of seed availability and poor yield lead to decrease importance in production and maintenance, especially in rice. Lack of awareness is another major problem for local cultivar conservation, as only a handful of farmers are indulging in cultivar conservation in forest and hilly areas. An increase in desire for a luxurious life and other job opportunities with the inflow of money from natives residing in other cities, states, and countries; agriculture, which was formerly the main occupation of the people, has taken a backseat. In alignment with this, farms and fields in the survey area have been turned into residential plots and commercial (retail) buildings, abandoning cultivation and agriculture 47 .

Deterioration of culture and traditions in rural areas, poor knowledge transfer from elders, increasing technology in cultivation and commercialization of agriculture has led to reduced desire for conservation by young farmers and migration to urban areas 48 . There are no defined and proper market chains for landraces in the Western Ghats region, which also affects the cultivation. Alwis et al. 10 discussed the marketing problems for tubers crops in Sri Lanka. The strengthening of international markets and export incentives for other products like HYVs and the commercialization of agriculture in the area results in further loss of landrace diversity in the future 48 , 49 .

The Western Ghats is a place of origin and diversity for many plants and animals including insects. Thus, a variety of pests and diseases attack on the plants are reported. Insect and non-insect pests like rodents, macaques, wild boars, peacocks and other birds are major threats during cropping and harvesting time 10 . Improper storage conditions lead to occurrence of storage pests and diseases, thereby enhancing the viability loss and poor germination 40 , 42 . Many of these threats and constraints can be overcome through proper strategies. Promoting self-interest and creating awareness on the importance of landraces would in turn boost the conservation, maintenance and cultivation of landraces.

Conclusion and future implications

India has one of the top three genebanks in the world conserving more than 400K accessions of agri-horticultural crops. About 550 germplasm accessions of seed propagated crops belonging to the surveyed area are conserved in the genebank at ICAR-NBPGR. Our study adds specific information related to use including the indigenous technical knowledge to the passport data of these accessions. This addition is expected to enhance their immediate utilization. Furthermore, the number of landraces that are conserved on-farm in various indigenous crops across the vast swathes of the huge country remains inadequately documented. This report represents only a cross-sectional study of on-farm conservation in Central Western Ghats. Similar studies in other regions of the country need to be carried out to document landraces, their diversity and determinants of on-farm conservation practices.

Current report has documented three significant issues:

On-Farm conservation is practiced by a very few custodian farmers. Younger generation appears to find no incentive to continue the conservation practices.

The on-farm conservation sites vary in size and crop-composition. Some landraces (particularly of rice) are conserved by multiple custodians signifying their culinary popularity.

Landraces being indivisible part of local cuisine and passion of custodian farmers are the most important reasons for con-farm conservation.

Possible ways to attract young farmers to on-farm conservation may include:

Registration of landraces as farmers’ variety (wherever applicable) with PPV&FRA

Popularization of the landraces among niche urban customers may increase demand and sale-price.

Development of improved versions (agronomic value) of these landraces by breeders and researchers may open avenues of benefit sharing by custodian farmers.

With enhanced and assured income generation, next-generation farmers may find incentives to continue on-farm conservation. Else, weakening of cultural traditions, declining economic returns, and changing climate may lead to erosion and ultimately irreversible loss of these invaluable landraces.

Data availability

The data that were generated during the study as well as those that support the findings are included in the paper. The data are also accessible from a database (under development and unpublished) at http://pgrinformatics.nbpgr.ernet.in/onfc/database.aspx .

Ford-Lloyd, B. & Jackson, M. Plant Genetic Resources: An Introduction to Their Conservation and Use 152 (Edward Arnold, London, 1986).

Google Scholar  

Maxted, N., Hunter, D. & Ríos, R. O. Plant Genetic Conservation (Cambridge University Press, Cambridge, 2020).

Book   Google Scholar  

Villa, T., Maxted, N., Scholten, M. & Ford-Lloyd, B. Defining and identifying crop landraces. Plant Genet. Resour. 3 (3), 373–384. https://doi.org/10.1079/PGR200591 (2005).

Article   Google Scholar  

Palni, L. M. S., Maikhuri, R. K. & Rao, K. S. in: Eco-Regional Cooperation for Biodiversity Conservation in the Himalaya 253–290 (UNDP, New York, 1998).

Frankel, O. H. Genetic dangers of the green revolution. World Agric. 19 , 9–13 (1970).

Harlan, J. R. Genetics of disaster. J. Environ. Qual. 1 (3), 212–215. https://doi.org/10.2134/jeq1972.00472425000100030002x (1972).

CGIAR. Guardians of diversity: The network of genebanks helping to feed the world (2019).

Maxted, N., Ford-Lloyd, B. V. & Hawkes, J. G. Complementary conservation strategies. In Plant Genetic Conservation, The In Situ Approach (eds Maxted, N. et al. ) 15–40 (Chapman & Hall, 1997).

Chapter   Google Scholar  

Veteläinen, M., Negri, V. & Maxted, N. European Landraces: On-Farm Conservation, Management and Use Vol. 15, 1–359 (Bioversity International, Rome, 2009).

Alwis, L. M. H. R., Dissanayakea, M. U. K. & Rathnayakea, R. M. W. M. C. Species diversity of local root and tuber crops in agricultural habitats in Uva region. J. Agric. Value Addit. 1 (1), 47–61 (2017).

Conversa, G. et al. Exploring on-farm agro-biodiversity: A study case of vegetable landraces from Puglia region (Italy). Biodivers. Conserv. 29 , 747–770. https://doi.org/10.1007/s10531-019-01908-3 (2020).

Ocimati, W. et al. Musa germplasm diversity status across a wide range of agro-ecological zones in Rwanda. J. Appl. Biosci. 73 , 5979–5990 (2014).

Sthapit, B., Lamers, H. & Rao, R. V. Custodian farmers of agricultural biodiversity: Selected profiles from South and South East Asia. In Proceedings of Workshop on Custodian Farmers of Agricultural Biodiversity, New Delhi, India (2013).

Thant, A. A. et al. On-farm rice diversity and farmers’ preferences for varietal attributes in Ayeyarwady Delta, Myanmar. J. Crop. Improv. 34 (4), 549–570. https://doi.org/10.1080/15427528.2020.1746457 (2020).

Scaria, R. Origin and evolution of peninsular India, Western Ghats, and its diverse life forms. In Microbial Biodiversity, Biotechnology and Ecosystem Sustainability (eds Aguilar, C. N. et al. ) 43–56 (Springer, 2023).

Myers, N., Mittermeier, R. A., Mittermeier, C. G., Da Fonseca, G. A. B. & Kent, J. Biodiversity hotspots for conservation priorities. Nature 403 (6772), 853–858. https://doi.org/10.1038/35002501 (2000).

Article   ADS   CAS   PubMed   Google Scholar  

Nayar, T. S., Rasiya-Beegam, A. & Sibi, M. Flowering Plants of the Western Ghats, India Vol. 2 (Jawaharlal Nehru Tropical Botanic Garden and Research Institute, Kerala, 2014).

Asha, K. I. et al. Survey and collection of tuber crops from Joida, Uttara Kannada: An unexplored biodiversity hotspot. J. Root Crops 43 (2), 102–106 (2017).

Gajanana, T. M. et al. Motivation for on-farm conservation of mango ( Mangifera indica L.) diversity in India—a case study. Indian J. Pl. Genet. Resour. 28 (1), 1–6. https://doi.org/10.5958/0976-1926.2015.00001.7 (2015).

Ramachandra, T. V., Subash-Chandran, M. D., Joshi, N. V., Prakash, N. M., Sreekantha & Gayatri, N. Agro-Biodiversity in Uttara Kannada, Sahyadri Conservation Series 38, ENVIS Technical Report 68, CES (Indian Institute of Science, Bangalore 560012, India, 2013).

Naderifar, M., Goli, H. & Ghaljaie, F. Snowball sampling: A purposeful method of sampling in qualitative research. Stride. Dev. Med. Educ. 14 (3), e67670 (2017).

Subedi, A. et al. Who maintains crop genetic diversity and how? Implications for on-farm conservation and utilization. Cult. Agric. 25 (2), 41–50 (2003).

Ramachandra, T. V., Bharath, S., Subash-Chandran, M. D. & Joshi, N. V. Salient ecological sensitive regions of central Western Ghats, India. Earth Syst. Environ. 2 , 15–34 (2018).

Jadhav, R., Datar, M. N. & Upadhye, A. S. Wild relatives of crop plants from Northern Western Ghats of Maharashtra: Diversity and distribution. In Advances in Plant Sciences and Biotechnology (eds Krishan, S. & Rodrigues, B. F.) 17–27 (Goa University Publisher, India, 2015).

Shannon, C. E. A mathematical theory of communication. Bell Syst. Tech. J. 27 (3), 379–423. https://doi.org/10.1002/j.1538-7305.1948.tb01338.x (1948).

Article   MathSciNet   Google Scholar  

Simpson, E. H. Measurement of diversity. Nature 163 , 688. https://doi.org/10.1038/163688a0 (1949).

Article   ADS   Google Scholar  

Vasugi, C. et al. Genetic diversity in unique indigenous mango accessions ( Appemidi ) of the Western Ghats for certain fruit characteristics. Curr. Sci. 103 (2), 199–207 (2012).

CAS   Google Scholar  

Pradeep, K., Joseph John, K., Latha, M. & Suma, A. Status of crop plants of agricultural importance in Kerala state India: An update. Genet. Resour. Crop. Evol. 68 (5), 1849–1873. https://doi.org/10.1007/s10722-020-01100-5 (2021).

Syamala, G. L. & Pravir, D. A Guidebook on Integration of Biodiversity: For Spice Sector in the Western Ghats (Confederation of Indian Industry, 2019).

Reshma, P., Neethu, R. S. & Sreekala, G. S. Genetic diversity of black pepper ( Piper nigrum L.) in India: A review. J. Pharm. Innov. 11 (10), 832–839 (2022).

Saji, K. V., Sasikumar, B., Rema, J., Aravind, S., & Nirmal Babu, K. in: Conservation and Utilization of Horticultural Genetic Resources 283–320. https://doi.org/10.1007/978-981-13-3669-0_9 (2019).

Latha, M., Pradheep, K. & Suma, A. Indigenous and minor vegetables of Western Ghats. Indian Hortic. 65 (3), 39–44 (2020).

Rathi, R. S. et al. On-farm conservation of Rainfed rice landrace diversity in Chhattisgarh, India. Indian J. Pl. Genet. Resour. 32 (1), 1–10. https://doi.org/10.5958/0976-1926.2019.00001.9 (2019).

Agnihotri, R. K. On-farm conservation of landraces of rice ( Oryza sativa L.) through cultivation in the Kumaon region of Indian central Himalaya. J. Mt. Sci. 4 (4), 354–360. https://doi.org/10.1007/s11629-007-0354-3 (2007).

Tandon, P., Abrol, Y. P. & Kumaria, S. Biodiversity and Its Significance 356 (I. K. International Publishing House, 2007).

Ashoka, N. et al. Comparative study of black pepper ( Piper nigrum L.) nursery raising in Karnataka: Traditional variety Sigandhini versus popular variety Panniyur -I. Curr. Sci. 121 (9), 1201–1207 (2021).

Appa Rao, S., Bounphanousay, C., Schiller, J. M., Alcantara, A. P. & Jackson, M. T. Naming of traditional rice varieties by farmers in the Lao PDR. Genet. Resour. Crop Evol. 49 , 83–88. https://doi.org/10.1023/A:1013843913975 (2002).

Magdum, S. A. & Khadke, S. G. Agro-morphological diversity in rice ( Oryza sativa L.) landraces from Chandoli region of Western Ghats. Int. J. Sci. Res. 9 (15), 43–48 (2022).

Mohana, G. S. & Manjunatha, B. Bio-prospecting studies in traditional rice varieties of central Western Ghats of Karnataka. J. Pharmacogn. Phytochem. 7 (1), 437–439 (2018).

Ocimati, W. et al. On-farm Musa germplasm diversity in different agro-ecologies of Burundi. Int. J. Biodivers. Conserv. 5 (11), 751–760 (2013).

Pusadee, T., Wongtamee, A., Rerkasem, B., Olsen, K. M. & Jamjod, S. Farmers drive genetic diversity of Thai purple rice ( Oryza sativa L.) landraces. Econ. Bot. 73 , 76–85. https://doi.org/10.1007/s12231-018-9436-0 (2019).

de Souza, R., Ogliar, J. B. & Pinto, T. T. Analysis of on farm conservation of sweet corn in a diversity microcenter of Zea mays L. in Southern Brazil. Maydica 65 (1), 11 (2020).

Wang, Y. et al. Influence of ethnic traditional cultures on genetic diversity of rice landraces under on-farm conservation in southwest China. J. Ethnobiol. Ethnomed. 12 , 51. https://doi.org/10.1186/s13002-016-0120-0 (2016).

Article   PubMed   PubMed Central   Google Scholar  

Poudel, D. & Johnsen, F. H. Valuation of crop genetic resources in Kaski, Nepal: Farmers’ willingness to pay for rice landraces conservation. J. Environ. Manage. 90 (1), 483–491 (2009).

Article   PubMed   Google Scholar  

Gauchan, D. Economic valuation of rice landraces diversity: a case study of Bara ecosite, Terai, Nepal. in A Scientific Basis of in situ Conservation of Agro-biodiversity on Farm: Nepal’s Contribution to Global Project, NP Working paper No. 1/99 (eds Sthapit, B., Upadhyay, M., Subedi, A. (Eds) (NARC/LI-BIRD/IPGRI, Nepal, Rome, Italy, 1999).

Hammer, K. & Teklu, Y. Plant genetic resources: Selected issues from genetic erosion to genetic engineering. J. Agric. Rural Dev. Trop. Subtrop. 109 , 15–50 (2008).

Kalyanaraman, J. & Koskimaki L. in Provincial Globalisation Research Report No. 6 (Bangalore, India 2013).

Fuentes, F. F., Bazile, D., Bhargava, A. & Martínez, E. A. Implications of farmers’ seed exchanges for on-farm conservation of quinoa, as revealed by its genetic diversity in Chile. J. Agric. Sci. 150 , 702–716. https://doi.org/10.1017/S0021859612000056 (2012).

Marti, N. & Pimbert, M. Barter markets for the conservation of agro-ecosystem multi-functionality: The case of the chalayplasa in the Peruvian Andes. Int. J. Agric. Sustain. 5 , 51–69. https://doi.org/10.1080/14735903.2007.9684813 (2007).

Download references

Acknowledgements

The authors thank the ICAR-NBPGR for the facilities. Puneeth was supported by ICAR-IARI Fellowship and Sunil Archak was supported by ICAR-National Fellowship. Authors would like to extend heartfelt gratitude towards the custodian farmers for their endeavors in landrace conservation.

Author information

Authors and affiliations.

Division of Plant Genetic Resources, ICAR - Indian Agricultural Research Institute, Pusa Campus, New Delhi, 110 012, India

G. M. Puneeth & Kerekoppa Manjunatha Laxmisha

ICAR - National Bureau of Plant Genetic Resources, Pusa Campus, New Delhi, 110 012, India

Ravi Gowthami, Gyanendra Pratap Singh & Sunil Archak

Division of Genetics, ICAR - Indian Agricultural Research Institute, Pusa Campus, New Delhi, 110 012, India

Ashvinkumar Katral

College of Forestry, University of Agricultural Sciences, Dharwad, Sirsi, 581 401, India

Ramesh Vasudeva

You can also search for this author in PubMed   Google Scholar

Contributions

SA and GMP designed the study. GMP carried out the study with the assistance of RV and GPS. AK and KML helped in data curation and analysis. GMP and RG wrote the manuscript. SA, GPS and RV edited and corrected the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Sunil Archak .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Puneeth, G.M., Gowthami, R., Katral, A. et al. On-farm crop diversity, conservation, importance and value: a case study of landraces from Western Ghats of Karnataka, India. Sci Rep 14 , 10712 (2024). https://doi.org/10.1038/s41598-024-61428-1

Download citation

Received : 26 October 2023

Accepted : 06 May 2024

Published : 10 May 2024

DOI : https://doi.org/10.1038/s41598-024-61428-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• On-farm conservation
• Documentation
• Western Ghats

By submitting a comment you agree to abide by our Terms and Community Guidelines . If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.