When assessing generalisability, focusing on differences in population or setting alone is insufficient
Open Access
- 20 March 2020
- journal article
- research article
- Published by Springer Science and Business Media LLC in Trials
- Vol. 21 (1), 1-4
- https://doi.org/10.1186/s13063-020-4178-6
Abstract
Generalisability is typically only briefly mentioned in discussion sections of evaluation articles, which are unhelpful in judging whether an intervention could be implemented elsewhere, with similar effects. Several tools to assess generalisability exist, but they are difficult to operationalise and are rarely used. We believe a different approach is needed. Instead of focusing on similarities (or more likely, differences) in generic population and setting characteristics, generalisability assessments should focus on understanding an intervention’s mechanism of action - why or how an intervention was effective. We believe changes are needed to four types of research. First, outcome evaluations should draw on programme theory. Second, process evaluations should aim to understand interventions’ mechanism of action, rather than simply ‘what happened’. Third, small scoping studies should be conducted in new settings, to explore how to enact identified mechanisms. Finally, innovative synthesis methods are required, in order to identify mechanisms of action where there is a lack of existing process evaluations.Funding Information
- Government of the United Kingdom (PR-R6-0113-1103)
This publication has 17 references indexed in Scilit:
- Using qualitative comparative analysis (QCA) in systematic reviews of complex interventions: a worked exampleSystematic Reviews, 2014
- A systematic review of barriers to and facilitators of the use of evidence by policymakersBMC Health Services Research, 2014
- The effect of moderate gestational alcohol consumption during pregnancy on speech and language outcomes in children: a systematic reviewSystematic Reviews, 2014
- Transformative Lifestyle Change: key to sustainable weight loss among women in a post-partum diet and exercise interventionMaternal & Child Nutrition, 2013
- Diet and exercise weight-loss trial in lactating overweight and obese womenThe American Journal of Clinical Nutrition, 2012
- GeneralizabilityNeurology, 2012
- How do we know when research from one setting can be useful in another? A review of external validity, applicability and transferability frameworksJournal of Health Services Research & Policy, 2011
- Methods for the thematic synthesis of qualitative research in systematic reviewsBMC Medical Research Methodology, 2008
- Using Programme Theory to Evaluate Complicated and Complex Aspects of InterventionsEvaluation, 2008
- Process evaluation in randomised controlled trials of complex interventionsBMJ, 2006