Evaluating the response effort and data quality of established political solidarity measures: a pre-registered experimental test in an online survey of the German adult resident population in 2021
Open Access
- 23 January 2023
- journal article
- research article
- Published by Springer Science and Business Media LLC in Quality & Quantity
- Vol. 57 (6), 5431-5447
- https://doi.org/10.1007/s11135-022-01594-4
Abstract
This experimental study aims to check and improve the quality of 16 established survey measures of political solidarities and related concepts, such as redistribution and social trust. Political solidarities are defined as one’s willingness to share the costs that result from public redistribution that favours people other than oneself and thus constitute a subset of welfare state attitudes. The pre-registered study plan included suggestions for the development of improved rating scales, which we defined as five-point, end verbalized rating scales without non-substantive answer options. The overall results from an experimental online survey in Germany indicate differences in response effort in terms of response times but almost no differences in data quality in terms of criterion validity. Thus, the 16 survey measures show solid instrument validity as well as minor improvements in respondents’ response times. Indeed, the measures are (at least) in the online survey world of Germany of high-quality and warrant inclusion in future surveys with small efficiency gains still attainable.Funding Information
- H2020 European Research Council (864818)
- Universität Duisburg-Essen
This publication has 37 references indexed in Scilit:
- Using Paradata to Explore Item Level Response Times in SurveysJournal of the Royal Statistical Society Series A: Statistics in Society, 2012
- Age‐based self‐interest, intergenerational solidarity and the welfare state: A comparative analysis of older people's attitudes towards public childcare in 12 OECD countriesEuropean Journal of Political Research, 2010
- Words, Numbers, and Visual Heuristics in Web Surveys: Is There a Hierarchy of Importance?Social Science Computer Review, 2010
- Cognitive burden of survey questions and response times: A psycholinguistic experimentApplied Cognitive Psychology, 2009
- Effects of Design in Web SurveysPublic Opinion Quarterly, 2008
- Color, Labels, and Interpretive Heuristics for Response ScalesPublic Opinion Quarterly, 2007
- Fast times and easy questions: the effects of age, experience and question complexity on web survey response timesApplied Cognitive Psychology, 2007
- Spacing, Position, and Order: Interpretive Heuristics for Visual Features of Survey QuestionsPublic Opinion Quarterly, 2004
- The Science of Asking QuestionsAnnual Review of Sociology, 2003
- Response strategies for coping with the cognitive demands of attitude measures in surveysApplied Cognitive Psychology, 1991