Reliability and validity of a nutrition and physical activity environmental self-assessment for child care
Open Access
- 1 January 2007
- journal article
- research article
- Published by Springer Science and Business Media LLC in International Journal of Behavioral Nutrition and Physical Activity
- Vol. 4 (1), 29
- https://doi.org/10.1186/1479-5868-4-29
Abstract
Few assessment instruments have examined the nutrition and physical activity environments in child care, and none are self-administered. Given the emerging focus on child care settings as a target for intervention, a valid and reliable measure of the nutrition and physical activity environment is needed. To measure inter-rater reliability, 59 child care center directors and 109 staff completed the self-assessment concurrently, but independently. Three weeks later, a repeat self-assessment was completed by a sub-sample of 38 directors to assess test-retest reliability. To assess criterion validity, a researcher-administered environmental assessment was conducted at 69 centers and was compared to a self-assessment completed by the director. A weighted kappa test statistic and percent agreement were calculated to assess agreement for each question on the self-assessment. For inter-rater reliability, kappa statistics ranged from 0.20 to 1.00 across all questions. Test-retest reliability of the self-assessment yielded kappa statistics that ranged from 0.07 to 1.00. The inter-quartile kappa statistic ranges for inter-rater and test-retest reliability were 0.45 to 0.63 and 0.27 to 0.45, respectively. When percent agreement was calculated, questions ranged from 52.6% to 100% for inter-rater reliability and 34.3% to 100% for test-retest reliability. Kappa statistics for validity ranged from -0.01 to 0.79, with an inter-quartile range of 0.08 to 0.34. Percent agreement for validity ranged from 12.9% to 93.7%. This study provides estimates of criterion validity, inter-rater reliability and test-retest reliability for an environmental nutrition and physical activity self-assessment instrument for child care. Results indicate that the self-assessment is a stable and reasonably accurate instrument for use with child care interventions. We therefore recommend the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) instrument to researchers and practitioners interested in conducting healthy weight intervention in child care. However, a more robust, less subjective measure would be more appropriate for researchers seeking an outcome measure to assess intervention impact.Keywords
This publication has 43 references indexed in Scilit:
- Nutrition and Physical Activity Self-assessment for Child Care (NAP SACC): Results from a Pilot InterventionJournal of Nutrition Education and Behavior, 2007
- Childhood Overweight, Obesity, and the Metabolic Syndrome in Developing CountriesEpidemiologic Reviews, 2007
- Prevalence of Overweight and Obesity in the United States, 1999-2004JAMA, 2006
- Two-year follow-up results for Hip-Hop to Health Jr.: A randomized controlled trial for overweight prevention in preschool minority childrenThe Journal of Pediatrics, 2005
- Orphanhood and childcare patterns in sub-Saharan AfricaAIDS, 2004
- Body Mass Index and Overweight in Adolescents in 13 European Countries, Israel, and the United StatesArchives of Pediatrics & Adolescent Medicine, 2004
- Interpretation of Kappa and B statistics measures of agreementJournal of Applied Statistics, 1997
- Increased incidence of non-insulin-dependent diabetes mellitus among adolescentsThe Journal of Pediatrics, 1996
- An Ecological Perspective on Health Promotion ProgramsHealth Education Quarterly, 1988
- A Coefficient of Agreement for Nominal ScalesEducational and Psychological Measurement, 1960