A Framework for the Evaluation of Internet-based Diabetes Management

Abstract

BACKGROUND: While still in its infancy, Internet-based diabetes management shows great promise for growth. However, the following aspects must be considered: what are the key metrics for the evaluation of a diabetes management site? how should these sites grow in the future and what services should they offer? OBJECTIVES: To examine the needs of the patient and the health care professional in an Internet-based diabetes management solution and how these needs are translated into services offered. METHODS: An evaluation framework was constructed based on a literature review that identified the requirements for an Internet-based diabetes management solution. The requirements were grouped into 5 categories: Monitoring, Information, Personalization, Communication, and Technology. Two of the market leaders (myDiabetes and LifeMasters) were selected and were evaluated with the framework. The Web sites were evaluated independently by 5 raters using the evaluation framework. All evaluations were performed from November 1, 2001 through December 15, 2001. RESULTS: The agreement level between raters ranged from 60% to 100%. The multi-rater reliability (kappa) was 0.75 for myDiabetes and 0.65 for LifeMasters, indicating substantial agreement. The results of the evaluations indicate that LifeMasters is a more-complete solution than myDiabetes in all dimensions except Information, where both sites were equivalent. LifeMasters satisfied 32 evaluation criteria while myDiabetes satisfied 24 evaluation criteria, out of a possible 40 in the framework. CONCLUSIONS: The framework is based on the recognition that the management of diabetes via the Internet is based on several integrated dimensions: Monitoring, Information, Personalization, Communication, and Technology. A successful diabetes management system should efficiently integrate all dimensions. The evaluation found that LifeMasters is successful in integrating the health care professional in the management of diabetes and that MyDiabetes is quite effective in providing a communication channel for community creation (however, communication with the health care professional is lacking). [J Med Internet Res 2002;4(1):e1]