JMIR Formative Research
Latest articles in this journal
Published: 31 March 2023
JMIR Formative Research, Volume 7; https://doi.org/10.2196/42452
Background: Sepsis is a leading cause of death in patients with trauma, and the risk of mortality increases significantly for each hour of delay in treatment. A hypermetabolic baseline and explosive inflammatory immune response mask clinical signs and symptoms of sepsis in trauma patients, making early diagnosis of sepsis more challenging. Machine learning–based predictive modeling has shown great promise in evaluating and predicting sepsis risk in the general intensive care unit (ICU) setting, but there has been no sepsis prediction model specifically developed for trauma patients so far. Objective: To develop a machine learning model to predict the risk of sepsis at an hourly scale among ICU-admitted trauma patients. Methods: We extracted data from adult trauma patients admitted to the ICU at Beth Israel Deaconess Medical Center between 2008 and 2019. A total of 42 raw variables were collected, including demographics, vital signs, arterial blood gas, and laboratory tests. We further derived a total of 485 features, including measurement pattern features, scoring features, and time-series variables, from the raw variables by feature engineering. The data set was randomly split into 70% for model development with stratified 5-fold cross-validation, 15% for calibration, and 15% for testing. An Extreme Gradient Boosting (XGBoost) model was developed to predict the hourly risk of sepsis at prediction windows of 4, 6, 8, 12, and 24 hours. We evaluated model performance for discrimination and calibration both at time-step and outcome levels. Clinical applicability of the model was evaluated with varying levels of precision, and the potential clinical net benefit was assessed with decision curve analysis (DCA). A Shapley additive explanation algorithm was applied to show the effect of features on the prediction model. In addition, we trained an L2-regularized logistic regression model to compare its performance with XGBoost. Results: We included 4603 trauma patients in the study, 1196 (26%) of whom developed sepsis. The XGBoost model achieved an area under the receiver operating characteristics curve (AUROC) ranging from 0.83 to 0.88 at the 4-to-24-hour prediction window in the test set. With a ratio of 9 false alerts for every true alert, it predicted 73% (386/529) of sepsis-positive timesteps and 91% (163/179) of sepsis events in the subsequent 6 hours. The DCA showed our model had a positive net benefit in the threshold probability range of 0 to 0.6. In comparison, the logistic regression model achieved lower performance, with AUROC ranging from 0.76 to 0.84 at the 4-to-24-hour prediction window. Conclusions: The machine learning–based model had good discrimination and calibration performance for sepsis prediction in critical trauma patients. Using the model in clinical practice might help to identify patients at risk of sepsis in a time window that enables personalized intervention and early treatment.
Published: 30 March 2023
JMIR Formative Research, Volume 7; https://doi.org/10.2196/39383
Background: Digital health tools such as electronic immunization registries (EIRs) have the potential to improve patient care and alleviate the challenges that arise from the use of paper-based clinic records for reporting. To address some of these challenges, the Kenya Ministry of Health and the International Training and Education Center for Health Kenya implemented an EIR system in 161 immunizing clinics in Siaya County between 2018 and 2019. The successful implementation of digital health tools depends on many factors, one of which is alignment between the technology and the context in which it is used. One important aspect of that implementation context is the perceptions of the health care workers (HCWs) using the EIR. Objective: This study aimed to evaluate HCWs’ perceptions of the usability and acceptability of multiple clinic workflows using the new EIR. Methods: We performed a mixed methods pre-post study using semistructured interviews with HCWs at 6 facilities in Siaya County, Kenya. We interviewed HCWs at each facility 4 times: at baseline and once after the implementation of 3 different workflow modifications (n=24 interviews). The baseline state was dual data entry with paper records and the EIR. We then implemented 3 workflow modifications for 1 full day each: fully paperless data entry, preparation of an appointment diary before patient visits for the day, and a combination of the 2 workflows. We compared ratings and themes across interviews after each of the 4 workflows to understand the changes in the usability and acceptability of the EIR. Results: HCWs considered the EIR clinic workflows to be usable and acceptable. Of the modified workflows, HCWs perceived the fully paperless workflow most favorably. In all workflows, HCWs’ perceived benefits included ease of clinical decision-making using the EIR, reduced mental burden of data entry when using the EIR, and ease of identification of errors. Perceived barriers to the workflow included contextual challenges such as staffing shortages and lack of network connectivity, EIR platform challenges such as errors in saving records and missing fields, and workflow challenges such as the dual data entry burden of using paper and digital tools simultaneously. Conclusions: Fully paperless EIR implementation shows great promise from a workflow acceptability standpoint, contingent upon the presence of supporting contextual clinic factors and the resolution of system performance and design challenges. Rather than trying to identify a singular best workflow, future efforts should provide adequate flexibility for HCWs to implement the new system in their unique clinic context. Future EIR implementation stands to benefit from continued monitoring of EIR adoption acceptability during implementation, both for Siaya’s program and for other efforts around the globe, as digital health interventions become more widely used.
Published: 30 March 2023
JMIR Formative Research, Volume 7; https://doi.org/10.2196/44575
Background: Remote photoplethysmography (rPPG) can record vital signs (VSs) by detecting subtle changes in the light reflected from the skin. Lifelight (Xim Ltd) is a novel software being developed as a medical device for the contactless measurement of VSs using rPPG via integral cameras on smart devices. Research to date has focused on extracting the pulsatile VS from the raw signal, which can be influenced by factors such as ambient light, skin thickness, facial movements, and skin tone. Objective: This preliminary proof-of-concept study outlines a dynamic approach to rPPG signal processing wherein green channel signals from the most relevant areas of the face (the midface, comprising the cheeks, nose, and top of the lip) are optimized for each subject using tiling and aggregation (T&A) algorithms. Methods: High-resolution 60-second videos were recorded during the VISION-MD study. The midface was divided into 62 tiles of 20×20 pixels, and the signals from multiple tiles were evaluated using bespoke algorithms through weighting according to signal-to-noise ratio in the frequency domain (SNR-F) score or segmentation. Midface signals before and after T&A were categorized by a trained observer blinded to the data processing as 0 (high quality, suitable for algorithm training), 1 (suitable for algorithm testing), or 2 (inadequate quality). On secondary analysis, observer categories were compared for signals predicted to improve categories following T&A based on the SNR-F score. Observer ratings and SNR-F scores were also compared before and after T&A for Fitzpatrick skin tones 5 and 6, wherein rPPG is hampered by light absorption by melanin. Results: The analysis used 4310 videos recorded from 1315 participants. Category 2 and 1 signals had lower mean SNR-F scores than category 0 signals. T&A improved the mean SNR-F score using all algorithms. Depending on the algorithm, 18% (763/4212) to 31% (1306/4212) of signals improved by at least one category, with up to 10% (438/4212) improving into category 0, and 67% (2834/4212) to 79% (3337/4212) remaining in the same category. Importantly, 9% (396/4212) to 21% (875/4212) improved from category 2 (not usable) into category 1. All algorithms showed improvements. No more than 3% (137/4212) of signals were assigned to a lower-quality category following T&A. On secondary analysis, 62% of signals (32/52) were recategorized, as predicted from the SNR-F score. T&A improved SNR-F scores in darker skin tones; 41% of signals (151/369) improved from category 2 to 1 and 12% (44/369) from category 1 to 0. Conclusions: The T&A approach to dynamic region of interest selection improved signal quality, including in dark skin tones. The method was verified by comparison with a trained observer’s rating. T&A could overcome factors that compromise whole-face rPPG. This method’s performance in estimating VS is currently being assessed. Trial Registration: ClinicalTrials.gov NCT04763746; https://clinicaltrials.gov/ct2/show/NCT04763746
JMIR Formative Research, Volume 7; https://doi.org/10.2196/45211
Background: Augmented reality (AR) and virtual reality (VR) have increasingly appeared in the medical literature in the past decade, with AR recently being studied for its potential role in remote health care delivery and communication. Recent literature describes AR’s implementation in real-time telemedicine contexts across multiple specialties and settings, with remote emergency services in particular using AR to enhance disaster support and simulation education. Despite the introduction of AR in the medical literature and its potential to shape the future of remote medical services, studies have yet to investigate the perspectives of telemedicine providers regarding this novel technology. Objective: This study aimed to understand the applications and challenges of AR in telemedicine anticipated by emergency medicine providers with a range of experiences in using telemedicine and AR or VR technology. Methods: Across 10 academic medical institutions, 21 emergency medicine providers with variable exposures to telemedicine and AR or VR technology were recruited for semistructured interviews via snowball sampling. The interview questions focused on various potential uses of AR, anticipated obstacles that prevent its implementation in the telemedicine area, and how providers and patients might respond to its introduction. We included video demonstrations of a prototype using AR during the interviews to elicit more informed and complete insights regarding AR’s potential in remote health care. Interviews were transcribed and analyzed via thematic coding. Results: Our study identified 2 major areas of use for AR in telemedicine. First, AR is perceived to facilitate information gathering by enhancing observational tasks such as visual examination and granting simultaneous access to data and remote experts. Second, AR is anticipated to supplement distance learning of both minor and major procedures and nonprocedural skills such as cue recognition and empathy for patients and trainees. AR may also supplement long-distance education programs and thereby support less specialized medical facilities. However, the addition of AR may exacerbate the preexisting financial, structural, and literacy barriers to telemedicine. Providers seek value demonstrated by extensive research on the clinical outcome, satisfaction, and financial benefits of AR. They also seek institutional support and early training before adopting novel tools such as AR. Although an overall mixed reception is anticipated, consumer adoption and awareness are key components in AR’s adoption. Conclusions: AR has the potential to enhance the ability to gather observational and medical information, which would serve a diverse set of applications in remote health care delivery and education. However, AR faces obstacles similar to those faced by the current telemedicine technology, such as lack of access, infrastructure, and familiarity. This paper discusses the potential areas of investigation that would inform future studies and approaches to implementing AR in telemedicine.
JMIR Formative Research, Volume 7; https://doi.org/10.2196/43222
Background: According to the World Health Organization, globally, one in seven 10- to 19-year-olds experiences a mental disorder, accounting for 13% of the global burden of disease in this age group. Half of all mental illnesses begin by the age of 14 years and some teenagers with severe presentations must be admitted to the hospital and assessed by highly skilled mental health care practitioners. Digital telehealth solutions can be useful for the assessment of young individuals remotely. Ultimately, this technology can save travel costs for the health service rather than assessing adolescents in person at the corresponding hospital. Especially in rural regions, where travel times can be high, this innovative approach can make a difference to patients by providing quicker assessments. Objective: The aim of this study is to share insights on how we developed a decision support tool to assign staff to days and locations where adolescent mental health patients are assessed face to face. Where possible, patients are seen through video consultation. The model not only seeks to reduce travel times and consequently carbon emissions but also can be used to find a minimum number of staff to run the service. Methods: To model the problem, we used integer linear programming, a technique that is used in mathematical modeling. The model features 2 objectives: first, we aim to find a minimum coverage of staff to provide the service and second, to reduce travel time. The constraints that are formulated algebraically are used to ensure the feasibility of the schedule. The model is implemented using an open-source solver backend. Results: In our case study, we focus on real-world demand coming from different hospital sites in the UK National Health Service (NHS). We incorporate our model into a decision support tool and solve a realistic test instance. Our results reveal that the tool is not only capable of solving this problem efficiently but also shows the benefits of using mathematical modeling in health services. Conclusions: Our approach can be used by NHS managers to better match capacity and location-dependent demands within an increasing need for hybrid telemedical services, and the aims to reduce traveling and the carbon footprint within health care organizations.
JMIR Formative Research, Volume 7; https://doi.org/10.2196/26077
Background: Despite bariatric surgery showing significant weight loss trajectories for many patients, a substantial proportion regain weight after the first year following surgery. The addition of telemedicine to standard care could support patients with engaging in a more active lifestyle and thus improve clinical outcomes. Objective: Our aim was to evaluate a telemedicine intervention program dedicated to the promotion of physical activity including digital devices, teleconsultation, and telemonitoring the first 6 months following bariatric surgery. Methods: This study employed a mixed methods design based on an open-label randomized controlled trial. Patients were included during the first week after bariatric surgery; then, they were randomized into 2 intervention groups: The TelePhys group received a monthly telemedicine consultation focusing on physical activity coaching, while the TeleDiet group received a monthly telemedicine consultation involving diet coaching. Data were collected using a watch pedometer and body weight scale, both of which were connected wirelessly. The primary outcome was the difference between the 2 groups in the mean numbers of steps at the first and sixth postoperative months. Weight change was also evaluated, and focus groups and interviews were conducted to enrich the results and capture perceptions of the telemedicine provided. Results: Among the 90 patients (mean age 40.6, SD 10.4 years; 73/90, 81% women; 62/90, 69% gastric bypass), 70 completed the study until the sixth month (n=38 TelePhys; n=32 TeleDiet), and 18 participants agreed to be interviewed (n=8 Telephys; n=10 TeleDiet). An increase in the mean number of steps between the first and sixth months was found in both groups, but this change was significant only in the TeleDiet group (P=.01). No difference was found when comparing both intervention groups. Interviewed participants reported having appreciated the teleconsultations, as the individualized tailored counseling helped them to make better choices about behaviors that could increase their likelihood of a daily life in better health. Weight loss followed by social factors (such as social support) were identified as the main facilitators to physical activity. Family responsibilities, professional constraints as well as poor urban policies promoting physical activity, and lack of accessibility to sport infrastructure were their major barriers to postoperative lifestyle adherence. Conclusions: Our study did not show any difference in mobility recovery after bariatric surgery related to a telemedicine intervention dedicated to physical activity. The early postoperative timing for our intervention may explain the null findings. eHealth interventions aiming to change behaviors and carried out by clinicians require support from structured public health policies that tackle patients’ obesogenic environment in order to be efficient in their struggle against sedentary lifestyle–related pathologies. Further research will need to focus on long-term interventions. Trial Registration: ClinicalTrials.gov NCT02716480, https://clinicaltrials.gov/ct2/show/NCT02716480
JMIR Formative Research, Volume 7; https://doi.org/10.2196/39894
Background: Veterans with posttraumatic stress disorder (PTSD) and moral injury can encounter several barriers to treatment, including limited access to care and low engagement with therapy. Furthermore, most treatment approaches focus on alleviating distress rather than cultivating positive experiences that could facilitate trauma recovery. A potential way to address these issues is through moral elevation: feeling uplifted and inspired by others’ virtuous actions. Objective: This study aimed to examine the feasibility and acceptability of a novel, web-based moral elevation intervention for veterans with PTSD symptoms and moral injury distress (Moral Elevation Online Intervention for Veterans Experiencing Distress Related to PTSD and Moral Injury [MOVED]). This mixed methods study also examined potential changes in PTSD symptoms, moral injury distress, quality of life, and prosocial behavior. Methods: In this pilot trial, 48 participants were randomized to a MOVED or control condition (24 participants per condition). Both conditions included 8 sessions and lasted 1 month. The MOVED intervention and all survey components across both conditions were administered online. Participants completed self-report measures that assessed PTSD symptoms, moral injury distress, quality of life, and prosocial behavior at baseline and follow-up. Veterans in the MOVED condition also completed individual qualitative interviews at follow-up. We coded qualitative responses to interviews and identified emergent themes. Results: Findings suggest the MOVED intervention was largely feasible, with evidence for moderate-to-high levels of participation, engagement, and retention in MOVED sessions. Both quantitative and qualitative results suggest veterans found MOVED to be acceptable and satisfactory at the overall treatment level. Furthermore, participants reported high scores for helpfulness and engagement at the session level. Veterans who completed MOVED reported large within-person decreases in PTSD symptoms (Cohen d=1.44), approximately twice that of veterans in the control condition (Cohen d=0.78). Those in MOVED also reported medium-sized increases in physical (Cohen d=0.71) and psychological domains of quality of life (Cohen d=0.74), compared with no meaningful changes in the control condition. Unexpectedly, MOVED veterans reported no decrease in moral injury distress, whereas veterans in the control condition endorsed a medium-sized decrease in the total score. There were no changes in prosociality for either condition. Qualitative feedback further supported high levels of perceived acceptability and satisfaction and positive treatment outcomes across a range of domains, including behaviors, cognitions, emotions, and social functioning. Veterans also recommended adaptations to enhance engagement and maximize the impact of intervention content. Conclusions: Overall, findings indicate that veterans with PTSD and moral injury distress were interested in an intervention based on exposure to and engagement with experiences of moral elevation. After further research and refinement guided by future trials, veterans may benefit from this novel approach, which may enhance treatment outcomes and increase treatment accessibility for those in need of additional trauma-focused care.
JMIR Formative Research, Volume 7; https://doi.org/10.2196/38540
Background: Although there is a need for gender-specific health care, especially within the context of vocational rehabilitation for youth with disabilities, clinicians, trainees, and community service providers commonly report lacking training in gender-sensitive approaches. Therefore, an educational tool designed for clinicians working with youth, that addresses how to approach such issues, could help clinicians to augment the care they provide. Objective: The objective of our study was to conduct a pilot evaluation of an educational simulation for health care and service providers focusing on gender-sensitive approaches within the context of supporting youth with disabilities in vocational rehabilitation. Methods: We conducted a survey from May to September 2021 to assess the relevance of the simulation content, preliminary perceived impact on gender-sensitive knowledge and confidence, and open-ended feedback of a web-based gender-sensitive educational simulation. A total of 12 health care providers from a variety of professions who had experience working with youth in the context of vocational rehabilitation participated in the survey (11 women and 1 man). Results: Most participants reported that the content of the simulation was relevant and comprehensive. The majority of participants reported that the simulation helped to increase their perceived knowledge or understanding of the topic, changed their perceived understanding of their intervention or approach, and informed their perceived confidence. Our qualitative findings from the open-ended questions highlighted three main themes: (1) relevance of the simulation content, (2) perceived impact for clinical practice (ie, gender-sensitive language and communication and building rapport with patients), and (3) perceived impact on organizational processes (ie, practices, policy, and privacy). Conclusions: Our educational simulation shows preliminary potential as an educational tool for service providers working with youth who have a disability within the context of vocational rehabilitation. Further research is needed to assess the impact of the tool with larger samples.
JMIR Formative Research, Volume 7; https://doi.org/10.2196/36562
Background: Hypertension is the leading modifiable risk factor for cardiovascular disease and mortality. Adopting lifestyle modifications, like increasing physical activity (PA), can be an effective strategy in blood pressure (BP) control, but many adults do not meet the PA guidelines. Financial incentive interventions have the power to increase PA levels but are often limited due to cost. Further, mobile health technologies can make these programs more scalable. There is a gap in the literature about the most feasible and effective financial incentive PA framework; thus, pay-per-minute (PPM) and self-funded investment incentive (SFII) frameworks were explored. Objective: The aims were to (1) determine the feasibility (recruitment, engagement, and acceptability) of an 8-week mobile-based PPM and SFII hypertension prevention PA program and (2) explore the effects of PPM and SFII interventions relative to a control on the PA levels, BP, and PA motivation. Methods: In total, 55 adults aged 40-65 years not meeting the Canadian PA guidelines were recruited from Facebook and randomized into the following groups: financial incentive groups, PPM or SFII, receiving up to CAD $20 each (at the time of writing: CAD $1=US $0.74), or a control group without financial incentive. PPM participants received CAD $0.02 for each minute of moderate-to-vigorous PA (MVPA) per week up to the PA guidelines and the SFII received CAD $2.50 for each week they met the PA guidelines. Feasibility outcome measures (recruitment, engagement, and acceptability) were assessed. Secondary outcomes included changes in PA outcomes (MVPA and daily steps) relative to baseline were compared among PPM, SFII, and control groups at 4 and 8 weeks using linear regressions. Changes in BP and relative autonomy index relative to baseline were compared among the groups at follow-up. Results: Participants were randomized to the PPM (n=19), SFII (n=18), or control (n=18) groups. The recruitment, retention rate, and engagement were 77%, 75%, and 65%, respectively. The intervention received overall positive feedback, with 90% of comments praising the intervention structure, financial incentive, and educational materials. Relative to the control at 4 weeks, the PPM and SFII arms increased their MVPA with medium effect (PPM vs control: η2p=0.06, mean 117.8, SD 514 minutes; SFII vs control: η2p=0.08, mean 145.3, SD 616 minutes). At 8 weeks, PPM maintained a small effect in MVPA relative to the control (η2p=0.01, mean 22.8, SD 249 minutes) and SFII displayed a medium effect size (η2p=0.07, mean 113.8, SD 256 minutes). Small effects were observed for PPM and SFII relative to the control for systolic blood pressure (SBP) and diastolic blood pressure (DBP) (PPM: η2p=0.12, Δmean SBP 7.1, SD 23.61 mm Hg; η2p=0.04, Δmean DBP 3.5, SD 6.2 mm Hg; SFII: η2p=0.01, Δmean SBP −0.4, SD 1.4 mm Hg; η2p=0.02, Δmean DBP −2.3, SD 7.7 mm Hg) and relative autonomy index (PPM: η2p=0.01; SFII: η2p=0.03). Conclusions: The feasibility metrics and preliminary findings suggest that a future full-scale randomized controlled trial examining the efficacy of PPM and SFII relative to a control is feasible, and studies with longer duration are warranted.
Published: 23 March 2023
JMIR Formative Research, Volume 7; https://doi.org/10.2196/43255
Background: Communication across language barriers is a particular challenge for health care providers. In emergency medical services, interpreters are mostly not available on rescue scenes, which jeopardizes safe and high-quality medical care. In a cocreative process together with paramedics and software designers, we developed a fixed-phrase translation app with 600 phrases and 18 supported languages that supports paramedics when providing care to foreign-language patients. This paper reports on the results of a pilot study to evaluate the app’s effect on paramedic-patient communication. Objective: This study aims to gain insights into the efficacy and feasibility of a multilingual app that helps paramedics to communicate with patients who are not proficient in the local language. Methods: A 3-armed nonrandomized interventional pilot study was conducted in 4 rescue stations in the German Federal State of Lower Saxony: 3 rural areas and 1 in urban environment. The intervention group comprised rescue missions with patients with limited German language proficiency (LGP) with whom the app was used; control group 1 comprised LGP patients without app usage; and control group 2 consisted of rescue missions with German-speaking patients. For each rescue operation with LGP patients, paramedics filled out questionnaires about the communications with patients. From standardized Rescue Service Case Protocols, we extracted information on patient demographics (age and sex), clinical aspects (preliminary diagnosis and Glasgow Coma Scale), and rescue operation characteristics (time spent on emergency scene and additional dispatch of emergency physicians). The primary outcome was the paramedics’ perceived quality of communication with LGP patients. The secondary outcome was the ability to obtain necessary information from patients and the ability to provide important information to patients. A linear regression model was applied to assess the impact of the app on perceived communication, controlling demographic factors, and severity of illness. Results: A total of 22 LGP patients were recruited into the intervention group and 122 into control group 1. The control group 2 included 27,212 German-speaking patients. LGP patients were more than 2 decades younger than German-speaking patients. App usage among LGP patients was associated with higher perceived overall quality of communication (0.7 points on a 5-point Likert scale, P=.03). Applying a linear regression model controlling for age, sex, and Glasgow Coma Scale, the quality of communication was associated with an increase of 0.9 points (95% CI 0.2-1.6, P=.01). Compared to either German-speaking patients or LGP patients, paramedics spent 6-7 minutes longer on an emergency scene when the app was used (P=.24). Conclusions: The use of the app suggests a relevant improvement in communication with patients with limited proficiency in the locally spoken language in paramedic care. The small sample size and the lack of randomization reduce the generalizability of the findings. Trial Registration: German Clinical Trials Register DRKS00016719; https://drks.de/search/de/trial/DRKS00016719