BMC Medical Research Methodology
Published by: Springer Science and Business Media LLC
Total articles ≅ 3,059
Latest articles in this journal
Published: 25 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-15; https://doi.org/10.1186/s12874-023-01892-x
Background: In many countries, the prevalence of non-communicable diseases risk factors is commonly assessed through self-reported information from health interview surveys. It has been shown, however, that self-reported instead of objective data lead to an underestimation of the prevalence of obesity, hypertension and hypercholesterolemia. This study aimed to assess the agreement between self-reported and measured height, weight, hypertension and hypercholesterolemia and to identify an adequate approach for valid measurement error correction. Methods: Nine thousand four hundred thirty-nine participants of the 2018 Belgian health interview survey (BHIS) older than 18 years, of which 1184 participated in the 2018 Belgian health examination survey (BELHES), were included in the analysis. Regression calibration was compared with multiple imputation by chained equations based on parametric and non-parametric techniques. Results: This study confirmed the underestimation of risk factor prevalence based on self-reported data. With both regression calibration and multiple imputation, adjusted estimation of these variables in the BHIS allowed to generate national prevalence estimates that were closer to their BELHES clinical counterparts. For overweight, obesity and hypertension, all methods provided smaller standard errors than those obtained with clinical data. However, for hypercholesterolemia, for which the regression model’s accuracy was poor, multiple imputation was the only approach which provided smaller standard errors than those based on clinical data. Conclusions: The random-forest multiple imputation proves to be the method of choice to correct the bias related to self-reported data in the BHIS. This method is particularly useful to enable improved secondary analysis of self-reported data by using information included in the BELHES. Whenever feasible, combined information from HIS and objective measurements should be used in risk factor monitoring.
Published: 23 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-21; https://doi.org/10.1186/s12874-023-01869-w
Getting access to administrative health data for research purposes is a difficult and time-consuming process due to increasingly demanding privacy regulations. An alternative method for sharing administrative health data would be to share synthetic datasets where the records do not correspond to real individuals, but the patterns and relationships seen in the data are reproduced. This paper assesses the feasibility of generating synthetic administrative health data using a recurrent deep learning model. Our data comes from 120,000 individuals from Alberta Health’s administrative health database. We assess how similar our synthetic data is to the real data using utility assessments that assess the structure and general patterns in the data as well as by recreating a specific analysis in the real data commonly applied to this type of administrative health data. We also assess the privacy risks associated with the use of this synthetic dataset. Generic utility assessments that used Hellinger distance to quantify the difference in distributions between real and synthetic datasets for event types (0.027), attributes (mean 0.0417), Markov transition matrices (order 1 mean absolute difference: 0.0896, sd: 0.159; order 2: mean Hellinger distance 0.2195, sd: 0.2724), the Hellinger distance between the joint distributions was 0.352, and the similarity of random cohorts generated from real and synthetic data had a mean Hellinger distance of 0.3 and mean Euclidean distance of 0.064, indicating small differences between the distributions in the real data and the synthetic data. By applying a realistic analysis to both real and synthetic datasets, Cox regression hazard ratios achieved a mean confidence interval overlap of 68% for adjusted hazard ratios among 5 key outcomes of interest, indicating synthetic data produces similar analytic results to real data. The privacy assessment concluded that the attribution disclosure risk associated with this synthetic dataset was substantially less than the typical 0.09 acceptable risk threshold. Based on these metrics our results show that our synthetic data is suitably similar to the real data and could be shared for research purposes thereby alleviating concerns associated with the sharing of real data in some circumstances.
Published: 17 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-10; https://doi.org/10.1186/s12874-023-01871-2
Background: The effectiveness of malaria vector control interventions is often evaluated using cluster randomized trials (CRT) with outcomes assessed using repeated cross-sectional surveys. A key requirement for appropriate design and analysis of longitudinal CRTs is accounting for the intra-cluster correlation coefficient (ICC). In addition to exchangeable correlation (constant ICC over time), correlation structures proposed for longitudinal CRT are block exchangeable (allows a different within- and between-period ICC) and exponential decay (allows between-period ICC to decay exponentially). More flexible correlation structures are available in statistical software packages and, although not formally proposed for longitudinal CRTs, may offer some advantages. Our objectives were to empirically explore the impact of these correlation structures on treatment effect inferences, identify gaps in the methodological literature, and make practical recommendations. Methods: We obtained data from a parallel-arm CRT conducted in Tanzania to compare four different types of insecticide-treated bed-nets. Malaria prevalence was assessed in cross-sectional surveys of 45 households in each of 84 villages at baseline, 12-, 18- and 24-months post-randomization. We re-analyzed the data using mixed-effects logistic regression according to a prespecified analysis plan but under five different correlation structures as well as a robust variance estimator under exchangeable correlation and compared the estimated correlations and treatment effects. A proof-of-concept simulation was conducted to explore general conclusions. Results: The estimated correlation structures varied substantially across different models. The unstructured model was the best-fitting model based on information criteria. Although point estimates and confidence intervals for the treatment effect were similar, allowing for more flexible correlation structures led to different conclusions based on statistical significance. Use of robust variance estimators generally led to wider confidence intervals. Simulation results showed that under-specification can lead to coverage probabilities much lower than nominal levels, but over-specification is more likely to maintain nominal coverage. Conclusion: More flexible correlation structures should not be ruled out in longitudinal CRTs. This may be particularly important in malaria trials where outcomes may fluctuate over time. In the absence of robust methods for selecting the best-fitting correlation structure, researchers should examine sensitivity of results to different assumptions about the ICC and consider robust variance estimators.
Published: 13 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-10; https://doi.org/10.1186/s12874-023-01878-9
Baseline imbalance in covariates associated with the primary outcome in clinical trials leads to bias in the reporting of results. Standard practice is to mitigate that bias by stratifying by those covariates in the randomization. Additionally, for continuously valued outcome variables, precision of estimates can be (and should be) improved by controlling for those covariates in analysis. Continuously valued covariates are commonly thresholded for the purpose of performing stratified randomization, with participants being allocated to arms such that balance between arms is achieved within each stratum. Often the thresholding consists of a simple dichotomization. For simplicity, it is also common practice to dichotomize the covariate when controlling for it at the analysis stage. This latter dichotomization is unnecessary, and has been shown in the literature to result in a loss of precision when compared with controlling for the covariate in its raw, continuous form. Analytic approaches to quantifying the magnitude of the loss of precision are generally confined to the most convenient case of a normally distributed covariate. This work generalises earlier findings, examining the effect on treatment effect estimation of dichotomizing skew-normal covariates, which are characteristic of a far wider range of real-world scenarios than their normal equivalents.
Published: 13 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-9; https://doi.org/10.1186/s12874-023-01884-x
Objectives: In evidence synthesis practice, dealing with studies with no cases in both arms has been a tough problem, for which there is no consensus in the research community. In this study, we propose a method to measure the potential impact of studies with no cases for meta-analysis results which we define as harms index (Hi) and benefits index (Bi) as an alternative solution for deciding how to deal with such studies. Methods: Hi and Bi are defined by the minimal number of cases added to the treatment arm (Hi) or control arm (Bi) of studies with no cases in a meta-analysis that lead to a change of the direction of the estimates or its statistical significance. Both exact and approximating methods are available to calculate Hi and Bi. We developed the “hibi” module in Stata so that researchers can easily implement the method. A real-world investigation of meta-analyses from Cochrane reviews was employed to evaluate the proposed method. Results: Based on Hi and Bi, our results suggested that 21.53% (Hi) to 26.55% (Bi) of Cochrane meta-analyses may be potentially impacted by studies with no cases, for which studies with no cases could not be excluded from the synthesis. The approximating method shows excellent specificity (100%) for both Hi and Bi, moderate sensitivity (68.25%) for Bi, and high sensitivity (80.61%) for Hi compared to the exact method. Conclusions: The proposed method is practical and useful for systematic reviewers to measure whether studies with no cases impact the results of meta-analyses and may act as an alternative solution for review authors to decide whether to include studies with no events for the synthesis or not.
Published: 10 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-13; https://doi.org/10.1186/s12874-023-01873-0
Background: Latent class models are increasingly used to estimate the sensitivity and specificity of diagnostic tests in the absence of a gold standard, and are commonly fitted using Bayesian methods. These models allow us to account for ‘conditional dependence’ between two or more diagnostic tests, meaning that the results from tests are correlated even after conditioning on the person’s true disease status. The challenge is that it is not always clear to researchers whether conditional dependence exists between tests and whether it exists in all or just some latent classes. Despite the increasingly widespread use of latent class models to estimate diagnostic test accuracy, the impact of the conditional dependence structure chosen on the estimates of sensitivity and specificity remains poorly investigated. Methods: A simulation study and a reanalysis of a published case study are used to highlight the impact of the conditional dependence structure chosen on estimates of sensitivity and specificity. We describe and implement three latent class random-effect models with differing conditional dependence structures, as well as a conditional independence model and a model that assumes perfect test accuracy. We assess the bias and coverage of each model in estimating sensitivity and specificity across different data generating mechanisms. Results: The findings highlight that assuming conditional independence between tests within a latent class, where conditional dependence exists, results in biased estimates of sensitivity and specificity and poor coverage. The simulations also reiterate the substantial bias in estimates of sensitivity and specificity when incorrectly assuming a reference test is perfect. The motivating example of tests for Melioidosis highlights these biases in practice with important differences found in estimated test accuracy under different model choices. Conclusions: We have illustrated that misspecification of the conditional dependence structure leads to biased estimates of sensitivity and specificity when there is a correlation between tests. Due to the minimal loss in precision seen by using a more general model, we recommend accounting for conditional dependence even if researchers are unsure of its presence or it is only expected at minimal levels.
Published: 2 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-12; https://doi.org/10.1186/s12874-023-01867-y
Background: Combinations of drugs are becoming increasingly common in oncology treatment. In some cases, patients can benefit from the interaction between two drugs, although there is usually a higher risk of developing toxicity. Due to drug–drug interactions, multidrug combinations often exhibit different toxicity profiles than those of single drugs, leading to a complex trial scenario. Numerous methods have been proposed for the design of phase I drug combination trials. For example, the two-dimensional Bayesian optimal interval design for combination drug (BOINcomb) is simple to implement and has desirable performance. However, in scenarios where the lowest and starting dose is close to being toxic, the BOINcomb design may tend to allocate more patients to overly toxic doses, and select an overly toxic dose combination as the maximum tolerated dose combination. Method: To improve the performance of BOINcomb in the above extreme scenarios, we widen the range of variation of the boundaries by setting the self-shrinking dose escalation and de-escalation boundaries. We refer to the new design as adaptive shrinking Bayesian optimal interval design for combination drug (asBOINcomb). We conduct a simulation study to evaluate the performance of the proposed design using a real clinical trial example. Results: Our simulation results show that asBOINcomb is more accurate and stable than BOINcomb, especially in some extreme scenarios. Specifically, in all ten scenarios, the percentage of correct selection is higher than the BOINcomb design within 30 to 60 patients. Conclusion: The proposed asBOINcomb design is transparent and simple to implement and can reduce the trial sample size while maintaining accuracy compared with the BOINcomb design.
Published: 1 March 2023
BMC Medical Research Methodology, Volume 23, pp 1-10; https://doi.org/10.1186/s12874-023-01875-y
Background: Science is becoming increasingly data intensive as digital innovations bring new capacity for continuous data generation and storage. This progress also brings challenges, as many scientific initiatives are challenged by the shear volumes of data produced. Here we present a case study of a data intensive randomized clinical trial assessing the utility of continuous pressure imaging (CPI) for reducing pressure injuries. Objective: To explore an approach to reducing the amount of CPI data required for analyses to a manageable size without loss of critical information using a nested subset of pressure data. Methods: Data from four enrolled study participants excluded from the analytical phase of the study were used to develop an approach to data reduction. A two-step data strategy was used. First, raw data were sampled at different frequencies (5, 30, 60, 120, and 240 s) to identify optimal measurement frequency. Second, similarity between adjacent frames was evaluated using correlation coefficients to identify position changes of enrolled study participants. Data strategy performance was evaluated through visual inspection using heat maps and time series plots. Results: A sampling frequency of every 60 s provided reasonable representation of changes in interface pressure over time. This approach translated to using only 1.7% of the collected data in analyses. In the second step it was found that 160 frames within 24 h represented the pressure states of study participants. In total, only 480 frames from the 72 h of collected data would be needed for analyses without loss of information. Only ~ 0.2% of the raw data collected would be required for assessment of the primary trial outcome. Conclusions: Data reduction is an important component of big data analytics. Our two-step strategy markedly reduced the amount of data required for analyses without loss of information. This data reduction strategy, if validated, could be used in other CPI and other settings where large amounts of both temporal and spatial data must be analysed.
Published: 27 February 2023
BMC Medical Research Methodology, Volume 23, pp 1-10; https://doi.org/10.1186/s12874-023-01863-2
Background: The study aimed to explore reporting characteristics of scoping reviews in dental public health and the impact of some factors on the reporting quality. Methods: This study searched for dental public health scoping reviews in PubMed and Scopus without year restrictions and restricted to English-language publications. Study selection was undertaken by two reviewers independently. One reviewer, after training, extracted data from included studies considering general study characteristics and reporting characteristics. The impact of PRISMA-ScR publication, journal endorsement, and use of study protocol on the reporting was explored. Results: Eighty-one scoping reviews were included. Five items presented rates of appropriate reporting higher than 80% considering the overall percentage. Related to the impact of PRISMA-ScR publication, six items were found more often in scoping reviews published after the publication of PRISMA-ScR than in scoping reviews published before the publication of PRISMA-ScR. With regards to journals endorsement, only two reporting characteristics were found more often in scoping reviews published in journals that endorse the PRISMA-ScR statement than in scoping reviews published in non-endorsers journals. Last, regarding the use of the pre-specified protocol, five reporting characteristics presented differences in studies reporting the use of pre-specified protocol than in studies that did not mention the use of a protocol. All differences were statistically significant. Conclusions: Important information is missing in the included scoping reviews demonstrating crucial reporting problems.
Published: 27 February 2023
BMC Medical Research Methodology, Volume 23, pp 1-10; https://doi.org/10.1186/s12874-023-01874-z
Background: Longitudinal studies are critical to informing evolving responses to COVID-19 but can be hampered by attrition bias, which undermines their reliability for guiding policy and practice. We describe recruitment and retention in the Optimise Study, a longitudinal cohort and social networks study that aimed to inform public health and policy responses to COVID-19. Methods: Optimise recruited adults residing in Victoria, Australia September 01 2020–September 30 2021. High-frequency follow-up data collection included nominating social networks for study participation and completing a follow-up survey and four follow-up diaries each month, plus additional surveys if they tested positive for COVID-19 or were a close contact. This study compared number recruited to a-priori targets as of September 302,021, retention as of December 31 2021, comparing participants retained and not retained, and follow-up survey and diary completion October 2020–December 2021. Retained participants completed a follow-up survey or diary in each of the final three-months of their follow-up time. Attrition was defined by the number of participants not retained, divided by the number who completed a baseline survey by September 302,021. Survey completion was calculated as the proportion of follow-up surveys or diaries sent to participants that were completed between October 2020–December 2021. Results: At September 302,021, 663 participants were recruited and at December 312,021, 563 were retained giving an overall attrition of 15% (n = 100/663). Among the 563 retained, survey completion was 90% (n = 19,354/21,524) for follow-up diaries and 89% (n = 4936/5560) for monthly follow-up surveys. Compared to participants not retained, those retained were older (t-test, p < 0.001), and more likely to be female (χ2, p = 0.001), and tertiary educated (χ2, p = 0.018). Conclusion: High levels of study retention and survey completion demonstrate a willingness to participate in a complex, longitudinal cohort study with high participant burden during a global pandemic. We believe comprehensive follow-up strategies, frequent dissemination of study findings to participants, and unique data collection systems have contributed to high levels of study retention.