Meta-analysis prediction intervals are under reported in sport and exercise medicine
Permanent link
https://hdl.handle.net/10037/35043Date
2024-03-19Type
Journal articleTidsskriftartikkel
Peer reviewed
Author
Borg, David N.; Impellizzeri, Franco M.; Borg, Samantha J.; Hutchins, Kate P.; Stewart, Ian B.; Jones, Tamara; Baguley, Brenton J.; Orssatto, Lucas B. R.; Bach, Aaron J. E.; Osborne, John Owen; McMaster, Benjamin S.; Buhmann, Robert L.; Bon, Joshua J.; Barnett, Adrian G.Abstract
Methods: We screened, at random, 1500 meta-analysis studies published between 2012 and 2022 in highly ranked sports medicine and medical journals. Articles that used a random effect meta-analysis model were included in the study. We randomly selected one meta-analysis from each article to extract data from, which included the number of estimates, the pooled effect, and the confidence and prediction interval.
Results: Of the 1500 articles screened, 866 (514 from sports medicine) used a random effect model. The probability of a prediction interval being reported in sports medicine was 1.7% (95% CI=0.9%, 3.3%). In medicine the probability was 3.9% (95% CI=2.4%, 6.6%). A prediction interval was able to be calculated for 220 sports medicine studies. For 60% of these studies, there was a discrepancy in study findings between the reported confidence interval and the calculated prediction interval. Prediction intervals were 3.4 times wider than confidence intervals.
Conclusion: Very few meta-analyses report prediction intervals and hence are prone to missing the impact of between-study heterogeneity on the overall conclusions. The widespread misinterpretation of random effect meta-analyses could mean that potentially harmful treatments, or those lacking a sufficient evidence base, are being used in practice. Authors, reviewers, and editors should be aware of the importance of prediction intervals.