The use and misuse of methods for publication bias assessment and adjustment in meta-analyses of psychotherapeutic interventions: A systematic survey of the literature

Abstract

Publication bias poses a threat to the validity of meta-analytic reviews, as it can lead to summary effect size estimates becoming inflated. Meta-analysts are advised to utilize multiple methods for detecting and controlling for publication bias. Our study aims to examine which and how many methods meta-analysts of psychotherapeutic interventions for depression, anxiety and PTSD utilize to identify and correct for publication bias, and to which extent they detect it. Additionally, we aim to provide some indication of the degree to which publication bias has (or has not) influenced meta-analytic estimates in this field, by reanalyzing meta-analyses for which study level data are available. 86 meta-analyses were included in our sample, and 37 meta-analyses also met the eligibility criteria for reanalysis. Findings demonstrate that 66 of 86 (76,7%) included meta-analyses utilized at least 1 publication bias method. 32 of 86 (37%) of the included meta-analyses utilized at least three publication bias methods. None of the included meta-analyses utilized a selection model approach. The funnel plot asymmetry tests varied from detecting publication bias on ~20% (Egger’s regression) to ~65% (trim-and-fill). The results from reanalyzes of study-level data indicates some inflation of effect size estimates, although the adjusted results generally do not considerably change the overall conclusions of these meta-analyses. Our overall findings indicate some degree of publication bias, that could go undetected because some meta-analysts do not sufficiently adhere to recommendations regarding publication bias methods.