Concerns about the Burden of Proof studies

**Background:** This article is a Matters Arising commentary responding to five Burden of Proof (BoP) studies published in Nature Medicine in 2022 (Zheng et al., Lescinsky et al., Stanaway et al., Razo et al., and Dai et al.). The BoP studies conducted meta-analyses of epidemiological evidence for several diet–disease and exposure–disease relationships and converted the results into a star-rating system (1–5 stars). Examples included five stars for smoking and lung cancer, two stars for low vegetable intake and ischemic heart disease (IHD), and two stars for unprocessed red meat and type 2 diabetes (T2D), colorectal cancer, and IHD. The authors of this commentary raise serious concerns about the methodological validity of the BoP approach and argue that the star ratings are overly simplistic, could undermine evidence-based public health recommendations, and fail to incorporate the full totality of evidence considered in standard expert reviews.

**Methods:** The commentary does not present new primary data but instead re-analyzes data from the BoP studies using established meta-analytic methods. For the association between unprocessed red meat and T2D, the authors used the Greenland and Longnecker method to estimate linear dose–response associations for each of the 15 studies included in Lescinsky et al. and then applied a standard random-effects meta-analysis that accounts for uncertainty in the between-study variance estimate. They also critique the BoP approach conceptually by comparing the rating assigned to smoking and IHD (3 stars based on 60 studies) with smoking and lower back pain (also rated 3 stars but based on only 6 studies), arguing this demonstrates the system fails to discriminate between relationships of vastly different evidentiary strength.

**Key Results:** The authors report that for unprocessed red meat and T2D, their standard random-effects meta-analysis yields a relative risk of 1.24 (95% CI 1.12–1.36; P<0.001), with confidence intervals only modestly wider than those from a fixed-effects model. They note this contrasts sharply with the BoP estimates, in which the uncertainty intervals were inflated approximately 2.5-fold relative to conventional confidence intervals, causing the lower boundary to include no association and resulting in a 2-star rating. For smoking and IHD, the authors note that the BoP assigned only 3 stars despite relative risks exceeding 5 for high-intensity smoking, a relationship recognized as causal by U.S. Surgeon General reports for over four decades with at least six documented mechanistic pathways. Additionally, in the meta-analysis of red meat and six chronic disease outcomes, none of the bias adjustments applied in the BoP framework were statistically significant, and there was minimal evidence of publication bias or influential outlier studies, suggesting the BoP assumption of extreme between-study heterogeneity is untenable.

**Clinical Implications:** The authors argue that the BoP star-rating system could mislead policy-makers and the public by implying that well-established relationships (such as smoking causing IHD, or red meat consumption increasing risk of T2D) are too uncertain to warrant strong public health recommendations. They contend that the BoP methodology, which produces uncertainty intervals so wide they encompass findings from almost all existing studies including those near the null, may be insensitive to new evidence. The authors advocate for approaches that: (1) use individual-level data from original studies with standardized exposure and covariate definitions; (2) employ dose–response meta-analyses of continuous exposures; (3) consider substitution analyses accounting for alternative foods given tightly regulated energy intake; and (4) incorporate the totality of evidence beyond epidemiology, including randomized controlled trials of intermediate risk factors (e.g., LDL cholesterol), animal studies, and biological mechanisms following Bradford Hill criteria. The commentary concludes that reviews by national and international bodies that integrate the full range of evidence provide a more reliable basis for personal and policy decisions than the BoP framework.