Strong recommendations from low certainty evidence: a cross-sectional analysis of a suite of national guidelines

**Background:** Clinical guidelines should be based on thorough evidence evaluation and typically include ratings of evidence quality and recommendation strength. The GRADE approach warns against making strong recommendations when evidence certainty is low or very low, but identifies five paradigmatic situations where this may be justified (e.g., life-threatening situations). This study aimed to characterize the strength of recommendations and certainty of evidence in Irish National Clinical Guidelines using GRADE.

**Methods:** This was a cross-sectional analysis of all National Clinical Guidelines from the National Clinical Effectiveness Committee (NCEC) website that used GRADE (fully or partially). As of June 30, 2022, 29 NCECs were available; 8 published between 2019 and 2022 met inclusion criteria. All recommendations and their corresponding certainty of evidence, strength of recommendations, and justifications were extracted. Authors classified instances of strong recommendations with low certainty evidence (discordant recommendations) into one of the five paradigmatic situations. Descriptive statistics were calculated. Data was extracted by one reviewer and cross-checked by another.

**Key Results:** From the 8 guidelines using GRADE, there were 240 total recommendations; 38 were excluded (18 best practice statements, others not using GRADE), leaving 202 recommendations. Of these, 151 (74.7%) were classified as strong and 51 (25.3%) as conditional. Of the 151 strong recommendations, 55 (36.4%) were supported by high or moderate certainty evidence and 96 (63.6%) by low or very low certainty evidence (considered discordant). Specifically, 60 (39.7%) had very low certainty, 18 (11.9%) had low certainty, and 18 (11.9%) had no studies. Of the 51 weak/conditional recommendations, 1 (2.0%) was based on high certainty evidence. The proportion of discordant recommendations varied across guidelines (6–100%). Of the 78 discordant recommendations based on low or very low certainty (excluding no studies), 55 (73.7%) were consistent with one of the five paradigmatic situations: 63.1% as life-threatening, 24.6% as uncertain benefit/certain harm, 10.5% as potential equivalence/less risky or costly, 1.7% as high certainty similar benefits/one option more risky, and 0% as potential catastrophic harm. None of the guidelines explicitly referenced any paradigmatic situation. Four of eight guidelines made a completed Evidence to Decision (EtD) framework available.

**Clinical Implications:** The proportion of discordant recommendations (63.6%) was higher than some previous international studies (range 30–50%) but similar to other guidelines focused on emergency situations. Half of the included guidelines focused on emergency situations, where robust evidence is often lacking. While most discordant recommendations could be mapped to a justifiable paradigmatic situation, the lack of explicit referencing within guidelines reduces transparency and trustworthiness. Guideline developers require further training and guidance to enable greater transparency in reporting reasons for discordant recommendations, particularly through explicit use of the five paradigmatic situations. The findings may reflect a learning curve with GRADE since its recommendation by the Irish Department of Health in 2019.