Risk stratification tools for predicting bleeding events in people with atrial fibrillation

National Guideline Centre (UK)

Risk stratification tools for predicting bleeding events in people with atrial fibrillation

Atrial fibrillation: diagnosis and management

Evidence reviews E&F

NICE Guideline, No. 196

Authors

National Guideline Centre (UK).

London: National Institute for Health and Care Excellence (NICE); 2021 Apr.

ISBN-13: 978-1-4731-4043-1

1. Effectiveness of risk stratification tools for predicting bleeding in people with atrial fibrillation

1.1. Review question: What is the most clinically and cost-effective risk stratification tool for predicting bleeding in people with atrial fibrillation?

1.2. Introduction

Anticoagulation is the therapy with the greatest influence on prognostic outcomes for patients with atrial fibrillation. Anticoagulation,however, is associated with significant risk for major haemorrhage, from one to seven per cent per annum in clinical trials. For the majority of patients with AF the benefits of anticoagulation outweigh this risk.

The risk of major haemorrhage varies among populations with AF and there is a potential to reduce harm further by identifying patients at high risk for whom to proceed with caution, particularly as many risk factors for haemorrhage on anticoagulation are modifiable. There are over twenty schemes & methods (including modifications), published, that attempt to quantify the risk of major haemorrhage on anticoagulation.The predicted risk of haemorrhage for an individual is not precise. It needs to be interpreted in context as many of the factors that increase risk of bleeding also increase the risk of embolic stroke.

The intention of this chapter is to evaluate which is the most clinical and cost effective method and to develop guidance as to how this informs clinical practice.

1.3. PICO table

For full details see the review protocol in appendix A.

Table 1

PICO characteristics of review question.

1.4. Methods and process

This evidence review was developed using the methods and process described in Developing NICE guidelines: the manual.⁸⁹Methods specific to this review question are described in the review protocol in appendix A.

This review is not a ‘prognostic accuracy’ review, but is instead a review of trials that have compared later health outcomes in people randomised to different prediction tools. Tools with differing prognostic accuracies may differ in their influence on later health outcomes through stimulating a more or less appropriate treatment approach. Whilst accuracy is not measured directly in such randomised trials, the advantage of such studies is that they demonstrate clinical efficacy. In contrast a prognostic accuracy study can only demonstrate the intrinsic predictive accuracy of the tool and is unable to show how that the accuracy affects health outcomes. However such randomised trials are not commonly undertaken, and may provide equivocal results, and so a prognostic accuracy review has also been undertaken.

Declarations of interest were recorded according to NICE’s 2018⁸⁹conflicts of interest policy.

1.5. Clinical evidence

1.5.1. Included studies

No relevant comparative clinical studies comparing bleeding risk tools with HAS-BLED were identified.

See also the study selection flow chart in appendix C, study evidence tables in appendix D, forest plots in appendix E and GRADE tables in appendix H.

1.5.2. Excluded studies

See the excluded studies list in appendix I.

1.5.3. Summary of clinical studies included in the evidence review

No studies were included

1.5.4. Quality assessment of clinical studies included in the evidence review

Not applicable.

See appendix F for full GRADE tables.

1.6. Economic evidence

1.7. Included studies

No relevant health economic studies were identified.

1.8. Excluded studies

No health economic studies that were relevant to this question were excluded due to assessment of limited applicability or methodological limitations.

See also the health economic study selection flow chart in appendix G.

1.8.1. Unit costs

Outlined in Table 2is a description of each risk tool and any additional healthcare resources required. As demonstrated in the table most risk tools require a review of the person’s medical history and in some cases computer access to complete algorithms. Only the ABC bleeding risk score required additional tests (biomarker assays), which would be an additional cost to the NHS.

Table 2

Bleeding risk tools.

2. Accuracy of risk stratification tools for predicting bleeding events in people with atrial fibrillation

2.1. Introduction

See evidence review E.

2.2. Review question: What is the most accurate risk stratification tool for predicting bleedingevents in people with atrial fibrillation?

For full details see review protocol in Appendix A.

Table 3

PICO characteristics of review question.

2.3. Clinical evidence

We searched for cohort studies covering the validation of risk assessment tools for bleeding in people with AF. 54studies evaluating the accuracy of bleedingrisk tools for people with atrial fibrillation were included in the review³^, ⁵^, ⁸^, ¹¹^, ¹⁴^, ¹⁹^-²¹^, ²³^, ²⁵^, ³⁰^-³³^, ³⁶^-³⁹^, ⁴¹^, ⁵²^, ⁵⁴^, ⁵⁶^-⁵⁸^, ⁶³^, ⁶⁵^, ⁷¹^, ⁷⁴^, ⁷⁷^, ⁸⁸^, ⁹⁰^, ⁹¹^, ⁹⁵^, ¹⁰³^, ¹¹⁰^, ¹¹³^-¹¹⁷^, ¹¹⁹^, ¹²⁰^, ¹²⁵^, ¹²⁶^, ¹²⁸^, ¹³⁵^-¹³⁸^, ¹⁴²^, ¹⁴⁶^, ¹⁴⁷^, ¹⁵⁴^, ¹⁵⁸whichare summarised in Table 4 below.The different risk schemes are outlined in Table 3.Evidence from these studies is summarised in the GRADE clinical evidence profilesbelow (Tables 4-13). See also the study selection flow chart in Appendix B, study evidence tables in Appendix E, forest plots in Appendix D, and excluded studies list in Appendix H.

This review evaluates the accuracy of the risk tools to predict bleeding, with reference to their discriminatory capabilities (sensitivity, specificity, and C statistics), calibration statistics and Atrial fibrillation update the Net Reclassification Index. The reference standard was the incidence (or not) of major bleeding (or other bleeding categories) at follow up.Only studies where all patients were anticoagulated (or where an anticoagulated sub-group were a separately analysed) were included; this was because the aim of the review is to establish which tool can best predict bleeding in those people who are taking anticoagulation.

Analyses were by cohort rather than study; that is, where a study included separate analyses for different OACs, these were analysed as separate cohorts (as if they were separate studies). This approach facilitated sub-grouping for different OACs if heterogeneity was detected.

For sub-grouping by OAC, cohorts were categorised into 1) VKA cohorts, 2) Mixed VKA/DOAC/unclear category cohorts and 3) DOACcohorts. For sub-grouping by antiplatelets use, cohorts were categorised into 1) cohorts with <33% on antiplatelets/NSAIDs/aspirin, 2)cohorts with >33%on antiplatelets, and 3) cohorts where the number on antiplatelets were not reported.

Separate analyses were performed for 1) major bleeding, 2) clinically relevant bleeding and 3) intracranial bleeding. Data concerning other forms of bleeding were not analysed in this review as they were deemed to overlap with these 3 categories, though available dataare outlined in the clinical evidence tables.

Summary of included studies

Table 4

Summary of studies included in the review.

Table 5

Summary of risk tools and their constituent variables.

2.3.1. Discriminationfor MAJOR BLEEDING

Table 6

Clinical evidence profile: accuracy of prediction of Major Bleedingin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I²to <50% in all sub-groups.

Table 7

Clinical evidence profile: sensitivity and specificityof prediction of Major Bleeding in all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results; for meta-analysed results the 95% credible intervals are given for (more...)

2.3.2. Calibrationfor MAJOR BLEEDING

Calibration waspredominantlyreportedwith graphical rather than numerical data. Hence this section has been dealt with narratively.

Several studies merely reported a non-comparative‘adequate’calibration, usuallybased on a Hosmer-Lemeshow p value >0.05. ‘Adequate’ goodness of fit was thus described for ATRIA⁴^, ¹⁴^, ⁶³, HAS-BLED⁴^, ¹⁴^, ⁶³^, ⁷¹, HEMORRHAGES⁴^, ¹⁴^, ⁶³^, ⁷¹, ORBIT¹⁴, Shireman⁷¹, mOBRI/Beyth⁷¹, Kuijer⁷¹and ABC¹¹^, ²³^, ⁵⁴. It was not possible, based on these data, to compare thelevels of calibration acrossthese tools.

However, some studies performed a relative, albeit qualitatively described,evaluation, which was based on inspection of calibration plots. Hilkens, 2017⁵⁸stated that ORBIT had a better calibration at 2 years than HEMORRHAGES, ATRIA, Shireman and HAS-BLED. ORBIT was also regarded as better calibrated than HAS-BLED and ATRIA by fourfurther studies,⁷⁷^, ⁹¹^, ¹¹⁴^, ¹⁵⁸although Mori, 2019⁸⁸did not note a difference.ATRIA was identified as the least wellcalibrated by twoof the studies⁹¹^, ¹⁵⁸but better than HAS-BLED by one¹¹⁴. Proietti 2018¹¹⁴noted that whilst ORBIT had the best calibration over all risk strata, HEMORRHAGES tended to underestimate risk, particularly in patients with a higher predicted risk, whereas ATRIA and HAS-BLED tended to over-estimate bleeding risk. Similarly, O’Brien⁹¹noted that whilst ORBIT was good at predicting risk in all risk strata, HAS-BLED tended to have worse calibration in low-risk strata, and ATRIA performed badly at mostrisk strata. Claxton, 2018²³evaluated the calibration of the Anticoagulation-specific bleeding score (ASBS) alone, demonstrating good calibration. Calibration plots are shown below.

Note that Lip, 2018⁷⁷, Mori, 2019⁸⁸and Yao, 2017¹⁵⁸only used DOACcohorts, but O’Brien, 2015⁹¹and Claxton, 2018²³used a mixed cohort. Both Hilkens, 2017⁵⁸and Proietti, 2018¹¹⁴contained separate cohorts of patients taking dabigatran and warfarin, but it appears that the plots reproduced below were from their total, mixed, cohort. It should also be noted that Proietti 2018¹¹⁴failed to specify if calibration data referredto major bleeding, although major bleedingis assumedto be the most likely bleeding

Source: Calibration plot in Claxton, 2018²³. This was for the Anticoagulation-specific bleeding score and was based on a mixed (VKA and DOAC) cohort.

Source: Calibration plot in Hilkens, 2017⁵⁸. This was based on a mixed (VKA and DOAC) cohort.

Source: Calibration plot in Proietti et al. 2018¹¹⁴(bleeding risk scores calibration between derivation cohorts and RE-LY cohort events rates). This probably relates to their total, mixed, cohort.

Figure 1

<Insert graphic title here>.

Source: Calibration plot in Lip, 2018⁷⁷. This was based on an exclusively DOAC-using cohort.

Source: Calibration plot in Yao, 2017¹⁵⁸. This was based on an exclusively DOAC-using cohort.

2.3.3. Net Reclassification improvementfor MAJOR BLEEDING

Several studies reported the Net Reclassification Improvement (NRI). This is expressed in terms of one (index) risk tool to another (comparator) risk tool, and gives a score between −2 and +2 (with +2 representing the best possible performance of the index tool relative to the comparator, and −2 the worst). The score represents the net improvement of the index test relative to the comparator in terms of the proportion of true cases (judged by later development of bleeding) that are correctly up-classified by the tool (relative to any false negative classifications yielded by the comparator), and the proportion of false cases (judged by the lack of later bleeding) that are correctly down-classified by the tool (relative to any false positive classifications yielded by the comparator). Meanwhile, incorrect up-classification or incorrect down-classification of the index relative to the comparator convey negative scores to the NRI, and so if a score is negative overall this indicates the index is less accurate than the comparator.

Table 8

NRI for major bleeding – HAS-BLED versus other tools.

Table 9

NRI for major bleeding – ATRIA versus other tools.

Table 10

NRI for major bleeding – HEMORRHAGES versus other tools.

Table 11

NRI for major bleeding – ORBIT versus other tools.

Table 12

NRI for major bleeding – CHADSVASC versus other tools.

2.3.4. Discrimination for CLINICALLY RELEVANT BLEEDING

Table 13

Clinical evidence profile: accuracy of prediction of CRBin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups.

Table 14

Clinical evidence profile: sensitivity and specificityof prediction of clinically relevant bleedingin all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results.

2.3.5. Calibration for CLINICALLY RELEVANT BLEEDING

Calibration was poorly reported in most papers, with all papers merely reporting the p value for Hosmer-Lemeshow statistics and proving a qualitative assessment of the relative calibration between tools. All studies simply reported a non-comparative ‘adequate’ calibration, usually based on a Hosmer-Lemeshow p value >0.05. ‘Adequate’ goodness of fit was thus described for ATRIA,⁴^, ¹⁴^, ⁶³HAS-BLED,⁴^, ¹⁴^, ⁶³^, ⁷¹HEMORRHAGES⁴^, ¹⁴^, ⁶³and ORBIT¹⁴. It was not possible, based on these data, to compare thelevels of calibration between these tools.

2.3.6. Net Reclassification improvement for CLINICALLY RELEVANT BLEEDING

Table 15

NRI for clinically relevant bleeding.

2.3.7. Discrimination for INTRACRANIAL HEMORRHAGE

Table 16

Clinical evidence profile: accuracy of prediction of ICHin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups.

Table 17

Clinical evidence profile: sensitivity and specificityof prediction of intracranial haemmorhagein all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results.

2.3.8. Calibration for INTRACRANIAL HEMORRHAGE

Proietti et al 2018¹¹⁴reported that the ORBIT score had best agreement between predicted and observed risks, that ATRIA had worst agreement and thatATRIA and HAS-BLED tended to overestimate the risk of bleeding. Meanwhile, HEMORRHAGES tended to underestimate bleeding risk. However it was unclear if this related specifically to intracranial bleeding.

2.3.9. Net Reclassification improvement for INTRACRANIAL HEMORRHAGE

Table 18

NRI for intracranial bleeding.

2.4. Economic evidence

2.4.1. Included studies

No relevant health economic studies were identified.

2.4.2. Excluded studies

No health economic studies that were relevant to this question were excluded due to assessment of limited applicability or methodological limitations.

See also the health economic study selection flow chart in appendix D.

2.4.3. Unit costs

See 1.8.1.

2.5. The committee’s discussion of the evidence

2.5.1. Interpreting the evidence

2.5.1.1. The outcomes that matter most

No clinical evidence was generated by thereviewon the effectiveness of risk stratification tool for predicting bleeding. The committee discussed the predictive accuracy evidence only, as this was felt to be sufficient to inform recommendations relevant to the most appropriate methods to predict bleeding in people with AF, without the need for any consensus recommendations or research recommendations pertaining to the effectivenessreview.

The committee agreed that the most critical predictive accuracy outcome measures for decision-making were calibration data. This was because the committee agreed that the best use of bleeding risk tools was as a means to guide a shared patient/clinician plan for alleviating reversible risk factors for bleeding; such a plan would require an accurate measure of absolute risk, the accuracy of which is best measured by calibration outcome data. Accurate binary decision-thresholds, such as those measured by discrimination outcome data (C statistics or sensitivity/specificity) were regarded as less critical, given that bleeding risk tools were not regarded as a decision aid for anticoagulant use (see second paragraph in section2.5.1.3). Net reclassification improvement (NRI) data, although also less critical than calibration data, was regarded as slightly more important than C statistics or sensitivity/specificity because of its propensity to sensitively differentiate the accuracy of different tools.

2.5.1.2. The quality of the evidence

Evidence was generally deemed low or very low quality. Risk of bias was serious or very serious due to unclear methodology in terms of blinding of risk tool and outcome data, and in many studies the follow up time was short (<5 years) or involved few events (<100). The quality was also affected by serious or very serious heterogeneity.

2.5.1.3. Benefits and harms

The benefit of an accurate estimation of bleeding risk is that this may prompt appropriate and directed alleviation of any reversible causes of bleeding, as well as allowing appropriate levels of vigilance during anticoagulation. One possible disadvantage (harm) of using bleeding risk tools is underestimating bleeding risk, which may lead to insufficient attention to preventable risk factors and insufficient monitoring. Another potential harm is over-estimating bleeding risk, which can lead to unnecessary over-vigilance and possibly reluctance on the part of the patient (and maybe clinician) to commence anticoagulation. Thus using accurate bleeding risk prediction tools was seen by the committee as vital to maximise benefits and minimise harms.

The committee discussed the commonly observed clinical practice of using the bleeding risk score as a counterbalance to the stroke risk score, which tends to be done in order to facilitate binary decisions about initiating anticoagulation. The drawbacks of this were discussed. Comparisons of the actual bleeding and stroke risk tool scores were regarded by the committee as largely meaningless, given the varying significance of scores across different tools. In addition, comparison of absolute stroke and bleeding risks (derived from the scores) was also regarded as potentially misleading in the context of a decision to anti-coagulate, because bleeding risk includes the risk of bleeding events of lower severity than a stroke. Thus, for example, the committee noted that an equal absolute risk of stroke and bleeding would not necessarily represent equipoise, as the two competing events might not be of comparable severity. Any assessment of risk must also weigh up the probability of an event occurring and consider the consequences of the event occurring. The committee reiterated the importance of using a bleeding risk tool to inform plans to reduce reversible causes of bleeding and to maintain appropriate levels of vigilanceduring anticoagulation, and that it should not be used as a threshold-based tool to determine if anticoagulation should take place.

The committee noted the importance of respecting any decision by an individual not to take anticoagulants. The committee were aware of the recommendations on tailoring healthcare services to the individual in the NICE guideline on patient experience of adult services (CG138).

Committee discussion focussed on tools where the weight of evidence was sufficient to warrant a recommendation. Therefore for tools that had been investigated in only one or two smaller studies, relatively little consideration was given to their possible useeven if predictive accuracy was encouraging. In addition, for those tools with larger amountsof evidence, the clearly less effective tools such as HEMORRHAGES(which had poorer calibration than ORBIT, HASBLED and ATRIA, as well as inferior discriminationand NRI)were given less consideration. Discussion focussed on three main tools: ORBIT, HAS-BLED and ATRIA, with the emphasis, as previously justified, on calibration data.

The calibration evidence suggested that ORBIT was better than HASBLED and ATRIA inaccurately predictingrisk of major bleeding. This was found in both mixed cohorts and DOAC-only cohorts. Importantly, ORBIT was better calibrated at all, and particularly higher, levels of risk. Given the relevance of calibration outcomes to the intended use of the tools - allowing an informed discussion about reversing modifiable risk factors and having an appropriate level of monitoring as a result of an accurate assessment of absolute risk - this finding was an important factor in the recommendation decision. Discrimination data were also discussed, and the committee agreed that the C statistics data supported the calibration data’s indication that ORBIT was the most appropriate tool. Although the C-statisticsevidence suggested little to choose between HAS-BLED, ATRIA and ORBIT for people on VKAs, the C statisticsevidence suggested that ORBIT was the most accurate tool to use for patients on DOACs. The committee noted that around 90% of patients were currently on DOACS, and that this proportion would continue to increase with time. Hence this supported ORBIT beingregarded as the most appropriate bleeding risk tool for current and future patients.The sensitivity and specificity data at the established thresholds suggested that HAS-BLED and other tools might be more sensitive than ORBIT in predicting who will bleed whilst on anticoagulants, but this was counterbalanced bythe greater specificity of ORBIT. In contrast to the situation when predicting strokes, reduced sensitivity of bleeding risk prediction was not regarded as a serious problem because failure to detect high bleeding risk would not necessarily change decisions. This was because prediction of bleeding would not be used to withhold anticoagulants; instead, the risk prediction would be used as an objective aid to discussion with the patient about the need to modify bleeding risks and to be vigilant about possible bleeding. Meanwhile, the NRI evidence was fairly equivocal, suggesting similarities between ORBIT and HAS-BLED, and the committee felt that it did not negate the calibration evidence that ORBIT was the most appropriate tool.

There was some discussion about a two-tier recommendation – recommending ORBIT for people on DOACs and continuing with HAS-BLED for those patients restricted to VKAs (given that HAS-BLED appears to be as accurate, based on discrimination data, as ORBIT and ATRIA in VKA populations). This idea was rejected, partly because it was believed that the people who would currently be given VKAs would tend to be different from the VKA populations in the included studies. The VKA study populations tended to be fairly typical samples of people with NVAF, because VKAs were the principal anticoagulant therapy available at the time of these studies. In contrast, patients currently being given VKAs would tend to be atypical (for example, people with serious renal dysfunction). The committee therefore believed that the evidence suggesting HAS-BLED might be appropriate for people on VKAs was not relevant to current users of VKAs. In addition, ORBIT was superior when measured by calibration outcomes in mixed cohorts. Given the greater relevance of calibration outcomes to the purported usage of bleeding risk tools, this strongly supported the decision to recommend ORBIT for all patients.

In addition to recommending ORBIT as a bleeding prediction tool, the committee also made recommendations on addressing the modifiable bleeding risk factors inherent in ORBIT, as well as the modifiable bleeding risk factors listed in the 2014 recommendations. Although the 2014 bleeding risk factors were related to the HAS-BLED, all were still thought to be relevant to a shared clinical decision on alleviating bleeding risk factors. Reversible causes of anaemia were listed as an additional modifiable risk factor as anaemia is a component of the ORBIT tool.

The committee were of the opinion that the decision to withhold anticoagulation because of concerns over bleeding risk meant depriving a patient of a treatment which, were it not for the bleeding risk, might have been of benefit in stroke prevention. As a number of factors contributing to bleeding risk are dynamic and also potentially correctable, the committee considered that the decision to withhold anticoagulation should not be made in perpetuity but should be subject to regular review and reconsideration as appropriate. They also thought it important that both the review and the outcome of the review should be documented.The committee expressed concern that anticoagulation was often erroneously not initiated due to a perceived high risk of falls, even though a very large number of falls (in excess of 300 per year) are known to be necessary to significantly increase the risk of bleeding. In addition, the committee noted that old age is often used as a reason to not anti-coagulate, even though age is already a factor in the bleeding risk tools used (and therefore would already be accounted for). Therefore the 2014 recommendation that anticoagulation should not be withheld because of the risk of falling was maintained, with an additional note that age should also not be a factor encouraging non-anticoagulation. The committee discussed referring to frailty in the recommendation but given it is so difficult to define they decided against this.

2.5.1.4. Cost effectiveness and resource use

No relevant health economic analyses were identified for this review. The committee discussed the different resource use for the different tests, in particular it was noted that ORBIT required knowledge of whether a patient had reduced haemoglobin or haematocrit. This was not part of the HAS-BLED score, the previously recommended bleeding risk tool, and so would be a change from current practice. The committee noted however that this should be available from patient history and so is unlikely to require additional NHS resource.

The committee also discussed the importance of using the most accurately calibratedbleeding tool as this would help to accurately identify individuals at higher risk of bleeding and therefore prompt the physicians to modify any bleeding risk factors and ensure adequate monitoring is provided. A more accurate tool, as demonstrated with the calibration data presented for ORBIT, would ensure the correct patients are being monitored and so NHS resources would be used more efficiently. That is only those who are truly at higher risk of bleeding are being monitored.

The committee agreed that there was sufficient clinical evidence of superiority for ORBIT to warrant an inevitablechange in practice.It involves measuring some parameters, such as haemoglobin and haematocrit, that are not included in the HAS-BLED tool used in current practice. However, the committee agreed that these factors would be measured routinely for people starting anticoagulation, regardless of the risk tool used, so extra resources are unlikely to be needed.

2.5.2. Other factors the committee took into account

Thecommitteenoted that people from black and ethnic minoritygroups do have a greater risk of stroke but the relationship with atrial fibrillation is unclear. For example, it is not clearif it Atrial fibrillation update isthe presence of comorbidities or ethnic group, or an interaction beween these, that increases the risk of stroke. The committee also noted that a greater proportion of people from black and ethnic minority groups are undiagnosed compared to the general population. This is in part related to who is targeted for screening which is outside of the remit of this guideline.

The use of the ORBIT score is a change in practice, and may lead to some implementation hurdles. One potential problem is that ORBIT does not measure all of the modifiable risk factors previously included in HAS-BLED. At first sight this appears to imply additional testing is needed to ensure that all modifiable risk factors are measured. We would argue that whether ORBIT or HAS-BLED are used does not actually change the amount of modifiable risk factor investigations that need to be carried out by the investigating clinician. For example, full blood count, labile INR, blood pressure, liver function tests and renal function tests will need to be carried out in either case to evaluate whether current bleeding, increased blood pressure or treatable liver or renal disorders are present, each of which can be treated if needed to reduce bleeding risk. The only difference is that the results of labile INR, blood pressure, liver function tests and renal function tests will feed into informing the HAS-BLED score whereas haemoglobin and renal function results (GFR) will feed into the ORBIT score. This does not make ORBIT any more costly in terms of clinician time and resources, as other variables in ORBIT do not require invasive investigations. It could be argued that if the modifiable risk factors are not part of the tool then clinicians will not be prompted to discuss their modification. This is unlikely provided good practice is observed, as knowledge of the modifiable risk factors of bleeding is a basic clinical skill for any clinician dealing with AF patients, and such prompting should not be necessary. Another potential problem is that recommended bleeding risk evaluation for other conditions (such as venous thromboembolism) does not use ORBIT. This means that if ORBIT is used for AF, another tool (such as HAS-BLED) has to be used for other conditions. We would argue that if other tools need to be used for other conditions this does not constitute a major hurdle for clinicians, as the use of these tools is not difficult, and access to the online versions is straightforward. Nevertheless, to avoid clinician confusion with the unfamiliar tool, there will be a need for an initial transition period when new practices are being learned. This may require re-education in both primary and secondary care, which will have a resource impact, although this will be a time-limited impact, as each clinician will require limited training. Finally, unlike HAS-BLED, ORBIT is not embedded in the GP system. This will initially lead to the need to work outside this system, causing some practical difficulties. It is hoped, however, that ORBIT will eventually become embedded in the GP system. Again, this will have a resource impact, but given that centralised software changes are unlikely to be too difficult, the impact is not believed to be too large. Whilst implementation of ORBIT will provide some challenges, these should be overcome by the advantages of a tool that can provide a more accurate measure of bleeding risk.

References

1.: Abumuaileq RRY, Abu-Assi E, Raposeiras-Roubin S, Lopez-Lopez A, Redondo-Dieguez A, Alvarez-Iglesias D et al. Comparative evaluation of HAS-BLED and ATRIA scores by investigating the full potential of their bleeding prediction schemes in non-valvular atrial fibrillation patients on vitamin-K antagonists. International Journal of Cardiology. 2014; 176(3):1259–1261 [PubMed: 25139324]

2.: Al-Turaiki AM, Al-Ammari MA, Al-Harbi SA, Khalidi NS, Alkatheri AM, Aldebasi TM et al. Assessment and comparison of CHADS2, CHA2DS2-VASc, and HAS-BLED scores in patients with atrial fibrillation in Saudi Arabia. Annals of Thoracic Medicine. 2016; 11(2):146–150 [PMC free article: PMC4854062] [PubMed: 27168864]

3.: Apostolakis S, Lane DA, Buller H, Lip GYH. Comparison of the CHADS2, CHA2DS2 VASc and HAS-BLED scores for the prediction of clinically relevant bleeding in anticoagulated patients with atrial fibrillation: The AMADEUS trial. Thrombosis and Haemostasis. 2013; 110(5):1074–1079 [PubMed: 24048467]

4.: Apostolakis S, Lane DA, Guo Y, Buller H, Lip GY. Performance of the HEMORR(2)HAGES, ATRIA, and HAS-BLED bleeding risk-prediction scores in patients with atrial fibrillation undergoing anticoagulation: the AMADEUS (evaluating the use of SR34006 compared to warfarin or acenocoumarol in patients with atrial fibrillation) study. Journal of the American College of Cardiology. 2012; 60(9):861–867 [PubMed: 22858389]

5.: Asuncion Esteve-Pastor M, Miguel Rivera-Caravaca J, Roldan V, Vicente V, Valdes M, Marin F et al. Long-term bleeding risk prediction in ‘real world’ patients with atrial fibrillation: comparison of the HAS-BLED and ABC-Bleeding risk scores. Thrombosis and Haemostasis. 2017; 117(10):1848–1858 [PubMed: 28799620]

6.: Atzema CL, Dorian P, Fang J, Tu JV, Lee DS, Chong AS et al. A clinical decision instrument to predict 30-day death and cardiovascular hospitalizations after an emergency department visit for atrial fibrillation: the atrial fibrillation in the emergency room, part 2 (AFTER2) study. American Heart Journal. 2018; 203:85–92 [PubMed: 30053692]

7.: Banerjee A, Fauchier L, Bernard-Brunet A, Clementy N, Lip GY. Composite risk scores and composite endpoints in the risk prediction of outcomes in anticoagulated patients with atrial fibrillation. The Loire Valley Atrial Fibrillation Project. Thrombosis and Haemostasis. 2014; 111(3):549–556 [PubMed: 24452108]

8.: Barnes GD, Gu X, Haymart B, Kline-Rogers E, Almany S, Kozlowski J et al. The predictive ability of the CHADS2 and CHA2DS2-VASc scores for bleeding risk in atrial fibrillation: the MAQI(2) experience. Thrombosis Research. 2014; 134(2):294–299 [PubMed: 24929840]

9.: Benezet-Mazuecos J, Cinza S, Marin F, Ruiz Diaz MA, Chaves J, Fernandez De Cabo S. Validation of a screening tool (ICUSI questionnaire) for AF patients receiving vitaminin K antagonist therapy. Value in Health. 2017; 20(9):A625

10.: Benito-Gonzalez T, Estevez-Loureiro R, de Prado AP, Minguito-Carazo C, Del Castillo Garcia S, Garrote-Coloma C et al. Incidence and prognostic implications of late bleeding events after percutaneous mitral valve repair. International Journal of Cardiology Heart and Vasculature. 2018; 21:16–21 [PMC free article: PMC6148729] [PubMed: 30255126]

11.: Berg DD, Ruff CT, Jarolim P, Giugliano RP, Nordio F, Lanz HJ et al. Performance of the ABC scores for assessing the risk of stroke or systemic embolism and bleeding in patients with atrial fibrillation in ENGAGE AF-TIMI 48. Circulation. 2019; 139(6):760–771 [PMC free article: PMC6363338] [PubMed: 30586727]

12.: Bernaitis N, Ching CK, Badrick T, Anoopkumar-Dukie S. Identifying warfarin control with stroke and bleed risk scores. Heart, Lung and Circulation. 2018; 27(6):756–759 [PubMed: 29233496]

13.: Bernaitis N, Ching CK, Chen L, Hon JS, Teo SC, Badrick T et al. A high HASBLED score identifies poor warfarin control in patients treated for non-valvular atrial fibrillation in Australia and Singapore. Basic and Clinical Pharmacology and Toxicology. 2017; 121(6):499–504 [PubMed: 28639436]

14.: Beshir SA, Aziz Z, Yap LB, Chee KH, Lo YL. Evaluation of the predictive performance of bleeding risk scores in patients with non-valvular atrial fibrillation on oral anticoagulants. Journal of Clinical Pharmacy and Therapeutics. 2018; 43(2):209–219 [PubMed: 29030869]

15.: Burgess S, Crown N, Louzada ML, Dresser G, Kim RB, Lazo-Langner A. Clinical performance of bleeding risk scores for predicting major and clinically relevant non-major bleeding events in patients receiving warfarin. Journal of Thrombosis and Haemostasis. 2013; 11(9):1647–1654 [PubMed: 23848301]

16.: Caldeira D, Costa J, Fernandes RM, Pinto FJ, Ferreira JJ. Performance of the HAS-BLED high bleeding-risk category, compared to ATRIA and HEMORR2HAGES in patients with atrial fibrillation: a systematic review and meta-analysis. Journal of Interventional Cardiac Electrophysiology. 2014; 40(3):277–284 [PubMed: 25012972]

17.: Camelo-Castillo A, Rivera-Caravaca JM, Marin F, Vicente V, Lip GYH, Roldan V. Predicting Adverse Events beyond Stroke and Bleeding with the ABC-Stroke and ABC-Bleeding Scores in Patients with Atrial Fibrillation: The Murcia AF Project. Thrombosis and Haemostasis. 2020; 120(8):1200–1207 [PubMed: 32506417]

18.: Candeias Faria D, Ferreira J, Beringuilho M, Augusto J, Roque D, Santos M et al. Atrial fibrillation stroke and bleeding risk scores as predictors of mortality at 3 years follow-up. European Heart Journal. 2018; 39:(Suppl 1):195

19.: Chang YT, Hu YF, Liao JN, Chern CM, Lin YJ, Chang SL et al. The assessment of anticoagulant activity to predict bleeding outcome in atrial fibrillation patients receiving dabigatran etexilate. Blood Coagulation and Fibrinolysis. 2016; 27(4):389–395 [PubMed: 26991859]

20.: Chao TF, Lip GYH, Lin YJ, Chang SL, Lo LW, Hu YF et al. Incident risk factors and major bleeding in patients with atrial fibrillation treated with oral anticoagulants: A comparison of baseline, follow-up and delta HAS-BLED scores with an approach focused on modifiable bleeding risk factors. Thrombosis and Haemostasis. 2018; 118(4):768–777 [PubMed: 29510426]

21.: Chao TF, Lip GYH, Lin YJ, Chang SL, Lo LW, Hu YF et al. Major bleeding and intracranial hemorrhage risk prediction in patients with atrial fibrillation: attention to modifiable bleeding risk factors or use of a bleeding risk stratification score? A nationwide cohort study. International Journal of Cardiology. 2018; 254:157–161 [PubMed: 29407081]

22.: Chia PL, Teoh X, Hua CM, Ching ME, Foo D. Anticoagulation use and predictors of stroke, bleeding and mortality in multi-ethnic Asian patients with atrial fibrillation: A single centre experience. Medical Journal of Malaysia. 2016; 71(5):256–258 [PubMed: 28064291]

23.: Claxton JS, MacLehose RF, Lutsey PL, Norby FL, Chen LY, O’Neal WT et al. A new model to predict major bleeding in patients with atrial fibrillation using warfarin or direct oral anticoagulants. PloS One. 2018; 13(9):e0203599 [PMC free article: PMC6130859] [PubMed: 30199542]

24.: Coleman CI, Vaitsiakhovich T, Nguyen E, Weeda ER, Sood NA, Bunz TJ et al. Agreement between coding schemas used to identify bleeding-related hospitalizations in claims analyses of nonvalvular atrial fibrillation patients. Clinical Cardiology. 2018; 41(1):119–125 [PMC free article: PMC6489698] [PubMed: 29360144]

25.: Dalgaard F, Pieper K, Verheugt F, Camm AJ, Fox KA, Kakkar AK et al. GARFIELD-AF model for prediction of stroke and major bleeding in atrial fibrillation: a Danish nationwide validation study. BMJ Open. 2019; 9(11):e033283 [PMC free article: PMC6858250] [PubMed: 31719095]

26.: Deitelzweig SB, Jing Y, Swindle JP, Makenbaeva D. Reviewing a clinical decision aid for the selection of anticoagulation treatment in patients with nonvalvular atrial fibrillation: applications in a US managed care health plan database. Clinical Therapeutics. 2014; 36(11):1566–1573.e1563 [PubMed: 25438725]

27.: Diemberger I, Fantecchi E, Reggiani MLB, Martignani C, Angeletti A, Massaro G et al. Atrial fibrillation and prediction of mortality by conventional clinical score systems according to the setting of care. International Journal of Cardiology. 2018; 261:73–77 [PubMed: 29572083]

28.: Donze J, Rodondi N, Waeber G, Monney P, Cornuz J, Aujesky D. Scores to predict major bleeding risk during oral anticoagulation therapy: a prospective validation study. American Journal of Medicine. 2012; 125(11):1095–1102 [PubMed: 22939362]

29.: Dukanovic A, Staerk L, Fosbol EL, Gadsboll K, Gislason GH, Olesen JB. Predicted risk of stroke and bleeding and use of oral anticoagulants in atrial fibrillation: Danish nationwide temporal trends 2011–2016. Thrombosis Research. 2017; 160:19–26 [PubMed: 29080549]

30.: Elvira-Ruiz G, Caro-Martinez C, Flores-Blanco PJ, Cerezo-Manchado JJ, Albendin-Iglesias H, Lova-Navarro A et al. Aortic valve stenosis provides complementary information to bleeding risk scores in non-valvular atrial fibrillation patients initiating anticoagulation. Journal of Geriatric Cardiology. 2020; 17(3):141–148 [PMC free article: PMC7118015] [PubMed: 32280330]

31.: Esteve-Pastor MA, Garcia-Fernandez A, Macias M, Sogorb F, Valdes M, Roldan V et al. Is the ORBIT bleeding risk score superior to the HAS-BLED Score in anticoagulated atrial fibrillation patients? Circulation Journal. 2016; 80(10):2102–2108 [PubMed: 27557850]

32.: Esteve-Pastor MA, Rivera-Caravaca JM, Shantsila A, Roldan V, Lip GYH, Marin F. Assessing bleeding risk in atrial fibrillation patients: comparing a bleeding risk score based only on modifiable bleeding risk factors against the has-bled score. The AMADEUS Trial. Thrombosis and Haemostasis. 2017; 117(12):2261–2266 [PubMed: 29212113]

33.: Fang MC, Go AS, Chang Y, Borowsky LH, Pomernacki NK, Udaltsova N et al. A new risk scheme to predict warfarin-associated hemorrhage: The ATRIA (Anticoagulation and Risk Factors in Atrial Fibrillation) Study. Journal of the American College of Cardiology. 2011; 58(4):395–401 [PMC free article: PMC3175766] [PubMed: 21757117]

34.: Fanola CL, Giugliano RP, Ruff CT, Trevisan M, Nordio F, Mercuri MF et al. A novel risk prediction score in atrial fibrillation for a net clinical outcome from the ENGAGE AF-TIMI 48 randomized clinical trial. European Heart Journal. 2017; 38(12):888–896 [PubMed: 28064150]

35.: Fauchier L, Chaize G, Gaudin AF, Vainchtock A, Rushton-Smith SK, Cotte FE. Predictive ability of HAS-BLED, HEMORR2HAGES, and ATRIA bleeding risk scores in patients with atrial fibrillation. A French nationwide cross-sectional study. International Journal of Cardiology. 2016; 217:85–91 [PubMed: 27179213]

36.: Fox KAA, Lucas JE, Pieper KS, Bassand JP, Camm AJ, Fitzmaurice DA et al. Improved risk stratification of patients with atrial fibrillation: an integrated GARFIELD-AF tool for the prediction of mortality, stroke and bleed in patients with and without anticoagulation. BMJ Open. 2017; 7(12):e017157 [PMC free article: PMC5778339] [PubMed: 29273652]

37.: Friberg L, Rosenqvist M, Lip GY. Evaluation of risk stratification schemes for ischaemic stroke and bleeding in 182 678 patients with atrial fibrillation: the Swedish Atrial Fibrillation cohort study. European Heart Journal. 2012; 33(12):1500–1510 [PubMed: 22246443]

38.: Gage BF, Yan Y, Milligan PE, Waterman AD, Culverhouse R, Rich MW et al. Clinical classification schemes for predicting hemorrhage: results from the National Registry of Atrial Fibrillation (NRAF). American Heart Journal. 2006; 151(3):713–719 [PubMed: 16504638]

39.: Gallego P, Roldan V, Torregrosa JM, Galvez J, Valdes M, Vicente V et al. Relation of the HAS-BLED bleeding risk score to major bleeding, cardiovascular events, and mortality in anticoagulated patients with atrial fibrillation. Circulation: Arrhythmia and Electrophysiology. 2012; 5(2):312–318 [PubMed: 22319005]

40.: Garcia-Fernandez A, Marin F, Roldan V, Galcera-Jornet E, Martinez-Martinez JG, Valdes M et al. The HAS-BLED score predicts long-term major bleeding and death in anticoagulated non-valvular atrial fibrillation patients undergoing electrical cardioversion. International Journal of Cardiology. 2016; 217:42–48 [PubMed: 27179207]

41.: Garcia-Fernandez A, Roldan V, Rivera-Caravaca JM, Hernandez-Romero D, Valdes M, Vicente V et al. Does von Willebrand factor improve the predictive ability of current risk stratification scores in patients with atrial fibrillation? Scientific Reports. 2017; 7:41565 [PMC free article: PMC5278507] [PubMed: 28134282]

42.: Geersing GJ. Balancing stroke and bleeding risk using CHA2DS2-VASc in treatment of primary care patients with Atrial Fibrillation. 2012. Available from: https://www.cochranelibrary.com/central/doi/10.1002/central/CN-00985604/full Last accessed: 21/01/2020.

43.: Giustozzi MG, Vedovati MC, Verso M, Conti S, Verdecchia P, Bogliari G et al. Predictors of major bleeding in patients aged 90 years or over with atrial fibrillation on anticoagulant treatment. European Heart Journal. 2018; 39:(Suppl 1):1309

44.: Gorman EW, Perkel D, Dennis D, Yates J, Heidel RE, Wortham D. Validation of the HAS-BLED tool in atrial fibrillation patients receiving rivaroxaban. Journal of Atrial Fibrillation. 2016; 9(2):1461 [PMC free article: PMC5129694] [PubMed: 27909541]

45.: Guo Y, Apostolakis S, Blann AD, Wang H, Zhao X, Zhang Y et al. Validation of contemporary stroke and bleeding risk stratification scores in non-anticoagulated Chinese patients with atrial fibrillation. International Journal of Cardiology. 2013; 168(2):904–909 [PubMed: 23167998]

46.: Guo Y, Lane DA, Chen Y, Lip GYH, m AFAIITi. Regular bleeding risk assessment associated with reduction in bleeding outcomes: the mAFA-II randomized trial. American Journal of Medicine. 2020; 133(10):1195–1202.e1192. [PubMed: 32289310]

47.: Guo Y, Zhu H, Chen Y, Lip GYH. Comparing bleeding risk assessment focused on modifiable risk factors only versus validated bleeding risk scores in atrial fibrillation. American Journal of Medicine. 2018; 131(2):185–192 [PubMed: 28943382]

48.: Guo YT, Zhang Y, Shi XM, Shan ZL, Wang CJ, Wang YT et al. Assessing bleeding risk in 4824 Asian patients with atrial fibrillation: The Beijing PLA Hospital Atrial Fibrillation Project. Scientific Reports. 2016; 6:31755 [PMC free article: PMC4997334] [PubMed: 27557876]

49.: Hijazi Z, Lindahl B, Oldgren J, Andersson U, Lindback J, Granger CB et al. Repeated measurements of cardiac biomarkers in atrial fibrillation and validation of the ABC stroke score over time. Journal of the American Heart Association. 2017; 6(6):23 [PMC free article: PMC5669148] [PubMed: 28645934]

50.: Hijazi Z, Lindback J, Alexander JH, Hanna M, Held C, Hylek EM et al. The ABC (age, biomarkers, clinical history) stroke risk score: a biomarker-based risk score for predicting stroke in atrial fibrillation. European Heart Journal. 2016; 37(20):1582–1590 [PMC free article: PMC4875560] [PubMed: 26920728]

51.: Hijazi Z, Lindback J, Siegbahn A, Alexander JH, Held C, Hanna M et al. A new biomarker based risk score for predicting major bleeding in atrial fibrillation-the ABC (age, biomarkers, current disease) risk score. European Heart Journal. 2014; 1:357

52.: Hijazi Z, Oldgren J, Andersson U, Connolly SJ, Eikelboom JW, Ezekowitz MD et al. Growth-differentiation factor 15 and risk of major bleeding in atrial fibrillation: insights from the Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) trial. American Heart Journal. 2017; 190:94–103 [PubMed: 28760218]

53.: Hijazi Z, Oldgren J, Lindback J, Alexander J, Connolly S, Eikelboom J et al. External validation of the biomarker-based ABC-bleeding risk score for atrial fibrillation. Journal of the American College of Cardiology. 2016; 67:(13 Suppl):893

54.: Hijazi Z, Oldgren J, Lindback J, Alexander JH, Connolly SJ, Eikelboom JW et al. The novel biomarker-based ABC (age, biomarkers, clinical history)-bleeding risk score for patients with atrial fibrillation: a derivation and validation study. Lancet. 2016; 387(10035):2302–2311 [PubMed: 27056738]

55.: Hijazi Z, Oldgren J, Lindback J, Alexander JH, Connolly SJ, Eikelboom JW et al. A biomarker-based risk score to predict death in patients with atrial fibrillation: the ABC (age, biomarkers, clinical history) death risk score. European Heart Journal. 2018; 39(6):477–485 [PMC free article: PMC5837352] [PubMed: 29069359]

56.: Hijazi Z, Siegbahn A, Andersson U, Granger CB, Alexander JH, Atar D et al. High-sensitivity troponin I for risk assessment in patients with atrial fibrillation: insights from the Apixaban for Reduction in Stroke and other Thromboembolic Events in Atrial Fibrillation (ARISTOTLE) trial. Circulation. 2014; 129(6):625–634 [PubMed: 24226808]

57.: Hijazi Z, Wallentin L, Siegbahn A, Andersson U, Alexander JH, Atar D et al. High-sensitivity troponin T and risk stratification in patients with atrial fibrillation during treatment with apixaban or warfarin. Journal of the American College of Cardiology. 2014; 63(1):52–61 [PubMed: 24055845]

58.: Hilkens NA, Algra A, Greving JP. Predicting major bleeding in ischemic stroke patients with atrial fibrillation. Stroke. 2017; 48(11):3142–3144 [PubMed: 28931618]

59.: Hippisley-Cox J, Coupland C. Predicting risk of upper gastrointestinal bleed and intracranial bleed with anticoagulants: cohort study to derive and validate the QBleed scores. BMJ. 2014; 349:g4606 [PMC free article: PMC4113281] [PubMed: 25069704]

60.: Hippisley-Cox J, Coupland C, Brindle P. The performance of seven QPrediction risk scores in an independent external sample of patients from general practice: a validation study. BMJ Open. 2014; 4(8):e005809 [PMC free article: PMC4156807] [PubMed: 25168040]

61.: Iwasaki Y. Clinical usefulness of ATRIA score to predict cardiovascular outcomes in patients with atrial fibrillation undergoing percutaneous coronary intervention with stenting. European Heart Journal. 2018; 39:(Suppl 1):1296

62.: Jaakkola S, Kiviniemi TO, Nuotio I, Hartikainen J, Mustonen P, Palomaki A et al. Usefulness of the CHADS2-VASc and HAS-BLED Scores in predicting the risk of stroke versus intracranial bleeding in patients with atrial fibrillation (from the FibStroke study). American Journal of Cardiology. 2018; 121(10):1182–1186 [PubMed: 29526276]

63.: Jaspers Focks J, van Vugt SPG, Albers-Akkers MTH, Lamfers EJP, Bloem-de Vries LM, Verheugt FWA et al. Low performance of bleeding risk models in the very elderly with atrial fibrillation using vitamin K antagonists. Journal of Thrombosis and Haemostasis. 2016; 14(9):1715–1724 [PubMed: 27172860]

64.: Jensen M, Skjoeth F, Nielsen PB, Larsen TB, Melgaard L, Lip GYH. Stroke and bleeding risk scores in patients with atrial fibrillation and valvular heart disease: Prospective validation of the EHRA classification in a nationwide cohort study. European Heart Journal. 2018; 39:(Suppl 1):25

65.: Jover E, Roldan V, Gallego P, Hernandez-Romero D, Valdes M, Vicente V et al. Predictive value of the CHA2DS2-VASc score in atrial fibrillation patients at high risk for stroke despite oral anticoagulation. Revista Española de Cardiología. 2012; 65(7):627–633 [PubMed: 22609214]

66.: Kearon C. In AF, ABC scores predicted stroke or major bleeding better than CHA2DS2-VASc and HAS-BLED scores, respectively. Annals of Internal Medicine. 2019; 170(12):JC71 [PubMed: 31207624]

67.: Lamberts M, Staerk L, Olesen JB, Fosbol EL, Hansen ML, Harboe L et al. Major bleeding complications and persistence with oral anticoagulation in non-valvular atrial fibrillation: Contemporary findings in real-life Danish patients. Journal of the American Heart Association. 2017; 6(2):e004517 [PMC free article: PMC5523754] [PubMed: 28196815]

68.: Lee KT, Chang SH, Yeh YH, Tu HT, Chan YH, Kuo CT et al. The CHA2DS2-VASc Score predicts major bleeding in non-valvular atrial fibrillation patients who take oral anticoagulants. Journal of Clinical Medicine. 2018; 7(10):1–9 [PMC free article: PMC6210214] [PubMed: 30304802]

69.: Li Kam Wa ME, Khouri A, Whitfield M, Amin R, Mohamed MO, McWilliams N et al. The ARDVAARC study: real-world outcomes and the utility of bleeding risk scores in patients who require anticoagulation following percutaneous coronary intervention (PCI). European Heart Journal. 2018; 39 (Suppl 1):1333

70.: Lip GY, Banerjee A, Lagrenade I, Lane DA, Taillandier S, Fauchier L. Assessing the risk of bleeding in patients with atrial fibrillation: the Loire Valley Atrial Fibrillation project. Circulation: Arrhythmia and Electrophysiology. 2012; 5(5):941–948 [PubMed: 22923275]

71.: Lip GY, Frison L, Halperin JL, Lane DA. Comparative validation of a novel risk score for predicting bleeding risk in anticoagulated patients with atrial fibrillation: the HAS-BLED (Hypertension, Abnormal Renal/Liver Function, Stroke, Bleeding History or Predisposition, Labile INR, Elderly, Drugs/Alcohol Concomitantly) score. Journal of the American College of Cardiology. 2011; 57(2):173–180 [PubMed: 21111555]

72.: Lip GY, Lin HJ, Hsu HC, Su TC, Chen MF, Lee YT et al. Comparative assessment of the HAS-BLED score with other published bleeding risk scoring schemes, for intracranial haemorrhage risk in a non-atrial fibrillation population: the Chin-Shan Community Cohort Study. International Journal of Cardiology. 2013; 168(3):1832–1836 [PubMed: 23336959]

73.: Lip GYH. Can we predict stroke in atrial fibrillation? Clinical Cardiology. 2012; 35(SUPPL. 1):21–27 [PMC free article: PMC6652729] [PubMed: 22246948]

74.: Lip GYH, Haguenoer K, Saint-Etienne C, Fauchier L. Relationship of the SAMe-TT2R2 score to poor-quality anticoagulation, stroke, clinically relevant bleeding, and mortality in patients with atrial fibrillation. Chest. 2014; 146(3):719–726 [PubMed: 24722973]

75.: Lip GYH, Jensen M, Melgaard L, Skjoth F, Nielsen PB, Larsen TB. Stroke and bleeding risk scores in patients with atrial fibrillation and valvular heart disease: evaluating ‘valvular heart disease’ in a nationwide cohort study. Europace: European Pacing, Arrhythmias, and Cardiac Electrophysiology. 2018; 06:1–8 [PubMed: 29986001]

76.: Lip GYH, Lane DA, Buller H, Apostolakis S. Development of a novel composite stroke and bleeding risk score in patients with atrial fibrillation: the AMADEUS Study. Chest. 2013; 144(6):1839–1847 [PubMed: 24009027]

77.: Lip GYH, Skjoth F, Nielsen PB, Kjaeldgaard JN, Larsen TB. The HAS-BLED, ATRIA, and ORBIT bleeding scores in atrial fibrillation patients using non-vitamin K antagonist oral anticoagulants. American Journal of Medicine. 2018; 131(5):574.e513–574.e527 [PubMed: 29274754]

78.: Lobos-Bejarano JM, Barrios V, Polo-Garcia J, Escobar C, Vargas-Ortega D, Marin-Montanes N et al. Evaluation of SAMe-TT2R2 score and other clinical factors influencing the quality of anticoagulation therapy in non-valvular atrial fibrillation: a nationwide study in Spain. Current Medical Research and Opinion. 2016; 32(7):1201–1207 [PubMed: 26967541]

79.: Loewen P, Dahri K. Risk of bleeding with oral anticoagulants: an updated systematic review and performance analysis of clinical prediction rules. Annals of Hematology. 2011; 90(10):1191–1200 [PubMed: 21670974]

80.: Lv MN, Zheng XC, Zhang HQ, Xu FD, Wu TT, Chen WJ et al. Comparison of clinical performance of four gastrointestinal bleeding risk scores in Chinese patients with atrial fibrillation receiving oral anticoagulants. Journal of Thrombosis and Thrombolysis. 2020; https://doi.org/10.1007/s11239-020-02152-1 [PubMed: 32462540]

81.: Maeda T, Nishi T, Funakoshi S, Tada K, Tsuji M, Satoh A et al. Risks of bleeding and stroke based on CHA2DS2-VASc scores in Japanese patients with atrial fibrillation: a large-scale observational study using real-world data. Journal of the American Heart Association. 2020; 9(5):e014574 [PMC free article: PMC7335551] [PubMed: 32106743]

82.: Marcucci M, Lip GY, Nieuwlaat R, Pisters R, Crijns HJ, Iorio A. Stroke and bleeding risk co-distribution in real-world patients with atrial fibrillation: the Euro Heart Survey. American Journal of Medicine. 2014; 127(10):979–986.e972 [PubMed: 24838192]

83.: Marcucci M, Nobili A, Tettamanti M, Iorio A, Pasina L, Djade CD et al. Joint use of cardio-embolic and bleeding risk scores in elderly patients with atrial fibrillation. European Journal of Internal Medicine. 2013; 24(8):800–806 [PubMed: 24035703]

84.: McAlister FA, Wiebe N, Jun M, Sandhu R, James MT, McMurtry MS et al. Are existing risk scores for nonvalvular atrial fibrillation useful for prediction or risk adjustment in patients with chronic kidney disease? Canadian Journal of Cardiology. 2017; 33(2):243–252 [PubMed: 27956042]

85.: McAlister FA, Wiebe N, Ronksley PE, Healey JS. Although non-stroke outcomes are more common, stroke risk scores can be used for prediction in patients with atrial fibrillation. International Journal of Cardiology. 2018; 269:145–151 [PubMed: 30077531]

86.: Methavigul K. Use of SAMe-TT2R2 score to predict the quality of anticoagulation control in patients with atrial fibrillation receiving warfarin in Thailand. Journal of the Medical Association of Thailand. 2020; 103(6):548–552

87.: Molnar AO, Sood MM. Predicting in a predicament: stroke and hemorrhage risk prediction in dialysis patients with atrial fibrillation. Seminars in Dialysis. 2018; 31(1):37–47 [PubMed: 28699181]

88.: Mori N, Sotomi Y, Hirata A, Hirayama A, Sakata Y, Higuchi Y. External validation of the ORBIT bleeding score and the HAS-BLED score in nonvalvular atrial fibrillation patients using direct oral anticoagulants (Asian data from the DIRECT registry). American Journal of Cardiology. 2019; 124(7):1044–1048 [PubMed: 31353002]

89.: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual [Updated October 2018]. London. National Institute for Health and Care Excellence, 2014. Available from: https://www.nice.org.uk/process/pmg20/chapter/introduction-and-overview [PubMed: 26677490]

90.: Nielsen PB, Larsen TB, Lip GYH. Recalibration of the HAS-BLED score: should hemorrhagic stroke account for one or two points? Chest. 2016; 149(2):311–314 [PubMed: 26356508]

91.: O’Brien EC, Simon DN, Thomas LE, Hylek EM, Gersh BJ, Ansell JE et al. The ORBIT bleeding score: a simple bedside score to assess bleeding risk in atrial fibrillation. European Heart Journal. 2015; 36(46):3258–3264 [PMC free article: PMC4670965] [PubMed: 26424865]

92.: O’Caoimh R, Igras E, Ramesh A, Power B, O’Connor K, Liston R. Assessing the appropriateness of oral anticoagulation for atrial fibrillation in advanced frailty: use of stroke and bleeding risk-prediction models. The Journal of Frailty and Aging. 2017; 6(1):46–52 [PubMed: 28244558]

93.: Okumura K, Inoue H, Atarashi H, Yamashita T, Tomita H, Origasa H et al. Validation of CHA2DS2-VASc and HAS-BLED scores in Japanese patients with nonvalvular atrial fibrillation: an analysis of the J-RHYTHM Registry. Circulation Journal. 2014; 78(7):1593–1599 [PubMed: 24759791]

94.: Oldgren J, Hijazi Z, Lindback J, Alexander JH, Connolly SJ, Eikelboom JW et al. Performance and validation of a novel biomarker-based stroke risk score for atrial fibrillation. Circulation. 2016; 134(22):1697–1707 [PubMed: 27569438]

95.: Olesen JB, Lip GY, Hansen PR, Lindhardsen J, Ahlehoff O, Andersson C et al. Bleeding risk in ‘real world’ patients with atrial fibrillation: comparison of two established bleeding prediction schemes in a nationwide cohort. Journal of Thrombosis and Haemostasis. 2011; 9(8):1460–1467 [PubMed: 21624047]

96.: Olesen JB, Lip GY, Lindhardsen J, Lane DA, Ahlehoff O, Hansen ML et al. Risks of thromboembolism and bleeding with thromboprophylaxis in patients with atrial fibrillation: a net clinical benefit analysis using a ‘real world’ nationwide cohort study. Thrombosis and Haemostasis. 2011; 106(4):739–749 [PubMed: 21789337]

97.: Olesen JB, Lip GYH, Hansen PR, Lindhardsen J, Ahlehoff O, Andersson C et al. Predicting bleeding risk in “real world” patients with atrial fibrillation with or without anticoagulation: A comparison of two established bleeding prediction schemes in a nationwide cohort study. European Heart Journal. 2011; 1:671 [PubMed: 21624047]

98.: Omran H, Bauersachs R, Rubenacker S, Goss F, Hammerstingl C. The HAS-BLED score predicts bleedings during bridging of chronic oral anticoagulation. Results from the national multicentre BNK Online bRiDging REgistRy (BORDER). Thrombosis and Haemostasis. 2012; 108(1):65–73 [PubMed: 22534746]

99.: Pardo Sanz A, Rincon LM, Tamayo A, De Lara G, Contreras H, Rueda A et al. Performance of atrial fibrillation ischemic and bleeding risk scores in patients with cancer. European Heart Journal. 2018; 39:(Suppl 1):305

100.: Parks AL, Fang MC. Scoring systems for estimating the risk of anticoagulant-associated bleeding. Seminars in Thrombosis and Hemostasis. 2017; 43(5):514–524 [PubMed: 28359135]

101.: Peacock WF, Tamayo S, Patel M, Sicignano N, Hopf KP, Yuan Z. CHA2DS2-VASc scores and major bleeding in patients with nonvalvular atrial fibrillation who are receiving rivaroxabans. Annals of Emergency Medicine. 2017; 69(5):541–550.e541 [PubMed: 27913059]

102.: Perez-Copete J, Esteve-Pastor MA, Roldan V, Valdes M, Marin F. Thromboembolic and bleeding risk scores in atrial fibrillation. Revista Espanola de Cardiologia Suplementos. 2016; 16:(Suppl 1):25–32

103.: Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJ, Lip GY. A novel user-friendly score (HAS-BLED) to assess 1-year risk of major bleeding in patients with atrial fibrillation: the Euro Heart Survey. Chest. 2010; 138(5):1093–1100 [PubMed: 20299623]

104.: Poli D, Antonucci E, Grifoni E, Carini U, Ciampa A, Da Col P et al. Low bleeding risk of very old atrial fibrillation women on VKA treatment: Results from a prospective collaborative study. on behalf of the ad hoc study group of FCSA. Journal of Thrombosis and Haemostasis. 2011; 2:886

105.: Poli D, Antonucci E, Grifoni E, Ciuti G, Marcucci R, Mannini L et al. Stroke risk in atrial fibrillation patients on warfarin: predictive ability of risk stratification schemes for primary and secondary prevention. Journal of Thrombosis and Haemostasis. 2009; 7:(Suppl 2):433 [PubMed: 19190823]

106.: Poli D, Antonucci E, Grifoni E, Di Gennaro L, Falanga A, Falco P et al. Bleeding risk in very old patients on VKA treatment: results of a prospective collaborative study. Journal of Thrombosis and Haemostasis. 2011; 2:747

107.: Poli D, Antonucci E, Grifoni E, Marcucci R, Mannini L, Abbate R et al. Bleeding risk during oral anticoagulation in atrial fibrillation patients older than 80 years. Journal of Thrombosis and Haemostasis. 2009; 7:(Suppl 2):433

108.: Poli D, Antonucci E, Marcucci R, Fatini C, Alterini B, Mannini L et al. Risk of bleeding in very old atrial fibrillation patients on warfarin: relationship with ageing and CHADS2 score. Thrombosis Research. 2007; 121(3):347–352 [PubMed: 17597186]

109.: Poli D, Antonucci E, Marongiu F, Pengo V, Testa S, Tripodi A et al. Similar performance of HAS-BLED, CHADS2 and CHA2DS2VASC scores in bleeding risk prediction of atrial fibrillation patients: the refined HAS-BED score results from the start register. Blood Transfusion. 2016; 14:(Suppl 5):S777

110.: Poli D, Antonucci E, Pengo V, Testa S, Palareti G. Comparison of HAS-BLED and HAS-BED versus CHADS2 and CHA2DS2VASC stroke and bleeding scores in patients with atrial fibrillation. American Journal of Cardiology. 2017; 119(7):1012–1016 [PubMed: 28237286]

111.: Poli D, Antonucci E, Testa S, Cosmi B, Palareti G, Ageno W et al. The predictive ability of bleeding risk stratification models in very old patients on vitamin K antagonist treatment for venous thromboembolism: results of the prospective collaborative EPICA study. Journal of Thrombosis and Haemostasis. 2013; 11(6):1053–1058 [PubMed: 23578305]

112.: Poli D, Antonucci E, Testa S, Tosetto A, Ageno W, Palareti G et al. Bleeding risk in very old patients on vitamin K antagonist treatment: results of a prospective collaborative study on elderly patients followed by Italian Centres for Anticoagulation. Circulation. 2011; 124(7):824–829 [PubMed: 21810658]

113.: Prochaska JH, Gobel S, Nagler M, Knopfler T, Eggebrecht L, Lamparter H et al. Sustained atrial fibrillation increases the risk of anticoagulation-related bleeding in heart failure. Clinical Research in Cardiology. 2018; 107(12):1170–1179 [PubMed: 29948286]

114.: Proietti M, Hijazi Z, Andersson U, Connolly SJ, Eikelboom JW, Ezekowitz MD et al. Comparison of bleeding risk scores in patients with atrial fibrillation: insights from the RE-LY trial. Journal of Internal Medicine. 2018; 283(3):282–292 [PubMed: 29044861]

115.: Proietti M, Rivera-Caravaca JM, Esteve-Pastor MA, Romiti GF, Marin F, Lip GYH. Predicting bleeding events in anticoagulated patients with atrial fibrillation: A comparison between the HAS-BLED and GARFIELD-AF bleeding scores. Journal of the American Heart Association. 2018; 7(18):e009766 [PMC free article: PMC6222935] [PubMed: 30371183]

116.: Proietti M, Senoo K, Lane DA, Lip GY. Major bleeding in patients with non-valvular atrial fibrillation: impact of time in therapeutic range on contemporary bleeding risk scores. Scientific Reports. 2016; 6:24376 [PMC free article: PMC4828703] [PubMed: 27067661]

117.: Quinn GR, Singer DE, Chang Y, Go AS, Borowsky LH, Fang MC. How well do stroke risk scores predict hemorrhage in patients with atrial fibrillation? American Journal of Cardiology. 2016; 118(5):697–699 [PMC free article: PMC5131634] [PubMed: 27394408]

118.: Rivera-Caravaca JM, Marin F, Esteve-Pastor MA, Rana-Miguez P, Anguita M, Muniz J et al. Usefulness of the 2MACE score to predicts adverse cardiovascular events in patients with atrial fibrillation. American Journal of Cardiology. 2017; 120(12):2176–2181 [PubMed: 29111209]

119.: Rivera-Caravaca JM, Marin F, Vilchez JA, Galvez J, Esteve-Pastor MA, Vicente V et al. Refining stroke and bleeding prediction in atrial fibrillation by adding consecutive biomarkers to clinical risk scores. Stroke. 2019; 50(6):1372–1379 [PubMed: 31084333]

120.: Rivera-Caravaca JM, Roldan V, Esteve-Pastor MA, Valdes M, Vicente V, Lip GYH et al. Importance of time in therapeutic range on bleeding risk prediction using clinical risk scores in patients with atrial fibrillation. Scientific Reports. 2017; 7(1):12066 [PMC free article: PMC5608893] [PubMed: 28935868]

121.: Rivera-Caravaca JM, Roldan V, Esteve-Pastor MA, Valdes M, Vicente V, Lip GYH et al. Long-term stroke risk prediction in ‘real world’ atrial fibrillation patients: a comparison of the ABC-stroke and CHA2DS2-VASc scores. Research and Practice in Thrombosis and Haemostasis. 2017; 1 (Suppl 1):335–336

122.: Rivera-Caravaca JM, Roldan V, Esteve-Pastor MA, Valdes M, Vicente V, Marin F et al. Prediction of long-term net clinical outcomes using the TIMI-AF score: comparison with CHA2DS2-VASc and HAS-BLED. American Heart Journal. 2018; 197:27–34 [PubMed: 29447781]

123.: Rivera Caravaca JM, Esteve-Pastor MA, Vilchez JA, Galvez J, Vicente V, Marin F et al. Refining stroke and bleeding risk prediction by adding consecutive biomarkers to CHA2DS2-VASc and HAS-BLED scores. European Heart Journal. 2018; 39:(Suppl 1):821–822

124.: Roldan V, Marin F, Diaz J, Gallego P, Jover E, Romera M et al. High sensitivity cardiac troponin T and interleukin-6 predict adverse cardiovascular events and mortality in anticoagulated patients with atrial fibrillation. Journal of Thrombosis and Haemostasis. 2012; 10(8):1500–1507 [PubMed: 22681487]

125.: Roldan V, Marin F, Fernandez H, Manzano-Fernandez S, Gallego P, Valdes M et al. Predictive value of the HAS-BLED and ATRIA bleeding scores for the risk of serious bleeding in a “real-world” population with atrial fibrillation receiving anticoagulant therapy. Chest. 2013; 143(1):179–184 [PubMed: 22722228]

126.: Roldan V, Marin F, Manzano-Fernandez S, Gallego P, Vilchez JA, Valdes M et al. The HAS-BLED score has better prediction accuracy for major bleeding than CHADS2 or CHA2DS2-VASc scores in anticoagulated patients with atrial fibrillation. Journal of the American College of Cardiology. 2013; 62(23):2199–2204 [PubMed: 24055744]

127.: Roldan V, Marin F, Muina B, Torregrosa JM, Hernandez-Romero D, Valdes M et al. Plasma von Willebrand factor levels are an independent risk factor for adverse events including mortality and major bleeding in anticoagulated atrial fibrillation patients. Journal of the American College of Cardiology. 2011; 57(25):2496–2504 [PubMed: 21497043]

128.: Roldan V, Rivera-Caravaca JM, Shantsila A, Garcia-Fernandez A, Esteve-Pastor MA, Vilchez JA et al. Enhancing the ‘real world’ prediction of cardiovascular events and major bleeding with the CHA2DS2-VASc and HAS-BLED scores using multiple biomarkers. Annals of Medicine. 2018; 50(1):26–34 [PubMed: 28892413]

129.: Rutherford OCW, Jonasson C, Ghanima W, Halvorsen S. A new score for assessing bleeding risk in patients with atrial fibrillation treated with NOACs. European Heart Journal. 2018; 39:(Suppl 1):1009–1010

130.: Sadeghi R, Mahjoob MP, Asadollahi M, Abbasi Z. Prevalence, main determinants, and early outcome of patients with atrial fibrillation hospitalized with ischemic stroke: evaluation of the value of risk assessment scores for predicting risk of stroke or major bleeding following anticoagulation therapy. Acta Bio-Medica de l Ateneo Parmense. 2015; 86(2):162–169 [PubMed: 26422431]

131.: Saito Y, Okumura Y, Nagashima K, Fukamachi D, Yokoyama K, Matsumoto N et al. Impact of the Fibrosis-4 Index on risk stratification of cardiovascular events and mortality in patients with atrial fibrillation: findings from a Japanese multicenter registry. Journal of Clinical Medicine. 2020; 9(2):21 [PMC free article: PMC7074173] [PubMed: 32098093]

132.: Salpagarova ZK, Chashkina M, Andreev DA, Bykova AA, Sychev DA, Kozlovskaya NL et al. The new warfarin dosing algorithm for patients with atrial fibrillation and severe chronic kidney disease. European Heart Journal. 2018; 39:(Suppl 1):614

133.: Sanders GD, Lowenstern A, Borre E, Chatterjee R, Goode A, Sharan L et al. Stroke prevention in patients with atrial fibrillation: a systematic review update. Rockville (MD). Agency for Healthcare Research and Quality, 2018. Available from: https://effectivehealthcare.ahrq.gov/products/stroke-afib-update/research-2018 [PMC free article: PMC534141] [PubMed: 30480925]

134.: Sani M, Ayubi E, Mansori K, Khazaei S. Predictive ability of HAS-BLED, HEMORR2HAGES, and ATRIA bleeding risk scores in patients with atrial fibrillation: methodological issues of prediction models. International Journal of Cardiology. 2016; 222:949 [PubMed: 27526365]

135.: Schwartz SM, Tedla YG, Greenland P, Yadlapati A, Passman RS. Discriminative ability of CHA2DS2-VASc and HAS-BLED score in whites and nonwhites. American Journal of Cardiology. 2019; 123(12):1949–1954 [PubMed: 30979410]

136.: Senoo K, Lip GY. Predictive abilities of the HAS-BLED and ORBIT bleeding risk scores in non-warfarin anticoagulated atrial fibrillation patients: an ancillary analysis from the AMADEUS trial. International Journal of Cardiology. 2016; 221:379–382 [PubMed: 27409565]

137.: Senoo K, Proietti M, Lane DA, Lip GYH. Evaluation of the HAS-BLED, ATRIA, and ORBIT bleeding risk scores in patients with atrial fibrillation taking warfarin. American Journal of Medicine. 2016; 129(6):600–607 [PubMed: 26482233]

138.: Serna MJ, Rivera-Caravaca JM, Gonzalez-Conejero R, Esteve-Pastor MA, Valdes M, Vicente V et al. Pharmacogenetics of vitamin K antagonists and bleeding risk prediction in atrial fibrillation. European Journal of Clinical Investigation. 2018; 48(6):e12929 [PubMed: 29577257]

139.: Shah RR, Pillai A, Omar A, Zhao J, Arora V, Kapoor D et al. Utility of the HAS-BLED score in risk stratifying patients on dual antiplatelet therapy post 12 months after drug-eluting stent placement. Catheterization and Cardiovascular Interventions. 2017; 89(4):E99–E103 [PubMed: 27184930]

140.: Shahid F, Lip GYH. Risk stratification models in atrial fibrillation. Seminars in Thrombosis and Hemostasis. 2017; 43(5):505–513 [PubMed: 28129663]

141.: Silva R, Silva PAE, Lima MC, Sant’Anna LT, Silva TC, Moreira PHV et al. Thromboembolism and bleeding risk scores and predictors of cardiac death in a population with atrial fibrillation. Arquivos Brasileiros de Cardiologia. 2017; 109(1):5–13 [PMC free article: PMC5524470] [PubMed: 28562832]

142.: Siu CW, Lip GY, Lam KF, Tse HF. Risk of stroke and intracranial hemorrhage in 9727 Chinese with atrial fibrillation in Hong Kong. Heart Rhythm. 2014; 11(8):1401–1408 [PubMed: 24747420]

143.: Sogaard M, Skjoth F, Kjaeldgaard JN, Lip GYH, Larsen TB. Bleeding complications in anticoagulated patients with atrial fibrillation and sepsis: a propensity-weighted cohort study. Journal of the American Heart Association. 2017; 6(11):09 [PMC free article: PMC5721800] [PubMed: 29122810]

144.: Somme D, Corvol A, Lazarovici C, Lahjibi-Paulet H, Gisselbrecht M, Saint-Jean O. Clinical usefulness in geriatric patients of combining CHADS2 and HEMORR2HAGES scores to guide antithrombotic prophylaxis in atrial fibrillation. Aging-Clinical and Experimental Research. 2010; 22(4):289–294 [PubMed: 19996707]

145.: Sood MM, Larkina M, Thumma JR, Tentori F, Gillespie BW, Fukuhara S et al. Major bleeding events and risk stratification of antithrombotic agents in hemodialysis: Results from the DOPPS. Kidney International. 2013; 84(3):600–608 [PMC free article: PMC3885984] [PubMed: 23677245]

146.: Steinberg BA, Shrader P, Kim S, Thomas L, Fonarow GC, Ansell J et al. How well does physician risk assessment predict stroke and bleeding in atrial fibrillation? Results from the Outcomes Registry for Better Informed Treatment of Atrial Fibrillation (ORBIT-AF). American Heart Journal. 2016; 181:145–152 [PubMed: 27823686]

147.: Suzuki M, Matsue Y, Nakamura R, Matsumura A, Hashimoto Y. Improvement of HAS-BLED bleeding score predictive capability by changing the definition of renal dysfunction in Japanese atrial fibrillation patients on anticoagulation therapy. Journal of Cardiology. 2014; 64(6):482–487 [PubMed: 24836929]

148.: Tchen S, Ryba N, Patel V, Cavanaugh J, Sullivan JB. Validation of bleeding risk prediction scores for patients with major bleeding on direct oral anticoagulants. Annals of Pharmacotherapy. 2020; 54(12):1175–1184 [PubMed: 32517484]

149.: Thomas IC, Sorrentino MJ. Bleeding risk prediction models in atrial fibrillation. Current Cardiology Reports. 2014; 16(1):432 [PubMed: 24264434]

150.: Toyoda K, Yasaka M, Uchiyama S, Iwade K, Koretsune Y, Nagata K et al. CHADS2 and CHA2DS2-VASc scores as bleeding risk indices for patients with atrial fibrillation: the Bleeding with Antithrombotic Therapy Study. Hypertension Research - Clinical and Experimental. 2014; 37(5):463–466 [PubMed: 24196199]

151.: van Doorn S, Rutten FH, O’Flynn CM, Oudega R, Hoes AW, Moons KGM et al. Effectiveness of CHA2DS2-VASc based decision support on stroke prevention in atrial fibrillation: a cluster randomised trial in general practice. International Journal of Cardiology. 2018; 273:123–129 [PubMed: 30224261]

152.: Van Mieghem W, Lancellotti P. CHADS2 risk score and rate of stroke or systemic embolism and major bleeding in patients with non-valvular atrial fibrillation receiving non-vitamin K antagonist oral anticoagulants. Acta Cardiologica. 2017; 72(4):390–396 [PubMed: 28681678]

153.: Wang C, Yu Y, Zhu W, Yu J, Lip GYH, Hong K. Comparing the ORBIT and HAS-BLED bleeding risk scores in anticoagulated atrial fibrillation patients: a systematic review and meta-analysis. Oncotarget. 2017; 8(65):109703–109711 [PMC free article: PMC5752553] [PubMed: 29312640]

154.: Wang SV, Franklin JM, Glynn RJ, Schneeweiss S, Eddings W, Gagne JJ. Prediction of rates of thromboembolic and major bleeding outcomes with dabigatran or warfarin among patients with atrial fibrillation: new initiator cohort study. BMJ. 2016; 353:i2607 [PMC free article: PMC4878389] [PubMed: 27221664]

155.: Wang SV, Rogers JR, Jin Y, Bates DW, Fischer MA. Use of electronic healthcare records to identify complex patients with atrial fibrillation for targeted intervention. Journal of the American Medical Informatics Association. 2017; 24(2):339–344 [PMC free article: PMC7651901] [PubMed: 27375290]

156.: Wang TK, Sathananthan J, Marshall M, Kerr A, Hood C. Relationships between anticoagulation, risk scores and adverse outcomes in dialysis patients with atrial fibrillation. Heart, Lung and Circulation. 2016; 25(3):243–249 [PubMed: 26481398]

157.: Wang Y, Bajorek B. Pilot of a Computerised Antithrombotic Risk Assessment Tool Version 2 (CARATV2.0) for stroke prevention in atrial fibrillation. Cardiology Journal. 2017; 24(2):176–187 [PubMed: 28070883]

158.: Yao X, Gersh BJ, Sangaralingham LR, Kent DM, Shah ND, Abraham NS et al. Comparison of the CHA2DS2-VASc, CHADS2, HAS-BLED, ORBIT, and ATRIA risk scores in predicting non-vitamin K antagonist oral anticoagulants-associated bleeding in patients with atrial fibrillation. American Journal of Cardiology. 2017; 120(9):1549–1556 [PubMed: 28844514]

159.: Zeng J, Yu P, Cui W, Wang X, Ma J, Zeng C. Comparison of HAS-BLED with other risk models for predicting the bleeding risk in anticoagulated patients with atrial fibrillation: a PRISMA-compliant article. Medicine. 2020; 99(25):e20782 [PMC free article: PMC7310965] [PubMed: 32569222]

160.: Zhu W, He W, Guo L, Wang X, Hong K. The HAS-BLED score for predicting major bleeding risk in anticoagulated patients with atrial fibrillation: a systematic review and meta-analysis. Clinical Cardiology. 2015; 38(9):555–561 [PMC free article: PMC6490831] [PubMed: 26418409]

161.: Ziviello F, Pilgrim T, Kroon H, Ooms JF, van Wiechen MP, Daemen J et al. Has-Bled score and actual bleeding in elderly patients undergoing transcatheter aortic valve implantation. JACC: Cardiovascular Interventions. 2019; 12:(Suppl 4):S49

162.: Zulkifly H, Lip GYH, Lane DA. Bleeding risk scores in atrial fibrillation and venous thromboembolism. American Journal of Cardiology. 2017; 120(7):1139–1145 [PubMed: 28800833]

Appendices

Appendix A. Review protocols

Table 19. Review question: What is the most clinically and cost-effective risk stratification tool for predicting bleeding in people with atrial fibrillation? (PDF, 425K)

Table 20. Review protocol:What is the most accurate risk stratification tool for predicting stroke or thromboembolic events in people with atrial fibrillation? (PDF, 416K)

Table 21. Health economic review protocol (PDF, 423K)

Appendix B. Literature search strategies

This literature search strategy was used for the following reviews:

What is the most clinically and cost-effective tool for assessing bleeding risk in people with atrial fibrillation?
What is the most accurate risk stratification tool for predicting bleeding events in people with atrial fibrillation?

The literature searches for this review are detailed below and complied with the methodology outlined in Developing NICE guidelines: the manual.⁸⁹

For more information, please see the Methods Report published as part of the accompanying documents for this guideline.

B.1. Clinical search literature search strategy (PDF, 452K)

B.2. Health Economics literature search strategy (PDF, 462K)

Appendix F. Forest plots

F.1. C statistics

Download PDF (1.1M)

Appendix G. Clinical evidence tables

Download PDF (1.4M)

Appendix H. Risk of bias (PROBAST)

Download PDF (504K)

Appendix I. Economic evidence tables

None.

Appendix J. Excluded clinical studies

No studies were excluded from the review on effectivess.

Table 92. Studies excluded from the clinical reviewRCT (PDF, 179K)

Table 93. Studies excluded from the clinical reviewaccuracy (PDF, 165K)

Appendix K. Excluded economic studies

No studies were excluded from the review on effectivenessof tools to predict bleeding.

No studies were excluded from the review on accuracy of tools to predict bleeding.

Final

Evidence review

Developed by the National Guideline Centre, Royal College of Physicians

Disclaimer: The recommendations in this guideline represent the view of NICE, arrived at after careful consideration of the evidence available. When exercising their judgement, professionals are expected to take this guideline fully into account, alongside the individual needs, preferences and values of their patients or service users. The recommendations in this guideline are not mandatory and the guideline does not override the responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patientand, where appropriate,their carer or guardian.

Local commissioners andproviders have a responsibility to enable the guideline to be applied when individual health professionals and theirpatients or service users wish to use it. They should do so in the context of local and national priorities for funding and developing services, and in light of their duties to have due regard to the need to eliminate unlawful discrimination, to advance equality of opportunity and to reduce health inequalities. Nothing in this guideline should be interpreted in a way that would be inconsistent with compliance with those duties.

NICE guidelines cover health and care in England. Decisions on how they apply in other UK countries are made by ministers in the Welsh Government, Scottish Government, and Northern Ireland Executive. All NICE guidance is subject to regular review and may be updated or withdrawn.

Bookshelf ID: NBK571344PMID: 34165928

Table 1PICO characteristics of review question

Population	People aged over 18 with a diagnosis of AF.
Interventions	Any bleeding risk tool (for example, ATRIA, HEMORRHAGES, ORBIT) [Note: treat each test using a different threshold as a separate intervention].
Comparison	HAS-BLED (the established method, as recommended by previous version of this guideline)
Outcomes	Critical health-related quality of life mortality stroke or thromboembolic complications major bleeding
Study design	Randomised controlled trials

Table 2Bleeding risk tools

Risk tool	Description	Additional tests required to complete risk tool
ABC bleeding score	- Age - Biomarkers (hematocrit, high sensitivity troponin T (hsTnT), GDF-15) - Clinical history (prior bleeding)	Biomarkers.
Orbit bleeding score	- older age (75+ years) - reduced haemoglobin/haematocrit/history of anaemia - bleeding history - insufficient kidney function - treatment with antiplatelet	None
ATRIA	- anaemia - severe renal disease - age ≥75 years - any prior haemorrhage diagnosis - hypertension history	None
HEMORR2HAGES	- hepatic or renal disease - ethanol (alcohol) abuse - malignancy history - age >75 years - platelet count or function - rebleeding risk - hypertension (uncontrolled) - anaemia - genetic factors (CYP2C9 single nucleotide polymorphisms) - excessive fall risk - stroke history	Genetic testing
HAS-BLED	- uncontrolled hypertension - renal disease - liver disease - stroke history - prior major bleeding or predisposition to bleeding - labile INR - age >65 - concomtant antiplatelets or NSAIDs - alcohol excess/abuse	None

Table 3PICO characteristics of review question

Question
Population	People aged >18 with a diagnosis of atrial fibrillation, who are on anticoagulants
Risk tool	Any bleedingrisk tool (e.g HAS-BLED, ORBIT, HEMORRHAGES, ATRIA, etc) Any other version of HAS-BLEDwith modifications
Target condition or Reference standard	Later major bleeding, or other bleeding
Outcomes (in terms of predictive test accuracy, calibration)	Simple diagnostic (prognostic) accuracy outcomes, such as sensitivity and specificity C-statistic(based on sensitivity and specificity but useful if >1 threshold used). Calibration outcomes Reclassification
Study types	cohort (external validation, internal validation)
Specific groups	Ethnic groups

Table 4Summary of studies included in the review

Study	Risk tool(s)	OAC	Concomitant antiplatelets or NSAIDS	Population	Number and type of outcome events	Follow up duration
Apostolakis 2012⁴	HAS-BLED HEMORRHAGES ATRIA	Warfarin	18%	2,293 patients with AF on VKAs, from AMADEUS RCT trial in UK. Age 70, 65% male, 77% hypertension, 20% DM, 13.5% previous stroke, 31% CAD, 18% antiplatelet treatment, TTR 0.57. Drops outs NR. No blinding reported.	39 MB 251 CRB	429 days
Apostolakis 2013³	HAS-BLED CHADS2 CHADSVASC	Warfarin	18%	As above	As above	As above
Barnes 2014⁸	CHADS2 CHADSVASC HEMORRHAGES HAS-BLED ATRIA	Warfarin	NR	2600 patients with NVAF and on warfarin were recruited. USA study. Age 70, 41.7% female, hypertension 75%, DM 25%, CAD 33%, CHF 24.2%, current smoking 6%, renal disease 12%, stroke 11.5%, bleeding diasthesis 31%, HAS-BLED score 2.6, CHADS2 score 3.4. TTR 59.3. Antiplatelets/NSAIDs not reported. No blinding. No data loss reported.	100 MB	1 year
Berg 2019¹¹	HAS-BLED ABC	Warfarin Edoxaban	NR	Patients enrolled on the ENGAGE AF-TIMI 48 trial, who were therefore taking VKAs or edoxaban. Participation in this sub-study was offered to all enrolled patients until recruitment reached 9000 participants	Unclear	3 years
Beshir 2018¹⁴	mOBRI CBRM HEMORRHAGES HAS-BLED ATRIA ORBIT	Warfarin, rivaroxaban, dabigatran	35%	1017 patients with NVAF and on Warfarin (INR 2–3), dabigatran or rivaroxaban between 2010 and 2015. Malaysia. Age >75: 27%, 52% male, hypertension 82%, IHD 33%, renal impairment 36%, DM 40%, prior stroke/TIA: 22%, CHF: 20%. CHADS2: 2. 35% on antiplatelets. No blinding. 291 lost to follow up from original sample of 1308 patients.	23 MB 76 CRNMB	1 year
Chang 2016¹⁹	HTI APTT Prothrombin time	dabigatran	12.50%	208 patients (213 enrolled and 5 lost to FU) with NVAF on dabigatran (either 100mg or 150mg/day). Taiwan. Age 74.7, 67.9% male, 36% history of stroke, 24.5% DM, 79.3% hypertension, 18.8% CAD, 16.3% HF, antiplatelets/NSAIDs 12.5%, renal disease 0.5%, history of GI bleeding 23.6%, HAS-BLED 1.8. 5 lost to follow up from original cohort of 213. No blinding.	17 MB	1 year
Chao 2018a²¹	Modifiable Bleeding Risk factors score (MBR) HEMORRHAGES HAS-BLED ATRIA ORBIT	Warfarin	22.70%	40,450 AF patients (defined as cases where there had been at least 2 confirmed outpatient diagnoses of AF) receiving warfarin between 1998 and 2011 in Taiwan. Age 67.3, male 55.7%, hypertension 67.4%, abnormal renal function 13.2%, stroke 43%, history of bleeding 18%, use of antiplatelets 22.7%, NSAIDs 7.2%, HAS-BLED 2.51. No loss to FU. No blinding reported.	6889 MB 1581 ICH	4.6 years
Chao 2018b²⁰	HAS-BLED baseline HAS-BLED change from baseline (Delta HAS-BLED) HAS-BLED follow up	Warfarin	2.30%	19,566 AF patients on Warfarin and a HAS_BLED score of <2 identified from the NHIRD of Taiwan (1998–2011). Age 63.8, male 57.4%, hypertension 52.6%, abnormal renal function 3.4%, stroke 22.6%, bleeding 6.9%, antiplatelet / NSAID drugs 2.3%. No loss to FU reported. No blinding reported.	3032 MB 671 ICH	4.8 years
Claxton 2018²³	Anticoagulation-Specific Bleeding Score (ABS) HAS-BLED ATRIA HEMORRHAGES ORBIT	Warfarin, dabigatran, rivaroxaban and apixaban	NR	81,285 NVAF patients on Warfarin or DOACs (initiated at baseline). Netherlands. This was an external validation cohort from the Optum Clinformatics database from 2009–2015. For warfarin group (largest) the demographics were: age 73.9, 44% woman, HAS-BLED 2.8, HF 45.5%, CHD: 47.3%, hypertension 89%, DM 39.9%, stroke 33.4%, PAD 25.7%, kidney disease 25.9%, prior GI bleed 16%, prior IC bleed: 2.1%, prior other bleed 16%. No blinding reported. No loss to follow up (as retrospective). No data on antiplatelets/NSAIDS	3238 MB	1 year
Dalgaard 2019²⁵	GARFIELD-AF HAS-BLED	Unclear	Unclear	51,180 Danish patients on OACs from the Danish Nationwide registries. Aged 18 or older with NVAF. Excluded patients with rheumatic valve disease or valve surgery.	1492 MB (but unclear if some had ICH)	1 year
Elvira-Ruiz, 2020³⁰	HAS-BLED ORBIT ATRIA HAS-BLEDwith existence of aortic stenosis (AS) ORBITwith AS ATRIAwith AS	Mixed VKA and DOACS (results not sub-grouped)	17.7%	2,880 NVAF patients initiating oral anticoagulants; age 77; 51.1% women; 49.3% permanent AF; hypertension 85.5%; DM 33.9%; CHADSVASC 4; HASBLED 2; ATRIA 3; ORBIT 1.	185 MB	18 months
Esteve Pastor 2016³¹	HAS-BLED ORBIT	VKA and DOACS	10.90%	1276 patients with chronic NVAF on VKA or DOAC for at least 6 months before enrolment (FANTASIIA population). SPAIN. There was another cohort of 406 patients in this paper that underwent electrical cardioversion, and they are not included in this extraction. Age 74, 44% male, 80.6% hypertensive, 30% HF, 29.3% DM, 6.6% VD, 12.9% previous embolism, 3.8% previous bleeding, 10% renal impairment, 1.3% liver impairment, 77.4% VKA, 22.6% DOACs, 10.9% on NSAIDS / antiplatelets. HAS-BLED score: 2. TTR 60.9. No blinding. No loss to FU reported.	46 MB	1 year
Esteve-Pastor 2017a⁵	ABC-bleedingCrC HAS-BLED	VKAs	NR	1,120 patients with paroxysmal, persistent or permanent AF, stable on VKAs (INR 2–3). Spain. Age 76, 49.5% male, 82% hypertension, 27%DM, 33% dyslipidaemia, 15.5% current smoker, 31.2% HF, 19.6% CAD, 19% previous stroke, 8.4% previous bleeding. TTR at 6 months 80, CHADSVASC 4, HAS-BLED 2, ABC 16.5. Number on antiplatelets – not reported. No loss to FU reported. No blinding.	207 MB 65 ICH 85 GIB	6.5 years
Esteve-Pastor 2017b³²	HAS-BLED Modifiable bleeding risk factors score	VKAs	21.40%	4576 patients with paroxysmal, persistent or permanent AF. 2283 on warfarin and 2293 on Idraparinux. Taken from the multinational AMADEUS database. Spain. Age 71, 66.5% male, 21.4% on anti-platelets or NSAID, 77% hypertensive, 20%DM, 23% HF, 31% CAD, 13% previous stroke, TTR 58, CHADSVASC 3, HAS-BLED 2, Modifiable bleeding risks score 1. No loss to FU reported. Assessors BLINDED.	113 MB 597 CRB	347 days
Fang 2011³³	ATRIA Outpatient Bleeding Index Kuijer et al. Kearon et al. HEMORRHAGES Shireman Riete risk scheme	Warfarin	NR	3063 patients in the validation cohort, taken from 9,186 patients with NVAF on warfarin (median exposure 3.5 years), taken from the ATRIA study (USA). AF defined as any ICD-9 codes. Demographic data not given for validation cohort. No blinding or loss to FU reported.	154 MB	3 years
Fox 2017³⁶	GARFIELD AF Risk HAS-BLED	VKA and DOAC	NR	25,285 patients with AF that were on OACs. 8804 on DOACs and 16,491 on VKAs. Details of the characteristics of these patients are not reported. No blinding reported.	625 MB	3 years
Friberg 2012³⁷	HAS-BLED HEMORRHAGES	Warfarin	NR	48, 599 patients with AF (defined by ICD-10 code 1489 with or without subscales A-F) using Warfarin at baseline identified from the Swedish National Discharge Registry. Demographic data stated to be in supplementary file but not available in that file who were on warfarin. This subset was taken from an overall cohort of 170 291 which included those not on anticoagulants. No blinding reported.	1.9 MB per 100 patient years	1.5 years
Gage 2006³⁸	Landefeld and Goldman and Beyth et al. Kuijer et al. Kearon et al. HEMORRHAGES	Warfarin	7.40%	1604 medicare beneficiaries on NRAF (USA) with chart-confirmed AF on warfarin. 69.2% aged > 75 years, 7.9% hepatic or renal disease, 4.8% malignancy, 37.2% previous stroke, 0.4% uncontrolled hypertension. Also on Aspirin: 7.04%. No blinding or loss to FU reported.	4.9 MB per 100 patient years	Unclear but approx. 1 year
Gallego 2012³⁹	HAS-BLED	Acenocoumarol	16.60%	965 consecutive anticoagulated people with permanent or paroxysmal AF, with at least 6 months of anticoagulation with acenocoumarol (INR 2–3). 50% male, mean age 76, hypertension 57%, DM 25.5%, HF 36.5%, prev. stroke/TIA 19%, renal impairment 10%, CAD 4%, hypercholesterolemia 31%, current smoking 14%, previous bleeding 8.5%, median HAS-BLED 2, CHADS2 score 2. Antiplatelet therapy 16.6%. 95 died during FU. No blinding reported.	75MB	861 days
Garcia-Fernandez 2017⁴¹	vWF HAS-BLED HAS-BLED + vWF	VKA	17.80%	1215 patients with NVAF on VKA at INR 2–3. Age 76, male 49.3%, hypertension 82.5%, DM 26.4%, HF 31.1%, IHD 19%, previous stroke 18.4%, previous bleeding 8.4%, renal disease 10.3%, antiplatelet drugs 17.8%, HAS-BLED score 2. No loss to FU or blinding reported.	222MB	2373 days
Hijazi 2014⁵⁶	CHADSVASC CHADSVASC with TnT	apixaban and warfarin	28–34%	14,897 patients with AF on apixaban or warfarin, from the ARISTOTLE trial. Likely to be a multinational multi-centre trail but not reported. Ranges of baseline data given as data given for different categories of TnT. Age 64–74, male 53.8–74.6%, CHF 28–47%, hypertension 87%, DM 18–32%, Prior stroke/TIA 16–21%, MI 6–19%. Aspirin 28–34%. Warfarin 53.2–55.7%. BLINDED ASSESORS of BLEEDING. No loss to FU reported.	674 MB	1.9 years
Hijazi 2014⁵⁶	HAS-BLED HAS-BLED with TnI	apixaban and warfarin	29–34%	14,821 patients with AF on apixaban or warfarin, from the ARISTOTLE trial. Overlap with Hijazi, 2014⁵⁷in terms of sample, but this study used a different risk tool. Likely to be a multinational multi-centre trial but not reported. Ranges of baseline data given as data given for different categories of TnI. Age 66–72, male 6–70%, CHF 24–51%, hypertension 87%, DM 21–28%, Prior stroke/TIA 16–21%, MI 6–19%. Aspirin 29–34%. Warfarin 49.9–56.5%. BLINDED assessors. No loss to FU reported.	674 MB	1.9 years
Hijazi 2016⁵⁴	HAS-BLED ORBIT ABC-bleeding ABC-bleeding (cTnl-hs) ABC-bleeding (cystatin C) ABC-bleeding (CKD-EPI)	warfarin and dabigatran (SEP ANALYSES)	44%	External validation in 8468 patients with AF (67% permanent or persistent) randomised to dabigatran and warfarin in the multinational RE-LY trial. Age 72, 26% women, 44% on antiplatelets or NSAISs, 8% current smokers, 22% DM, 79% hypertension, 29% CHF, 13% previous clinically relevant bleeding, 19% previous stroke/TIA, 17% previous MI, 4% previous PAD, 19% vascular disease, Renal function CKD-EPI 68.2. ASSESSOR BLINDING. No loss to FU reported.	463 MB	1.9 years
Hijazi 2017⁵²	HAS-BLED ORBIT (with or without GDF-15)	warfarin and dabigatran	36–41%	8,474 AF patients (with at least 1 additional risk factor for stroke) taken from the RE-LY study, on dabigatran or warfarin. Baseline characteristics given as ranges as sub-grouped by GDF-15. Age 69–75, male 61–67%, sbp 130, DM 11–35%, HF 25–34%, hypertension 78–80%, previous stroke/TIA 20–22%, prior MI 12–21%, prev PAD/MI/CAD 23–38%, aspirin 36–41%. CHADS2 >3 22–43%. No blinding/loss to FU reported.	458 MB	1.9 years
Hilkens 2017⁵⁸	HEMORRHAGERS Shireman HAS_BLED ATRIA ORBIT (score) ORBIT (equation)	warfarin and dabigatran (SEP ANALYSES)	NR	3623 patients with AF on warfarin or dabigatran, from the RE-LY trial in Holland. No baseline data available. No report of blinding/loss to FU.	266 MB	2 years
Jaspers Focks 2016⁶³	HAS-BLED ATRIA HEMORRHAGES	VKA	4.10%	1157 AF patients aged >80 years, using a VKA from 2011–2014 in the Netherlands. Median age 84, 42.6% male, 37 months on VKA, 65.8% hypertension, 22% previous stroke/TIA, 9.8% LVEF<40%, 26.6% CAD, 25.7% DM, 21.8% previous bleeding, 5.3% recent or active malignancy, 4.1% on antiplatelets and 2.1% on NSAIDS. HAS-BLED score 2.23. No blinding reported. 735 completed 3 year follow up (367 patients died and 55 patients moved out of the area or discontinued VKA treatment	77 MB	30 months
Jover 2012⁶⁵	CHADSVASC	acenocoumarol	17%	933 patients with permanent or paroxysmal NVAF on acenocoumarol OAC (INR 2–3) for at least 6 months. Age 76, 46% male, 85% hypertension, 27% DM, 32% hypercholesterolemia, 14% current smokers, 39% CHF, 20% prior stroke/TIA, 20% CAD, 9% PAD, 17% on antiplatelets. CHADS2 score 2, CHADSVASC score 4. No blinding reported. No loss to FU reported.	80 MB	2.5 years
Lip 2011⁷¹	HAS-BLED Shireman HEMORRHAGE Beyth et al. Kuijer et al.	warfarin	NR	7,329 people with NVAF on warfarin or ximelagatran. Taken from the SPORTIF III and V cohorts (Multinational cohort). Following data are for those who developed a major bleed/no major bleed: age 73.9/70.9, female 31/31%, paroxysmal AF 11/12%, hypertension 77/77%, DM 29/23%, CAD 50/45%, LV dysfunction 44/36%, stroke/TIA 26/21%, CHADS 2.6/2.2.Blinded assessors.	136 MB	499 days
Lip 2014⁷⁴	SAME-TT2R2	VKAs	17%	4,637 patients with AF (n=572 had valvular AF) who were receiving OACs. FRANCE. Mean age 71, 35% female, 60% HF, 28% CAD, 12% previous MI, 6% previous CABG, 44% hypertensive, 9% previous stroke, 9% renal insufficiency. 17% on antiplatelets, 15% on Aspirin, 6% clopidogrel, 4% DAT. Mean CHADSVASC score 3.2, Mean HAS-BLED score 1.6. Not blinded.	144 MB	1016 days
Lip 2018⁷⁷	HAS-BLED ATRIA ORBIT	DOACS	39.10%	57,930 patients with NVAF on DOACs. Taken from 3 Danish nationwide databases. Age 73.5, female 44.6%, HF 22.5%, DM 15.2%, Vascular diseases 16.2%, hypertension 59%, CPD 13.3%, prior bleeding 14.2%, kidney diseases 3.4%, Aspirin use 39.1%, NSAIDs 22.4%. Not blinded. Loss to FU not reported.	2.41 /100 person-years	1 year
Mori, 2019⁸⁸	ORBIT HAS-BLED	DOACS	21.5%	2216 patients with NVAF using DOACs; 63.6% male; median age 73 years; median CHADS2 2; hypertension 73.5%; DM 27.9%; Dyslipidaemia 65.2%; eGFR 64.9; CAD 19.8%; PAD 7.1%; HF 23.7%; prior stroke 20.2%; prior bleeding 27.1%; antiplatelets 21.5%	93 MB	315 days
Nielsen 2016⁹⁰	HAS-BLED Recalibrated HAS-BLED (2 points for previous haemorrhagic stroke instead of 1 point)	unclear	NR	Unknown number of OAC-treated patients from a cohort of 210,299 patients with AF taken from 3 Danish patient registries from 1999 to 2013. Demographic data for the sub-group having OACs is not reported	4.73 MB per 100 person years	Unclear
O’Brien 2015⁹¹	ORBIT HAS-BLED ATRIA-bleeding	rivaroxaban and warfarin	NR	14,264 patients with AF on either rivaroxaban (20mg daily) or Warfarin. This was the external validation cohort, comprising patients from the ROCKET-AF. Demographics of this external validation sample not reported.	772 MB	1.9 years
Olesen 2011⁹⁵	HAS-BLED HEMORRHAGES	VKA	33%	44, 771 patients with AF receiving OACs in Denmark during 1997–2006. Demographic data given as two values as separate data for those with major bleeding / those without. Age 74.6 / 71.2, male 66.8 / 61.2 %, HASBLED score 2.5–2, HF 24.4/19.8%, hypertension 51.6/49.5%, DM 11.4/9.5%, Stroke 22.3/17.4, Renal disease 8.2/4.6%, Vascular disease 18.6/14.8%, Bleeding history 22.6/8.2%, antiplatelet drugs 33% / 25.5%, NSAIDs 22.8/19.1%.	2051 MB	1 year
Pisters 2010¹⁰³	HAS-BLED HEMORRHAGES	Unspecified OACs	NR	1956 patients on OACs only with NVAF (validation cohort). Data not given for this validation cohort subset.	1.75 MB/100 patients years	1 year
Poli 2017¹¹⁰	HAS-BLED HAS-BED (HAS-BLED but without labile INR score) CHADS2 CHADSVASC	warfarin and DOACs	16.50%	4579 patients with AF on DOACS (n=1048) or VKAs (n=3531) on START register in Italy. Age 76, 55% men, 15% HF, 80% hypertensive, 20% DM, 18% CAD, 6% PAD, 43% moderate renal impairment (eGFR 30–60 ml/min), 15% previous stroke/TIA, 3.4% history of major bleeding, TTR 67, concomitant antiplatelet drugs 16.5%, dual antiplatelet therapy 1.3%.	115 MB	1.4 years
Prochaska 2018¹¹³	HAS-BLED HAS-BLED with a point for sustained AF Simplified HAS-BLED	VKA - phenprocoumon	18.30%	1089 patients with medical and electrophysiological evidence of AF, and on VKAs, as part of the thrombEVAL cohort. Denmark. The following baseline data is separated into paroxysmal (n=398) and sustained (n=691) sub-groups by the paper: male 63/63%, age 72/75, DM 30/33%, Family history of MI/stroke 44.5/42%, hypertension 83/81.6%, CKD 24/27%, CAD 43.6/46.7%, HF 43.5/55.2%, history of major bleeding 6.8/6.2%, history of stroke/TIA 16.7/18.7%, MI 21.8/20.8%, PAD 16.1/17.5%, aspirin 18.3/15.1	150 CRB (includes MB and CRNMB)	3 years
Proietti 2016¹¹⁶	HAS-BLED ORBIT ATRIA HEMORRAGES ORBIT with TTR <65% (adding one point to score if <65%) ATRIA with TTR <65% (adding one point to score if <65%) HEMORRAGES with TTR <65% (adding one point to score if <65%)	warfarin	19.90%	3551 patients receiving warfarin in the pooled population dataset from the SPORTIF III and V studies with AF. De-identified datasets with patient-level information for the SPORTIF trials were obtained directly from Astra Zeneca, and all the analyses were performed independent of the company. All patients assigned to the warfarin treatment arms and with available data for the clinical variables used to calculate the four bleeding prediction scores were included in the present analysis. The majority of patients were male (69.5%) and the median [IQR] age was 72 [66–77] years. HAS-BLED score >3: 71%. 706/3551 (19.9%) treated concomitantly with aspirin. 20.1% VKA naïve at baseline prior to VKA initiation.	162 MB	1.6 years
Proietti 2018a¹¹⁴	HAS-BLED ORBIT ATRIA HEMORRHAGES	dabigatran 110mg, 150mg and warfarin (SEP ANALYSESfor C statistics but mixed for sensitivity/spe cificity)	40%	18,113 patients with AF on dabigatran (110 or 150 mg) or warfarin in the RE-LY trial. Multinational cohort. Age 72, 36% female, 79% hypertension, DM 23%, CAD 28%, prev stroke 22%, symptomatic HF 27%, VKA naïve 50%, anti-platelets 40%, CHADS2 2. BLINDED ASSESSORS.	1182 MB	2 years
Proietti 2018b¹¹⁵	HAS-BLED GARFIELD	warfarin	19.90%	3550 AF patients enrolled on the SPORTIF III trial who were on Warfarin. Age 72, 30.5% female, 76.7% hypertension, 23.5% DM, 44.3% CAD, 20.6% stroke/TIA, 37.3% HF, 5.6% previous bleeding, 25.9% CKD, 19.9% aspirin use. TTR 68.1. HAS-BLED: 3. 804 patients interrupted Warfarin during the follow up period. BLINDED ASSESSORS.	127 MB 168 major/CRNMB	1.56 years
Quinn 2016¹¹⁷	CHADS2 CHADSVASC ATRIA HAS-BLED	warfarin	NR	13,559 patients with AF who were on and off warfarin. No demographic data provided.	unclear	unclear
Rivera-Caravaca 2017¹²⁰	HEMORRHAGES HAS-BLED ATRIA ORBIT	VKAs	18%	1361 patients – same patients as Roldan 2017¹²⁸- with AF who were taking VKA OACs (acenocoumarol), in Spain. Age 76, 49% male, 82% hypertensive, 27% DM, 19% previous stroke/TIA, 19% CAD, 31% HF, 7% PAD, 10% renal impairment, 33% hypercholesterolemia, 8% previous bleeding episode, 4% alcohol abuse, 1% hepatic disease, 8% cancer. Median HAS-BLED score of 2	250 MB	6.5 years
Rivera-Caravaca, 2019¹¹⁹	HAS-BLED HAS-BLED with 1 to 6 added biomarkers	VKAs	18.4%	940 patients who were taking VKA OACs (IRR 2–3), in Spain. Age 76, 50.6% male, 82% hypertensive, 26.2% DM, 18.8% previous stroke/TIA, 19.8% CAD, 30.4% HF, 10.6% renal impairment, 33.3% hypercholesterolemia, Median HAS-BLED score of 2	172MB	6.5 years
Roldan 2013a¹²⁵	HAS-BLED ATRIA	acenocoumarol	17%	937 consecutive patients with AF receiving anticoagulant therapy with INR from 2–3. 49% male, mean age 76, 82% hypertension, 25% DM, 37% HF, 19% stroke, 10% renal impairment, 19% CAD, 9% previous bleeding, 17% antiplatelet therapy. Median HAS-BLED score of 2, median CHADS2 score of 2.	79 MB	952 days
Roldan 2013b¹²⁶	HAS-BLED CHADS CHADSVASC	acenocoumarol	18%	1370 consecutive patients with AF receiving anticoagulant therapy with INR from 2–3. 49% male, mean age 76, 19% stroke, 10% renal impairment, 18% CAD, 9% previous bleeding, 18% antiplatelet therapy. Median HAS-BLED score of 2, median CHADS2 score of 2.	114 MB	996 days
Roldan 2017¹²⁸	HAS-BLED Modified HAS-BLED (including vWF, high sensitivity troponin T, N-terminal fragment B-type natriuretic peptide, high sensitivity IL-6, time in therapeutic range and modification of diet in renal disease CHADS-VASC Modified CHADSVASC (as above)	VKAs	18%	1361 consecutive patients with AF who were taking VKA OACs (acenocoumarol), in Spain. Age 76, 49% male, 82% hypertensive, 27% DM, 19% previous stroke/TIA, 19% CAD, 31% HF, 7% PAD, 10% renal impairment, 33% hypercholesterolemia, 8% previous bleeding episode, 4% alcohol abuse, 1% hepatic disease, 8% cancer. 18% antiplatelet therapy. Median HAS-BLED score of 2	250 MB	7.49 years
Schwartz, 2019¹³⁵	Modified HAS-BLED	VKAs and DOACS	NR	Data from 9819 patients with AF who were on DOACs or VKAs were retrieved from the Northwestern Healthcare system’s Enterprise Database Warehouse. The data allowed identification of bleeding outcomes, and calculation of prior HAS-BLED scores. Mean age 67.6 for white patients and 63.1 for non-white patients. Mean CHADSVASC was 2.4 in whites and 2.2 in non-whites	604 MB	971 days
Senoo 2016a¹³⁶	HAS-BLED ORBIT	Idraparinux	NR	2283 patients with AF on non-warfarin OAC. UK. Age 70. No other details of demographics reported.	74 MB 346 CRB	311 days
Senoo 2016b¹³⁷	HAS-BLED ORBIT ATRIA Also with TTR for NRI analysis of ORBIT and ATRIAS only	warfarin	16.50%	2293 patients with AF warfarin OAC. UK. Age 71, 65.5% male, paroxysmal AF 35.5%, persistent AF 9.3%, permanent AF 54.9%, hypertension 77%, HF 24%, DM 20%, CAD 31%, Stroke/TIA 25%, TTR 58%, Aspirin 16.5%;NSAIDS 5.4%.CHASVASC of 0–2: 28.8%, HAS-BLED 2.	39 MB 251 CRB	Unclear but probably < 1 year
Serna 2018¹³⁸	HAS-BLED GEN /HAS-BLED (added point if patient carrying VKORC1 allele and CYP2C9*3 polymorphisms)	acenocoumarol (VKA)	NR	652 consecutive ASF patients stable on VKAs (INR 2–3) for 6 months. Spain. Age 76, 48.6% male, 82.8% hypertension, 24.2% DM, 18.7% history of stroke/TIA, 18.4% CAD, 31.9% hypercholesterolemia, 34.5% HF, 9.2% renal impairment, 1.5% hepatic impairment, 8.3% previous bleeding. HAS-BLED score 2. No data on antiplatelets.	106 MB	7.6 years
Siu 2014¹⁴²	HAS-BLED	warfarin	NR	1912 patients with NVAF (not defined) who received OACs (Warfarin). Mean age 73, 47% female, 55.8% hypertensive, 24% DM, 1.8% renal failure on dialysis, 24% HF, 24% CAD, 6.3% PAD, 29.6% prior stroke/TIA, prior IC haemorrhage 2.1%. Mean CHADSVASC 3.3. No data on antiplatelets	30 ICH	3.19 years
Steinberg 2016¹⁴⁶	ATRIA HAS-BLED	warfarin and dabigatran	NR	7420 AF patients on OACs, out of an original cohort of 9715 from the ORBIT-AF trial. USA. Ranges for baseline data given as different data given for people in low, intermediate and high risk categories. Age 73–77, female 40–46%, hypertension 83–87%, diabetes 28–38%, previous GI bleed 5.7–16%, CAD 32–48%, Prior stroke/TIA 14–26%, CHF 30–46%, HAS-Bled 1.61–2.17, CHADS2 2.17–2.81. No data on antiplatelets.	632 MB	Unclear
Suzuki 2014¹⁴⁷	HAS-BLED Modified HAS_BLED (renal dysfunction defined by eGFR <60, with exclusion of the ‘elderly’ factor because eGFR is calculated based on patient age)	warfarin	36.9–50%	231 NVAF patients on warfarin for at least 1 year. Demographics given as ranges as only reported for sub-groups of eGFR: age 68–74, 63.1–80% male, hypertension 53.2 to 64.4%, CAD 14.4 to 16.7%, CHF: 20 to 25.2%, dyslipidaemia 28.8 to 36.7%, eGFR 12.7 to 74.3 mL/min/1.73m2) antiplatelet drugs 36.9 to 50%. TTR 56.9 to 65.1%.	44 MB	7.1 years
Wang 2016¹⁵⁴	HAS-BLED	dabigatran and warfarin (SEP ANALYSES)	NR	21,934 adults with AF who were starting dabigatran (30%) or Warfarin. Patients were on a healthcare claims database in USA. Demographic data given for those on Warfarin (n=15418): Age 65, female 34%, 27% CHF, 31% DM, 93% hypertensive, 20% prior stroke, 22% PVD. 43% with HAS-BLED score of 3 or more. 32% with CHADS2 score of 3 or more.	4.6 MB per 100 patient years	5 months
Yao 2017¹⁵⁸	CHADSVASC CHADS HAS-BLED ORBIT ATRIA	DOACS (results not sub-grouped)	7%	39, 539 patients with NVAF from USA insurance database (OptumsLabs Data Warehouse) who had started DOACs between 2010 and 2015. Age 71, 42% female, 20% non-white, 28% HF, 86% hypertension, 34% DM, 14% previous strokes/TIA, 48% vascular disease, 7% stage II or IV CKD, 4% abnormal liver function, 9% previous major bleeding, 7% using antiplatelets, 5% using NSAIDs, 28% had had previous warfarin exposure. HAS-BLED: 2	115 MB	0.6 years

: MB=major bleeding, CRB= clinically relevant bleeding, CRNMB= clinically relevant non-major bleeding, ICH= Intracranial hemorrhage

Table 5Summary of risk tools and their constituent variables

Risk tool	Variables and scoring	Bleeding risk interpretation (where applicable)
ABC-bleeding	Prior bleeding, age, hs-troponin, GDF-15 and Hb. Continuous values inputted (where appropriate) and a probability score derived by algorithm.	Score is the 1 year risk of major bleeding
ABC bleeding CrC	ABC-bleeding with creatinine clearance replacing GDF-15
ABC-bleeding CKD-EPI	ABC-bleeding with CKD-EPI biomarker added to the scheme
ABC-bleeding cTnl-hs	ABC-bleeding with cTnl-hs biomarker added to the scheme
ABC-bleeding cystatin C	ABC-bleeding with cystatin C biomarker added to the scheme
Anticoagulation-specific Bleeding Score (ABS)	The 1-year risk of bleeding can be calculated as 1 - (0.98101) Exp[0.02306(Age - 70.1736) + 0.29958(Kidney Disease −0.13244) + 0.19215(COPD −0.31286)+ 0.23529(Prior Bleed −0.21338) +0.32257(Anemia −0.24892) + 0.21811(Heart Failure-0.33899)+ 0.22599(Antiplatelet-0.16341) + 0.15944 (Diuretics-0.4518) + 0.2111(Diabetes Mellitus-0.31686) + 0.16806 (Cancer-0.16955) - 0.28572 (Antiarrhythmic −0.11919) + 0.13743(Ischemic stroke - 0.26681) + 0.10269(Coronary Artery Disease −0.40768) - 0.04775(Male Sex-0.59637) −0.30127 (Dabigatran) + 0.01299(Rivaroxaban) - 0.52426(Apixaban)]	1 year risk of bleeding yielded
APTT	Biomarker: activated partial thromboplastin time	No pre-set thresholds provided in paper
ATRIA	Anaemia (3 points), severe renal disease (eGFR <30) (3 points), age >75 years (2 points), any prior bleeding (1 point), hypertension history (1 point)	Low: 0–3 Moderate: 4 High: 5 or more
ATRIA with AS	ATRIA with existence of aortic stenosis added inas a risk factor to the scheme
ATRIA with TTR (<65% TTR)	ATRIA with time in therapeutic range of <65% added in as a risk factor to the scheme
Beyth	See mOBRI
CBRM	See Shireman
CHADS2	One point each for CHF, hypertension, age 75 of older, and DM, and 2 points for prior stroke or TIA.	Score 0=low risk; score 1–2=intermediate risk; score 3 to 6=high risk
CHADSVASC	One point for female sex, history of CHF, history of hypertension, history of vascular disease or history of DM. 2 points for history of stroke/TE. Age <65=0 points, 65–74=1 point, >75=2 points. Maximum score 9 points.	Low risk =0 points; 1 point=low/moderate; >2 points moderate/high
CHADSVASC with TnT	CHADSVASC with TnT levels added in to the scheme
GARFIELD/ GARFIELD AF	Age, pulse, systolic blood pressure, history of vascular disease, history of bleeding, heart failure, renal disease and use of OACs.	Score is a measure of bleeding risk
GDF-15	Biomarker: levels of Growth Differentiation Factor 15
GEN/HAS-BLED	HAS-BLED with added point if patient carrying VKORC1 allele and CYP2C9*3 polymorphisms
HAS-BED	HAD-BLED with elimination of labile INR factor.
HAS-BLED	Hypertension, abnormal renal/liver function (1 point each), stroke, bleeding history or predisposition, labile INR, elderly drugs/alcohol concomitantly (1 point each). Maximum 9 points	Low: 0 Moderate: 1–2 High: 3 or more
HAS-BLED with AS	HAS-BLED with existence of aortic stenosis added inas a risk factor to the scheme
HAS-BLED with GDF-15	HAS-BLED with GDF biomarker added to the scheme
HAS-BLED with point for sustained AF	HAS-BLED with additional factor of ‘sustained AF in the presence of HF’.
HAS-BLED with TnI	HAS-BLED with TnT levels added in to the scheme
HAS-BLED with VWF	HAS-BLED with Van Willebrandlevels added into the scheme
HAS-BLED with no labile INR and no stroke/TIA component	HAS-BLED with no labile INR and no stroke/TIA component
HAS-BLED + VWF + NT-proBNP	HAS-BLED with Van Willebrand levels and N-terminal pro-B-type natriuretic peptide added into the scheme
HAS-BLED + VWF + NT-proBNP + IL-6	HAS-BLED with Van Willebrand levels and N-terminal pro-B-type natriuretic peptide and Interleukin-6 added into the scheme
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T	HAS-BLED with Van Willebrand levels and N-terminal pro-B-type natriuretic peptide and Interleukin-6 and Troponin T added into the scheme
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP	HAS-BLED with Van Willebrand levels and N-terminal pro-B-type natriuretic peptide and Interleukin-6 and Troponin T and Beta trace protein added into the scheme
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP + soluble fibrin monomer complex	HAS-BLED with Van Willebrand levels and N-terminal pro-B-type natriuretic peptide and Interleukin-6 and Troponin T and Beta trace protein and soluble fibrin monomer complex added into the scheme
HEMORRHAGES	Hepatic or renal disease (1 point) Ethanol abuse (1 point)* Malignancy (1 point) Older age >75 yrs (1 point) Reduced platelet count or function (1 point) Re-bleeding risk (2 points) Hypertension (1 point) Anaemia (1 point) Genetic factors (1 point) Excessive fall risk or neuropsychiatric disease (1 point) Stroke (1 point)	Low: 0–1 Intermediate: 2–3 High: 4 and above
HEMORRHAGES with TTR (<65% TTR)	HEMORRHAGES with time in therapeutic range of <65% added in as a risk factor to the scheme
HTI	Biomarker: Hemoclot thrombin inhibitor levels	No pre-set thresholds provided in paper
Kearon 2003	Age >65yrs (1 point) Prior stroke (1 point) Prior peptic ulcer disease (1 point) Prior GI bleeding (1 point) Creatinine >1.5 mg/dl (1 point) Anemia or thrombocytopenia (1 point) Liver disease (1 point) Diabetes mellitus (1 point) Antiplatelet therapy (1 point)	Low: 0–1 Intermediate:2 High 3 or more
Kuijer 1999	Age >60 yrs (1.6 points) Female (1.3 points) Malignancy (2.2 points)	Low: 0 Intermediate 1–2 High 3 or more
Landefield and Goldman and Beyth	See mOBRI
MBRFS	See MBR
mOBRI (also known as Landefield and Goldman and Beyth, or simply Beyth)	Age > 65 years, GI bleed in past 2 weeks, previous stroke, comorbidities (recent MI, Hct <30%, diabetes, creatinine >1.5 ml/l) with 1 point for presence of each risk factor	Low: 0 Moderate; 1–2 High: 3 or more
MBR (Modifiable Bleeding Risk factors score)	Defined as the cumulative number of modifiable bleeding risk factors of each patient according to the 2016 ESC guideline, including hypertension, medication predisposing to bleeding, and excess alcohol. 1 point for each.	Score ranges from 0–3.
Modified CHADSVASC	CHADSVASC with vWF, high sensitivity troponin T, N-terminal fragment B-type natriuretic peptide, high sensitivity IL-6, time in therapeutic range and modification of diet in renal disease
Modified HAS-BLED (multiple additions using biomarkers)	HAS-BLED with addition ofvWF, high sensitivity troponin T, N-terminal fragment B-type natriuretic peptide, high sensitivity IL-6, time in therapeutic range and modification of diet in renal disease
Modified HAS-BLED (single change of renal dysfunction threshold)	HAS-BLED with modification of the renal impairment factor (from eGFR <30 to eGFR <60)
ORBIT	Older age (75 years and above) (1point), reduced hemoglobin, hematocrit, or history of anemia (2 points), bleeding history: (2 points), insufficient kidney function (eGFR below 60 mL/min/1.73 m2)(1 point), treatment with an antiplatelet agent (1 point).	Low: 0–2 Moderate:3 High: 4 or more
ORBIT with AS	ORBIT with existence of aortic stenosis added inas a risk factor to the scheme
ORBIT with GDF-15	ORBIT with GDF-15 levels added into the scheme
ORBIT with TTR (<65% TTR)	ORBIT with time in therapeutic range of <65% added in as a risk factor to the scheme
Outpatient bleeding Index (OBI)	Age >65 yrs (1 point) Prior stroke (1 point) Prior GI bleeding (1 point) Recent MI, diabetes mellitus, hematocrit <30%, creatinine >1.5 mg/dl (1 point if any of the above)	Low: 0 Intermediate 1–2 High 3 or more
Prothrombin time	Biomarker: Prothrombin time	No pre-set thresholds provided in paper
Riete	Recent major bleeding (□15 days before thrombotic event) (2 points) Creatinine >1.2 mg/dl (1.5 points) Anemia (1.5 points) Malignancy (1 point) Clinically overt pulmonary embolism (1 point) Age >75 yrs (1 point)	Low: 0 Intermediate: 1–4 High: >4
Same TTR	Sum of points after addition of one point for female sex, age <60 years, medical history of >2 comorbidities (amongst hypertension, DM, CAD/MI, PAD, CHF, previous CVA, pulmonary disease and hepatic/renal disease, treatment and 2 points each for smoking and non-white race.	Low:0–1 Moderate: 2 High >2
Shireman 2006 (also known as CBRM)	Age >70 yrs Female Remote bleeding event Recent bleeding event Alcohol or drug abuse Diabetes mellitus Anemia (Hct <30% during index hospitalization) Antiplatelet drugs (aspirin, clopidogrel, or ticlodipine at discharge) Risk score = 0.49 (age >70) + 0.32 (female) + 0.58 (remote bleed) + 0.62 (recent bleed) + 0.71 (alcohol/drug abuse) + 0.27 (diabetes) + 0.86 (anemia) + 0.32 (antiplatelet use)	Low <1.07 Intermediate >1.07, <2.19 High >2.19
Simplified HAS-BLED	HAS-BLED, containing only the factors of age >65 years, history of major bleeding, and sustained AF in the presence of heart failure
TnI	Biomarker: Troponin I levels
TnT	Biomarker: Troponin T levels
vWF	Biomarker: levels of plasma glycoprotein von Willebrand factor

Table 6Clinical evidence profile: accuracy of prediction of Major Bleedingin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I²to <50% in all sub-groups

Risk tool	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	Area Under Curve Individual study effects [point estimate (95% Cis) ] Pooled effect/range/median	Quality
HAS-BLED	47	532,442	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.62 (0.61–0.64) [I²=94%]	VERY LOW
Modified HASBLED¹³⁵	1	9819	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	0.60(0.55–0.66)(‘Non-white’ participants) 0.57(0.55–0.60) (‘white’ participants)	VERY LOW
HAS-BLED with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.68(0.66–0.70)	MODERATE
HAS-BLED with GDF-15	1	8474	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision	0.69(0.67–0.72)	VERY LOW
HAS-BLED with vWF	2	1215	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	POOLED RESULT: Fixed effect: 0.62 (0.60–0.64) [I²=6%]	MOD
HAS-BLED + VWF + NT-proBNP	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.64(0.61–0.67)	MOD
HAS-BLED + VWF + NT-proBNP + IL-6	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.64(0.61–0.67)	MOD
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.64(0.61–0.67)	MOD
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.64(0.60–0.67)	MOD
HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP + soluble fibrin monomer complex	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.64(0.60–0.67)	MOD
GEN/HAS-BLED	1	652	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.65(0.61–0.68)	MOD
Modified HAS-BLED (multiple additions using biomarkers)	1	1361	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.60(0.56–0.64)	MOD
Modified HAS-BLED (single change of renal dysfunction threshold)	1	231	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.67(0.57–0.75)	VERY LOW
HAS-BED	1	4579	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.58(0.53–0.64)	LOW
HAS-BLED with TnI	1	14,821	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.63	LOW
HEMORRHA GES	19	240,995	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.63 (0.60–0.66) [I²=97%]	VERY LOW
HEMORRHA GES with TTR (<65% TTR)	2	4912	Serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Median: 0.65	VERY LOW
ATRIA	23	286,664	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.64 (0.61–0.66) [I²=97%]	VERY LOW
ATRIA with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.67(0.66–0.69)	MODERATE
ATRIA with TTR (<65% TTR)	2	4912	Serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Median: 0.68	VERY LOW
ORBIT	21	270,606	Very serious risk of bias^a	Very serious riskof inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.64 (0.61–0.67) [I²=97%]	VERY LOW
ORBIT with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.68(0.67–0.70)	MODERATE
ORBIT with TTR (<65% TTR)	2	4912	Serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Median: 0.67	VERY LOW
ORBIT with GDF-15	1	8474	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.71(0.68–0.73)	VERY LOW
CHADS2	5	61,647	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.61 (0.57–0.64) [I²=85%]	VERY LOW
CHADSVASC	8	24,402	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED RESULT: Random effect: 0.59 (0.54–0.64) [I²=92%]	VERY LOW
Modified CHADSVASC	1	1361	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.56(0.53–0.60)	MOD
CHADSVASC with TnT	1	14,897	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.63(0.61–0.65)	LOW
GARFIELD	3	62,172	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random effects 0.60 (0.56–0.65); I2=96%	VERY LOW
GARFIELD subgrouped by OAC - VKA	1	3550	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.56(0.54–0.58)	LOW
GARFIELD subgrouped by OAC – Mixed VKA/DOACs	1	7442	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.59–0.63)	LOW
GARFIELD subgrouped by antiplatelets - <33% with antiplatelets	1	3550	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.56(0.54–0.58)	LOW
GARFIELD subgrouped by antiplatelets – unknown % with antiplatelets	1	7442	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.59–0.63)	LOW
ABC-bleeding	3	16869	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	Serious imprecision^c	POOLED RESULT: Random effect: 0.69(0.65–0.74) [I²=85%]	VERY LOW
ABC-bleeding Subgrouped by OAC - VKA	1	2814	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.65(0.61–0.70)	VERY LOW
ABC-bleeding Subgrouped by OAC - Mixed	1	8705	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.69(0.66–0.71)[Mixed]	VERY LOW
ABC-bleeding Subgrouped by OAC - NOACs	1	5350	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.74(0.71–0.76)[DOAC]	VERY LOW
ABC-bleeding CrC	1	1120	Serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.52(0.49–0.55)	LOW
ABC-bleeding cTnl-hs	2	8164	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	Serious imprecision^c	POOLED RESULT: Random effect: 0.70 (0.61–0.78) [I2=92%]	VERY LOW
ABC-bleeding cTnl-hs subgrouped by OAC - VKA	1	2814	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.65(0.61–0.70	VERY LOW
ABC-bleeding cTnl-hs subgrouped by OAC - DOAC	1	5350	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.74(0.71–0.76)	LOW
ABC-bleeding cystatin C	2	8164	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	Serious imprecision^c	POOLED RESULT: Random effect: 0.68 (0.65–0.72) [I2=90.6%]	VERY LOW
ABC-bleeding cystatin C subgrouped by OAC - VKA	1	2814	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.60(0.54–0.66)	LOW
ABC-bleeding cystatin C subgrouped by OAC - DOAC	1	5350	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.72(0.68–0.75)	VERY LOW
ABC-bleeding CKD-EPI	2	8164	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	Serious imprecision^c	POOLED RESULT: Random effect: 0.70 (0.68–0.72) [I2=79%]	VERY LOW
ABC-bleeding CKD-EPI subgrouped by OAC - VKA	1	2814	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.65(0.60–0.69)	LOW
ABC-bleeding CKD-EPI subgrouped by OAC - DOAC	1	5350	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.71(0.69–0.74)	VERY LOW
vWF	1	1215	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.61(0.57–0.65)	MOD
ABS	1	81285	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.67(0.65–0.68)[warfarin], 0.72(0.69–0.76)[dabigatran] 0.70(0.68–0.73)[rivaroxaban] 0.72(0.67–0.77) [apixaban]	VERY LOW
OBI	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.59(0.58–0.611	LOW
Kuijer	3	8332	Very serious risk of bias^a	Serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED EFFECT: Random effects: 0.54 (0.51–0.58) [I²=72%]	VERY LOW
Kearon	2	4667	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	Median: 0.675	LOW
Riete	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.68(0.65–0.70)	LOW
Shireman / CBRM	5	12385	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	POOLED EFFECT: Random effect: 0.64(0.59–0.69) [I²=80%]	VERY LOW
mOBRI/Lande field and Goldman and Beyth / Beyth	3	8762	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	POOLED EFFECT: Fixed effect: 0.56(0.51–0.60) [I²=0%].	LOW
TnT	1	14,897	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.62(0.60–0.64)	LOW
TnI	1	14,821	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.60	LOW
GDF-15	1	8474	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.67(0.65–0.69)	LOW
MBR	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.53(0.52–0.53)	LOW
HTI	1	208	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.65	LOW
Prothrombin time	1	208	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.54(0.47–0.62)	VERY LOW
Same TTR	1	4637	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.55 (0.54–0.57)	LOW
APTT	1	208	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	0.58(0.50–0.69)	LOW

: Pooling (meta-analysis) was carried out if there were at least two studies per risk tool with confidence intervals. RevMan was used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist(see Appendix F).Risk of bias was serious for some risk tools because fewof the studiesreported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because manystudieswith the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, an I²of 50–74% was deemed serious inconsistency and an I²of 75% or above was deemed very serious inconsistency. If no pooling were possible, inconsistency was assessed by inspection of the degree of overlap of confidence intervals between studies: if one of more Cis did not overlap then a rating of serious inconsistency was given. Reasons for heterogeneity between studiesmay include geographical/cultural/ethnic differences. Clinically the studies appeared reasonably homogeneous, with similar rates of hypertension, diabetes and former stroke.

c): The judgement of precision was based on the spread of confidence interval around two clinical thresholds: C statisticsof 0.5 and 0.7. The threshold of 0.5 marked the boundary between no predictive value better than chance and a predictive value better than chance. The threshold of 0.7 marked the boundary above which the committee might consider recommendations. If the 95% Cis crossed one of these thresholds a rating of serious imprecision was given and if they crossed both of these thresholds a rating of very serious imprecision as given.

Table 7Clinical evidence profile: sensitivity and specificityof prediction of Major Bleeding in all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results; for meta-analysed results the 95% credible intervals are given for the pooled effect only

Risk tool	No of COHORTS	n	Sensitivity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Specificity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Risk of bias	Inconsistency	Indirectness	Imprecision	Quality
HAS-BLED at threshold of ≥1	7	128791	Pooled sensitivity: 0.979(0.941–0.993)	Pooled specificity: 0.070(0.027–0.174)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Serious imprecision^c	VERY LOW
HAS-BLED at threshold of ≥2	10	177728	Pooled sensitivity: 0.793(0.570–0.919)	Pooled specificity: 0.396(0.207–0.624)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Very serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Serious imprecision^c	VERY LOW
HAS-BLED at threshold of ≥3	13	170197	Pooled sensitivity: 0.512(0.385–0.637)	Pooled specificity: 0.679(0.554–0.782)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
HAS-BLED at threshold of ≥4	1	3525	0.543(0.453–0.632)	0.591(0.575–0.608)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
Modified HASBLED¹³⁵at threshold of ≥1	1	9819	0.925 (0.902–0.945)	0.1504(0.143–0.158)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
Modified HASBLED¹³⁵at threshold of ≥2	1	9819	0.644(0.604–0.682)	0.4937(0.483–0.5040	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
Modified HASBLED¹³⁵at threshold of ≥3	1	9819	0.311(0.275–0.349)	0.826(0.819–0.834)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
HEMORRHAGES at threshold of ≥1	3	7406	Pooled sensitivity: 0.919(0.658–0.985)	Pooled specificity: 0.167(0.037–0.5207)	Sensitivity
					Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
HEMORRHAGES at threshold of ≥2	6	60023	Pooled sensitivity: 0.631(0.417–0.798)	Pooled specificity: 0.549(0.349–0.734))	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
HEMORRHAGES at threshold of ≥3	2	5138	0.478(0.354–0.603) 0.171 (0.112–0.250)	0.739(0.716–0.761) 0.886(0.874–0.896)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecision	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecision	VERY LOW
ATRIA at threshold of ≥1	4	103289	Pooled sensitivity: 0.955(0.864–0.986)	Pooled specificity: 0.132(0.061–0.259)	Sensitivity
					Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecison^c	VERY LOW
ATRIA at threshold of >2	5	103289	Pooled sensitivity: 0.685(0.450–0.848)	Pooled specificity: 0.539(0.354–0.716)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecison^c	VERY LOW
ATRIA at threshold of ≥3	3	101023	Pooled sensitivity: 0.571(0.212–0.856)	Pooled specificity: 0.638(0.35446–0.861)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecision	VERY LOW
ATRIA at threshold of ≥4	6	111338	Pooled sensitivity: 0.259(0.096–0.513)	Pooled specificity: 0.874(0.714–0.941)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecison	VERY LOW
ORBIT at threshold of ≥1	4	103302	Pooled sensitivity: 0.804(0.610–0.916)	Pooled specificity: 0.381(0.217–0.574)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Very serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
ORBIT at threshold of ≥2	4	103302	Pooled sensitivity: 0.460(0.233–0.692)	Pooled specificity: 0.716(0.528–0.849)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	Serious imprecison^c	VERY LOW
ORBIT at threshold of ≥3	8	114895	Pooled sensitivity: 0.340(0.213–0.493)	Pooled specificity: 0.845(0.766–0.900)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecison	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^a	No serious indirectness	No serious imprecison	VERY LOW
CHADS2 at threshold of ≥1	1	39539	0.991(0.981–0.998)	0.084(0.081–0.086)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
CHADS2 at threshold of ≥2	1	39539	0.865(0.836–0.889))	0.341(0.336–0.346)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
CHADS2 at threshold of ≥3	1	39539	0.552(0.513–0.590)	0.776(0.775–0.779)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
CHADSVASC at threshold of ≥1	1	39539	0.998(0.992–1.00)	0.385(0.366–0.404)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
CHADSVASC at threshold of ≥2	1	39539	0.984(0.970–0.992)	0.129(0.125–0.132)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
CHADSVASC at threshold of ≥3	1	39539	0.929(0.907–0.948)	0.271(0.267–0.276)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
ABC-bleedingCrCat threshold of ≥2%	1	1120	0.835(0.778–0.884)	0.194(0.169–0.221)	Sensitivity
					Serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
					Specificity
					Serious risk of bias^a	NA	No serious indirectness	No serious imprecison	LOW
HTIat threshold >117 ng/ml	1	208	0.59[no raw data or 95% Cis reported in paper]	0.71[no raw data or 95% Cis reported in paper]	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	NA	LOW
					Specificity
					Very serious risk of bias^a	NAS	No serious indirectness	NA	LOW

: Pooling (meta-analysis) was carried out if there were at least three studies per risk tool with confidence intervals. RevMan and WinBugs were used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist. Risk of bias was serious for some risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because many studies with the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, inconsistency was assessed by visual inspection of the sensitivity/specificity plots, or data (if 2 studies). The evidence was downgraded by 1 increment if there was no overlap of 95% confidence intervals. For single studies no evaluation was made and ‘not applicable’ was recorded.Subgrouping to attempt to resolve heterogeneity was not carried out because there would always be <3 studies in any of the constituent sub-group categories, making it not possible to do a further meta-analysis within each sub-group.

c): Imprecision was assessed based on inspection of the confidence region in the meta-analysis or, where meta-analysis has not been conducted, assessed according to the range of confidence intervals in the individual studies. The evidence was downgraded by 1 increment when the confidence interval around the point estimate crossed one of the clinical thresholds (0.90 or 0.60 for sensitivity and 0.5 and 0.1 for specificity), and downgraded by 2 increments when the confidence interval around the point estimate crossed both of the clinical thresholds. The upper clinical threshold marked the point above which recommendations would be possible, and the lower clinical threshold marked the point below which the tool would be regarded as of little clinical use.

Source: Calibration plot in Claxton, 2018²³. This was for the Anticoagulation-specific bleeding score and was based on a mixed (VKA and DOAC) cohort.

Source: Calibration plot in Hilkens, 2017⁵⁸. This was based on a mixed (VKA and DOAC) cohort.

Figure 1<Insert graphic title here>

Source: Calibration plot in O’Brien 2015⁹¹. This was a mixed cohort.

Source: Calibration plot in Lip, 2018⁷⁷. This was based on an exclusively DOAC-using cohort.

Source: Calibration plot in Yao, 2017¹⁵⁸. This was based on an exclusively DOAC-using cohort.

Table 8NRI for major bleeding – HAS-BLED versus other tools

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
HAS-BLED v HEMORRHAGES	5	50,051	Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled: Random effects NRI: + 0.080(−0.030to +0.190); I²= 69%	VERY LOW
HAS-BLED v ATRIA	6	50,988	Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled: Random effects NRI: + 0.070(−0.020to +0.160); I²= 52%	VERY LOW
HAS-BLED v MBR	1	40450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.056 (0.043 to 0.068)	LOW
HAS-BLED v CHADS2	3	17529	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	Pooled fixed effect NRI: +0.440(+0.250to +0.630); I²=0%	LOW
HAS-BLED v ORBIT	3	46284	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	Pooled fixed effect NRI: +0.050(+0.040to +0.070); I²=0%	LOW
HAS-BLED v CHADSVASC	3	5518	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	Pooled fixed effect NRI: +0.37 (+0.21 to +0.52); I²=0%	LOW
HAS-BLED v ABC	1	8705	Serious risk of bias^a	Noserious inconsistency	No serious indirectness	Serious imprecision^c	−0.138(−0.080to 0.228)	LOW
HAS-BLED v ABC CrC	1	1120	Serious risk of bias^a	Noserious inconsistency	No serious indirectness	Serious imprecision^c	+0.137(−0.010to 0.290)	LOW
HAS-BLED v GARFIELD	1	3550	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.042(−0.087 to 0.189)	VERY LOW
HAS-BLED v HAS-BLED with vWF	2	2155	Serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled random effect NRI: −0.12 (−0.33 to +0.09); I²=92%	VERY LOW
HAS-BLED v HAS-BLED + VWF + NT-proBNP	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.201(−0.329 to −0.002)	MOD
HAS-BLED v HAS-BLED + VWF + NT- proBNP + IL-6	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.192(−0.325to −0.001)	MOD
HAS-BLED v HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.194(−0.337 to −0.003)	MOD
HAS-BLED v HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.196(−0.327 to −0.005)	MOD
HAS-BLED v HAS-BLED + VWF + NT-proBNP + IL-6 + Troponin T + BTP + soluble fibrin monomer complex	1	940	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.203(−0.342 to −0.004)	MOD
HAS-BLED v Recalibrated HAS-BLED	1	Unknown	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.090(−0.123 to −0.0480)	LOW
HAS-BLED v modified HAS-BLED (including multiple biomarkers)	1	1361	Serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.062 (−0.020to 0.140)	LOW
HAS-BLED v modified HAS-BLED (including new renal dysfunction definition)	1	231	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.500(−0.820to −0.180)	LOW
HAS-BLED v GEN/HAS_BLES	1	652	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.044(0.010to 0.080)	MOD
HAS-BLED vs HAS-BLED with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.0481(p=0.034)	MOD

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 9NRI for major bleeding – ATRIA versus other tools

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
ATRIA v CHADS2	2	16159	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	MEDIAN: +0.43	LOW
ATRIA v ORBIT	1	3551	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.0355	LOW
ATRIA v CHADSVASC	2	42139	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	MEDIAN:+0.32	LOW
ATRIA v HEMORRHAGES	5	12664	Very serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled random effect NRI: +0.090(−0.080to +0.207); I2=83%	VERY LOW
ATRIA v OBI	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.505	LOW
ATRIA v Kuijer	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.566	LOW
ATRIA v Kearon	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.277	LOW
ATRIA v Shireman	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.344	LOW
ATRIA v Riete	1	3063	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.448	LOW
ATRIA v ATRIA with TTR<65%	3	4005	Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	Pooled random effect NRI: −0.230(−0.410to −0.040); I²=64%	VERY LOW
ATRIA v MBR	1	40450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision	+0.007 (−0.014 to 0.027)	LOW
ATRIA vs ATRIA with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.0645(p=0.025)	MOD

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 10NRI for major bleeding – HEMORRHAGES versus other tools

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
HEMORRHAGES v CHADS2	1	2600	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.540(0.220to 0.860)	LOW
HEMORRHAGES v CHADSVASC	1	2600	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.590(0.240to 0.940)	LOW
HEMORRHAGES v ORBIT	1	3551	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	−0.216	LOW
HEMORRHAGES v HEMORRHAGES with TTR<65%	2	1712	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	MEDIAN: −0.161	MOD
HEMORRHAGES v MBR	1	40450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.012 (−0.007 to 0.032)	VERY LOW

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 11NRI for major bleeding – ORBIT versus other tools

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
ORBIT v ORBIT with TTR<65%	3	4009	Very serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled random effect NRI: −0.21(−0.44 to 0.02); I²=77%	VERY LOW
ORBIT v CHADSVASC	1	39539	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	NA	+0.010	LOW
ORBIT v MBR	1	40450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	0.000 (−0.021 to 0.021)	VERY LOW
ORBIT vs ORBIT with AS	1	2880	Serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision	−0.014(p=0.170)	VERY LOW

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 12NRI for major bleeding – CHADSVASC versus other tools

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
CHADSVASCv CHADS2	3	55698	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	MEDIAN: +0.040	VERY LOW
CHADSVASC v modified CHADSVASC (including multiple biomarkers)	1	1361	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.0026 (−0.020to 0.030)	VERY LOW

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 13Clinical evidence profile: accuracy of prediction of CRBin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups

Risk tool	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	Area Under Curve Individual study effects [point estimate (95% Cis) ] Pooled effect/range/median	Quality
HAS-BLED	8	18258	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled result: Random effect: 0.56(0.54–0.59). I²=83%	VERY LOW
HEMORRHAGES	3	4467	Very serious risk of bias^a	Serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random effects 0.56 (0.52–0.60); I2=64%	VERY LOW
HEMORRHAGES subgrouped by OAC - VKA	2	3450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: fixed effect 0.54(0.51–0.56); I2=0%	LOW
HEMORRHAGES subgrouped by OAC – Mixed VKA/D OAC	1	1157	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.55–0.68)	LOW
HEMORRHAGES subgrouped by antiplatelets - <33%	2	3450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: fixed effects 0.54(0.51–0.56); I2=0%	LOW
HEMORRHAGES subgrouped by antiplatelets - >33%	1	1157	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.55–0.68)	LOW
ATRIA	4	6760	Very serious risk of bias^a	Serious risk of inconsistency^b	No serious indirectness	Serious imprecision	Pooled effect: Random Effects 0.52 (0.49–0.56); I²=63%	VERY LOW
ATRIA subgrouped by OAC - VKA	3	5743	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	Pooled effect: Fixed effects 0.51(0.49–0.53); I²=0%	VERY LOW
ATRIA subgrouped by OAC – Mixed VKA/DOACs	1	1017	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.54–0.67)	LOW
ATRIA subgrouped by antiplatelets – <33%	3	5743	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	Pooled effect: Fixed effects 0.51(0.49–0.53); I²=0%	VERY LOW
ATRIA subgrouped by antiplatelets – >33%	1	1017	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.54–0.67)	LOW
ORBIT	3	5593	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random Effects 0.57(0.52–0.61); I²=73%	VERY LOW
ORBIT subgrouped by antiplatelets - <33%	1	2293	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	0.52(0.48–0.56)	VERY LOW
ORBIT subgrouped by antiplatelets - >33%	1	1017	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.54–0.68)	LOW
ORBIT subgrouped by antiplatelets – not reported	1	2283	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.58(0.55–0.61)	LOW
CHADS2	1	2293	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	0.51(0.47–0.55)	VERY LOW
CHADSVASC	1	2293	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	0.53(0.49–0.57)	VERY LOW
GARFIELD	1	3550	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.57(0.55–0.58)	LOW
MBRFS	1	4576	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.53(0.52–0.54)	LOW
mOBRI	1	1017	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.56(0.50–0.62)	LOW
CBRM/Shireman	1	1017	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.58(0.54–0.62)	LOW
Simplified HAS-BLED	1	1089	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.642(0.60–0.68)	LOW
HAS-BLED with point for sustained AF	1	1089	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.61(0.57–0.65)	LOW

: GRADE was conducted with emphasis on C statistics as this was the primary measure discussed in decision making.

: Pooling (meta-analysis) was carried out if there were at least two studies per risk tool with confidence intervals. RevMan was used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for some risk tools because few of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because many studies with the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, an I²of 50–74% was deemed serious inconsistency and an I²of 75% or above was deemed very serious inconsistency. If no pooling were possible, inconsistency was assessed by inspection of the degree of overlap of confidence intervals between studies: if one of more Cis did not overlap then a rating of serious inconsistency was given. Reasons for heterogeneity between studies may include geographical/cultural/ethnic differences. Clinically the studies appeared reasonably homogeneous, with similar rates of hypertension, diabetes and former stroke.

c): The judgement of precision was based on the spread of confidence interval around two clinical thresholds: C statisticsof 0.5 and 0.7. The threshold of 0.5 marked the boundary between no predictive value better than chance and a predictive value better than chance. The threshold of 0.7 marked the boundary above which the committee might consider recommendations. If the 95% Cis crossed one of these thresholds a rating of serious imprecision was given and if they crossed both of these thresholds a rating of very serious imprecision as given.

Table 14Clinical evidence profile: sensitivity and specificityof prediction of clinically relevant bleedingin all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results

Risk tool	No of COHORTS	n	Sensitivity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Specificity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Risk of bias	Inconsistency	Indirectness	Imprecision	Quality
HAS-BLED at threshold ≥1	2	4566	Median^d: 0.913(0.880–0.940)	Median^d: 0.171(0.160–0.190	Sensitivity
					Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
HAS-BLED at threshold ≥2	2	4566	Median^d: 0.496(0.440–0.550)	Median^d: 0.686(0.670–0.710)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
HAS-BLED at threshold ≥3	2	4566	Median^d: 0.110(0.080–0.150)	Median^d: 0.950(0.940–0.960)	Sensitivity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
					Specificity
					Very serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision	VERY LOW
ATRIA at threshold ≥1	1	2268	0.879(0.832–0.917)	0.113(0.099–0.128)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
ATRIA at threshold ≥2	1	2268	0.411(0.349–0.475)	0.583(0.561–0.605)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
Hemmorhages at threshold ≥1	1	2268	0.742(0.683–0.795)	0.353(0.332–0.374)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
Hemmorhages at threshold ≥2	1	2268	0.266(0.212–0.326)	0.779(0.770–0.788)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
ORBIT at threshold ≥1	1	2283	0.734(0.684–0.779)	0.388(0.367–0.411)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
ORBIT at threshold ≥2	1	2283	0.283(0.236–0.334	0.812(0.793–0.829)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
CHADS2 at threshold ≥1	1	2293	0.972(0.943–0.988)³	0.0230(0.170–0.305)³	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
CHADS2 at threshold ≥2	1	2293	0.637(0.575–0.697)	0.385(0.364–0.406)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
CHADSVASC at threshold ≥2	1	2293	0.936(0.899–0.963)	0.079(0.069–0.093)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	Serious imprecision^c	VERY LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
CHADSVASC at threshold ≥3	1	2293	0.753(0.695–0.805)	0.292(0.273–0.313)	Sensitivity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW
					Specificity
					Very serious risk of bias^a	NA	No serious indirectness	No serious imprecision	LOW

: Pooling (meta-analysis) was carried out if there were at least three studies per risk tool with confidence intervals. RevMan and WinBugs were used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist. Risk of bias was serious for some risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because many studies with the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, inconsistency was assessed by visual inspection of the sensitivity/specificity plots, or data (if 2 studies). The evidence was downgraded by 1 increment if there was no overlap of 95% confidence intervals. For single studies no evaluation was made and ‘not applicable’ was recorded.

c): Imprecision was assessed based on inspection of the confidence region in the meta-analysis or, where meta-analysis has not been conducted, assessed according to the range of confidence intervals in the individual studies. The evidence was downgraded by 1 increment when the confidence interval around the point estimate crossed one of the clinical thresholds (0.90 or 0.60 for sensitivity and 0.5 and 0.1 for specificity), and downgraded by 2 increments when the confidence interval around the point estimate crossed both of the clinical thresholds. The upper clinical threshold marked the point above which recommendations would be possible, and the lower clinical threshold marked the point below which the tool would be regarded as of little clinical use.

d): For unpooled data the median value was given (of data with 95% CIs). If there were an even number of data points in the unpooled data, the data point chosen in the central pair was the one with lower sensitivity, with its paired specificity.

Table 15NRI for clinically relevant bleeding

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
HAS-BLED v HEMORRHAGES	2	3450	Very serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled: Random effects NRI: + 0.030(−0.130to +0.180); I²= 89%	VERY LOW
HAS-BLED v ATRIA	2	3450	Very serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled: Random effects NRI: + 0.040(−0.150to +0.220); I²= 92%	VERY LOW
ATRIA v HEMORRHAGES	2	3450	Very serious risk of bias^a	Very serious inconsistency^b	No serious indirectness	Serious imprecision^c	Pooled: Random effects NRI: + 0.060(−0.060to +0.190); I2 = 81%	VERY LOW
HAS-BLED v CHADS2	1	2293	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.130(0.050to 0.210)	LOW
HAS-BLED v GARFIELD	1	3550	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	−0.033(−0.129 to 0.094)	VERY LOW
HAS-BLED v CHADSVASC	1	2293	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.130(0.050to 0.210)	LOW
HAS-BLED v ORBIT	1	2283	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.156(0.043 to 0.27)	MOD
ATRIA v ATRIA +TTR	1	2293	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.260 (−0.480to −0.040)	LOW
ORBIT v ORBIT + TTR	1	2293	Serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.260 (−0.480to −0.040)	MOD

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Table 16Clinical evidence profile: accuracy of prediction of ICHin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups

Risk tool	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	Area Under Curve Individual study effects [point estimate (95% Cis) ] Pooled effect/range/median	Quality
HAS-BLED	7	110,194	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random effects 0.56(0.53–0.60); I²=83%	VERY LOW
HAS-BLED subgrouped by antiplatelets - <33%	1	40,450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.53(0.51–0.54)	LOW
HAS-BLED subgrouped by antiplatelets - >33%	3	18.113	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: Fixed effects 0.56(0.52–0.60); I2=0%	LOW
HAS-BLED subgrouped by antiplatelets – not reported	3	51631	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: Fixed effects 0.59(0.58–0.61); I2=0%	LOW
HEMORRHAGES	5	107,162	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random effects: 0.58(0.52–0.64); I2=93%	VERY LOW
HEMORRHAGES subgrouped by antiplatelets – <33%	1	40,450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.53(0.51–0.54)	LOW
HEMORRHAGES subgrouped by antiplatelets – >33%	3	18,113	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: Fixed effects 0.59(0.55–0.63); I2=0%	LOW
HEMORRHAGES subgrouped by antiplatelets – not reported	1	48,599	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.62(0.60–0.64)	LOW
ATRIA	4	58,563	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effect: Random effects 0.56(0.50–0.61); I2=75%	VERY LOW
ATRIA subgrouped for antiplatelets - <33%	1	40,450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	0.50(0.49–0.52)	VERY LOW
ATRIA subgrouped for antiplatelets - >33%	3	18.113	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: Fixed effects 0.58(0.54–0.63); I2=0%	LOW
ORBIT	4	58,563	Very serious risk of bias^a	Very serious risk of inconsistency^b	No serious indirectness	No serious imprecision	Pooled effectRandom effects 0.58(0.50–0.67); I2=91%	VERY LOW
ORBIT subgrouped for antiplatelets - <33%	1	40,450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	serious imprecision^c	0.50(0.48–0.51)	VERY LOW
ORBIT subgrouped for antiplatelets - >33%	3	18,113	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	Pooled effect: Fixed effects 0.62(0.58–0.66); I2=0%	LOW
ABCBleeding CrC	1	1120	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	Serious imprecision^c	0.47(0.40–0.53)	VERY LOW
MBR	1	40450	Very serious risk of bias^a	No serious risk of inconsistency	No serious indirectness	No serious imprecision	0.52(0.50–0.53)	LOW

: GRADE was conducted with emphasis on C statisticsas this was the primary measure discussed in decision making.

: Pooling (meta-analysis) was carried out if there were at least two studies per risk tool with confidence intervals. RevMan was used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for some risk tools because few of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because many studies with the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, an I²of 50–74% was deemed serious inconsistency and an I²of 75% or above was deemed very serious inconsistency. If no pooling were possible, inconsistency was assessed by inspection of the degree of overlap of confidence intervals between studies: if one of more Cis did not overlap then a rating of serious inconsistency was given. Reasons for heterogeneity between studies may include geographical/cultural/ethnic differences. Clinically the studies appeared reasonably homogeneous, with similar rates of hypertension, diabetes and former stroke.

c): The judgement of precision was based on the spread of confidence interval around two clinical thresholds: C statistics of 0.5 and 0.7. The threshold of 0.5 marked the boundary between no predictive value better than chance and a predictive value better than chance. The threshold of 0.7 marked the boundary above which the committee might consider recommendations. If the 95% Cis crossed one of these thresholds a rating of serious imprecision was given and if they crossed both of these thresholds a rating of very serious imprecision as given.

Table 17Clinical evidence profile: sensitivity and specificityof prediction of intracranial haemmorhagein all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results

Risk tool	No of COHORTS	Sensitivity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Specificity (threshold denotes the ‘positive’ score – i.e. the score indicating a high risk of bleeding)	Risk of bias	Inconsistency	Indirectness	Imprecision	Quality
HAS-BLEDat threshold ≥3	1	0.538(0.410–0.660)	0.572(0.540–0.600)	Sensitivity
				Serious risk of bias^a	NA	No serious indirectnes	Serious imprecision^c	LOW
				Specificity
				Serious risk of bias^a	NA	No serious indirectnes	No serious imprecision	MOD
ABCCrC at threshold ≥2%	1	0.785(0.670–0.880)	0.186(0.160–0.210)	Sensitivity
				Serious risk of bias^a	NA	No serious indirectnes	No serious imprecision	MOD
				Specificity
				Serious risk of bias^a	NA	No serious indirectnes	No serious imprecision	MOD

: Pooling (meta-analysis) was carried out if there were at least three studies per risk tool with confidence intervals. RevMan and WinBugs were used to carry out the analyses. If pooling was not possible for risk tools with >1 data point then the range and median value of the study point estimates were recorded. If there were only one data point then only the result from the study was recorded.

a): Risk of bias was assessed using the PROBAST checklist. Risk of bias was serious for some risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the rest of the risk tools because many studies with the aforementioned limitations also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Where data were pooled, inconsistency was assessed by visual inspection of the sensitivity/specificity plots, or data (if 2 studies). The evidence was downgraded by 1 increment if there was no overlap of 95% confidence intervals. For single studies no evaluation was made and ‘not applicable’ was recorded.

c): Imprecision was assessed based on inspection of the confidence region in the meta-analysis or, where meta-analysis has not been conducted, assessed according to the range of confidence intervals in the individual studies. The evidence was downgraded by 1 increment when the confidence interval around the point estimate crossed one of the clinical thresholds (0.90 or 0.60 for sensitivity and 0.5 and 0.1 for specificity), and downgraded by 2 increments when the confidence interval around the point estimate crossed both of the clinical thresholds. The upper clinical threshold marked the point above which recommendations would be possible, and the lower clinical threshold marked the point below which the tool would be regarded as of little clinical use.

Table 18NRI for intracranial bleeding

Prediction tool comparison	No of COHORTS	n	Risk of bias	Inconsistency	Indirectness	Imprecision	NRI(95% CI)	Quality
HAS-BLED v HEMORRHAGES	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.030(−0.001 to 0.060)	VERY LOW
HAS-BLED v ATRIA	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.060(0.026 to 0.093)	LOW
HAS-BLED V ORBIT	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	+0.048(0.013 to 0.082)	LOW
HAS-BLED v MBR	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.007(−0.018 to 0.033)	VERY LOW
HAS-BLED v ABCCrC	1	1120	Serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	+0.139(−0.010to 0.290)	LOW
MBR v HEMORRHAGES	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	−0.022(−0.062 to 0.017)	VERY LOW
MBR v ATRIA	1	40,450	Very serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision	−0.052(−0.094 to −0.011)	LOW
MBR v ORBIT	1	40,450	Serious risk of bias^a	No serious inconsistency	No serious indirectness	Serious imprecision^c	−0.040(−0.083 to 0.002)	LOW

a): Risk of bias was assessed using the PROBAST checklist (see Appendix F).Risk of bias was serious for most risk tools because none of the studies reported any blinding of assessors for risk tool data and outcome status, and most did not report loss to follow up, although follow up and number of events were appropriate. Risk of bias was very serious for the Framingham risk tool because the study concerned also had insufficient numbers of events (<100) and/or inappropriately short follow up times (<5 years) to be able to accurately predict risk.

b): Inconsistency was serious if I2 was 50–74% and very serious if 75% of higher

c): Imprecision serious if the 95% CIs crossed zero.

Risk stratification tools for predicting bleeding events in people with atrial fibrillation

Authors

1. Effectiveness of risk stratification tools for predicting bleeding in people with atrial fibrillation

1.1. Review question: What is the most clinically and cost-effective risk stratification tool for predicting bleeding in people with atrial fibrillation?

1.2. Introduction

1.3. PICO table

1.4. Methods and process

1.5. Clinical evidence

1.5.1. Included studies

1.5.2. Excluded studies

1.5.3. Summary of clinical studies included in the evidence review

1.5.4. Quality assessment of clinical studies included in the evidence review

1.6. Economic evidence

1.7. Included studies

1.8. Excluded studies

1.8.1. Unit costs

2. Accuracy of risk stratification tools for predicting bleeding events in people with atrial fibrillation

2.1. Introduction

2.2. Review question: What is the most accurate risk stratification tool for predicting bleedingevents in people with atrial fibrillation?

2.3. Clinical evidence

Summary of included studies

2.3.1. Discriminationfor MAJOR BLEEDING

2.3.2. Calibrationfor MAJOR BLEEDING

2.3.3. Net Reclassification improvementfor MAJOR BLEEDING

2.3.4. Discrimination for CLINICALLY RELEVANT BLEEDING

2.3.5. Calibration for CLINICALLY RELEVANT BLEEDING

2.3.6. Net Reclassification improvement for CLINICALLY RELEVANT BLEEDING

2.3.7. Discrimination for INTRACRANIAL HEMORRHAGE

2.3.8. Calibration for INTRACRANIAL HEMORRHAGE

2.3.9. Net Reclassification improvement for INTRACRANIAL HEMORRHAGE

2.4. Economic evidence

2.4.1. Included studies

2.4.2. Excluded studies

2.4.3. Unit costs

2.5. The committee’s discussion of the evidence

2.5.1. Interpreting the evidence

2.5.1.1. The outcomes that matter most

2.5.1.2. The quality of the evidence

2.5.1.3. Benefits and harms

2.5.1.4. Cost effectiveness and resource use

2.5.2. Other factors the committee took into account

References

Appendices

Appendix A. Review protocols

Appendix B. Literature search strategies

Appendix C. Clinical article selection

Appendix D. Economic article selection

Appendix E. Full GRADE tables(Including individual study data)

Appendix F. Forest plots

F.1. C statistics

Appendix G. Clinical evidence tables

Appendix H. Risk of bias (PROBAST)

Appendix I. Economic evidence tables

Appendix J. Excluded clinical studies

Appendix K. Excluded economic studies

Table 1PICO characteristics of review question

Table 2Bleeding risk tools

Table 3PICO characteristics of review question

Table 4Summary of studies included in the review

Table 5Summary of risk tools and their constituent variables

Table 6Clinical evidence profile: accuracy of prediction of Major Bleedingin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2to <50% in all sub-groups

Table 7Clinical evidence profile: sensitivity and specificityof prediction of Major Bleeding in all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results; for meta-analysed results the 95% credible intervals are given for the pooled effect only

Figure 1<Insert graphic title here>

Table 8NRI for major bleeding – HAS-BLED versus other tools

Table 9NRI for major bleeding – ATRIA versus other tools

Table 10NRI for major bleeding – HEMORRHAGES versus other tools

Table 11NRI for major bleeding – ORBIT versus other tools

Table 12NRI for major bleeding – CHADSVASC versus other tools

Table 13Clinical evidence profile: accuracy of prediction of CRBin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups

Table 14Clinical evidence profile: sensitivity and specificityof prediction of clinically relevant bleedingin all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results

Table 15NRI for clinically relevant bleeding

Table 16Clinical evidence profile: accuracy of prediction of ICHin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I2 to <50% in all sub-groups

Table 17Clinical evidence profile: sensitivity and specificityof prediction of intracranial haemmorhagein all risk tools featured in the studies (see table 3). 95% CIs are given for non-pooled results

Table 18NRI for intracranial bleeding

Table 6Clinical evidence profile: accuracy of prediction of Major Bleedingin all risk tools featured in the studies (see table 3). Outcomes split across subgroups are only shown if sub-grouping was able to reduce I²to <50% in all sub-groups