Evidence review for CVD risk assessment tools: primary prevention

Evidence review for CVD risk assessment tools: primary prevention

Cardiovascular disease: risk assessment and reduction, including lipid modification

Evidence review A

NICE Guideline, No. 238

London: National Institute for Health and Care Excellence (NICE); 2023 May.

ISBN-13: 978-1-4731-5203-8

1. Cardiovascular risk assessment tools in adults without established cardiovascular disease

1.1. Review question

What is the most accurate tool for determining 10-year and lifetime cardiovascular risk in adults without established cardiovascular disease?

1.1.1. Introduction

A number of risk tools, using a combination of modifiable and non-modifiable risk factors, have been developed to assess a person’s risk of experiencing a cardiovascular event. Previous iterations of this guideline have assessed these for accuracy and at present recommend QRISK2 for risk assessment in those who have not experienced a cardiac event (the primary prevention population). There are annual updates of the QRISK tool adding in new clinical variables, and other tools continue to be developed with a view to improve the tools to better predict events and more accurately assess risk in different population subgroups that were either absent in previous tools derivation and validation populations, or in which the prior existing tools performed less well.

Risk tools have been developed to predict both 10 year and lifetime risk of adverse events. The previous guideline recommends that 10-year risk is calculated as there was insufficient evidence to recommend that lifetime risk assessment tools be recommended. Research into lifestyle risk assessment tools has also progressed, with new tools being developed and those existing ones being enhanced by additional clinical variables in their equations. It is also suggested that lifetime risk tools may better facilitate communication of risk to people having their cardiovascular risk assessed.

This evidence review therefore intends to update the previous review with the new evidence that has been published in both risk tools for predicting both 10-year and lifetime cardiovascular risk for primary prevention to determine whether the newer tools are superior to QRISK2.

1.1.2. Summary of the protocol

For full details see the review protocol in Appendix A.

Table 1

PICO characteristics of review question.

1.1.3. Methods and process

This evidence review was developed using the methods and process described in Developing NICE guidelines: the manual. Methods specific to this review question are described in the review protocol in appendix A and the methods document.

Declarations of interest were recorded according to NICE’s conflicts of interest policy.

When a specific risk assessment tool was validated in multiple publications using the same data source and population, only the most recent study or study with the largest applicable sample size was included if the patient registration dates overlap. Therefore, earlier reports/reports of smaller cohorts from the same database were excluded to avoid double counting.

1.2. Risk prediction evidence

Evidence was available for all risk tools included in the protocol. The predictor variables included, and the outcomes predicted in these tools are summarised in Table 2 and Table 3, respectively. Full details of the predictor variables can be found in Appendix D.1.

Table 2

Predictor variables included in CVD risk assessment tools.

Table 3

Outcomes predicted by CVD risk assessment tools.

1.2.1. Included studies

A search for cohort studies assessing the validation of risk assessment tools for cardiovascular disease (CVD) events and mortality was undertaken. Only tools that have UK validation and studies of adults without established CVD were included. Sixteen cohort studies on 11 risk tools, reported in 17 papers, were included in the review.^{1–3, 5–11, 13–16, 19, 21, 23}

Evidence from these studies on the discriminative ability of the tools is summarised in the overview tables (Table 5, Table 6, and Table 7), and the clinical evidence summary (Table 8) below. Evidence on their calibration and on reclassification is summarised in sections 1.2.5 and 1.2.6, respectively.

The results of one study⁶ are not included in the summary, but are available in Appendix D. They are not included in the evidence summary because this is the original derivation study for the ASCVD tool in an American population, and so is included for reference only because UK validation studies are available for this tool.

See also the study selection flow chart in Appendix A, study evidence tables in Appendix D, and forest plots and summary ROC curves in Appendix E.

1.2.2. Excluded studies

One Cochrane review¹² was identified but excluded because none of the included studies used a tool specified in the review protocol.

One study²² from the 2014 update of CG181 was excluded, although it assessed a tool within the protocol for this update of the review, because it was based on a simulated population and did not provide any data of relevance for decision making.

See the excluded studies list in Appendix I.

1.2.3. Summary of studies included in the prognostic evidence

The included study characteristics are summarised in Table 4 below.

Table 4

Summary of studies included in the evidence review.

See Appendix D for full evidence tables. See Appendix J for List of abbreviations used in Table 4.

1.2.4. Summary of prognostic evidence: discrimination

1.2.4.1. Overview of outcome data

Table 5

Summary of results: AUC (95% CI).

Table 6

Summary of results: D statistics.

Table 7

Summary of results: sensitivity and specificity.

1.2.4.2. Clinical evidence profile for C statistic data

Table 8

Clinical evidence profile: Discriminative capacity of selected CVD risk prediction tools.

1.2.5. Summary of prognostic evidence: calibration

No calibration statistics matching the protocol were reported in the included studies, so GRADE assessment was not possible. However, available calibration curves and ratios of predicted to observed are provided below.

1.2.5.1. Calibration curves and predicted:observed events

QRISK2–2011, QRISK2–2010 and QRISK2–2008

Figure 1 shows the calibration plots for the 3 versions of QRISK2 and the NICE version of the Framingham equation. All 3 versions of the QRISK2 prediction models show good calibration in all 10^ths of risk, with the exception of the highest 10^th of risk in both men and women (calibration slope, range 0.92–0.95).

Figure 1

Calibration curves: observed versus predicted 10-year risk of CVD (from Collins 2012).

Reproduced from Predicting the 10 year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2, Gary S Collins, Douglas G Altman, 344:e4181, copyright 2012 with permission from BMJ Publishing Group Ltd.

QRISK2–2012

Figure 2, Figure 3 and Table 9 show the ratio of predicted to observed events for QRISK2–2012 (Tillin 2014²³). This shows under-prediction for all ethnic groups in men, and in European white and South Asian groups in women, as well as large overprediction in African Caribbean women.

QRISK2 showed a closer relationship with observed risk in African Caribbean men, but a marked under-prediction of observed risk in South Asian women.

Table 9

QRISK2-2012 predicted : observed events.

Figure 2

Calibration curves for QRISK2: observed versus predicted 10-year risk of CVD in men.

Figure 3

Calibration curves for QRISK2: observed versus predicted 10-year risk of CVD in women.

Reproduced from Ethnicity and prediction of cardiovascular disease: performance of QRISK2 and Framingham scores in a U.K. tri-ethnic prospective cohort study (SABRE--Southall And Brent REvisited) T Tillin et al, Heart 2014 Jan;100(1):60–7, Open Access article.

QRISK2–2014

Figure 4 shows the calibration plots for QRISK2–2014, comparing the mean predicted risks and observed risks for each score across each 10th of predicted risk. The QRISK2–2014 prediction model shows good calibration in all 10^ths of risk, except for the highest 10^th of risk in both men and women.

Figure 4

Calibration curves: observed versus predicted 10-year risk of CVD (from Hippisley-Cox 2014).

Reproduced from The performance of seven QPrediction risk scores in an independent external sample of patients from general practice: a validation study, Julia Hippisley-Cox, Carol Coupland, Peter Brindle, vol 4, copyright 2014, with permission from BMJ Publishing Group Ltd.

QRISK3–2017

Figure 5 shows the calibration plots for QRISK3–2017, comparing the mean predicted risks and observed risks for each score across each 10th of predicted risk (Hippisley-Cox 2017⁷). In women, the mean 10 year predicted risk was 4.7% and the observed 10 year risk was 5.8% (95% CI: 5.8% to 5.9%). In men, the mean 10 year predicted risk was 6.4% and the observed 10 year risk was 7.5% (95% CI: 7.5% to 7.6%).

QRISK3–2017 shows good calibration in all 10^ths of risk across all age groups, except for those aged 25–39 where mean predicted risks were slightly higher than observed risks.

Figure 5

Calibration curves: observed versus predicted 10-year risk of CVD (from Hippisley-Cox 2017).

Reproduced from Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study, Julia Hippisley-Cox, Carol Coupland, Peter Brindle, BMJ 2017;357:j2099, Open Access article.

QRISK3 external validation

Figure 6 and Figure 7 show the calibration plots for QRISK3 in women and men, respectively (Livingstone 2021¹⁵).

In women, when not considering competing mortality risks, calibration was excellent for the whole cohort, and also excellent for those aged 25–44 years. However, QRISK3 over-predicted CVD risk in older age groups. When competing mortality risks were accounted for (Figure 6), there was over-prediction of risk at higher levels of predicted CVD risk in all women. The same pattern of increasing over-prediction with increasing age was observed, but in greater magnitude, and calibration was poor in older age groups.

In men, when not considering competing mortality risks, calibration was excellent, although with somewhat greater over-prediction at higher levels of predicted CVD risk than in women (Figure 7). Calibration was excellent for men aged 25–44 years, but QRISK3 progressively over-predicted CVD risk with increasing age. When competing mortality risks were accounted for, QRISK3 over-predicted risk at higher levels of predicted CVD risk in all men. Calibration was poor, with large over-prediction in older age groups.

Figure 6

QRISK3 calibration in women.

Reproduced from Effect of competing mortality risks on predictive performance of the QRISK3 cardiovascular risk prediction tool in older people and those with comorbidity: external validation population cohort study, S Livingstone et al, The Lancet VOLUME 2, ISSUE 6, E352-E361, Open Access article.

Figure 7

QRISK3 calibration in men.

CRISK, CRISK-CCI and QRISK3

Figure 8 shows the calibration plots for QRISK3, CRISK and CRISK-CCI from Livingstone 2022¹⁶. Figure 9 shows the calibration stratified by age groups.

In women overall, there was some overprediction with CRISK at higher levels of predicted risk, but CRISK was better calibrated than QRISK3, whilst calibration with CRISK-CCI was excellent. In younger women, there was some underprediction with CRISK and CRISK-CCI that was similar to QRISK3. In older women, CRISK modestly over-predicted CVD risk, particularly at higher levels of predicted risk but was still better calibrated than QRISK3 whilst calibration with CRISK-CCI was excellent.

In men overall, calibration using CRISK-CCI was better than CRISK which showed some underprediction, whilst QRISK3 overpredicted CVD risk. In younger men, there was some underprediction with CRISK and QRISK3, but calibration with CRISK-CCI was excellent. In older men at lower levels of predicted risk, calibration with CRISK and CRISK-CCI was good, whilst there was overprediction with QRISK3. However, all models overpredicted risk at higher levels of predicted risk.

Figure 8

Calibration plot of QRISK3, CRISK and CRISK-CCI.

Reproduced from Predictive performance of a competing risk cardiovascular prediction tool CRISK compared to QRISK3 in older people and those with comorbidity: population cohort study, S Livingstone et al, BMC Medicine volume 20, Article number: 152 (2022), unadapted, Open Access article.

Figure 9

Calibration plots of QRISK3, CRISK and CRISK-CCI stratified by age.

PRIMROSE-lipid and -BMI tools

Figure 10 shows the calibration plots from the PRIMROSE tools (Osborn 2015²¹). In men, the PRIMROSE models showed over-prediction in those with 7.5–20% predicted risk and underprediction of risk in the highest risk group. In women, the PRIMROSE models were well calibrated, except for some underprediction of risk in the highest risk group for PRIMROSE-BMI.

Among those estimated to be at high-risk (risk score >20%), the following proportions were observed to have developed CVD:

PRIMROSE BMI 531/2989 (17.8%)
PRIMROSE lipid 570/2991 (19.1%)

Among those estimated to be at low risk (risk score <20%) the following proportions were observed to have developed CVD:

PRIMROSE BMI 641/17 418 (3.7%)
PRIMROSE lipid 602/17 416 (3.5%)

Figure 10

Calibration plots for PRIMROSE tools.

This figure reproduced from Cardiovascular risk prediction models for people with severe mental illness: results from the prediction and management of cardiovascular risk in people with severe mental illnesses (PRIMROSE) research program, DPJ Osborn et al, JAMA Psychiatry 2015 Feb;72(2):143–51, has been redacted pending copyright approval from JAMA Psychiatry.

SCORE2

Figure 11 shows the ratio of predicted to observed events for SCORE2². This shows over-prediction in younger age groups and under-prediction in older age groups, particularly in men.

Figure 11

Calibration of SCORE2 in CPRD data by age groups (SCORE2 working group 2021).

This figure reproduced from SCORE2 risk prediction algorithms: new models to estimate 10-year risk of cardiovascular disease in Europe, SCORE2 working group and ESC Cardiovascular risk collaboration, European Heart Journal, Volume 42, Issue 25, 1 July 2021, Pages 2439–2454, adapted (cropped to show only SCORE2 fatal + non-fatal risk), has been redacted pending copyright approval from Oxford University Press.

SCORE2-OP

Figure 12 shows the ratio of predicted to observed events for SCORE2-OP¹. This shows good calibration, with a slight underprediction at 10–20% predicted risk and a slight overprediction at >20% predicted risk.

Figure 12

Calibration plot of observed versus estimated (O/E) risk within deciles of the CPRD cohort.

This figure reproduced from SCORE2-OP risk prediction algorithms: estimating incident cardiovascular event risk in older persons in four geographical risk regions, SCORE2-OP working group and ESC Cardiovascular risk collaboration, European Heart Journal, Volume 42, Issue 25, 1 July 2021, Pages 2455–2467, has been redacted pending copyright approval from Oxford University Press.

QRISK lifetime

Table 10 shows the ratio of predicted to observed events for QRISK lifetime (Hippisley-Cox 2010⁹). This shows minor under-prediction in those at low predicted risk but good calibration in the highest 10th of risk.

Table 10

Predicted and observed lifetime risk of cardiovascular disease by 10th of predicted lifetime risk in the validation cohort of 1,267,159 patients.

Reproduced from Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database, Julia Hippisley-Cox, Carol Coupland, John Robson, Peter Brindle, BMJ 2010 Dec 9;341:c6624, with permission from BMJ Publishing Group Ltd.

LIFE-CVD

Figure 13 shows the calibration plot for the LIFE-CVD model (Jaspers 2020 ¹¹). This shows some over prediction at lower risk and under prediction at higher predicted risk levels.

Figure 13

External calibration of predicted vs. observed 10-year risk using the LIFE-CVD model.

This figure reproduced from Prediction of individualized lifetime benefit from cholesterol lowering, blood pressure lowering, antithrombotic therapy, and smoking cessation in apparently healthy people, NEM Jaspers et al, European Heart Journal 2020 Mar 14;41(11):1190–1199, has been redacted pending copyright approval from Oxford University Press.

1.2.6. Summary of prognostic evidence: reclassification

No reclassification statistics were reported in the included studies. Therefore, a narrative summary of the available information is provided below where both the proportion reclassified and the observed risk in this subset of patients are reported.

1.2.6.1. QRISK3 vs QRISK2 (Hippisley-Cox 2017 ⁷)

There were 458,263 (17.2%) people classified as high risk (risk ≥10% over 10 years) with QRISK2–2017; 458 869 (17.2%) using QRISK3 without SBP variance, and 458 868 (17.2%) using QRISK3 with SBP variance.

Of the 458,263 people classified as high risk on QRISK2–2017, 10,948 (2.4%) would be reclassified as low risk using QRISK3 without SBP variance. The 10-year observed risk among these reclassified patients was 10.3% (95% CI: 9.6% to 11.1%). Conversely, of the 2,213,035 classified as low risk using QRISK2–2017, 11,554 (0.5%) would be reclassified as high risk using QRISK3 without SBP variance. The 10-year observed risk among these reclassified patients was 12.2% (95% CI: 11.4% to 13.1%).

Of the 458,869 patients with a 10-year predicted risk score of 10% or more using QRISK3 without SBP variance, 9,102 (2.0%) would be reclassified as low risk using QRISK3 with SBP variance. The 10-year observed risk among these reclassified individuals was 9.6% (95% CI: 8.9% to 10.5%). Conversely, of the 2,213,429 with a 10-year predicted risk score of less than 10% using QRISK3 without SBP variance, 9,101 (2.4%) would be reclassified as high risk using QRISK3 with SBP variance. The 10-year observed risk among these reclassified patients was 10.7% (95% CI: 9.9% to 11.6%).

1.2.7. Economic evidence

1.2.7.1. Included studies

One health economic study with a relevant comparison was included in this review.²⁴ This is summarised in the health economic evidence profile below (Table 11) and the health economic evidence table in Appendix G.

1.2.7.2. Excluded studies

No relevant health economic studies were excluded due to assessment of limited applicability or methodological limitations.

See also the health economic study selection flow chart in Appendix F.

1.2.8. Summary of included economic evidence

Table 11

Health economic evidence profile: risk assessment tools.

1.2.9. Economic model

This area was not prioritised for new cost-effectiveness analysis.

1.2.10. Evidence statements

1.2.10.1. Economic

One cost-utility analysis found that risk assessment using an SMI-specific BMI algorithm (PRIMROSE) was the dominant strategy (lowest cost and highest QALYs) in people with serious mental illness compared to an SMI-specific lipid algorithm (PRIMROSE) and a UK-adjusted Framingham general population BMI or lipid algorithm. This analysis was assessed as partially applicable with potentially serious limitations.

1.2.11. The committee's discussion and interpretation of the evidence

1.2.11.1. The outcomes that matter most

The committee agreed that the clinical outcomes that the tools of relevance to this review should predict were CVD events, in particular cardiovascular mortality, non-fatal MI and stroke. The accuracy of prediction tools to estimate the risk of CVD events at 10-year or lifetime thresholds was measured using the following metrics:

Discrimination

Area under the ROC curve (c-index, c-statistic).
Classification measures at 5%, 7.5%, 10%, 15% and 20% predicted risk thresholds: sensitivity, and specificity.
D statistic

Calibration

Calibration plots
Predicted risk versus observed risk
Statistical tests for agreement between predicted and observed events (e.g., Hosmer-Lemeshow or Nam–D'Agostino statistics)

Reclassification / revalidation

net classification improvement
integrated discrimination index

The committee agreed that a good risk tool should accurately predict the true CVD risk (either 10-year or lifetime risk), that is it needs to be well calibrated; over- or under- prediction would lead to over- or under- treatment, which could result in harm. Discrimination is important to correctly classify individuals into risk groups to inform decisions on pharmacological treatment. Clinically relevant re-classification decisions are also important to compare the utility of the tools.

The committee noted that very little evidence was available for the sensitivity and specificity of the tools at specific thresholds and that no reclassification statistics were reported.

1.2.11.2. The quality of the evidence

The quality of the evidence ranged from low to high, with the majority being of moderate quality. Downgrading of the evidence was mainly due to risk of bias; some tools having internal validation only, cohorts having less than 100 events and studies not reporting calibration data. For some tools with both internal and external validation there was inconsistency in the findings between the cohorts. It was noted that some of the studies included softer end points that may be more difficult to define in their models (for example TIA or angina), or outcomes subject to practice variation (e.g., revascularisation). However, it was agreed that this should not be considered as a reason for downgrading the quality of the evidence as the model development criteria in these studies appeared sufficiently robust to predict the primary outcomes of interest.

Data sources

The committee discussed the differences in the validation databases used in the different UK studies and whether they could be considered as distinct cohorts. It was noted that these are all UK primary care data, but that they are drawn from distinct sets of GP practices, and so can be considered different cohorts while still being representative of the UK primary care population.

1.2.11.3. Benefits and harms

Discrimination

QRISK2 and QRISK3 showed similar ability to classify individuals into risk groups based on AUC data. CRISK and CRISK-CCI tools also showed similar discrimination but did not have any external validation data. These tools all have a higher discriminative capacity in women than in men. Other tools included in the review were inferior in terms of this assessment metric.

Calibration

QRISK2 and QRISK3 were demonstrated to be generally well calibrated but showed some overprediction in the highest risk groups. Calibration of QRISK3 was also less accurate when accounting for competing mortality risk. CRISK, and especially CRISK-CCI, were better calibrated than QRISK3, but all 3 models overpredicted risk at higher levels of predicted risk in those aged 75–84 years. The committee noted that although overprediction could result in unnecessary treatment and anxiety, underprediction would have worse consequences in this context as the tools are used to identify those people who will be offered statins. This means people who would benefit from statins due to their high risk of CVD events may not be identified.

The QRISK2 and 3 and CRISK and CRISK-CCI tools were agreed to be good tools for ranking people from likely highest to lowest risk based on their calibration performance. This was agreed to be useful from a population health perspective because a tool is needed to help triage people. However, it was also noted that most CVD events occur among people who are not perceived to be at high risk because this is often the largest group; therefore, the greatest impact on population health will be based on what threshold is chosen to define those at high risk and, in turn, who should be offered statins. It was noted that whatever threshold is chosen it is likely that events will still be missed.

Given the poorer discriminative ability of other tools considered in the review, their calibration data was considered of less value for decision making and did not inform the committee discussions. However, it was noted that PRIMROSE and LIFE-CVD showed under-prediction in the highest risk groups, which significantly limits their utility. SCORE2 also showed under-prediction in older age groups and SCORE2-OP showed slight under-prediction at 10–20% predicted risk.

The evidence demonstrated that QRISK3, CRISK and CRISK-CCI over-predict in people aged over 75 and that their discriminative ability reduces with increasing age, even when accounting for competing mortality risk. However, the committee did not consider this a problem in terms of the use of the tool for determining when to offer treatment in clinical practice, as people aged over 75 would already have a greater than 10% risk. The need to assess risk was therefore agreed as important both to inform treatment threshold, but also to inform discussions with a person about risk. Calibration and discrimination data for different age subgroups were not available for QRISK2.

Reclassification

Limited data were available for reclassification, and no reclassification statistics were reported. However, data showed that QRISK3 correctly reclassified to high risk 0.5% of those who were low risk on QRISK2. The committee discussed that this reflects the benefit of QRISK3 for correctly assessing risk in people for whom the clinical variables added since QRISK2 apply. There was also evidence that a higher proportion of those with observed CVD events were correctly classified as high risk with QRISK3 than with CRISK-CCI.

Lifetime risk tools

The committee queried the value of studies assessing lifetime risk tools over a 10-year period. It was agreed this evidence was very limited in terms of how it could be used to inform accuracy of the tool over a lifetime. However, the committee discussed the potential utility of lifetime risk estimates in younger people, who may not cross the threshold for being considered high risk based on 10-year estimates. In this group, the use of lifetime risk estimates could help inform discussions about CVD risk and the importance of lifestyle modification at an earlier age. They highlighted that these tools may underestimate the effects of treatment however, as they assume the cholesterol levels entered are the value someone has always had, rather than using RCT data to estimate the impact medicines may have on reducing cholesterol. They agreed that while they should not be used for that purpose, this was not needed explicitly in the recommendation as this was worded so as not to imply this was where they could help conversations and that did not override their benefit in aiding discussions about risk. They therefore agreed to include a recommendation in the guideline within the section on communication about risk, giving the example of QRISK-lifetime of one such tool that could be used. Although this tool performed best from the limited evidence of lifetime risk tools, it was agreed the recommendation should not be restricted to that tool as newer evidence may emerge and so this was just provided as an example.

Summary

The committee agreed that all of the tools have limitations. They tend to be quite well calibrated, but less accurate in terms of discrimination therefore none are very good screening tests for predicting those who will and will not get disease, but they can be useful in splitting into low, medium and high risk, or ordering likelihood of events occurring. The committee discussed that one use of risk assessment tools for CVD is to help decide on suitability for treatment (See evidence review C for further discussion on this topic). The committee agreed that using an appropriate risk assessment tool should not replace clinical judgement and that risk score interpretation should be individualised.

The committee agreed that overall the evidence suggests that QRISK3 performs better than QRISK2, although the difference in performance was marginal. The evidence that QRISK3 appropriately reclassified 0.5% of those low risk on QRISK2 to the high-risk category was agreed to be important and reflects the added accuracy of this version of the tool for classifying people with conditions not included in the QRISK2 algorithm, such as severe mental illness and systemic lupus erythematosus. The committee raised concerns that QRISK3 would take longer to complete in practice as QRISK2 is embedded in clinical systems and so pulls the necessary data from medical records. Any additional time taken to complete such a tool would lead to a risk that it wouldn’t be fully completed, particularly when considering the current context is people working in very busy clinics when healthcare professionals are already very limited by time. The committee were aware that QRISK3 had been incorporated into the NHS health check and that discussions were ongoing at the time of development of the guideline regarding the continuation of inclusion within clinical systems. The committee however agreed the best risk assessment tool should still be recommended within the guideline as the implementation in systems would apply to all tools. It was agreed that the best tool should be recommended, but recognised that it may be necessary to use QRISK2 until QRISK3 is available in clinical systems. However, this should not be the case for people who use corticosteroids or atypical antipsychotics or have a diagnosis of systemic lupus erythematosus, migraine, severe mental illness, or erectile dysfunction because QRISK2 may underestimate their 10-year CVD risk because, unlike QRISK3, it does not include these variables. In these cases, where QRISK2 is still the version within the care providers electronic system, the web version of QRISK3 should be used. It was acknowledged that QRISK3 is now the standard version of this tool and that the annual remodelling of the algorithm to the latest version of the QResearch database will be applied to QRISK3. Therefore, earlier versions of QRISK, including QRISK2, may not be subject to this annual remodelling and their performance may decay. This was agreed to be another reason in support of recommending QRISK3.

It was agreed that an important aspect of the use of any tool is the conversation that is had about risk between the healthcare professional and the person, and how risk is communicated.

Subgroups

Overall, it was agreed that QRISK3 appears to perform reasonably well in terms of discrimination for subgroups with comorbidities including people with CKD, type 1 diabetes and severe mental illnesses, although not so well for people with type 2 diabetes. However, it was further noted that all of this evidence was from internal validation studies only and performance was not as good as it was in the whole population cohort. Furthermore, the models considered perform relatively poorly in terms of discrimination for people with type 2 diabetes. The committee noted that this could be due to some variables associated with type 2 diabetes that would affect CVD risk not being captured in the risk tool, including the length of time someone has had diabetes, their blood sugar control and the therapies that they receive, some of which reduce CVD risk. Additionally, as the CVD event rate is already high in this population risk discrimination is more difficult.

No calibration data were available for any of these subgroups and the AUC statistic was lower than that for the overall cohort in all subgroups.

The previous update of this guideline also considered evidence for UKPDS (a type 2 diabetes specific risk calculator). They noted that the UKPDS is based on a historical cohort and had not been updated. At that time, the former committee noted that QRISK2 included diabetes as a risk factor and the development cohort included more than 40,000 people with prevalent type 2 diabetes compared to 4540 newly diagnosed type 2 diabetes patients in the UKPDS derivation cohort and the accuracy results overall were better than UKPDS (although there was no direct head-to-head comparison). They discussed that there was some suggestion that people with diabetes were of equivalent risk to a secondary prevention population, but on balance the committee consensus was that although incidence of CVD events was increased in people with type 2 diabetes, it was not quite as high as a secondary prevention and so use of a risk tool was still of value. They therefore agreed it was appropriate to recommend QRISK3 for people with type 2 diabetes despite the fact that there had not been external validation of QRISK3 in a type 2 diabetes population. The committee’s opinion was that it is still difficult to persuade some people to try statin treatment, even when they know they have diabetes, and so continued use of a risk tool could help the communication of risk and improve uptake of statins, even knowing it performs less well in this group. The committee agreed that was an important factor and that a risk tool should continue to be recommended for people with type 2 diabetes, although raised that communication of risk may be better informed, in their opinion, by lifetime risk tools. In line with the previous update of this guidance, QRISK is still the best tool for this population and QRISK3 should replace QRISK2 as it is the current version of this tool.

The committee discussed whether it was appropriate to recommend the use of a risk tool for people with either chronic kidney disease (CKD) or type 1 diabetes, in whom risk tools have not previously been recommended. Although type 1 diabetes was included within QRISK3 and there was internal validation data available, the committee noted that people with type 1 diabetes are at very high risk of CVD events. As discussed in the previous version of this guideline, features of the metabolic syndrome are highly relevant to the occurrence of CVD events in type 1 diabetes and these risk factors will be recognised by specialists in diabetes who will treat people accordingly. Like QRISK2, QRISK3 only includes a tick box for type 1 diabetes, which does not include factors considered clinically important such as length of time the person has had diabetes or urine albumin. As evidence in this population is still limited the committee agreed that a recommendation not to use a risk tool in this group should be retained.

They acknowledged that QRISK3 has expanded the definition of CKD to include stage 3, and that there is now internal validation data which shows reasonable discriminative power for both population subgroups, although no calibration data were available. However, the committee agreed people with CKD are often at high CVD risk, including those with stage 1 or 2 CKD which is not captured in QRISK3 and in whom risk can actually be higher than in many people with stage 3 without albuminuria. Therefore, they considered that QRISK3 is likely to significantly underestimate CVD risk, especially those with CKD stage 1 or 2. They also noted that the AUC for this group was lower than the general population sample and was only available from an internal validation cohort. Therefore, the committee agreed that a recommendation not to use a risk tool in people with CKD should be retained. They noted that people with albuminuria (A2 or A3) or with eGFR <60 ml/min/1.73m² with or without albuminuria should be considered at greater risk of CVD and CVD risk modification should be considered within this group.

As QRISK3 includes consideration of more population subgroups than QRISK2, the committee agreed that these factors could be removed from the 2014 recommendation highlighting where risk tools may underestimate 10-year risk. They acknowledged that the evidence for the performance of the tool in these subgroups had not been validated in separate groups of people to those analysed for its development, nor was calibration data available for these subgroups. However, they agreed that the tool should still be recommended in these groups as the risk tool is used to determine a threshold for treatment and therefore use of QRISK3 for someone with in these subgroups could impact treatment decisions. Based on their clinical experience, the committee agreed that it remained important to highlight that risk tools may still underestimate CVD risk in certain groups of people that are not adequately reflected in the tool. These included autoimmune disorders and other systemic inflammatory disorders as although systemic lupus erythematosus and rheumatoid arthritis are included in QRISK3, this does not adequately reflect the other related conditions that are associated with an increased risk of CVD and that this should still be noted. Furthermore, it was noted that the definition of severe mental illness used in the cohort to derive and validate QRISK3 differed from that in many electronic record systems. The cohort included a large proportion with moderate to severe depression, who are not consistently defined as having severe mental illness. The committee were aware that people with severe mental illness defined as schizophrenia, bipolar disorder and other psychoses are known to be at higher risk of CVD than people with moderate to severe depression. While risk may be increased in this group compared to the general population, the likely impact of including a large proportion in the cohort is that risk may be still slightly underestimated in people with severe mental illness. The committee also noted there was the potential for risk to be overestimated in people with moderate to severe depression. However, they noted this was not evidenced and as recommendations reinforce the importance of shared decision making in CVD risk management, the impact of minimal risk of overestimation was low. The committee agreed that the QRISK3 tool did provide the best estimate of risk for people with severe mental illness, but noted it was important to retain the recommendation that risk tools may underestimate risk in people with severe mental illness. The committee agreed that clinical judgement should inform interpretation of the risk score, based on the individual’s circumstances.

1.2.11.4. Cost effectiveness and resource use

One cost-effectiveness analysis was included that compared severe mental illness (SMI)-specific risk assessment using the PRIMROSE algorithm to a general population risk assessment tool in a population with SMI and without established CVD. This analysis found risk assessment using the PRIMROSE BMI algorithm was the most cost-effective option however the general population comparator was based on a UK adapted Framingham equation that was excluded from the guideline update clinical review protocol as QRISK2 was concluded as better for risk assessment in the 2014 CG181 update. In addition, QRISK3 includes fields related to SMI and so should reflect risk in people with SMI better than the general population algorithm used in this analysis. This limited the conclusions that could be drawn from this analysis. It was also noted that the PRIMROSE risk tool had not been externally validated and the clinical review did not provide evidence that this tool would perform better than QRISK3 (although no direct comparison was available).

No other cost-effectiveness analyses were identified. The committee discussed whether the different risk tools would require different resource use and so have different costs to use. The tools included in the clinical review were considered to require similar information. It was noted that QRISK3 has additional fields to complete over QRISK2 (which is currently recommended): whether the individual has a diagnosis of migraine, systemic lupus erythematosus, severe mental illness or erectile dysfunction, whether they have a prescription for corticosteroids or atypical antipsychotics, and a measure of systolic blood pressure variability. It was noted that this information can mostly be elicited quickly by asking the patient or from patient records and it was not considered likely to require additional or longer appointment times if QRISK3 was integrated into clinical systems in the same way as QRISK2 currently is, however the committee noted this is currently under discussion by the relevant parties. The committee noted that the measure of blood pressure variability may not be completed unless it was calculated within IT systems automatically but much of the clinical validation data was for QRISK3 without this field completed and the tool would still calculate risk if this was omitted.

QRISK3 is available as a web tool but the committee highlighted that QRISK2 has to-date usually been integrated into clinical IT systems and that using QRISK3 would be more time consuming to complete if it was not similarly integrated. It was noted that in August 2021 Public Health England issued guidance about using QRISK3 in NHS health checks (responsibility for the NHS Health Check programme has now transferred to the Office for Health Improvement and Disparities). This guidance includes information about integration of QRISK3 and noted that at the time of publication QRISK3 was already incorporated into one system. Also, since QRISK3 is now the standard version of QRISK provided in ClinRisk Ltd software development kits, as software updates are deployed it will become the current version by default over time. The committee were aware that there was some uncertainty about future provision of risk tools in clinical systems but that a statement had been made by EMIS in Pulse Today stating that they are working to offer the QIRSK2 calculator beyond April. Although no information was available about QRISK3, the committee agreed that if integrated into systems, use of QRISK3 was not considered likely to require additional resources over QRISK2.

Assuming risk tools continue to be integrated in clinical systems and that significant differences in resource use are not expected related to carrying out risk assessment, whatever risk tool is used, the cost effectiveness of using a risk assessment tool will therefore be related to its effectiveness in correctly predicting risk. The committee discussed what influence risk assessment will have on the treatment and outcomes in the rest of the treatment pathway. It was noted that risk assessment is currently used to determine who starts statin treatment. It is also used when considering starting other treatments including blood pressure lowering medication for people with stage 1 hypertension and type 2 diabetes treatment. The committee also highlighted that if individuals have a better understanding of their CVD risk and its implications this could also improve their willingness to start treatment, adhere to treatment and make lifestyle modifications.

Theoretically, the consequence of inaccurate risk assessment could be that a group of people incorrectly calculated as being above the selected risk threshold are prescribed medication but do not get sufficient benefit to justify their use; and/or a group of people incorrectly calculated as being below the selected risk threshold are not prescribed medication and health benefits and cost savings of avoiding future health events are missed. It was noted that statins were shown to be cost-effective even at low risk levels. Therefore, overestimation of risk by a tool will be less of an issue than underestimation or misclassification from a statins cost-effectiveness perspective. Overestimation will lead to more people being treated and lower absolute benefit in the additional people treated but is likely to still be cost effective. For other treatments not looked at in this guideline however this may not always be the case.

The clinical review found that although QRISK3 performed better than QRISK2, the tools’ performance did not vary substantially overall and so changing to QRISK3 may not have a large impact to costs or outcomes on a population level. However, it was noted that for people in the specific population groups that have been added to QRISK3 it will increase their risk estimate and so may change their risk category which could affect the treatments they are offered and therefore the health benefits they receive.

The committee discussed that calculating lifetime risk is likely to require healthcare professionals to enter data into an online calculator as it is not currently incorporated into clinical IT systems. This would take some additional time however it is not clear whether this would result in longer consultations or not. In addition, it would not be done for everyone. If integrated into clinical systems time impact would be minimal. Lifetime risk calculation is likely to be useful in younger people who do not meet conventional criteria for being high risk but who do have risk factors for cardiovascular disease that could confer a high lifetime risk. Lifetime risk estimates could also be useful in some people for whom additional information about cardiovascular risk is deemed helpful to fully inform the patient and encourage them to make lifestyle changes or start or adhere to risk reducing treatments. Given this, any additional time costs were considered likely to improve management of cardiovascular risk and so reduce clinical events.

1.2.11.5. Other factors the committee took into account

It was noted that QRISK3 is only validated for use in people aged 25–84 inclusive. The committee therefore agreed it was important to retain the 2014 recommendation highlighting that people aged 85 years or older should be considered at high risk due to age alone. There are no risk tools validated in people aged under 25, and as the majority of people of this age group would not be high risk, the committee agreed no separate recommendation was required.

The committee were aware that hormone therapies used for gender reassignment may impact a person’s risk of CVD. They were aware however that the NHS Health Check best practice guidance states that gender should be recorded as reported by the individual. If the individual discloses gender reassignment, they should be provided with CVD risk calculations based on both genders and advised to discuss with their GP which calculation is most appropriate for them as an individual. They agreed that healthcare professionals should follow this guidance when undertaking formal risk assessments.

The committee discussed other equalities issues that were highlighted when starting development of the update. They noted that the factors included in QRISK3 do address consideration of many relevant factors, for example severe mental illness (as mentioned above), ethnicity and socio-economic status. They also agreed that when full formal CVD risk assessments were first introduced some factors were not consistently recorded in people’s medical records, however this was no longer a particular issue, and so they agreed recommendations to highlight these as risk factors for CVD, or areas in which risk might be underestimated, were no longer required in the guideline.

The committee discussed how sudden death was captured in the databases used. Some committee members raised that in the past where sudden death was listed as the cause of death on a death certificate, it was listed as MI in medical records, leading to an innate bias. The committee were unsure if this was still true. They queried whether the databases used in the development of these models included sudden death in cardiovascular mortality. It was noted that the committee’s knowledge of these databases was that if the sudden death was 30 days within an MI, then this was listed within cardiovascular mortality (due to MI). The committee considered this was appropriate.

It was noted when a cut off for a tool is selected (for example, using a 10% risk on QRISK2) it corresponds to a particular point on the area under the curve, and therefore a particular sensitivity and specificity. The committee discussed that it would be useful to know the detection rate at the threshold that was being considered as that in which statin treatment should be offered to a person. This data was not reported in the included papers, but it was possible to calculate this for QRISK3 from an external validation cohort. The committee noted that the sensitivity improved as the high risk threshold was lowered from 10% to 7.5%, but at the expense of an increased false positive rate. They noted it was important to be aware of the trade-off between these metrics when considering whether it was appropriate to lower the threshold for treatment.

The committee discussed whether cardiovascular risk assessment was needed at all and whether risk assessment could be stopped if all people over a certain age were offered statins given that they were found to be cost effective for most people between 40 and 80 years of age and they considered that age was the largest single determinant of risk. However, age alone had not been considered as part of the review, and although it may be possible to determine at what age everyone was over a defined high-risk threshold for a particular tool, there were concerns that this would be detrimental to a person’s understanding of their individualised CVD risk and the importance of risk factor modification. It was noted that statins are not the only primary prevention treatment where initiation is influenced by CV-risk. In addition, it was agreed that it was important to be able to assess level of risk to aid discussions about lifestyle changes and treatment initiation because people at higher risk were likely to be more motivated to make changes or start treatment and would also receive the largest benefit of doing so. The committee were aware of reports indicating that the uptake of statins in those at greater than 10% risk is currently less than 50%. They raised concerns that without a risk assessment or good communication about risk in absolute terms on an individual level, this could be even lower. Furthermore, it was noted that there could be an equalities consideration regarding engagement with lipid-lowering strategies. In the committee’s experience, people with lower levels of education and from lower socio-economic groups may be less likely to take statins, even when they are at high risk. Not informing people of their risk score as a motivator of change, would likely negatively impact this as they may be even less likely to engage in lifestyle modification or consider treatment if they are unaware of their risk. This was not evidenced by recent audit data that the committee were aware of, but the committee agreed it was nevertheless important to be aware of with a view to not negatively impacting this. A further equalities consideration was the ability to reach people who are not registered with a GP, who are likely to also overlap with the above group. The committee agreed this is a particular challenge in reducing health inequalities, as NICE guidelines apply where NHS care is commissioned or delivered, they agreed that this should equally be considered by outreach services that may also include people not registered with GPs in order to try to help all people have a better understanding of CVD risk. Therefore, they agreed it is beneficial to recommend that a risk assessment tool is used to inform a threshold for treatment to enable effective communication of risk and avoid reinforcing health inequalities.

Overall, it was agreed, risk assessment as a starting point for risk management is beneficial irrespective of the treatment initiation threshold for statins (the treatment initiation threshold is discussed in the statins evidence report C).

The committee also noted that healthcare professionals may be familiar with the JBS3 tool for assessing lifetime risk. They discussed that this tool was based using the QRISK-Lifetime algorithm and therefore it was not included separately within the review. QRISK-Lifetime was provided as an example of a lifetime risk calculator in the new recommendation for communicating risk, but the recommendation was not restricted to QRISK-Lifetime.

1.2.12. Recommendations supported by this evidence review

This evidence review supports recommendations 1.1.7 to 1.1.11 and 1.1.16.

1.2.13. References

1.: Anonymous. SCORE2-OP risk prediction algorithms: Estimating incident cardiovascular event risk in older persons in four geographical risk regions. European Heart Journal. 2021; 42(25):2455–2467 [PMC free article: PMC8248997] [PubMed: 34120185]

2.: Anonymous. SCORE2 risk prediction algorithms: New models to estimate 10-year risk of cardiovascular disease in Europe. European Heart Journal. 2021; 42(25):2439–2454 [PMC free article: PMC8248998] [PubMed: 34120177]

3.: Collins GS, Altman DG. Predicting the 10 year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2. BMJ. 2012; 344:e4181 [PMC free article: PMC3380799] [PubMed: 22723603]

4.: D'Agostino RB Sr., Vasan RS, Pencina MJ, Wolf PA, Cobain M, Massaro JM et al General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation. 2008; 117(6):743–753 [PubMed: 18212285]

5.: Dziopa K, Asselbergs FW, Gratton J, Chaturvedi N, Schmidt AF. Cardiovascular risk prediction in type 2 diabetes: a comparison of 22 risk scores in primary care settings. Diabetologia. 2022; 65(4):644–656 [PMC free article: PMC8894164] [PubMed: 35032176]

6.: Goff DC, Lloyd-Jones DM, Bennett G, Coady S, D?Agostino RB, Gibbons R et al 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk. Circulation. 2014; 129(25suppl2):49–s73 [PubMed: 24222018]

7.: Hippisley-Cox J, Coupland C, Brindle P. Development and validation of QRISK3 risk prediction algorithms to estimate future risk of cardiovascular disease: prospective cohort study. BMJ. 2017; 357:j2099 [PMC free article: PMC5441081] [PubMed: 28536104]

8.: Hippisley-Cox J, Coupland C, Brindle P. The performance of seven QPrediction risk scores in an independent external sample of patients from general practice: a validation study. BMJ Open. 2014; 4(8):e005809 [PMC free article: PMC4156807] [PubMed: 25168040]

9.: Hippisley-Cox J, Coupland C, Robson J, Brindle P. Derivation, validation, and evaluation of a new QRISK model to estimate lifetime risk of cardiovascular disease: cohort study using QResearch database. BMJ. 2010; 341:c6624 [PMC free article: PMC2999889] [PubMed: 21148212]

10.: Hippisley-Cox J, Coupland C, Vinogradova Y, Robson J, Minhas R, Sheikh A et al Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2. BMJ. 2008; 336(7659):1475–1482 [PMC free article: PMC2440904] [PubMed: 18573856]

11.: Jaspers NE M, Blaha MJ, Matsushita K, van der Schouw YT, Wareham NJ, Khaw K- T et al Prediction of individualized lifetime benefit from cholesterol lowering, blood pressure lowering, antithrombotic therapy, and smoking cessation in apparently healthy people. European Heart Journal. 2020; 41(11):1190–1199 [PMC free article: PMC7229871] [PubMed: 31102402]

12.: Karmali K, Persell S, Perel P, Lloyd-Jones D, Berendsen M, Huffman M. Risk scoring for the primary prevention of cardiovascular disease. Cochrane Database of Systematic Reviews 2017, Issue 3. Art. No.: CD006887. DOI: 10.1002/14651858.CD006887.pub4. [PMC free article: PMC6464686] [PubMed: 28290160] [CrossRef]

13.: Lindbohm JV, Sipila PN, Mars N, Knuppel A, Pentti J, Nyberg ST et al Association between change in cardiovascular risk scores and future cardiovascular disease: analyses of data from the Whitehall II longitudinal, prospective cohort study. The Lancet Digital health. 2021; 3(7):e434–e444 [PMC free article: PMC8474012] [PubMed: 34167764]

14.: Lindbohm JV, Sipila PN, Mars NJ, Pentti J, Ahmadi-Abhari S, Brunner EJ et al 5-year versus risk-category-specific screening intervals for cardiovascular disease prevention: a cohort study. The lancet Public Health. 2019; 4(4):e189–e199 [PMC free article: PMC6472327] [PubMed: 30954144]

15.: Livingstone S, Morales DR, Donnan PT, Payne K, Thompson AJ, Youn J- H et al Effect of competing mortality risks on predictive performance of the QRISK3 cardiovascular risk prediction tool in older people and those with comorbidity: external validation population cohort study. The Lancet Healthy longevity. 2021; 2(6):e352–e361 [PMC free article: PMC8175241] [PubMed: 34100008]

16.: Livingstone SJ, Guthrie B, Donnan PT, Thompson A, Morales DR. Predictive performance of a competing risk cardiovascular prediction tool CRISK compared to QRISK3 in older people and those with comorbidity: population cohort study. BMC Medicine. 2022; 20(1):152 [PMC free article: PMC9066924] [PubMed: 35505353]

17.: National Institute for Health and Care Excellence. Developing NICE guidelines: the manual [updated January 2022]. London. National Institute for Health and Care Excellence, 2014. Available from: http://www.nice.org.uk/article/PMG20/chapter/1%20Introduction%20and%20overview

18.: National Institute for Health and Clinical Excellence. The guidelines manual. London. National Institute for Health and Clinical Excellence, 2011. Available from: http://www.nice.org.uk/aboutnice/howwework/developingniceclinicalguidelines/clinicalguidelinedevelopmentmethods/GuidelinesManual2009.jsp

19.: Osborn D, Burton A, Walters K, Atkins L, Barnes T, Blackburn R et al Primary care management of cardiovascular risk for people with severe mental illnesses: the Primrose research programme including cluster RCT. NIHR Journals Library Programme Grants for Applied Research. 2019; 4:4 [PubMed: 31034192]

20.: Osborn D, Burton A, Walters K, Atkins L, Barnes T, Blackburn R et al Primary care management of cardiovascular risk for people with severe mental illnesses: the Primrose research programme including cluster RCT. 2019. Available from: http://www.epistemonikos.org/documents/a6be67124f180e8a97bee68f0a4fc3b9a0053d17 [PubMed: 31034192]

21.: Osborn DP J, Hardoon S, Omar RZ, Holt RI G, King M, Larsen J et al Cardiovascular risk prediction models for people with severe mental illness: results from the prediction and management of cardiovascular risk in people with severe mental illnesses (PRIMROSE) research program. JAMA psychiatry. 2015; 72(2):143–151 [PMC free article: PMC4353842] [PubMed: 25536289]

22.: Simmonds MC, Wald NJ. Risk estimation versus screening performance: a comparison of six risk algorithms for cardiovascular disease. Journal of Medical Screening. 2012; 19(4):201–205 [PubMed: 23293165]

23.: Tillin T, Hughes AD, Whincup P, Mayet J, Sattar N, McKeigue PM et al Ethnicity and prediction of cardiovascular disease: performance of QRISK2 and Framingham scores in a U.K. tri-ethnic prospective cohort study (SABRE-Southall And Brent REvisited). Heart (British Cardiac Society). 2014; 100(1):60–67 [PMC free article: PMC4869829] [PubMed: 24186564]

24.: Zomer E, Osborn D, Nazareth I, Blackburn R, Burton A, Hardoon S et al Effectiveness and cost-effectiveness of a cardiovascular risk prediction algorithm for people with severe mental illness (PRIMROSE). BMJ Open. 2017; 7(9):e018181 [PMC free article: PMC5588956] [PubMed: 28877952]

Appendices

Appendix A. Review protocols

A.1. Review protocol for CVD risk assessment tools: primary prevention

Download PDF (187K)

A.2. Health economic review protocol

Download PDF (169K)

Appendix B. Literature search strategies

Cardiovascular risk assessment tools in adults without established cardiovascular disease

The literature searches detailed below are for the review:

What is the most accurate tool for determining 10-year and lifetime cardiovascular risk in adults without established cardiovascular disease?

They complied with the methodology outlined in Developing NICE guidelines: the manual.¹⁷

For more information, please see the Methodology review published as part of the accompanying documents for this guideline.

B.1. Clinical search literature search strategy

Searches were constructed using a PICO framework where population (P) terms were combined with Intervention (I) and in some cases Comparison (C) terms. Outcomes (O) are rarely used in search strategies as these concepts may not be indexed or described in the title or abstract and are therefore difficult to retrieve. Search filters were applied to the search where appropriate.

Download PDF (209K)

B.2. Health Economics literature search strategy

Health economic evidence was identified by conducting literature searches as below. The following databases were searched: NHS Economic Evaluation Database (NHS EED - this ceased to be updated after 31st March 2015), Health Technology Assessment database (HTA - this ceased to be updated from 31st March 2018) and The International Network of Agencies for Health Technology Assessment (INAHTA). Searches for recent evidence were run on Medline and Embase from 2014 onwards for health economics, and all years for quality-of-life studies.

Download PDF (171K)

Appendix C. Prognostic evidence study selection

Download PDF (104K)

Appendix D. Prognostic evidence

Appendix E. Forest plots and summary ROC curves

E.1. Summary of C statistic data

Download PDF (150K)

E.2. Sensitivity and specificity data

Download PDF (135K)

E.3. Summary ROC curves

Download PDF (141K)

Appendix F. Economic evidence study selection

Download PDF (164K)

Appendix G. Economic evidence tables

Download PDF (155K)

Appendix H. Health economic model

This area was not prioritised for new cost-effectiveness analysis.

Appendix I. Excluded studies

I.1. Clinical studies

Table 26

Studies excluded from the clinical review.

I.2. Health Economic studies

Published health economic studies that met the inclusion criteria (relevant population, comparators, economic study design, published 2007 or later and not from non-OECD country or USA) but that were excluded following appraisal of applicability and methodological quality are listed below. See the health economic protocol for more details.

Table 27

Studies excluded from the health economic review.

Appendix J. List of abbreviations

Table 28

List of abbreviations.

Final

Evidence review underpinning recommendations 1.1.7 to 1.1.11 and 1.1.16 in the NICE guideline

Developed by National Institute for Health and Care Excellence

Disclaimer: The recommendations in this guideline represent the view of NICE, arrived at after careful consideration of the evidence available. When exercising their judgement, professionals are expected to take this guideline fully into account, alongside the individual needs, preferences and values of their patients or service users. The recommendations in this guideline are not mandatory and the guideline does not override the responsibility of healthcare professionals to make decisions appropriate to the circumstances of the individual patient, in consultation with the patient and/or their carer or guardian.

Local commissioners and/or providers have a responsibility to enable the guideline to be applied when individual health professionals and their patients or service users wish to use it. They should do so in the context of local and national priorities for funding and developing services, and in light of their duties to have due regard to the need to eliminate unlawful discrimination, to advance equality of opportunity and to reduce health inequalities. Nothing in this guideline should be interpreted in a way that would be inconsistent with compliance with those duties.

NICE guidelines cover health and care in England. Decisions on how they apply in other UK countries are made by ministers in the Welsh Government, Scottish Government, and Northern Ireland Executive. All NICE guidance is subject to regular review and may be updated or withdrawn.

Bookshelf ID: NBK603622PMID: 38723137

Table 1PICO characteristics of review question

Population	Adults (18 years and over) without established CVD, including adults with chronic kidney disease, type 1 diabetes, and type 2 diabetes Validation studies in a UK population Derivation studies from the UK, or non-UK cohorts if the tool has subsequently been validated in a UK population.
Risk tools	10-year risk QRISK 2 QRISK 3 SCORE 2 SCORE 2 – OP AHA/ASCVD risk engine LIFE-CVD PRIMROSE (BMI model and lipid model) CCRISK CRISK Lifetime risk QRISK lifetime AHA/ASCVD risk engine LIFE-CVD
Patient outcomes	Overall CVD events, including: All-cause mortality CV mortality Non-fatal myocardial infarction Non-fatal stroke
Statistical outcomes	Discrimination: Area under the ROC curve (c-index, c-statistic). Classification measures at 5%, 7.5%, 10%, 15% and 20% predicted risk thresholds: sensitivity, and specificity. D statistic Calibration: Calibration plots Predicted risk versus observed risk Statistical tests for agreement between predicted and observed events (E.g. Hosmer-Lemeshow or Nam–D'Agostino statistics) Reclassification / revalidation net classification improvement integrated discrimination index
Study design	Cohort (external validation, internal validation)
Specific groups	Subgroups that will be investigated: presence of type 1 diabetes presence of CKD (eGFR <60 ml/min/1.73 m² and/or albuminuria)

Table 2Predictor variables included in CVD risk assessment tools

Risk Score	Age	Sex	Ethnicity	BMI	Total cholesterol	LDL- cholesterol	HDL- cholesterol	Non-HDL- cholesterol	Systolic blood pressure	Blood pressure medication	Diabetes	Smoking	Family history of CVD	Social deprivation	Chronic kidney disease	Rheumatoid arthritis	SBP variability	Migraine	Corticosteroids	Systemic lupus erythematosus	Erectile dysfunction	Antipsychotics	Severe mental illness	HIV/AIDS	Antidepressants	History of heavy drinking
ASCVD	X	X	X		X		X		X	X	X	X
CRISK	X	X	X	X	X		X		X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
CRISK-CCI	X	X	X	X	X		X		X	X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
LIFE-CVD	^*	X		X				X	X		X	X	X
PRIMROSE-BMI	X	X		X					X		X	X		X								X	X		X	X
PRIMROSE-lipids	X	X			X		X		X		X	X		X								X	X		X	X
QRISK2	X	X	X	X	X		X		X	X	X	X	X	X	X	X
QRISK3	X	X	X	X	X		X		X	X	X	X	X	X	X	X	X	X	X	X	X	X	X
QRISK-lifetime	^*	X	X	X	X	X	X		X	X	X	X	X	X	X	X
SCORE2	X	X			X		X		X		X	X
SCORE-OP	X	X			X		X		X		X	X

*: Age considered as the underlying time function of the model, not as a predictor variable

: Definitions: ASCVD; atherosclerotic cardiovascular disease score derived in US cohorts, CRISK; Competing risk model, CRISK-CCI; Competing risk model with Charlson comorbidity index, LIFE-CVD; prediction algorithm for cardiovascular disease derived from a US cohort (MESA). QRISK; prediction algorithm for cardiovascular disease derived from UK cohort, QResearch; large consolidated database derived from the anonymised health records from general practices using Egton Medical Information Systems clinical computer system in the UK, PRIMROSE; Prediction risk score for people with severe mental illnesses derived from a European cohort, SCORE; risk prediction algorithm for cardiovascular disease in Europe, SCORE-OP; risk prediction algorithm estimating incident cardiovascular event risk in older persons in four geographical risk regions

Table 3Outcomes predicted by CVD risk assessment tools

Risk Score	Derivation cohort and region	Publication year	Myocardial infarction	Coronary heart disease death	Stroke	Stroke death	Transient ischaemic attack	Coronary revascularisation	Angina pectoris	Unstable angina
ASCVD	ARIC (Atherosclerosis Risk in Communities), CARDIA (Coronary Artery Risk Development in Young Adults), CHS (Cardiovascular Health Study), Framingham USA	2013	X	X	X	X
CRISK	CPRD (UK Clinical Practice Research Datalink) Gold UK	2017	X	X	X	X	X		X	X
CRISK-CCI	CPRD Gold UK	2017	X	X	X	X	X		X	X
LIFE-CVD	MESA (Multi-Ethnic Study of Atherosclerosis) USA	2020	X	X	X	X
PRIMROSE-BMI	THIN (The Health Improvement Network) UK	2015	X	X	X	X	X	X	X	X
PRIMROSE-lipids	THIN UK	2015	X	X	X	X	X	X	X	X
QRISK2	QRESEARCH UK	2009	X	X	X	X	X		X	X
QRISK3	QRESEARCH UK	2017	X	X	X	X	X		X	X
QRISK-lifetime	QRESEARCH UK	2010	X	X	X	X	X		X	X
SCORE2	45 prospective cohorts Europe, Canada, USA	2021	X	X	X	X
SCORE-OP	ARIC, CPRD, HYVET (Hypertension in the Very Elderly Trial), MESA, PROSPER (PROspective study of pravastatin in the elderly at risk), SPRINT (Systolic Blood Pressure Intervention Trial) Europe, USA	2021	X	X	X	X

Table 4Summary of studies included in the evidence review

Study (cohort)	Risk tool(s)	Population, N (Country)	Age, years (range)	Outcomes (including definitions)	No. of CVD events
From 2014 update of CG181
Collins 2012B³ (THIN) External validation of QRISK2	QRISK2–2008 QRISK2–2010 QRISK2–2011	2,084,445 UK	30–84	Fatal or non-fatal CVD: myocardial infarction, angina, CHD, stroke, transient ischaemic attacks	93,563
Hippisley-Cox 2008¹⁰ (QResearch) Development and validation of QRISK2 (10-year risk)	QRISK2–2008	2,285,815 UK	35–74	Fatal or non-fatal CVD: coronary heart disease (angina and myocardial infarction), stroke, or transient ischaemic attacks.	96,709
Hippisley-Cox 2010⁹ (QResearch) Development and validation of QRISK2 (lifetime risk)	QRISK2–2010 lifetime	3,601,918 UK	30–84	Fatal or non-fatal CVD: coronary heart disease (angina and myocardial infarction), stroke, or transient ischaemic attacks.	121,623
From update search
Anonymous 2021 (SCORE2 working group)² (CPRD) Development, internal and external validation	SCORE2	677,684 (derivation) 30 plus countries (ERFC) and UK 1,133,181 (validation) 15 European countries: Denmark, Finland, France, Germany, Italy, Netherlands, Norway, Spain, Sweden, UK, Czech Republic, Estonia, Poland, Lithuania, Russia Validation cohort: MORGAM project, BiomarCaREConsortium, EPIC-CVD, CPRD, HNR, Estonian Biobank, HAPIEE study, HUNT study, DETECT study, Gutenberg Health Study	40–69	Fatal or non-fatal CVD. Cause-specific mortality due to hypertensive disease, ischemic heart disease, arrhythmias, heart failure, cerebrovascular disease: atherosclerosis/abdominal aortic aneurysm, sudden death and death within 24h of symptom onset Non-fatal cardiovascular disease: non-fatal myocardial infarction, non-fatal stroke	30,121 (derivation cohort) 43,492 (validation cohort)
Anonymous 2021 (SCORE2-OP working group)¹ (CPRD) Development, internal and external validation	SCORE2-OP ASCVD	28,503 (derivation) Norway 338,615 (validation) USA, Europe, and UK Validation cohort: ARIC, MESA, and CPRD cohorts, and the combined study populations of the HYVET, PROSPER, and SPRINT trial	65 and older	Cause-specific mortality due to: hypertensive disease, ischemic heart disease, arrhythmias, heart failure, cerebrovascular disease, atherosclerosis/AAA, sudden death and death within 24, h of symptom onset Non-fatal cardiovascular disease: non-fatal myocardial infarction, non-fatal stroke	10,089 (derivation cohort) 33,219 (validation cohort)
Dziopa 2022⁵ (CPRD) External validation	QRISK2 QRISK3 ASCVD	168,871 (type 2 diabetes) UK	Range: NR Mean (SD): 59.3 (13.9)	CVD: the first occurrence of fatal or non-fatal myocardial infarction, sudden cardiac death, ischaemic heart disease, fatal or non-fatal stroke, or PAD since diagnosis of type 2 diabetes. Additional outcomes (CVD+) included all of the above plus heart failure and atrial fibrillation	38,335
Goff 2014⁶ Development, internal and external validation	ASCVD	24,626 USA Validation cohort: NHLBI-sponsored cohort studies, including the ARIC study Cardiovascular Health Study, CARDIA study combined with applicable data from the Framingham Original and Offspring Study cohort	40–79	CVD: nonfatal myocardial infarction, CHD death, or fatal or nonfatal stroke	2689
Hippisley-Cox 2014⁸ (CPRD) External validation	QRISK2–2014	3,271,512 UK	25–99	CVD: defined as a composite outcome of coronary heart disease, ischaemic stroke, or transient ischaemic attack	139,485
Hippisley-Cox 2017⁷ (QResearch) Development and internal validation of QRISK3	QRISK2–2017 QRISK-3	7,889,803 (derivation) 2,671,298 (validation) UK	25–84	CVD: coronary heart disease, ischaemic stroke, or transient ischaemic attack	363,565 (derivation)
Jaspers, 2020¹¹ (EPIC-Norfolk) Development, internal and external validation	LIFE-CVD	6715 (MESA derivation) 23548 (validation) Europe	45–80	CVD: fatal or non-fatal MI or stroke, resuscitated cardiac arrest, and coronary heart disease death	621 (MESA derivation)
Lindbohm 2019¹⁴ (Whitehall II cohort) External validation	ASCVD Revised ASCVD	6964 UK	40–64	CVD: fatal coronary heart disease, non-fatal myocardial infarction, fatal or non-fatal stroke	617
Lindbohm 2021¹³ (Whitehall II cohort) External validation	ASCVD	7996 UK	40–63	CVD: nonfatal myocardial infarction, CHD death, or fatal or nonfatal stroke	1840
Livingstone 2021¹⁵ (CPRD Gold database) External validation	QRISK3	2,904,773 UK	25–84	CVD: coronary heart disease, ischaemic stroke, or transient ischaemic attack	95,517
Livingstone 2022¹⁶ (CPRD Gold database) Development and internal validation of CRISK tools External validation of QRISK3	CRISK CRISK-CCI QRISK3	1,936,516 (derivation) 968,257 (validation) UK	25–84	CVD: coronary heart disease, ischaemic stroke, or transient ischaemic attack	31,839
Osborn 2015²¹ and 2019^{19, 20} (THIN) Development and internal validation	PRIMROSE BMI PRIMROSE lipid	38,824 UK – adults with severe mental illness	30–90	CVD: myocardial infarction, angina pectoris, cerebrovascular accidents, or major coronary surgery	2324
Tillin 2014²³ (SABRE) External validation	QRISK2–2012	3674 UK	40–69	CVD: myocardial infarction, coronary revascularisation, angina, transient ischaemic attack or stroke	465

Table 5Summary of results: AUC (95% CI)

Tool and subgroup	AUC (95% CI)
Tool and subgroup	Women	Men
Hippisley-Cox 2008 ¹⁰ . QRISK2–2008; QResearch database
QRISK2–2008	0.817 (0.814–0.820)	0.792 (0.789–0.794)
Collins 2012 ³ . QRISK2; THIN database
QRISK2–2011. Age 30–84	0.835 (0.834–0.837)	0.809 (0.807–0.811)
QRISK2–2010. Age 30–84	0.835 (0.833–0.837)	0.811 (0.809–0.812)
QRISK2–2011. Age 35–74	0.802 (0.800–0.804)	0.771 (0.769–0.773)
QRISK2–2008. Age 35–74	0.800 (0.798–0.803)	0.772 (0.769–0.774)
Hippisley-Cox 2014 ⁸ . QRISK2–2014; CPRD database
QRISK2–2014	0.883 (0.882–0.884)	0.859 (0.858–0.861)
Tillin 2014 ²³ . QRISK2–2012; SABRE cohort
QRISK2–2012 European White	0.750 (0.670–0.820)	0.700 (0.660–0.740)
QRISK2–2012 South Asian	0.750 (0.660–0.840)	0.730 (0.690–0.770)
QRISK2–2012 African Caribbean	0.650 (0.540–0.760)	0.670 (0.570–0.770)
Hippisley-Cox 2017 ⁷ . QRISK2–2017 and QRISK3; QResearch database
QRISK2–2017: full cohort	0.879 (0.878–0.88)	0.858 (0.856–0.859)
QRISK3 – with SBP variation: full cohort	0.880 (0.879–0.882)	0.858 (0.857–0.860)
QRISK3 – without SBP variation: full cohort	0.880 (0.878–0.881)	0.858 (0.857–0.859)
QRISK3 – without SBP variation: CKD stage 3–5	0.742 (0.720–0.764)	0.737 (0.715–0.776)
QRISK3 – without SBP variation: type 1 diabetes	0.823 (0.789–0.857)	0.804 (0.760–0.832)
QRISK3 – without SBP variation: type 2 diabetes	0.701 (0.691–0.711)	0.696 (0.687–0.704)
QRISK3 – without SBP variation: SMI	0.844 (0.837–0.851)	0.817 (0.809–0.852)
QRISK3 – without SBP variation: age <40	0.747 (0.728–0.766)	0.781 (0.771–0.792)
QRISK3 – without SBP variation: age 40–59	0.752 (0.747–0.757)	0.732 (0.728–0.736)
QRISK3 – without SBP variation: age 60+	0.692 (0.689–0.695)	0.659 (0.656–0.663)
Dziopa 2022 ⁵ . QRISK2, QRISK3 &ASCVD; CPRD database
QRISK2: type 2 diabetes	0.664 (0.660–0.668)
QRISK3: type 2 diabetes	0.664 (0.660–0.667)
ASCVD: type 2 diabetes	0.668 (0.664–0.671)
Lindbohm 2019¹⁴ and 2021¹³. ASCVD; Whitehall II cohort
ASCVD (original version) Age 40–64	0.71
ASCVD (revised for Whitehall II cohort) Age 40–64	0.72
ASCVD (original version) Age 40–75 (cohort overlaps with above)	0.699
Livingstone 2021¹⁵ and 2022¹⁶; QRISK3, CRISK and CRISK-CCI; CPRD database
QRISK3: in full CPRD cohort	0.865 (0.861–0.868)	0.834 (0.831–0.837)
QRISK3: in full CPRD cohort; age 25–44	0.758 (0.747–0.769)	0.757 (0.749–0.764)
QRISK3: in full CPRD cohort; age 45–64	0.707 (0.702–0.713)	0.681 (0.677–0.685)
QRISK3: in full CPRD cohort; age 65–74	0.641 (0.635–0.647)	0.612 (0.606–0.617)
QRISK3: in full CPRD cohort; age 75–84	0.611 (0.605–0.616)	0.585 (0.579–0.591)
QRISK3: in CRISK validation cohort (subset of above cohort)	0.863 (0.858–0.869)	0.832 (0.827–0.836)
QRISK3: in CRISK validation cohort; age 25–44	0.765 (0.747–0.783)	0.740 (0.727–0.753)
QRISK3: in CRISK validation cohort; age 45–64	0.708 (0.698–0.717)	0.679 (0.672–0.686)
QRISK3: in CRISK validation cohort; age 65–74	0.641 (0.631–0.652)	0.606 (0.596–0.615)
QRISK3: in CRISK validation cohort; age 75–84	0.614 (0.605–0.622)	0.590 (0.580–0.601)
CRISK	0.863 (0.858–0.869)	0.833 (0.828–0.837)
CRISK; age 25–44	0.761 (0.743–0.779)	0.744 (0.731–0.757)
CRISK; age 45–64	0.710 (0.701–0.720)	0.683 (0.676–0.690)
CRISK; age 65–74	0.645 (0.634–0.655)	0.610 (0.600–0.619)
CRISK; age 75–84	0.614 (0.605–0.622)	0.594 (0.583–0.604)
CRISK-CCI	0.864 (0.859–0.869)	0.819 (0.815–0.824)
CRISK-CCI; age 25–44	0.763 (0.745–0.781)	0.733 (0.720–0.746)
CRISK-CCI; age 45–64	0.713 (0.703–0.722)	0.661 (0.654–0.668)
CRISK-CCI; age 65–74	0.647 (0.637–0.658)	0.591 (0.581–0.600)
CRISK-CCI; age 75–84	0.616 (0.607–0.624)	0.570 (0.559–0.580)
Osborn 2015 ²¹. PRIMROSE (internal validation); UK THIN database
PRIMROSE-BMI	0.779 (0.749–0.810)	0.784 (0.735–0.833)
PRIMROSE-lipid	0.790 (0.755–0.824)	0.796 (0.758–0.833)
SCORE2 working group 2021 ² . SCORE2; CPRD database
SCORE2: full cohort	0.720 (0.717–0.724)
SCORE2: age 40–50	0.698 (0.689–0.706)
SCORE2: age 50–59	0.653 (0.647–0.659)
SCORE2: age 60–69	0.620 (0.614–0.625)
SCORE2-OP working group 2021 ¹ . SCORE2-OP & ASCVD; CPRD database
SCORE2-OP.	0.657 (0.655–0.662)
ASCVD	0.663 (0.659–0.666)
Hippisley-Cox 2010 ⁹ . Lifetime QRISK2; QRESEARCH database
QRISK2- lifetime (at 10 years)	0.842 (0.840–0.844)	0.828 (0.826–0.830)
Jaspers 2020 ¹¹ . LIFE-CVD; EPIC-Norfolk
LIFE-CVD	0.76 (0.75–0.76)

Table 6Summary of results: D statistics

Tool and subgroup	D statistics
Tool and subgroup	Women	Men
Hippisley-Cox 2008 ¹⁰ . QRISK2–2008; QResearch database
QRISK2–2008	1.795 (1.769–1.820)	1.615 (1.594–1.637)
Collins 2012 ³ . QRISK2; THIN database
QRISK2–2011 (aged 30–84)	1.98 (1.96–1.99)	1.73 (1.71–1.75)
QRISK2–2010 (aged 30–84)	1.97 (1.95–1.99)	1.76 (1.74–1.77)
QRISK2–2011 (aged 35–74)	1.67 (1.65–1.69)	1.44 (1.42–1.46)
QRISK2–2008 (aged 35–74)	1.66 (1.56–1.76)	1.45 (1.31–1.59)
Hippisley-Cox 2014 ⁸ . QRISK2–2014; CPRD database
QRISK2–2014	2.328 (2.313–2.343)	2.085 (2.071–2.098)
Tillin 2014 ²³ . QRISK2–2012; SABRE cohort
QRISK2 - European White	1.33 (0.79 to 1.87)	1.06 (0.82 to 1.30)
QRISK2 - South Asian	1.55 (0.91 to 2.19)	1.22 (0.99 to 1.45)
QRISK2 - African Caribbean	0.74 (0 to 1.63)	0.96 (0.32 to 1.59)
Hippisley-Cox 2017 ⁷ . QRISK2–2017 and QRISK3; QResearch database
QRISK2–2017	2.48 (2.46 to 2.5)	2.25 (2.24 to 2.27)
QRISK3-with SBP variability	2.49 (2.47 to 2.51)	2.26 (2.25 to 2.28)
QRISK3-without SBP variability	2.48 (2.46 to 2.5)	2.26 (2.24 to 2.27)
QRISK3 – without SBP variation: CKD stage 3–5	1.32 (1.17 to 1.47)	1.28 (1.13 to 1.44)
QRISK3 – without SBP variation: type 1 diabetes	1.94 (1.66 to 2.22)	1.87 (1.64 to 2.11)
QRISK3 – without SBP variation: type 2 diabetes	1.19 (1.12 to 1.25)	1.12 (1.06 to 1.17)
QRISK3 – without SBP variation: SMI	2.16 (2.1 to 2.22)	1.94 (1.87 to 2.02)
QRISK3 – without SBP variation: age <40	1.66 (1.55 to 1.76)	1.75 (1.69 to 1.82)
QRISK3 – without SBP variation: age 40–59	1.48 (1.44 to 1.51)	1.33 (1.31 to 1.36)
QRISK3 – without SBP variation: age 60+	1.11 (1.09 to 1.13)	.903 (.883 to .922)
Livingstone 2021¹⁵ (Royston’s D) QRISK3; CPRD database
QRISK3 (full cohort)	2.43 (2.41 to 2.45)	2.1 (2.08 to 2.12)
QRISK3 (age 25–44)	1.69 (1.63 to 1.76)	1.57 (1.52 to 1.61)
QRISK3 (age 45–64)	1.25 (1.22 to 1.28)	1.04 (1.02 to 1.07)
QRISK3 (age 65–74)	0.82 (0.77 to 0.86)	0.63 (0.59 to 0.66)
QRISK3 (age 75–84)	0.61 (0.56 to 0.66)	0.46 (0.42 to 0.51)
Osborn 2015 ²¹. PRIMROSE (internal validation); UK THIN database
PRIMROSE BMI	1.8 (1.7 to 1.9)	1.84 (1.73 to 1.96)
PRIMROSE lipid	1.87 (1.76 to 1.98)	1.92 (1.8 to 2.03)
Hippisley-Cox 2010. Lifetime QRISK2 ⁹. Lifetime QRISK2; QRESEARCH database
QRISK2 lifetime	NR	NR
Abbreviation: NR; not reported

Table 7Summary of results: sensitivity and specificity

Tool	Threshold	Sensitivity, % (95% CI)	Specificity, % (95% CI)
Hippisley-Cox 2014 ⁸ . QRISK2–2014; CPRD database
QRISK2–2014	20.7% (top decile of predicted risk) Observed risk 31.8%	49.9	91.9
Livingstone 2022 ¹⁶ . QRISK3 and CRISK-CCI; CPRD database ^*
QRISK3	7.5%	Women: 75.0 Men: 79.5	Women: 81.2 Men: 71.5
	10%	Women: 68.3 Men: 71.3	Women: 85.3 Men: 77.9
	20%	Women: 47.0 Men: 45.1	Women: 93.1 Men: 90.9
CRISK-CCI	7.5%	Women: 73.3 Men: 77.9	Women: 82.5 Men: 72.5
	10%	Women: 65.9 Men: 69.1	Women: 69.1 Men: 79.0
	20%	Women: 41.2 Men: 37.6	Women: 94.5 Men: 92.3

*: Sensitivity and specificity values have been calculated from data available in the study report and are therefore approximate. See also Appendix E.2 and E.3.

Table 8Clinical evidence profile: Discriminative capacity of selected CVD risk prediction tools

Risk tool	No of studies	N	Risk of bias	Inconsistency	Indirectness	Imprecision	Area Under Curve: Individual study effects [point estimate (95% CI)]	Confidence
QRISK2–2008 (internal and external validation)	2	Women: 1441890 Men: 1392787	No serious risk of bias^a	Serious inconsistency^b	No serious indirectness	No serious imprecision^c	Women 30–84: 0.817 (0.814–0.820) Men 30–84: 0.792 (0.789–0.794) Women aged 35–74: 0.800 (0.798–0.803) Men aged 35–74: 0.772 (0.769–0.774)	MODERATE
QRISK2–2010	1	Women: 1066127 Men: 1018318	No serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women aged 30–84: 0.835 (0.833–0.837) Men aged 30–84: 0.811 (0.809–0.812)	HIGH
QRISK2–2011	1	Women: 1066127 Men: 1018318	No serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women aged 30–84: 0.835 (0.834–0.837) Men aged 30–84: 0.809 (0.807–0.811)	HIGH
QRISK2–2012	1	European White Women: 444 South Asian Women: 241 African Caribbean Women: 247 African Caribbean Men: 307	Serious risk of bias^a,d	No serious inconsistency	No serious indirectness	serious imprecision^c	European white women: 0.750 (0.670–0.820) South Asian women: 0.750 (0.660–0.840) African Caribbean women: 0.650 (0.540–0.760) European white men: 0.700 (0.660–0.740) South Asian men: 0.730 (0.690–0.770) African Caribbean men: 0.670 (0.570–0.770)	LOW
QRISK2–2012	1	European White Men: 1359 South Asian Men: 1076	No serious risk of bias^a	No serious inconsistency	No serious indirectness	serious imprecision^c	European white men: 0.700 (0.660–0.740) South Asian men: 0.730 (0.690–0.770)	MODERATE
QRISK2–2014	1	Women: 1682709 Men: 1588803	No serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.883 (0.882–0.884) Men: 0.859 (0.858–0.861)	HIGH
QRISK2–2017	1	Women: 1360457 Men: 1310841	Serious risk of bias^a,e	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.879 (0.878–0.88) Men: 0.858 (0.856–0.859)	MODERATE
QRISK2-year not specified	1	168871	No serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision^c	Type 2 diabetes: 0.664 (0.660–0.668)	HIGH
QRISK3-year not specified	1	168871	No serious risk of bias^a	No serious inconsistency	No serious indirectness	No serious imprecision^c	Type 2 diabetes: 0.664 (0.660–0.667)	HIGH
QRISK3–2017 internal and external validation (with SBP variability)	2	Women: 2845054 Men: 2731017	No serious risk of bias^a	serious inconsistency^b	No serious indirectness	No serious imprecision^c	Women: 0.880 (0.879–0.882) 0.865 (0.861–0.868) Men: 0.858 (0.857–0.860) 0.834 (0.831–0.837)	MODERATE
QRISK3– 2017 internal validation (without SBP variability)	1	Women: 1360457 Men: 1310841	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.880 (0.878–0.881) Men: 0.858 (0.857–0.859)	MODERATE
QRISK3–2017 (Type 1 diabetes subgroup)	1	Women: 3351 Men: 3932	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.823 (0.789–0.857) Men: 0.804 (0.760–0.832)	MODERATE
QRISK3–2017 (CKD stage 3–5 subgroup)	1	Women: 6949 Men: 4232	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.742 (0.720–0.764) Men: 0.737 (0.715–0.776)	MODERATE
ASCVD	4	Type 2 diabetes: 168871 Age≥65: 319390 Age 40–64: 6964 Age 40–75: 7996	Serious risk of bias^a,g	No serious inconsistency	No serious indirectness	No serious imprecision^c	Type 2 diabetes: 0.668 (0.664–0.671) Age≥65: 0.663 (0.659–0.666) Age 40–64: 0.71 Age 40–75: 0.72	MODERATE
ASCVD revised for Whitehall II cohort	1	6964	Serious risk of bias^a,h	No serious inconsistency	No serious indirectness	No serious imprecision^c	Age 40–75: 0.699	MODERATE
CRISK internal validation	1	Women: 494865 Men: 473392	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.863 (0.858–0.869) Men: 0.833 (0.828–0.837)	MODERATE
CRISK-CCI internal validation	1	Women: 494865 Men: 473392	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.864 (0.859–0.869) Men: 0.819 (0.815–0.824)	MODERATE
PRIMROSE-BMI internal validation	1	Women: 2041 Men: 1842	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.779 (0.749–0.810) Men: 0.784 (0.735–0.833)	MODERATE
PRIMROSE-lipid internal validation	1	Women: 2041 Men: 1842	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.790 (0.755–0.824) Men: 0.796 (0.758–0.833)	MODERATE
SCORE2	1	927079	No serious risk of bias	No serious inconsistency	No serious indirectness	No serious imprecision^c	0.720 (0.717–0.724)	HIGH
SCORE2-OP	1	319390	No serious risk of bias	No serious inconsistency	No serious indirectness	No serious imprecision^c	0.657 (0.655–0.662)	HIGH
QRISK lifetime internal validation (assessed over 10 years)	1	Women: 645012 Men: 622147	Serious risk of bias^a,f	No serious inconsistency	No serious indirectness	No serious imprecision^c	Women: 0.842 (0.840–0.844) Men: 0.828 (0.826–0.830)	MODERATE
LIFE-CVD	1	23548	No serious risk of bias	No serious inconsistency	No serious indirectness	No serious imprecision^c	0.760 (0.750–0.760)	HIGH

: GRADE was conducted with emphasis on area under the curve, as this was the primary measure for decision making

a): Risk of bias was assessed using the PROBAST checklist. Downgraded by 1 increment if the majority of the evidence was at high risk of bias, and downgraded by 2 increments if the majority of the evidence was at very high risk of bias. Risk of bias was serious for some risk tools because of low event rate, insufficient reporting of outcomes, lack of calibration data, or having internal validation only.

b): If no pooling were possible, inconsistency was assessed by inspection of the degree of overlap of confidence intervals between studies: if one of more CIs did not overlap then a rating of serious inconsistency was given. Reasons for heterogeneity between studies may include geographical/cultural/ethnic differences.

c): The judgement of precision was based on the spread of confidence interval across two clinical thresholds: C statistics of 0.5 and 0.7. The threshold of 0.5 marked the boundary between no predictive value better than chance and a predictive value better than chance. The threshold of 0.7 marked the boundary above which the committee might consider recommendations. If the 95% CIs crossed one of these thresholds a rating of serious imprecision was given and if they crossed both of these thresholds a rating of very serious imprecision as given.

d): Event rate <100 in each subgroup

e): Same data source as internal validation cohort

f): Internal validation only

g): Insufficient reporting (point estimate only) in 2/4 and no calibration data in 1/4 studies

h): Insufficient reporting (point estimate only)

Figure 1Calibration curves: observed versus predicted 10-year risk of CVD (from Collins 2012)

Table 9QRISK2-2012 predicted : observed events

Tillin 2014	Men	Women
QRISK2 - European White	0.78 (0.72 to 0.85)	0.73 (0.65 to 0.80)
QRISK2 - South Asian	0.71 (0.64 to 0.78)	0.52 (0.34 to 0.72)
QRISK2 - African Caribbean	0.95 (0.80 to 1.00)	1.22 (1.04 to 1.84)

Figure 2Calibration curves for QRISK2: observed versus predicted 10-year risk of CVD in men

Source: Tillin 2014 ²³

Figure 3Calibration curves for QRISK2: observed versus predicted 10-year risk of CVD in women

Source: Tillin 2014 ²³

Figure 4Calibration curves: observed versus predicted 10-year risk of CVD (from Hippisley-Cox 2014)

Source: Hippisley-Cox 2014 ⁸

Figure 5Calibration curves: observed versus predicted 10-year risk of CVD (from Hippisley-Cox 2017)

Source: Hippisley-Cox 2017 ⁷

Figure 6QRISK3 calibration in women

Source: Livingstone 2021 ¹⁵

Figure 7QRISK3 calibration in men

Source: Livingstone 2021 ¹⁵

Figure 8Calibration plot of QRISK3, CRISK and CRISK-CCI

Source: Livingstone 2022 ¹⁶

Figure 9Calibration plots of QRISK3, CRISK and CRISK-CCI stratified by age

Source: Livingstone 2022 ¹⁶

Figure 10Calibration plots for PRIMROSE tools

Source: Osborn 2015 ²¹

Figure 11Calibration of SCORE2 in CPRD data by age groups (SCORE2 working group 2021)

Source: SC0RE2 working group 2021 ²

Figure 12Calibration plot of observed versus estimated (O/E) risk within deciles of the CPRD cohort

Source: SCORE2-OP working group 2021 ¹

Table 10Predicted and observed lifetime risk of cardiovascular disease by 10th of predicted lifetime risk in the validation cohort of 1,267,159 patients

Model decile	Mean lifetime risk (%)		Ratio of predicted to observed
Model decile	Predicted	Observed	Ratio of predicted to observed
Women:
1	18.5	22.4	0.83
2	21.3	25.9	0.82
3	22.9	27.3	0.84
4	24.4	28.5	0.86
5	26.0	29.4	0.88
6	27.8	31.9	0.87
7	30.2	34.8	0.87
8	33.7	36.8	0.92
9	39.5	41.3	0.96
10	51.9	50.8	1.02
Men:
1	22.5	25	0.90
2	27.2	32.1	0.85
3	29.8	34.9	0.85
4	32.0	37.3	0.86
5	34.2	39.3	0.87
6	36.6	42.1	0.87
7	39.5	44.9	0.88
8	43.5	47.5	0.92
9	49.9	51	0.98
10	64.4	63.7	1.01

Figure 13External calibration of predicted vs. observed 10-year risk using the LIFE-CVD model

Source: Jaspers 2020 ¹¹

Table 11Health economic evidence profile: risk assessment tools

Study

Applicability

Limitations

Other comments

Incremental cost

Incremental effects

Cost effectiveness

Uncertainty

Zomer 2017²⁴ (UK)

Partially applicable^(a)

Potentially serious limitations^(b)

Patient-level simulation model
Cost-utility analysis (QALYs)
Population: people with SMI and no CVD
Comparators^(c):
1. General population lipid algorithm
2. General population BMI algorithm
3. SMI-specific lipid algorithm
4. SMI-specific BMI algorithm
Time horizon: 10 years

2–1: £11

3–1: £5

4–1: −£7^(d)

2–1: −0.002

3–1: −0.001

4–1: 0.002

SMI-specific BMI algorithm is dominant (lower costs and higher QALYs than all other options)

Probability cost effective (£20K/30K threshold):

~22%/~22%
~17%/~17%
~13%/~13%
~43%/~43%

In some deterministic sensitivity analyses the general population lipid algorithm became the most cost-effective option (when statin compliance was reduced to 50%, when utility in the SMI population was reduced, and in some of the scenarios when costs were doubled).

: Abbreviations: BMI = body mass index; CVD = cardiovascular disease; ICER = incremental cost-effectiveness ratio; QALY = quality-adjusted life years; SMI = serious mental illness.

(a): Doesn’t include comparison to general population algorithms used in current practice (general population algorithms were UK adjusted Framingham equations which don’t meet the update review protocol [QRISK2 recommended in the 2014 CG181 update over Framingham-based assessments]). 2012/13 cost year and some based on resource use before 2007 may not reflect current NHS context. Cost of blood test excluded for BMI-based algorithms but would be required in patients starting statin therapy so can monitor impact of treatment.

(b): The PRIMROSE SMI-specific risk tool has not been externally validated (see clinical review). Time horizon of 10 years may not fully reflect the impact on costs and QALYs.

(c): General population algorithms were UK adjusted Framingham (D’Agostino 2008) – not included in update review protocol; SMI-specific algorithms were PRIMROSE. For all groups, people assessed as >10% 10-year CV risk receive and statin treatment (20mg atorvastatin). People already on statin therapy (in THIN) remained on treatment irrespective of risk level. A ’No risk assessment’ group without additional statin treatment was also estimated but is not presented here as did not meet the protocol.

(d): 2012/13 costs. Cost components incorporated: risk assessment (GP time and blood tests); statins; CVD event costs (first and subsequent years).

Table 26Studies excluded from the clinical review

Study	Exclusion reason(s)
Abeles Robin D, Mullish Benjamin H, Forlano Roberta et al (2019) Derivation and validation of a cardiovascular risk score for prediction of major acute cardiovascular events in nonalcoholic fatty liver disease; the importance of an elevated mean platelet volume. Alimentary pharmacology & therapeutics 49(8): 1077–1085 [PMC free article: PMC6519040] [PubMed: 30836450]	- Analysis not relevant to this protocol: prediction of 1-year risk only
Albarqouni Loai, Doust Jennifer A, Magliano Dianna et al (2019) External validation and comparison of four cardiovascular risk prediction models with data from the Australian Diabetes, Obesity and Lifestyle study. The Medical journal of Australia 210(4): 161–167 [PubMed: 30656697]	- Population not relevant to this review protocol: Australia
Alemao Evo, Cawston Helene, Bourhis Francois et al (2017) Comparison of cardiovascular risk algorithms in patients with vs without rheumatoid arthritis and the role of C-reactive protein in predicting cardiovascular outcomes in rheumatoid arthritis. Rheumatology (Oxford, England) 56(5): 777–786 [PMC free article: PMC8344293] [PubMed: 28087832]	- Analysis not relevant to this protocol: Prediction of 5 and 3-year risk only
Arts E E A, Popa C D, Den Broeder A A et al (2016) Prediction of cardiovascular risk in rheumatoid arthritis: performance of original and adapted SCORE algorithms. Annals of the rheumatic diseases 75(4): 674–80 [PubMed: 25691119]	- Study does not contain a risk tool relevant to this review protocol: SCORE - Population not relevant to this review protocol: Netherlands
Arts E E A, Popa C, Den Broeder A A et al (2015) Performance of four current risk algorithms in predicting cardiovascular events in patients with early rheumatoid arthritis. Annals of the rheumatic diseases 74(4): 668–74 [PubMed: 24389293]	- Population not relevant to this review protocol: Netherlands
Ashraf Tariq, Mengal Muhammad Naeem, Muhammad Atif Sher et al (2020) Ten years risk assessment of atherosclerotic cardiovascular disease using Astro-CHARM and pooled cohort equation in a south Asian subpopulation. BMC public health 20(1): 403 [PMC free article: PMC7099772] [PubMed: 32220240]	- Population not relevant to this review protocol: Pakistan - Study design not relevant to this review protocol: cross sectional
Aspelund Thor, Thorgeirsson Gudmundur, Sigurdsson Gunnar et al (2007) Estimation of 10-year risk of fatal cardiovascular disease and coronary heart disease in Iceland with results comparable with those of the Systematic Coronary Risk Evaluation project. European journal of cardiovascular prevention and rehabilitation: official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology 14(6): 761–8 [PubMed: 18043296]	- Population not relevant to this review protocol: Iceland
Bae Jae Hyun, Moon Min Kyong, Oh Sohee et al (2020) Validation of Risk Prediction Models for Atherosclerotic Cardiovascular Disease in a Prospective Korean Community-Based Cohort. Diabetes & metabolism journal 44(3): 458–469 [PMC free article: PMC7332332] [PubMed: 31950769]	- Population not relevant to this review protocol: Korea
Bell Katy J L, White Sam, Hassan Omar et al (2022) Evaluation of the Incremental Value of a Coronary Artery Calcium Score Beyond Traditional Cardiovascular Risk Assessment: A Systematic Review and Meta-analysis. JAMA internal medicine 182(6): 634–642 [PMC free article: PMC9039826] [PubMed: 35467692]	- Population not relevant to this review protocol: US, Netherlands, Germany and South Korea
Bertomeu-Gonzalez Vicente, Maldonado Soriano, Cristina Bleda-Cano, Jesus et al (2019) Predictive validity of the risk SCORE model in a Mediterranean population with dyslipidemia. Atherosclerosis 290: 80–86 [PubMed: 31593904]	- Population not relevant to this review protocol: Spain - Study does not contain a risk tool relevant to this review protocol: SCORE
Cacciapaglia Fabio, Fornaro Marco, Venerito Vincenzo et al (2020) Cardiovascular risk estimation with 5 different algorithms before and after 5 years of bDMARD treatment in rheumatoid arthritis. European journal of clinical investigation 50(12): e13343 [PubMed: 32654116]	- Population not relevant to this review protocol: Italy - Study design not relevant to this review protocol:
Campos-Staffico Alessandra M, Cordwin David, Murthy Venkatesh L et al (2021) Comparative performance of the two pooled cohort equations for predicting atherosclerotic cardiovascular disease. Atherosclerosis 334: 23–29 [PMC free article: PMC8527545] [PubMed: 34461391]	- Population not relevant to this review protocol: USA
Cauwenberghs Nicholas, Hedman Kristofer, Kobayashi Yukari et al (2019) The 2013 ACC/AHA risk score and subclinical cardiac remodeling and dysfunction: Complementary in cardiovascular disease prediction. International journal of cardiology 297: 67–74 [PubMed: 31623873]	- Population not relevant to this review protocol: Belgium
Mora Cedeno, Santiago Goicoechea, Marian Torres, Esther et al (2017) Cardiovascular risk prediction in chronic kidney disease patients. Nefrologia: publicacion oficial de la Sociedad Espanola Nefrologia 37(3): 293–300 [PubMed: 28495396]	- Population not relevant to this review protocol: Spain
Chew K W, Bhattacharya D, Horwich T B et al (2017) Performance of the Pooled Cohort atherosclerotic cardiovascular disease risk score in hepatitis C virus-infected persons. Journal of viral hepatitis 24(10): 814–822 [PMC free article: PMC5589479] [PubMed: 28273386]	- Population not relevant to this review protocol: USA
Chia Yook Chin; Lim Hooi Min; Ching Siew Mooi (2014) Validation of the pooled cohort risk score in an Asian population - a retrospective cohort study. BMC cardiovascular disorders 14: 163 [PMC free article: PMC4246627] [PubMed: 25410585]	- Population not relevant to this review protocol: Malaysia
Chlabicz Malgorzata, Jamiolkowski Jacek, Laguna Wojciech et al (2021) A Similar Lifetime CV Risk and a Similar Cardiometabolic Profile in the Moderate and High Cardiovascular Risk Populations: A Population-Based Study. Journal of clinical medicine 10(8) [PMC free article: PMC8069041] [PubMed: 33918620]	- Study design not relevant to this review protocol: cross sectional study with no calibration or discrimination data - Population not relevant to this review protocol: Poland
Clark Christopher E, Warren Fiona C, Boddy Kate et al (2021) Associations Between Systolic Interarm Differences in Blood Pressure and Cardiovascular Disease Outcomes and Mortality: Individual Participant Data Meta-Analysis, Development and Validation of a Prognostic Algorithm: The INTERPRESS-IPD Collaboration. Hypertension (Dallas, Tex.: 1979) 77(2): 650–661 [PMC free article: PMC7803446] [PubMed: 33342236]	- Population not relevant to this review protocol: Validation not in a UK population (USA, China, Spain and Netherlands); 18.3% had established CVD - Study does not contain a risk tool relevant to this review protocol: Validation only for prediction of fatal events
Colaco Keith, Ocampo Vanessa, Ayala Ana Patricia et al (2020) Predictive Utility of Cardiovascular Risk Prediction Algorithms in Inflammatory Rheumatic Diseases: A Systematic Review. The Journal of rheumatology 47(6): 928–938 [PubMed: 31416923]	- Systematic review used as source of primary studies
Colantonio Lisandro D, Richman Joshua S, Carson April P et al (2017) Performance of the Atherosclerotic Cardiovascular Disease Pooled Cohort Risk Equations by Social Deprivation Status. Journal of the American Heart Association 6(3) [PMC free article: PMC5524046] [PubMed: 28314800]	- Population not relevant to this review protocol: USA - Analysis not relevant to this protocol: prediction of 5-year risk only
Collins Gary S and Altman Douglas G (2009) An independent external validation and evaluation of QRISK cardiovascular risk prediction: a prospective open cohort study. BMJ (Clinical research ed.) 339: b2584 [PMC free article: PMC2714681] [PubMed: 19584409]	- Study does not contain a risk tool relevant to this review protocol: QRISK
Collins Gary S and Altman Douglas G (2010) An independent and external validation of QRISK2 cardiovascular disease risk score: a prospective open cohort study. BMJ (Clinical research ed.) 340: c2442 [PMC free article: PMC2869403] [PubMed: 20466793]	- Validation cohort overlaps with an included study with also reports on QRISK2–2008 using data from THIN, and includes a larger, more-applicable sample
Conroy R M, Pyorala K, Fitzgerald A P et al (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. European heart journal 24(11): 987–1003 [PubMed: 12788299]	- Study does not contain a risk tool relevant to this review protocol: SCORE: fatal events only
Cooney Marie Therese, Selmer Randi, Lindman Anja et al (2016) Cardiovascular risk estimation in older persons: SCORE O.P. European journal of preventive cardiology 23(10): 1093–103 [PubMed: 26040999]	- Study does not contain a risk tool relevant to this review protocol: SCORE-OP
Corrales Alfonso, Vegas-Revenga Nuria, Atienza-Mateo Belen et al (2021) Combined use of QRISK3 and SCORE as predictors of carotid plaques in patients with rheumatoid arthritis. Rheumatology (Oxford, England) 60(6): 2801–2807 [PubMed: 33249513]	- Study design not relevant to this review protocol: diagnostic accuracy for carotid plaques
Courand Pierre-Yves, Lenoir Jerome, Grandjean Adrien et al (2022) SCORE underestimates cardiovascular mortality in hypertension: insight from the OLD-HTA and NEW-HTA Lyon cohorts. European journal of preventive cardiology 29(1): 136–143 [PubMed: 33580796]	- Population not relevant to this review protocol: France - Study does not contain a risk tool relevant to this review protocol: SCORE: fatal events only
Crowson Cynthia S, Gabriel Sherine E, Semb Anne Grete et al (2017) Rheumatoid arthritis-specific cardiovascular risk scores are not superior to general risk scores: a validation analysis of patients from seven countries. Rheumatology (Oxford, England) 56(7): 1102–1110 [PMC free article: PMC5850220] [PubMed: 28339992]	- Population not relevant to this review protocol: Includes 7 countries (UK, Norway, Netherlands, USA, South Africa, Canada and Mexico) and proportions are unclear.
Dalton Jarrod E, Perzynski Adam T, Zidar David A et al (2017) Accuracy of Cardiovascular Risk Prediction Varies by Neighborhood Socioeconomic Position: A Retrospective Cohort Study. Annals of internal medicine 167(7): 456–464 [PMC free article: PMC6435027] [PubMed: 28847012]	- Population not relevant to this review protocol: USA - Study design not relevant to this review protocol: prediction of 5-year risk only
De Bacquer Dirk, De Backer Guy (2010) Predictive ability of the SCORE Belgium risk chart for cardiovascular mortality. International journal of cardiology 143(3): 385–90 [PubMed: 19386372]	- Population not relevant to this review protocol: Belgium - Study does not contain a risk tool relevant to this review protocol: SCORE: fatal events only
de la Iglesia Beatriz, Potter John F, Poulter Neil R et al (2011) Performance of the ASSIGN cardiovascular disease risk score on a UK cohort of patients from general practice. Heart (British Cardiac Society) 97(6): 491–9 [PubMed: 21097820]	- Study does not contain a risk tool relevant to this review protocol: ASSIGN and Framingham
De Las Heras Gala T., Geisel M.H., Peters A. et al (2016) Recalibration of the ACC/AHA risk score in two population-based German cohorts. PLoS ONE 11(10): e0164688 [PMC free article: PMC5061315] [PubMed: 27732641]	- Population not relevant to this review protocol: Germany
DeFilippis Andrew P, Young Rebekah, Carrubba Christopher J et al (2015) An analysis of calibration and discrimination among multiple cardiovascular risk scores in a modern multiethnic cohort. Annals of internal medicine 162(4): 266–75 [PMC free article: PMC4414494] [PubMed: 25686167]	- Population not relevant to this review protocol: USA
DeFilippis Andrew Paul, Young Rebekah, McEvoy John W et al (2017) Risk score overestimation: the impact of individual cardiovascular risk factors and preventive therapies on the performance of the American Heart Association-American College of Cardiology-Atherosclerotic Cardiovascular Disease risk score in a modern multi-ethnic cohort. European heart journal 38(8): 598–608 [PMC free article: PMC5837662] [PubMed: 27436865]	- Population not relevant to this review protocol: USA
Di Battista, Marco, Tani, Chiara, Elefante, Elena et al (2020) Framingham, ACC/AHA or QRISK3: which is the best in systemic lupus erythematosus cardiovascular risk estimation?. Clinical and experimental rheumatology 38(4): 602–608 [PubMed: 31694741]	- Population not relevant to this review protocol: Italy
Drosos George C, Konstantonis George, Sfikakis Petros P et al (2020) Underperformance of clinical risk scores in identifying vascular ultrasound-based high cardiovascular risk in systemic lupus erythematosus. European journal of preventive cardiology: 2047487320906650 [PubMed: 32122200]	- Population not relevant to this review protocol: Greece
Edwards N., Langford-Smith A.W. W., Parker B.J. et al (2018) QRISK3 improves detection of cardiovascular disease risk in patients with systemic lupus erythematosus. Lupus Science and Medicine 5(1): e000272 [PMC free article: PMC6109811] [PubMed: 30167314]	- Data not reported in an extractable format or a format that can be analysed no accuracy data - Study design not relevant to this review protocol: cross sectional
Emdin Connor A, Khera Amit V, Natarajan Pradeep et al (2017) Evaluation of the Pooled Cohort Equations for Prediction of Cardiovascular Risk in a Contemporary Prospective Cohort. The American journal of cardiology 119(6): 881–885 [PubMed: 28061997]	- Population not relevant to this review protocol: USA
Fan W., Wong D.N., Li X. et al (2020) Cardiovascular Risk Prediction in Diabetes from Machine Learning: The ACCORD Study. Circulation 142(suppl3)	- Conference abstract
Fausto S., Marina C., Marco D.C. et al (2018) The expanded risk score in rheumatoid arthritis (ERS-RA): Performance of a disease-specific calculator in comparison with the traditional prediction scores in the assessment of the 10-year risk of cardiovascular disease in patients with rheumatoid arthritis. Swiss Medical Weekly 148(3334): w14656 [PubMed: 30141517]	- Population not relevant to this review protocol: Italy
Giavarina Davide, Barzon Elena, Cigolini Massimo et al (2007) Comparison of methods to identify individuals at increased risk of cardiovascular disease in Italian cohorts. Nutrition, metabolism, and cardiovascular diseases: NMCD 17(4): 311–8 [PubMed: 17434054]	- Study design not relevant to this review protocol: cross-sectional
Gidlow C.J., Ellis N.J., Cowap L. et al (2021) Cardiovascular disease risk communication in nhs health checks using qrisk 2 and jbs3 risk calculators: The rico qualitative and quantitative study. Health Technology Assessment 25(50): vii-102 [PubMed: 34427556]	- Study design not relevant to this review protocol: qualitative study with no predictive accuracy data
Goh Louise Gek Huang; Welborn Timothy Alexander; Dhaliwal Satvinder Singh (2014) Independent external validation of cardiovascular disease mortality in women utilising Framingham and SCORE risk models: a mortality follow-up study. BMC women's health 14: 118 [PMC free article: PMC4181599] [PubMed: 25255986]	- Population not relevant to this review protocol: Australia - Study does not contain a risk tool relevant to this review protocol SCORE and Framingham: fatal events only
Gopal Dipesh P and Usher-Smith Juliet A (2016) Cardiovascular risk models for South Asian populations: a systematic review. International journal of public health 61(5): 525–34 [PubMed: 26361963]	- Systematic review used as source of primary studies
Grammer Tanja B, Dressel Alexander, Gergei Ingrid et al (2019) Cardiovascular risk algorithms in primary care: Results from the DETECT study. Scientific reports 9(1): 1101 [PMC free article: PMC6355969] [PubMed: 30705337]	- Population not relevant to this review protocol: Germany
Graversen Peter, Abildstrom Steen Z, Jespersen Lasse et al (2016) Cardiovascular risk prediction: Can Systematic Coronary Risk Evaluation (SCORE) be improved by adding simple risk markers? Results from the Copenhagen City Heart Study. European journal of preventive cardiology 23(14): 1546–56 [PubMed: 26976846]	- Population not relevant to this review protocol: Denmark - Study does not contain a risk tool relevant to this review protocol SCORE
Hageman Steven H J, McKay Ailsa J, Ueda Peter et al (2022) Estimation of recurrent atherosclerotic cardiovascular event risk in patients with established cardiovascular disease: the updated SMART2 algorithm. European heart journal 43(18): 1715–1727 [PMC free article: PMC9312860] [PubMed: 35165703]	- Population not relevant to this review protocol: Secondary prevention - Study does not contain a risk tool relevant to this review protocol SMART2
Hippisley-Cox J, Coupland C, Vinogradova Y et al (2008) Performance of the QRISK cardiovascular risk prediction algorithm in an independent UK sample of patients from general practice: a validation study. Heart (British Cardiac Society) 94(1): 34–9 [PubMed: 17916661]	- Study does not contain a risk tool relevant to this review protocol QRISK and Framingham
Hippisley-Cox Julia, Coupland Carol, Vinogradova Yana et al (2007) Derivation and validation of QRISK, a new cardiovascular disease risk score for the United Kingdom: prospective open cohort study. BMJ (Clinical research ed.) 335(7611): 136 [PMC free article: PMC1925200] [PubMed: 17615182]	- Study does not contain a risk tool relevant to this review protocol QRISK, Framingham and ASSIGN
Johns I., Moschonas K.E., Medina J. et al (2018) Risk classification in primary prevention of CVD according to QRISK2 and JBS3 - - heart age', and prevalence of elevated high-sensitivity C reactive protein in the UK cohort of the EURIKA study. Open Heart 5(2): e000849 [PMC free article: PMC6269641] [PubMed: 30564373]	- Study design not relevant to this review protocol: cross sectional and no predictive accuracy outcome data
Jorstad Harald T, Colkesen Ersen B, Minneboo Madelon et al (2015) The Systematic COronary Risk Evaluation (SCORE) in a large UK population: 10-year follow-up in the EPIC-Norfolk prospective population study. European journal of preventive cardiology 22(1): 119–26 [PubMed: 24002125]	- Study does not contain a risk tool relevant to this review protocol SCORE: fatal events only
Jung K.J., Jang Y., Oh D.J. et al (2015) The ACC/AHA 2013 pooled cohort equations compared to a Korean Risk Prediction Model for atherosclerotic cardiovascular disease. Atherosclerosis 242(1): 367–375 [PubMed: 26255683]	- Population not relevant to this review protocol: Korea
Karmali KN, Persell SD, Perel P et al (2017) Risk scoring for the primary prevention of cardiovascular disease. Cochrane Database of Systematic Reviews [PMC free article: PMC6464686] [PubMed: 28290160]	- Systematic review used as source of primary studies none of the included studies used a tool specified in the review protocol
Karmali Kunal N, Goff David C Jr, Ning, Hongyan et al (2014) A systematic examination of the 2013 ACC/AHA pooled cohort risk assessment tool for atherosclerotic cardiovascular disease. Journal of the American College of Cardiology 64(10): 959–68 [PubMed: 25190228]	- Study design not relevant to this review protocol: no predictive accuracy data reported
Khera Rohan, Pandey Ambarish, Ayers Colby R et al (2020) Performance of the Pooled Cohort Equations to Estimate Atherosclerotic Cardiovascular Disease Risk by Body Mass Index. JAMA network open 3(10): e2023242 [PMC free article: PMC7596579] [PubMed: 33119108]	- Population not relevant to this review protocol: USA
Kim Tae Hyuk, Choi Hoon Sung, Bae Ji Cheol et al (2014) Subclinical hypothyroidism in addition to common risk scores for prediction of cardiovascular disease: a 10-year community-based cohort study. European journal of endocrinology 171(5): 649–57 [PubMed: 25184283]	- Population not relevant to this review protocol: Korea
Kuragaichi Takashi, Kataoka Yuki, Miyakoshi Chisato et al (2019) External validation of pooled cohort equations using systolic blood pressure intervention trial data. BMC research notes 12(1): 271 [PMC free article: PMC6518641] [PubMed: 31088530]	- Population not relevant to this review protocol: USA
Lengele Jean-Philippe, Vinck Wouter J, De Plaen Jean-Francois et al (2007) Cardiovascular risk assessment in hypertensive patients: major discrepancy according to ESH and SCORE strategies. Journal of hypertension 25(4): 757–62 [PubMed: 17351366]	- Study design not relevant to this review protocol: cross sectional and no predictive accuracy outcome data
Li Yan, Sperrin Matthew, Ashcroft Darren M et al (2020) Consistency of variety of machine learning and statistical models in predicting clinical risks of individual patients: longitudinal cohort study using cardiovascular disease as exemplar. BMJ (Clinical research ed.) 371: m3919 [PMC free article: PMC7610202] [PubMed: 33148619]	- Data not reported in an extractable format or a format that can be analysed Confidence intervals not reported for accuracy data - Validation cohort overlaps with an included study of more direct relevance
Loprinzi P D (2016) Predictive validity of the ACC/AHA pooled cohort equations in predicting cancer-specific mortality in a National Prospective Cohort Study of Adults in the United States. International journal of clinical practice 70(8): 691–5 [PubMed: 27384232]	- Study does not contain a risk tool relevant to this review protocol: cancer-specific mortality
Loprinzi Paul D and Addoh Ovuokerie (2016) Predictive Validity of the American College of Cardiology/American Heart Association Pooled Cohort Equations in Predicting All-Cause and Cardiovascular Disease-Specific Mortality in a National Prospective Cohort Study of Adults in the United States. Mayo Clinic proceedings 91(6): 763–9 [PubMed: 27180122]	- Population not relevant to this review protocol: USA
Lucaroni Francesca, Modica Cicciarella, Domenico Macino, Mattia et al (2019) Can risk be predicted? An umbrella systematic review of current risk prediction models for cardiovascular diseases, diabetes and hypertension. BMJ open 9(12): e030234 [PMC free article: PMC6937066] [PubMed: 31862737]	- Systematic review used as source of primary studies
Mancini G.B. J. and Ryomoto A. (2014) Comparison of cardiovascular risk assessment algorithms to determine eligibility for statin therapy: Implications for practice in Canada. Canadian Journal of Cardiology 30(6): 661–666 [PubMed: 24882538]	- Study design not relevant to this review protocol: no predictive accuracy data
Mansoor Hend, Jo Ara, Beau De Rochars V Madsen et al (2019) Novel Self-Report Tool for Cardiovascular Risk Assessment. Journal of the American Heart Association 8(24): e014123 [PMC free article: PMC6951080] [PubMed: 31818214]	- Population not relevant to this review protocol: USA
Matsushita K., Jassal S.K., Sang Y. et al (2020) Incorporating kidney disease measures into cardiovascular risk prediction: Development and validation in 9 million adults from 72 datasets. EClinicalMedicine 27: 100552 [PMC free article: PMC7599294] [PubMed: 33150324]	- Study design not relevant to this review protocol: only estimates 5-year risk for the UK cohort
McKay Ailsa J, Gunn Laura H, Ference Brian A et al (2022) Is the SMART risk prediction model ready for real-world implementation? A validation study in a routine care setting of approximately 380 000 individuals. European journal of preventive cardiology 29(4): 654–663 [PubMed: 34160035]	- Population not relevant to this review protocol: secondary prevention
Mora Samia, Wenger Nanette K, Cook Nancy R et al (2018) Evaluation of the Pooled Cohort Risk Equations for Cardiovascular Risk Prediction in a Multiethnic Cohort From the Women's Health Initiative. JAMA internal medicine 178(9): 1231–1240 [PMC free article: PMC6142964] [PubMed: 30039172]	- Population not relevant to this review protocol: USA
Moral Pelaez I., Brotons Cuixart C., Fernandez Valverde D. et al (2021) External validation of the European and American equations for calculating cardiovascular risk in a Spanish working population. Revista Clinica Espanola 221(10): 561–568 [PubMed: 34147422]	- Population not relevant to this review protocol: Spain
Mosepele M., Hemphill L.C., Palai T. et al (2017) Cardiovascular disease risk prediction by the American College of Cardiology (ACC)/American Heart Association (AHA) Atherosclerotic Cardiovascular Disease (ASCVD) risk score among HIV-infected patients in sub-Saharan Africa. PLoS ONE 12(2): e0172897 [PMC free article: PMC5325544] [PubMed: 28235058]	- Population not relevant to this review protocol: Botswana - Study design not relevant to this review protocol: Cross sectional and no predictive accuracy outcome data
Motamed N, Ajdarkosh H, Perumal D et al (2021) Comparison of risk assessment tools for cardiovascular diseases: results of an Iranian cohort study. Public health 200: 116–123 [PubMed: 34717165]	- Population not relevant to this review protocol: Iran
Nanna Michael G, Peterson Eric D, Wojdyla Daniel et al (2020) The Accuracy of Cardiovascular Pooled Cohort Risk Estimates in U.S. Older Adults. Journal of general internal medicine 35(6): 1701–1708 [PMC free article: PMC7280419] [PubMed: 31667745]	- Population not relevant to this review protocol: USA
Navarini Luca, Caso Francesco, Costa Luisa et al (2020) Cardiovascular Risk Prediction in Ankylosing Spondylitis: From Traditional Scores to Machine Learning Assessment. Rheumatology and therapy 7(4): 867–882 [PMC free article: PMC7695785] [PubMed: 32939675]	- Population not relevant to this review protocol: Italy
Navarini Luca, Margiotta Domenico Paolo Emanuele, Caso Francesco et al (2018) Performances of five risk algorithms in predicting cardiovascular events in patients with Psoriatic Arthritis: An Italian bicentric study. PloS one 13(10): e0205506 [PMC free article: PMC6181379] [PubMed: 30308025]	- Population not relevant to this review protocol: Italy
Nguyen Q.D., Odden M.C., Peralta C.A. et al (2020) Predicting risk of atherosclerotic cardiovascular disease using pooled cohort equations in older adults with frailty, multimorbidity, and competing risks. Journal of the American Heart Association 9(18): e016003 [PMC free article: PMC7727000] [PubMed: 32875939]	- Population not relevant to this review protocol: USA
Ozen Gulsen, Sunbul Murat, Atagunduz Pamir et al (2016) The 2013 ACC/AHA 10-year atherosclerotic cardiovascular disease risk index is better than SCORE and QRisk II in rheumatoid arthritis: is it enough?. Rheumatology (Oxford, England) 55(3): 513–22 [PubMed: 26472565]	- Study design not relevant to this review protocol: Cross sectional
Pandey Ambarish, Mehta Anurag, Paluch Amanda et al (2021) Performance of the American Heart Association/American College of Cardiology Pooled Cohort Equations to Estimate Atherosclerotic Cardiovascular Disease Risk by Self-reported Physical Activity Levels. JAMA cardiology 6(6): 690–696 [PMC free article: PMC8082430] [PubMed: 33909016]	- Population not relevant to this review protocol: USA
Pate Alexander, Emsley Richard, Ashcroft Darren M et al (2019) The uncertainty with using risk prediction models for individual decision making: an exemplar cohort study examining the prediction of cardiovascular disease in English primary care. BMC medicine 17(1): 134 [PMC free article: PMC6636064] [PubMed: 31311543]	- Study does not contain a risk tool relevant to this review protocol: Unvalidated models based on QRISK2 and QRISK3
Patel Aniruddh P, Wang Minxian, Kartoun Uri et al (2021) Quantifying and Understanding the Higher Risk of Atherosclerotic Cardiovascular Disease Among South Asian Individuals: Results From the UK Biobank Prospective Cohort Study. Circulation 144(6): 410–422 [PMC free article: PMC8355171] [PubMed: 34247495]	- Data not reported in an extractable format or a format that can be analysed
Pennells Lisa, Kaptoge Stephen, Wood Angela et al (2019) Equalization of four cardiovascular risk algorithms after systematic recalibration: individual-participant meta-analysis of 86 prospective studies. European heart journal 40(7): 621–631 [PMC free article: PMC6374687] [PubMed: 30476079]	- Population not relevant to this review protocol: Mixed cohorts: <80% UK-based
Piccininni Marco, Rohmann Jessica L, Huscher Dorte et al (2020) Performance of risk prediction scores for cardiovascular mortality in older persons: External validation of the SCORE OP and appraisal. PloS one 15(4): e0231097 [PMC free article: PMC7144969] [PubMed: 32271825]	- Population not relevant to this review protocol: Germany - Study does not contain a risk tool relevant to this review protocol: SCORE-OP
Plante T.B., Juraschek S.P., Zakai N.A. et al (2019) Pooled Cohort Equation performance in primary and secondary prevention subgroups of the systolicblood pressure intervention trial (SPRINT). Circulation 139(supplement1) [PMC free article: PMC7240131] [PubMed: 31575423]	- Conference abstract
Prausmuller Suriya, Resl Michael, Arfsten Henrike et al (2021) Performance of the recommended ESC/EASD cardiovascular risk stratification model in comparison to SCORE and NT-proBNP as a single biomarker for risk prediction in type 2 diabetes mellitus. Cardiovascular diabetology 20(1): 34 [PMC free article: PMC7856811] [PubMed: 33530999]	- Population not relevant to this review protocol: Austria - Study does not contain a risk tool relevant to this review protocol: SCORE
Preiss David and Kristensen Soren L (2015) The new pooled cohort equations risk calculator. The Canadian journal of cardiology 31(5): 613–9 [PubMed: 25843167]	- Review article but not a systematic review
Raiko Juho R H, Magnussen Costan G, Kivimaki Mika et al (2010) Cardiovascular risk scores in the prediction of subclinical atherosclerosis in young adults: evidence from the cardiovascular risk in a young Finns study. European journal of cardiovascular prevention and rehabilitation: official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology 17(5): 549–55 [PMC free article: PMC2907448] [PubMed: 20354441]	- Population not relevant to this review protocol: Finland - Study does not contain a risk tool relevant to this review protocol:
Ramsay Sheena E, Morris Richard W, Whincup Peter H et al (2011) Prediction of coronary heart disease risk by Framingham and SCORE risk assessments varies by socioeconomic position: results from a study in British men. European journal of cardiovascular prevention and rehabilitation: official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology 18(2): 186–93 [PubMed: 21450664]	- Study does not contain a risk tool relevant to this review protocol: Framingham and SCORE
Rana Jamal S, Tabada Grace H, Solomon Matthew D et al (2016) Accuracy of the Atherosclerotic Cardiovascular Risk Equation in a Large Contemporary, Multiethnic Population. Journal of the American College of Cardiology 67(18): 2118–2130 [PMC free article: PMC5097466] [PubMed: 27151343]	- Population not relevant to this review protocol: USA
Read Stephanie H, van Diepen Merel, Colhoun Helen M et al (2018) Performance of Cardiovascular Disease Risk Scores in People Diagnosed With Type 2 Diabetes: External Validation Using Data From the National Scottish Diabetes Register. Diabetes care 41(9): 2010–2018 [PubMed: 30002197]	- Analysis not relevant to this review protocol: 5-year risk estimate only
Romanens Michel, Adams Ansgar, Sudano Isabella et al (2021) Prediction of cardiovascular events with traditional risk equations and total plaque area of carotid atherosclerosis: The Arteris Cardiovascular Outcome (ARCO) cohort study. Preventive medicine 147: 106525 [PubMed: 33745952]	- Population not relevant to this review protocol: Switzerland and Germany - Study does not contain a risk tool relevant to this review protocol:
Saar Aet, Lall Kristi, Alver Maris et al (2019) Estimating the performance of three cardiovascular disease risk scores: the Estonian Biobank cohort study. Journal of epidemiology and community health 73(3): 272–277 [PubMed: 30635435]	- Population not relevant to this review protocol: Estonia
Santos-Ferreira Catia, Baptista Rui, Oliveira-Santos Manuel et al (2020) A 10- and 15-year performance analysis of ESC/EAS and ACC/AHA cardiovascular risk scores in a Southern European cohort. BMC cardiovascular disorders 20(1): 301 [PMC free article: PMC7304198] [PubMed: 32560700]	- Population not relevant to this review protocol: Portugal
Sawano Mitsuaki, Kohsaka Shun, Okamura Tomonori et al (2016) Validation of the european SCORE risk chart in the healthy middle-aged Japanese. Atherosclerosis 252: 116–121 [PubMed: 27521900]	- Population not relevant to this review protocol: Japan - Study does not contain a risk tool relevant to this review protocol:
Schiborn Catarina, Kuhn Tilman, Muhlenbruch Kristin et al (2021) A newly developed and externally validated non-clinical score accurately predicts 10-year cardiovascular disease risk in the general adult population. Scientific reports 11(1): 19609 [PMC free article: PMC8490374] [PubMed: 34608230]	- Population not relevant to this review protocol: Germany
Schulz C.-A., Mavarani L., Reinsch N. et al (2021) Prediction of future cardiovascular events by Framingham, SCORE and asCVD risk scores is less accurate in HIV-positive individuals from the HIV-HEART Study compared with the general population. HIV Medicine 22(8): 732–741 [PubMed: 34028959]	- Population not relevant to this review protocol: Germany
Siontis George C M, Tzoulaki Ioanna, Siontis Konstantinos C et al (2012) Comparisons of established risk prediction models for cardiovascular disease: systematic review. BMJ (Clinical research ed.) 344: e3318 [PubMed: 22628003]	- Systematic review used as source of primary studies
Sivakumaran J., Harvey P., Omar A. et al (2021) Assessment of cardiovascular risk tools as predictors of cardiovascular disease events in systemic lupus erythematosus. Lupus Science and Medicine 8(1): e000448 [PMC free article: PMC8162102] [PubMed: 34045359]	- Population not relevant to this review protocol: Canada
Tang Xun, Zhang Dudan, He Liu et al (2019) Performance of atherosclerotic cardiovascular risk prediction models in a rural Northern Chinese population: Results from the Fangshan Cohort Study. American heart journal 211: 34–44 [PubMed: 30831332]	- Population not relevant to this review protocol: China
Tolunay Hatice and Kurmus Ozge (2016) Comparison of coronary risk scoring systems to predict the severity of coronary artery disease using the SYNTAX score. Cardiology journal 23(1): 51–6 [PubMed: 26503075]	- Population not relevant to this review protocol: Turkey - Study does not contain a risk tool relevant to this review protocol:
Tralhao Antonio, Ferreira Antonio M, Goncalves Pedro de Araujo et al (2016) Accuracy of Pooled-Cohort Equation and SCORE cardiovascular risk calculators to identify individuals with high coronary atherosclerotic burden - implications for statin treatment. Coronary artery disease 27(7): 573–9 [PubMed: 27285280]	- Population not relevant to this review protocol: Portugal - Analysis not relevant to this review protocol: predicting risk of coronary atherosclerotic burden
Triant Virginia A, Perez Jeremiah, Regan Susan et al (2018) Cardiovascular Risk Prediction Functions Underestimate Risk in HIV Infection. Circulation 137(21): 2203–2214 [PMC free article: PMC6157923] [PubMed: 29444987]	- Population not relevant to this review protocol: USA
Ueda Peter, Woodward Mark, Lu Yuan et al (2017) Laboratory-based and office-based risk scores and charts to predict 10-year risk of cardiovascular disease in 182 countries: a pooled analysis of prospective cohorts and health surveys. The lancet. Diabetes & endocrinology 5(3): 196–213 [PMC free article: PMC5354360] [PubMed: 28126460]	- Population not relevant to this review protocol: No accuracy data for UK
van der Heijden Amber A W A, Monica M Ortegon, Niessen Louis W et al (2009) Prediction of coronary heart disease risk in a general, prediabetic, and diabetic population during 10 years of follow-up: accuracy of the Framingham, SCORE, and UKPDS risk functions: The Hoorn Study. Diabetes care 32(11): 2094–8 [PMC free article: PMC2768197] [PubMed: 19875606]	- Population not relevant to this review protocol: Netherlands
van Dis Ineke, Kromhout Daan, Geleijnse Johanna M et al (2010) Evaluation of cardiovascular risk predicted by different SCORE equations: the Netherlands as an example. European journal of cardiovascular prevention and rehabilitation: official journal of the European Society of Cardiology, Working Groups on Epidemiology & Prevention and Cardiac Rehabilitation and Exercise Physiology 17(2): 244–9 [PubMed: 20195155]	- Population not relevant to this review protocol: Netherlands - Analysis not relevant to this review protocol: Fatal events only
Vassy Jason L, Lu Bing, Ho Yuk-Lam et al (2020) Estimation of Atherosclerotic Cardiovascular Disease Risk Among Patients in the Veterans Affairs Health Care System. JAMA network open 3(7): e208236 [PMC free article: PMC7361654] [PubMed: 32662843]	- Population not relevant to this review protocol: USA
Vega Alonso A.T., Ordax Diez A., Lozano Alonso J.E. et al (2019) Validation of the SCORE index and SCORE for old people in the Castilla y Leon cardiovascular disease risk cohort. Hipertension y Riesgo Vascular 36(4): 184–192 [PubMed: 30926254]	- Study not reported in English - Population not relevant to this review protocol: Spain
Verweij Lotte, Peters Ron J G, Scholte Op Reimer Wilma J M et al (2019) Validation of the Systematic COronary Risk Evaluation - Older Persons (SCORE-OP) in the EPIC-Norfolk prospective population study. International journal of cardiology 293: 226–230 [PubMed: 31324398]	- Study does not contain a risk tool relevant to this review protocol: SCORE-OP (not latest version)
Wang M., Wang W., Liu J. et al (2017) Updating 10-year atherosclerotic cardiovascular risk assessment equation for Chinese adults. Journal of the American College of Cardiology 70(16supplement1): c74–c75	- Conference abstract
Welsh Paul, Hart Carole, Papacosta Olia et al (2016) Prediction of Cardiovascular Disease Risk by Cardiac Biomarkers in 2 United Kingdom Cohort Studies: Does Utility Depend on Risk Thresholds For Treatment?. Hypertension (Dallas, Tex.: 1979) 67(2): 309–15 [PMC free article: PMC4716288] [PubMed: 26667414]	- Study does not contain a risk tool relevant to this review protocol: Model based on QRISK2 and modifications
WHO CVD Risk Chart Working, Group (2019) World Health Organization cardiovascular disease risk charts: revised models to estimate risk in 21 global regions. The Lancet. Global health 7(10): e1332–e1345 [PMC free article: PMC7025029] [PubMed: 31488387]	- Study does not contain a risk tool relevant to this review protocol:
Xu Yu, Li Mian, Qin Guijun et al (2021) Cardiovascular Risk Based on ASCVD and KDIGO Categories in Chinese Adults: A Nationwide, Population-Based, Prospective Cohort Study. Journal of the American Society of Nephrology: JASN [PMC free article: PMC8017537] [PubMed: 33788701]	- Population not relevant to this review protocol: China
Yang Xueli, Li Jianxin, Hu Dongsheng et al (2016) Predicting the 10-Year Risks of Atherosclerotic Cardiovascular Disease in Chinese Population: The China-PAR Project (Prediction for ASCVD Risk in China). Circulation 134(19): 1430–1440 [PubMed: 27682885]	- Population not relevant to this review protocol: China
Yu Zhi, Yang Nicole, Everett Brendan M et al (2018) Impact of Changes in Inflammation on Estimated Ten-Year Cardiovascular Risk in Rheumatoid Arthritis. Arthritis & rheumatology (Hoboken, N.J.) 70(9): 1392–1398 [PMC free article: PMC6115296] [PubMed: 29676517]	- Population not relevant to this review protocol: USA
Zafrir Barak, Saliba Walid, Widder Rachel Shay Li et al (2021) Value of addition of coronary artery calcium to risk scores in the prediction of major cardiovascular events in patients with type 2 diabetes. BMC cardiovascular disorders 21(1): 541 [PMC free article: PMC8590310] [PubMed: 34773970]	- Population not relevant to this review protocol: Israel
Zhu Lisa, Singh Manpreet, Lele Sonia et al (2022) Assessing the validity of QRISK3 in predicting cardiovascular events in systemic lupus erythematosus. Lupus science & medicine 9(1) [PMC free article: PMC8867320] [PubMed: 35193947]	- Population not relevant to this review protocol: USA

Table 27Studies excluded from the health economic review

Reference	Reason for exclusion
None

Table 28List of abbreviations


BiomarCaREConsortium	Biomarker for Cardiovascular Risk Assessment across Europe consortium
DETECT	Dynamic Electronic Tracking and Escalation to reduce Critical Care Transfers
EHR	Electronic Health Records
EPIC-CVD	European Prospective Investigation into Cancer and Nutrition-cardiovascular disease
ERFC	Emerging Risk Factors Collaboration
ESC	European Society of Cardiology
GHS	Gutenberg Health Study
h	hours
HAPIEE	Health, Alcohol and Psychosocial factors In Eastern Europe
HNR	Heinz-Nixdorf Recall
HUNT	The Trøndelag Health Study
MORGAM	MOnica Risk, Genetics, Archiving and Monograph
NR	not reported
NHLBI	National Heart, Lung, and Blood Institute
PAD	Peripheral arterial disease
SABRE	Southall and Brent Revisited cohort

Evidence review for CVD risk assessment tools: primary prevention

1. Cardiovascular risk assessment tools in adults without established cardiovascular disease

1.1. Review question

1.1.1. Introduction

1.1.2. Summary of the protocol

1.1.3. Methods and process

1.2. Risk prediction evidence

1.2.1. Included studies

1.2.2. Excluded studies

1.2.3. Summary of studies included in the prognostic evidence

1.2.4. Summary of prognostic evidence: discrimination

1.2.4.1. Overview of outcome data

1.2.4.2. Clinical evidence profile for C statistic data

1.2.5. Summary of prognostic evidence: calibration

1.2.5.1. Calibration curves and predicted:observed events

QRISK2–2011, QRISK2–2010 and QRISK2–2008

QRISK2–2012

QRISK2–2014

QRISK3–2017

QRISK3 external validation

CRISK, CRISK-CCI and QRISK3

PRIMROSE-lipid and -BMI tools

SCORE2

SCORE2-OP

QRISK lifetime

LIFE-CVD

1.2.6. Summary of prognostic evidence: reclassification

1.2.6.1. QRISK3 vs QRISK2 (Hippisley-Cox 2017 7)

1.2.7. Economic evidence

1.2.7.1. Included studies

1.2.7.2. Excluded studies

1.2.8. Summary of included economic evidence

1.2.9. Economic model

1.2.10. Evidence statements

1.2.10.1. Economic

1.2.11. The committee's discussion and interpretation of the evidence

1.2.11.1. The outcomes that matter most

Discrimination

Calibration

Reclassification / revalidation

1.2.11.2. The quality of the evidence

Data sources

1.2.11.3. Benefits and harms

Discrimination

Calibration

Reclassification

Lifetime risk tools

Summary

Subgroups

1.2.11.4. Cost effectiveness and resource use

1.2.11.5. Other factors the committee took into account

1.2.12. Recommendations supported by this evidence review

1.2.13. References

Appendices

Appendix A. Review protocols

A.1. Review protocol for CVD risk assessment tools: primary prevention

A.2. Health economic review protocol

Appendix B. Literature search strategies

B.1. Clinical search literature search strategy

B.2. Health Economics literature search strategy

Appendix C. Prognostic evidence study selection

Appendix D. Prognostic evidence

D.1. Risk factors and variables included in the risk assessment tools

D.2. Evidence tables from the 2014 version of CG181

D.3. Evidence tables from update search

Appendix E. Forest plots and summary ROC curves

E.1. Summary of C statistic data

E.2. Sensitivity and specificity data

E.3. Summary ROC curves

Appendix F. Economic evidence study selection

Appendix G. Economic evidence tables

Appendix H. Health economic model

Appendix I. Excluded studies

I.1. Clinical studies

I.2. Health Economic studies

Appendix J. List of abbreviations

Table 1PICO characteristics of review question

Table 2Predictor variables included in CVD risk assessment tools

Table 3Outcomes predicted by CVD risk assessment tools

Table 4Summary of studies included in the evidence review

1.2.6.1. QRISK3 vs QRISK2 (Hippisley-Cox 2017 ⁷)