Can biomarkers improve predictions of the RCRI tool to predict heart-related complications in patients undergoing surgery other than heart surgery?

Background and review question

Although patients undergo surgery to maintain or increase life expectancy or to improve quality of life, surgery is not without risks. Some patients will develop a heart-related complication after surgery other than heart surgery, such as a heart infarction. Several tools try to predict someone's chance of developing a heart complication after surgery using information collected in the period before surgery. The Revised Cardiac Risk Index (RCRI) is such a tool that tries to estimate the chance of developing a heart complication during hospital admission in patients undergoing surgery other than heart surgery. It uses information on whether the patient has in the past experienced a heart infarction, heart failure and/or a stroke during his/her life, their use of insulin for the treatment of diabetes mellitus, their current renal (kidney) function and whether he/she will undergo high or non-high risk surgery. The RCRI is commonly used by physicians, but the predictions are not always very accurate. Therefore, several researchers have attempted to improve these predictions by adding extra information to this tool. This information can be derived from so-called biomarkers, which are, for example, measurements from blood, imaging techniques or other characteristics, such as age, smoking status or physical condition of the patient.

The aim of this systematic review was to investigate whether the addition of such biomarkers to the RCRI improves predictions of heart-related complications during hospitalisation in patients undergoing surgery other than heart surgery. In addition, we investigated whether biomarkers and other prediction tools resulted in better predictions of heart-related complications during hospitalisation compared to the predictions of the RCRI in patients undergoing surgery other than heart surgery.

Key results

We identified 69 different predictors that were added to the RCRI tool to improve predictions of these heart-related complications. The evidence is current to 25 June 2020. Predictions seem to improve with the addition of some biomarkers derived from blood. These are troponin (which measures muscular damage of the heart), brain natriuretic peptide (BNP) and (NT-pro)brain natriuretic peptide (NT-proBNP) (which both measure severity of heart failure).

In addition, there were 60 biomarkers that were studied to compare their predictions to the RCRI. Other studies included in this review suggest that BNP and NT-proBNP alone may predict heart-related complications even better than the RCRI. Sixty-five prediction tools other than the RCRI tried to improve its predictions. The American College of Surgeons National Surgical Quality Improvement (ACS-NSQIP) and ACS-NSQIP-MICA (myocardial infarction or cardiac arrest) surgical risk score tools could make better predictions than the RCRI, but this was only true for certain outcomes, and not for heart-related complications. However, for all of these research questions, we are not confident in the results due to large variation in the research methods applied and signs of less accurate research approaches having been used.

Authors' conclusions

Troponin, BNP and NT-proBNP may improve the ability of the RCRI to predict heart-related complications. The ACS-NSQOP-MICA and ACS-NSQIP surgical risk score tools seem to be better at predicting postoperative complications than the RCRI tool, but not heart-related complications. However, due to deficiencies in how the studies were conducted, we are uncertain whether the results we found apply to all patients undergoing surgery other than heart surgery. We need more and better research on biomarkers with promising predictive performance in other settings. 

Authors' conclusions: 

Studies included in this review suggest that the predictive performance of the RCRI in predicting MACE is improved when NT-proBNP, troponin or their combination are added. Other studies indicate that BNP and NT-proBNP, when used in isolation, may even have a higher discriminative performance than the RCRI. There was insufficient evidence of a difference between the predictive accuracy of the RCRI and other prediction models in predicting MACE. However, ACS-NSQIP-MICA and ACS-NSQIP-SRS outperformed the RCRI in predicting myocardial infarction and cardiac arrest combined, and all-cause mortality, respectively. Nevertheless, the results cannot be interpreted as conclusive due to high risks of bias in a majority of papers, and pooling was impossible due to heterogeneity in outcomes, prediction horizons, biomarkers and studied populations.

Future research on the added prognostic value of biomarkers to existing prediction models should focus on biomarkers with good predictive accuracy in other settings (e.g. diagnosis of myocardial infarction) and identification of biomarkers from omics data. They should be compared to novel biomarkers with so far insufficient evidence compared to established ones, including NT-proBNP or troponins. Adherence to recent guidance for prediction model studies (e.g. TRIPOD; PROBAST) and use of standardised outcome definitions in primary studies is highly recommended to facilitate systematic review and meta-analyses in the future. 

Read the full abstract...
Background: 

The Revised Cardiac Risk Index (RCRI) is a widely acknowledged prognostic model to estimate preoperatively the probability of developing in-hospital major adverse cardiac events (MACE) in patients undergoing noncardiac surgery. However, the RCRI does not always make accurate predictions, so various studies have investigated whether biomarkers added to or compared with the RCRI could improve this.

Objectives: 

Primary: To investigate the added predictive value of biomarkers to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery.

Secondary: To investigate the prognostic value of biomarkers compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery.

Tertiary: To investigate the prognostic value of other prediction models compared to the RCRI to preoperatively predict in-hospital MACE and other adverse outcomes in patients undergoing noncardiac surgery.

Search strategy: 

We searched MEDLINE and Embase from 1 January 1999 (the year that the RCRI was published) until 25 June 2020. We also searched ISI Web of Science and SCOPUS for articles referring to the original RCRI development study in that period.

Selection criteria: 

We included studies among adults who underwent noncardiac surgery, reporting on (external) validation of the RCRI and:

- the addition of biomarker(s) to the RCRI; or

- the comparison of the predictive accuracy of biomarker(s) to the RCRI; or

- the comparison of the predictive accuracy of the RCRI to other models.

Besides MACE, all other adverse outcomes were considered for inclusion.

Data collection and analysis: 

We developed a data extraction form based on the CHARMS checklist. Independent pairs of authors screened references, extracted data and assessed risk of bias and concerns regarding applicability according to PROBAST. For biomarkers and prediction models that were added or compared to the RCRI in ≥ 3 different articles, we described study characteristics and findings in further detail. We did not apply GRADE as no guidance is available for prognostic model reviews.

Main results: 

We screened 3960 records and included 107 articles.  

Over all objectives we rated risk of bias as high in ≥ 1 domain in 90% of included studies, particularly in the analysis domain. Statistical pooling or meta-analysis of reported results was impossible due to heterogeneity in various aspects: outcomes used, scale by which the biomarker was added/compared to the RCRI, prediction horizons and studied populations. 

Added predictive value of biomarkers to the RCRI

Fifty-one studies reported on the added value of biomarkers to the RCRI. Sixty-nine different predictors were identified derived from blood (29%), imaging (33%) or other sources (38%). Addition of NT-proBNP, troponin or their combination improved the RCRI for predicting MACE (median delta c-statistics: 0.08, 0.14 and 0.12 for NT-proBNP, troponin and their combination, respectively). The median total net reclassification index (NRI) was 0.16 and 0.74 after addition of troponin and NT-proBNP to the RCRI, respectively. Calibration was not reported. To predict myocardial infarction, the median delta c-statistic when NT-proBNP was added to the RCRI was 0.09, and 0.06 for prediction of all-cause mortality and MACE combined. For BNP and copeptin, data were not sufficient to provide results on their added predictive performance, for any of the outcomes.

Comparison of the predictive value of biomarkers to the RCRI 

Fifty-one studies assessed the predictive performance of biomarkers alone compared to the RCRI. We identified 60 unique predictors derived from blood (38%), imaging (30%) or other sources, such as the American Society of Anesthesiologists (ASA) classification (32%). Predictions were similar between the ASA classification and the RCRI for all studied outcomes. In studies different from those identified in objective 1, the median delta c-statistic was 0.15 and 0.12 in favour of  BNP and NT-proBNP alone, respectively, when compared to the RCRI, for the prediction of MACE. For C-reactive protein, the predictive performance was similar to the RCRI. For other biomarkers and outcomes, data were insufficient to provide summary results. One study reported on calibration and none on reclassification.

Comparison of the predictive value of other prognostic models to the RCRI  

Fifty-two articles compared the predictive ability of the RCRI to other prognostic models. Of these, 42% developed a new prediction model, 22% updated the RCRI, or another prediction model, and 37% validated an existing prediction model. None of the other prediction models showed better performance in predicting MACE than the RCRI. To predict myocardial infarction and cardiac arrest, ACS-NSQIP-MICA had a higher median delta c-statistic of 0.11 compared to the RCRI. To predict all-cause mortality, the median delta c-statistic was 0.15 higher in favour of ACS-NSQIP-SRS compared to the RCRI. Predictive performance was not better for CHADS2, CHA2DS2-VASc, R2CHADS2, Goldman index, Detsky index or VSG-CRI compared to the RCRI for any of the outcomes. Calibration and reclassification were reported in only one and three studies, respectively.