Scoring Systems and Patient Prognosis Case File

Scoring Systems and Patient Prognosis Case File

Eugene C. Toy, MD, Manuel Suarez, MD, FACCP, Terrence H. Liu, MD, MPH

Case 3

A 78-year-old man was admitted to the intensive care unit  (ICU) for exacerbation of congestive heart failure  (CHF).  During the first 2 days of his ICU stay,  his clini­cal  condition further deteriorated,  and he subsequently required intubation and mechanical ventilation. At  a patient care  conference  with family  members, his family members inquired about his prognosis and your assessment of his chances for recovery with meaningful survival. 

⯈ What can be used to predict recovery  or death in this patient? 

⯈ What are the types of prognostic systems to determine severity of conditions in the ICU?

ANSWER TO CASE 3:

Scoring Systems and Patient Prognosis

Summary: A 78-year-old man is admitted to the ICU for CHF exacerbation. By day 2 in the ICU, the patient has shown signs of continued deterioration and is placed on mechanical ventilation. The family is hoping that you can help them understand his potential for meaningful recovery. 

• Predicting outcome: A variety of models are available for disease severity strati­fication and outcomes prognostication. These models are necessary for quality control and management in the ICU. Although these systems are helpful for outcomes prediction in various populations of ICU patients, the models gener­ally are not intended to be used for outcomes prediction in individual patients. 
• Prognostic systems available for  severity determination in the ICU: ICU scoring systems are generally categorized into 4 groups: ( 1) General risk-prog­nostic scores (eg, acute physiology and chronic health evaluation [APACHE], APACHE II, APACHE III, APACHE IV, mortality prediction model [MPM], simplified acute physiology score [SAPS II and III]). (2) Disease-and organ­specific prognostic scores (eg, Glasgow coma score [GCS], Child-Pugh clas­sification, model for end-stage liver disease [MELD] score,  risk, injury, loss, and end-stage kidney [RIFLE] classification for acute kidney injury [AKI], heart fail­ure scores). (3) Organ dysfunction scores ( eg, sepsis-related organ failure score [SOFA], multiple-organ dysfunction score [MODS], logistic organ dysfunction system [LODS]). (4) Trauma scoring (eg, injury severity score [ISS], revised trauma score [RTS]). 

ANALYSIS

Objectives

1. Learn the various scoring systems that are applicable to patient populations in the ICU. 
2. Learn the applicability and limitations of prognostic systems in clinical practice. 

Considerations

As critical care providers, we are frequently approached by patients' families to pro­vide them with our opinion or "best guess" regarding what will happen to their loved ones. In these situations, it is important to help the family understand that there are no tools that would allow anyone to reliably predict the clinical course and outcome of an individual patient. However, several models are available to esti­mate the probability of in-hospital mortality and 1-year mortality in hospitalized heart failure patients, and these models may help provide some insight regarding what event could transpire during his ICU and hospital stay (Table 3-1). These heart failure outcome predictive models take into account a variety of clinical and

SBP,  systolic blood  pressure;  HR,  heart  rate; HF,  heart failure; COPD, chronic obstructive pulmonary disease; CVA, cerebral vascular accident; LV EF, left ventricular ejection fraction

laboratory variables such as patient age, comorbid conditions, vital signs, labora­tory values such as serum sodium and BUN, and the presence of left ventricular dysfunction. For this patient, we can enter the pertinent data into one or more of these risk-prediction models and calculate his probabilities of in-hospital and long ­term survival. Ultimately, it is our responsibility as ICU care providers to remind the patient's family that the projected probabilities are our best estimation based on prior observations of cohorts of heart failure patients, and the projections may or may not hold true for our patient.

Approach To:

Scoring Systems and Patient Prognosis

DEFINITIONS

GENERAL RISK, PROGNOSTICATION SYSTEMS: These systems were developed based on the assumption that acute disease severity can be measured by the patients' characteristics and degree of abnormality of various physiologic variables. The general risk, prognostication systems include the APACHE II model introduced in 1985 and is one of the most commonly applied prognostic systems of this type in the ICU. These systems are designed to help determine outcomes in populations and are useful for quality assurance and outcome assessment in cohorts of patients. These scoring systems can provide good estimate of the number of patients who are predicted to die in a population of similar patients; however, these systems cannot be used to predict exactly which of the patients will die. 

DISEASE, AND ORGAN,SPECIFIC PROGNOSTIC SCORES: These are scor­ing systems used to quantify single-organ failure or disease-specific outcomes. Examples of these include the GCS, Child-Pugh classification, MELD, RIFLE classification, and the variety of heart failure predictive models discussed earlier. Unlike the general risk­ prognostic scores, these scoring systems are reasonably accurate for organ-specific risk prognostication and are commonly applied for clinical decision-making. 

ORGAN DYSFUNCTION SCORING SYSTEMS: There are several scoring sys­tems that are used to quantify and monitor the progression of patients with multiple organ dysfunction syndromes. These include sepsis-related organ failure score (SOFA), multiple-organ dysfunction score  (MODS), and  logistic organ dysfunction system (LODS). These systems are not only useful at the bedside for clinical decision-making, but are also useful for quality assurance and outcome assessment in cohorts of patients. 

TRAUMA SCORES: There are 2  commonly  used scoring systems for  trauma patients. These include the  revised trauma score (RTS)  and the injury severity score (ISS). The RTS is based on 3 physiological parameters (GCS, systolic blood pressure, and respiratory rate), whereas the ISS is an anatomy-based scoring system quantifying the number and severity of injuries in 6 distinct body regions (head, face, chest, abdomen, extremity, and external). The RTS is a  useful system for the serial assessment of trauma patients, particularly during the initial assessment phase. The ISS is more useful for quality assurance and outcomes assessment in cohorts of patients, and this scoring system is not used for clinical decision-making.

CLINICAL APPROACH

Quantitative assessment of disease severity and prognostication of outcomes in the ICU have become increasingly important. From the standpoint of individual patient care, the ability to accurately assess the severity of illness and accurately determin­ing the level of function of specific organs are very helpful in clinical decision ­making and the determination of medical resources that the individual patient may require. Scoring systems that have become integral parts of day-to-day care 

of patients in the ICU include the GCS, MODS, MELD, and SOFA. The major reasons that these scoring systems have gained popularity in direct patient care are that these scores can be easily calculated by care providers and the calculations are reproducible between observers. The drawback of using any scoring system for out­come prognostication is that while all scoring systems are useful to indicate when a patient's condition is improving or deteriorating and help project probabilities of clinical outcomes, the scores are in no way predictive of individual outcomes. As all scoring systems are mathematical models, usefulness of each model is influenced by its discrimination and calibration. 

The discrimination of a predictive model  measures the ability to distinguish patients who would have one outcome versus another (eg, live or die). Discrimina­tion is most commonly expressed by a receiver operating curve (ROC). The ROC plots the sensitivity of a test (y-axis) against 1 specificity (x-axis). The area under the ROC (aROC)  represents the combined performance of the model. A perfect model has aROC of 1; whereas an aROC of 0.5 suggests that the model is no better than chance alone. Most predictive models that are useful have an aROC> 0. 7, an aROC >0.8 is considered good, and an aROC of >0.9 is considered excellent. With further discrimination, the curve will have a more vertical initial rise followed by a more horizontal extension (Figure 3-1). 

The calibration of a scoring system is a  measurement of its accuracy at different levels of risk. The calibration of a system can be examined using goodness-of-fit statistics, which looks at the difference between the observed frequency and the expected frequency for a wide range of groups of patients. The Hosmer-Lemeshow test is a statistical goodness-of-fit test for logistical regression models used to test

Figure 3-1.  Example of ROC curve. 

Figure 3-2.. An example of goodness-of-fit curve. (In this example, the observed outcomes are closer to the expected outcome in the low-risk cohorts in comparison to the cohorts in the higher risk groups.)

whether the expected outcomes match the observed outcomes in various popula­tion subgroups. A P-value can be calculated, and if   it is large then the model is well calibrated or fits the data well (Figure 3-2).

APPLICATIONS OF SCORING SYSTEMS

Organ-specific prognostic scores are frequently used to follow an ICU patient's prog­ress and guide clinical decisions. These systems are easy to use and have good inter­observer reproducibility. Most available scoring systems will provide good outcome estimates in a population of patients; however, they are not designed to predict out­comes for individual patients. Mortality prediction using serial severity scores have been evaluated by a number of groups. These results have demonstrated that with the assessment of serial scores such as APACHE II and APACHE III, the specificity of the predictive models can be improved. The values of most scoring systems are for the evaluation of ICU performance and for the determination of patient eligibility for enrollment in clinical trials (see Table 3-2). When scoring systems are used for ICU performance evaluation, it is  important that choice of model for outcome pre­diction is appropriate. For example, an ICU with a  lower actual mortality compared with the expected mortality does not necessarily indicate that the care is better. Multiple factors can influence ICU mortality including case-mix, admission and discharge policies, and resource availability (such as staffing). 

a ROC, area under receiver operating curve; APACHE,  acute physiology and chronic health evaluation; SAPS, simplified acute  physiology score; MPM, mortality prediction model; ROC, receiver operating curve. 

CLINICAL CASE CORRELATION

• See  also Case 1  (Early Awareness of Critical Illness), Case 2  (Transfer of Critically Ill Patients), and Case 4 (Monitoring). 

COMPREHENSION QUESTIONS

3.1  A fourth year medical student is beginning the ICU rotation and is assigned to research the applicability of various scoring systems to clinical usefulness. There are several patients in the ICU including those with trauma, stroke, sepsis, and heart disease. Which of the following scoring system has been found to directly correlate with change in a patient's condition and is most useful for bedside decision-making? 

A. Injury severity score (ISS) 

B. Revised trauma score (RTS) 

C. Multiple organ dysfunction score (MODS) 

D.  APACHEII E.  APACH Ill 

3.2  Which of the following is an organ-specific scoring scheme rather than a general-risk prognostication scoring system? 

A. APACHE II 

B. SAPS II 

C. MPM II 

D. APACHE III 

E. GCS 

3.3  An ICU director is initiating a new quality improvement process for a general medical-surgical I CU. Which of these scoring systems is most useful for mainte­nance of quality control in this unit? 

A.  MODS 

B.  ISS 

C.  MELD score 

D.  RIFLE E.  APACHE III 

ANSWERS TO QUESTIONS

3.1 B. The revised trauma score (RTS) is a physiologic scoring system. This is based on the patient's GCS, systolic blood pressure, and respiratory rate. A  drop in RTS of >2 during the early post-injury observational period generally indicates a worsening condition and would require reassessment of the patient. The ISS is an anatomy-based trauma scoring system that is    most useful when assessing pop­ulation outcomes and quality management of trauma services. The MODS is a scoring system to document the severity of organ dysfunction. It is used to direct care, when there is a pattern of change that is unexpected. It is however not as useful as the RTS in direct decision-making in the ICU. The APACHE II and APACHE III scores are general prognostication scores that are mainly applied for quality management. Some groups have shown that serial measurements of APACHE II and  III scores can be used for individual outcome prognostication. 

3.2 E. The Glasgow coma score is not a general-risk prognostication scoring system; whereas, all the other scoring systems listed are general-risk prognostication scores. 

3.3 E. In a combined medical-surgical ICU population, the APACHE III scoring system would be the most useful for outcome prognostication and quality con­trol. The MODS score is an organ dysfunction scoring system that may not capture all the patient outcomes in a mixed ICU population. Similarly, ISS is for trauma severity grading. The MELD and RIFLE are organ-specific scoring systems for liver and kidney injury quantification, and these scoring systems would not capture all the important variables that might influence outcome. 

CLINICAL PEARLS

⯈ Most general-risk prognostic scoring systems are designed for population risk-prognostication and lack specificity for individual risk prognostication.

⯈ Some of the disease-specific scoring systems are  useful  for  bedside decision-making, and these include GCS, RIFLE, and MELD scores. 

⯈ THE APACHE score  is probably the most commonly  used system  in the general ICU. The higher the score, the greater the mortality: APACHE  II score of25 = predicted mortality of SO%, and a score of over 35 = predicted mortality of 80%. 

⯈ The RIFLE score applies to acute kidney injury and can  be remembered since RIFLE begins with "R" for "Renal." 

⯈ The M ELD score is used to assess liver failure, the "LD" of MELD means "Liver Disease." 

⯈ The RTS score is useful for bedside decision making of trauma patients. It is weighed heavily toward the GCS (significant head trauma)  and sig­nificant physiological problems (SBP and RR). 

References

Capuzzo M, Moreno RP, Le Gall J-R. Outcome prediction in critical care: the simplified acute physiology score models. Curr Opin Crit Care. 2008;14:485-490. 

Gartman EJ, Casserly BP, Martin 0, Ward NS. Using serial severity scores to predict death in ICU patients: a validation study and review of the literature. Curr Opin Crit Care. 2009;15:5 78-582. 

Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anesthesia! Scan. 2008;52:467 -478. 

Vicent J-L, Bruzzi de Carvalho R Severity of illness. Semin Resp Crit Care Med. 2010;31:31-38. 

Vincent J-L, Moreno R. Clinical review: scoring systems in the critically ill. Critical Care. 2010;14:207-215.

HomeCritical Care Case FileScoring Systems and Patient Prognosis Case File