Cumulative Sum Analysis : A Simple and Practical Tool for Monitoring and Auditing Clinical Performance

Abstract Monitoring daily performance, identification of decrease of performance and evaluation of improvement actions are three essential point in quality evaluation and clinical auditing. The cumulative sum analysis, charts (CUSUM) based on sequential monitoring of cumulative performance over a time period, results in a real-time monitoring of performance. An important benefit of CUSUM analysis is that this analysis is able to identify subtle, slow, sustained degradation in a process thought to be under control. Beside the principle of CUSUM analysis, two examples how CUSUM analysis can be used in daily practice are presented. Because of the simplicity to perform this analysis and the clear insight of the charts this method can widespread be used.


Introduction
Monitoring daily practice performance, identification of decrease of performance and evaluation of improvement actions are three essential point in clinical medical audits [1,2]. However more important is that these three points becomes a part of daily practice of each medical department. The cumulative sum analysis (CUSUM) is popular in cardiac surgery, certainly for the evaluation of performed cardiac surgery based on risk adjusted mortality [3,4]. CUSUM analysis and charts are based on sequential monitoring of cumulative performance over a time period. Each procedure must be updated and there is a real-time monitoring of performance [4]. Important is that for using a CUSUM analysis, the event must be defined as a binary variable. This can be a clear binary event, death or alive, had the patient a complication during hospital stay yes or no. But also did you meet a standard, a benchmark, yes or no. An important benefit of CUSUM analysis is that this analysis is able to identify subtle, slow, sustained degradation in a process thought to be under control. CUSUM analysis is easy to perform and the intention of this manuscript is to familiarize doctors and everybody involve with quality audits with the principles of this CUSUM analysis, completed with two practical examples. The examples are based on real situations, however the context is changed.

The CUSUM Analysis/Chart
The standard (non risk-adjusted) CUSUM works with a constant risk of failure for each case (procedure, patient). The basic principle of the analysis is the principle of reward or punishment in function of the risk [4]. For example we know that in a department 10% of the patients has at least one complication during their treatment. So the calculated risk for each patient to have a complication during his treatment in this department is 10%. When a patient has no complication, we are rewarded with 0.10 points. However when a patient have a complication during his treatment we are punished with -(1-0.10) =-0.90 points. By summing the values of each patient over time, we construct a graph. X t =max (0,X t-1 +W t ) where t=1,2,3,… case number, X 0 =0 and w t is the score, reward of punishment for each patient.
The horizontal axis represents the cases over the time; let us say the date the treatment starts. The vertical axis indicates the number of patients without a complication compared to expected number patients

Abstract
Monitoring daily performance, identification of decrease of performance and evaluation of improvement actions are three essential points in quality evaluation and clinical auditing. The cumulative sum analysis, charts (CUSUM) based on sequential monitoring of cumulative performance over a time period, results in a real-time monitoring of performance. An important benefit of CUSUM analysis is that this analysis is able to identify subtle, slow, sustained degradation in a process thought to be under control. Beside the principle of CUSUM analysis, two examples how CUSUM analysis can be used in daily practice are presented. Because of the simplicity to perform this analysis and the clear insight of the charts this method can widespread be used. When as accepted, because we have a risk of 10%, the last tenth patients have a complication, the graph decreases for ten times 0.90 points=9 points so normally the CUSUM graph makes a wave-like movement around the null line. An upwards slope indicates that the observed number of patients with a complication are fewer than expected, this means an improvement of the process. A downward slope indicates that the observed number of patients with a complication is higher than expected, thus a worsening of the process. This means that in these graphs cumulative expected minus observed mortality is plotted on the vertical axis.

CUSUM as Self-Monitoring Tool
A department knows that the last year 7% of the patients had a complication during their hospital stay. Since 2009 a CUSUM analysis is performed and the CUSUM chart shows an upward slope over the year 2009 ( Figure 2). This means less complication than expected. In 2009 only 4% of the treated patients had a complication. Interesting is now that this department can be satisfied with their results and continues their analysis with the expected risk of 7% during the following years. However in the aim of improvement their performance the department can recalculate the risk based on the last two previous years. This means a risk of 5% for the further CUSUM analysis what is resulting in a minor upward slope if they do better but also a major downward slope if there are more complication than expected.

CUSUM as Tool for Auditing: Indicator for Sustained Degradation in a Process thought to be under Control
During a medical audit the waiting time for the outpatient clinic was evaluated. The target is 6 days. The department presented a plot of the waiting time of their last 500 patients visiting the outpatient clinic with a mean waiting time of 7.2 days, a median of 6 days and a stable trend line (Figure 3). Based on the waiting time of the year before it was calculated that 50% of the patients were seen within the 6 days after their announcement for the outpatient clinic. So the risk that a patient is not seen on the outpatient clinic within the target of 6 days was calculated on 50%. Based on this risk a CUSUM analysis was performed for the 500 patients presented by the department. This CUSUM analysis shows that for the first 60 patients there was an upward slope indicating better than expected, however since patient number 138 there is a downward slope, meaning that the during these last period less and less patients on the outpatient clinic were seen within 6 days ( Figure 4). So this CUSUM analysis show already that there is a problem not visualized or detected by the presented plot of the department. Of course the opposite is also possible, improvement of processes can also early be detected, by an upwards slope, interesting for the early evaluation of improvement initiative.

Discussion
For the interpretation of a CUSUM analysis it is important to realize that a downwards slope indicates that in the studied process there are more events than predicted, so the process is going worse. However the reason why is important, so a downward slope is an alarm signal. A signal asking for further investigation and identification of possible reasons why the process is going worse. An upwards slope indicate that there are less observed events than predicted, so that the process is going better. Doing better is of course important in health care processes. However if we want to know if an improvement results in a significant improved process, CUSUM charts with confidence limits must be constructed [4]. However this is beyond the scope of this article. In the first place it is important to know if the measured outcome shows improvement of quality.
Important is also to realize that if the curve is above the null line, it means that your global process is performing better than expected. However when there is a downward slope, even in a curve above the null line, this means that at that specific moment your process is performing less good than expected. Of course the opposite can be stated for an upward slope under the null line. So you can say that a CUSUM analysis has a kind of 'memory'. This is strength of this analysis, even at the moment that the total process is apparently under control; the downward slope indicates some debilitation.
In conclusion CUSUM analysis is a simple and practical tool useful for monitoring and auditing health care processes. It provides real time monitoring, and can easily be performed. An important benefit, certainly for audits is the ability to detect slow degradation in a process though to be under control.    figure 3. The calculated risk that a patient has to wait longer than 6 days is 50%. The curve shows a downwards slope, indicating that more and more patients had to wait more than 6 days.