The Impact of Pharmacist Led Vancomycin Order Set Implementation in a Computerized-Prescriber-Order-Entry (CPOE) System at a Tertiary Care Centre: A Quasi Experimental Study

Introduction: Our hospital has developed a vancomycin protocol in order to improve vancomycin dosing and prevent improper sampling time of vancomycin levels and treatment failure. The study objective is to evaluate the impact of implementing a vancomycin order set integrated into a Computerized-Prescriber-Order-Entry (CPOE) system by pharmacist in adult surgical wards to optimize vancomycin dosing and monitoring. Primary objective: Evaluation of the correct sampling time of vancomycin trough levels by implementing a vancomycin order set guiding nurses and physicians on the appropriate sampling time. Secondary objectives: Evaluation of the impact of vancomycin order set integrated into CPOE system on vancomycin initial doses, appropriate use based on indication and time to reach target therapeutic level. Method: This study was a prospective quasi-experimental study. Patients who are >18 years, admitted to surgical wards, receiving new vancomycin orders, creatinine clearance ≥15 ml/min and weighing 40-120 kg were eligible. We excluded patients receiving vancomycin loading doses, prophylaxis, treatment course <5 days and random or peak vancomycin levels. Educational sessions were provided to the surgical residents on the use the order set by pharmacy resident, subsequently vancomycin orders were then reassessed after the order set implementation. Results: Total of 272 vancomycin trough levels were collected (136 levels in the pre-phase and 136 levels in the post-implementation phase) from 33 patients who met the study eligibility criteria. A 10% reduction in inappropriate vancomycin trough levels in the post-implementation phase was observed compared to the pre-phase (p=0.478) Appropriate vancomycin initial doses for the study patients were 2 out of 11 doses (18%) in the pre-phase and 22 doses in the post-phase (50%) (p=0.078). Conclusion: The implementation of an institutional vancomycin order set did not result in a significant change in appropriate vancomycin initial dosing and trough level sampling time in the adult surgical wards in our hospital. Citation: Damfu N, Aseeri M, , Davis A, Hasan H, Ismail S (2016) The Impact of Pharmacist Led Vancomycin Order Set Implementation in a Computerized-Prescriber-Order-Entry (CPOE) System at a Tertiary Care Centre: A Quasi Experimental Study. J Pharmacovigil 4: 219. doi:10.4172/2329-6887.1000219


Introduction
Vancomycin is the most widely used antibiotic for the treatment of serious gram-positive infections involving methicillin-resistant Staphylococcus aureus (MRSA) [1]. According to Infectious Disease Society of America (IDSA), the initial vancomycin dose is 15-20 mg/kg/ dose based on the actual body weight, this dose is given intravenously every 8-12 h for adult patients with normal renal function [1]. Doses are adjusted according to the patient's estimated creatinine clearance and the maximum single dose of vancomycin is 2 g [1,2]. Inappropriate vancomycin dosing may lead to delays in therapeutic serum trough concentrations, the emergence of vancomycin resistant pathogens, and may ultimately culminate in treatment failure [3].
Trough serum concentrations have been recommended by IDSA for monitoring vancomycin nephrotoxicity [1]. Vancomycin serum or plasma trough concentrations are recommended as a surrogate marker of pharmacodynamics target attainment to predict vancomycin efficacy [4].
In patients with normal renal function, trough levels should be obtained once the patient has reached steady state prior to the fourth dose [1]. Minimum trough concentrations of vancomycin should be maintained above 10 mcg/ml in order to avoid the development of resistance. For most infections, trough serum concentrations of 10-15 mcg/ml are sufficient. However, for complicated infections including: bacteraemia (caused by Staphylococcus aureus), endocarditis, osteomyelitis, meningitis, and hospital-acquired pneumonia, higher serum concentrations of 15-20 mcg/ml are required [1].
Improper sampling time can cause suboptimal dosing, lead to increased vancomycin resistance, prolonged hospital stay, and treatment failure. Morrison et al. retrospectively analyzed 2597 vancomycin trough levels during 13 months at a large medical centre. The results showed that 41.3% (n=1075) of vancomycin trough levels were drawn too early (defined as levels taken <10 h after administration of the previous dose). Consequently, in level which drawn too early, clinicians were more likely to decrease, discontinue, or hold a patient's vancomycin dose by (25.6%), or repeat the vancomycin level (29.2%). This may lead to an overestimation of patients' true trough levels, possible under dosing of vancomycin, or a high rate of repeated tests for vancomycin trough levels [5].
McCluggage et al., evaluated the implementation of a vancomycin nomogram in a Computerized-Prescriber-Order-Entry (CPOE) system on 522 vancomycin orders. A significant difference in the percentage of initial vancomycin orders that met nomogram recommendations in the post-implementation group (n=243) were observed compared to the pre-implementation group (n=279) (36% vs. 24%, p=0.0028). Therefore, integrating the vancomycin nomogram into the CPOE system increased the likelihood of patients receiving an initial vancomycin regimen that coincided with the nomogram's recommendations [6].
Traugott et al., evaluated the effects of Therapeutic Drug Monitoring (TDM) criteria in a CPOE system on the appropriateness of orders for vancomycin levels on 545 vancomycin serum levels in 200 patients (310 serum levels in 100 patients in the pre-implementation group and 235 serum levels in 100 patients in the post-implementation group) [7]. The percentage of appropriate orders for vancomycin levels significantly increased after criteria implementation (58% vs. 68% in the pre-implementation and post-implementation groups, respectively. p=0.02). The majority of inappropriate levels were due to improper timing of sample collections (55%) [7].
A vancomycin protocol was approved by the hospital antimicrobial committee; however, the dosing protocol has not been fully implemented or integrated into a CPOE order which might resulted in improper dosing, sampling time, and treatment failure. Therefore, we aim to evaluate the impact of implementing a vancomycin order set integrated into CPOE system in adult surgical patients to optimize vancomycin dosing and monitoring for better therapeutic outcomes.

Primary objective
Evaluate the correct sampling time of vancomycin trough levels (identified from the hospital information system and patient charts).

Secondary objectives
1. Evaluate the impact of the vancomycin order set in a CPOE system on appropriateness of initial vancomycin dosing.
2. Evaluate the appropriateness of vancomycin use based on indication by reviewing cultures and patient charts.
3. Evaluate the time to reach vancomycin target trough level (by looking at the administration time of vancomycin from medication administration records, and sampling time of vancomycin levels and level results from the hospital information system).

Study design and setting
It is a quasi-experimental study conducted from July 2014 to May 2015 at adult surgical wards (90 beds) at a Tertiary Care Teaching Center, Jeddah Saudi Arabia.

Inclusion criteria
1. All adult patients, who are 18 years of age or older, admitted to surgical wards and receiving new vancomycin orders.
3. Patients with total body weight of 40-120 kg.

Exclusion criteria
1. Patients with acute kidney injury or unstable kidney function.
2. Patients who received surgical prophylactic vancomycin doses.

Patients who had a random or peak level drawn.
This study was conducted in three phases ( Figure 1).

First phase (pre-implementation phase over 5 months)
A daily report was generated by the Health Information System (HIS) to identify patients receiving vancomycin in adult surgical wards. Patients whose vancomycin orders met the eligibility criteria were included and data was collected using the vancomycin data collection sheet by pharmacy practice resident.

Second phase (electronic order set implementation and educational sessions for surgical residents over 1 month)
The hospital vancomycin dosing protocol was redesigned into an order set following Institute for Safe Medication Practices (ISMP) Standards for order sets [8], and subsequently integrated into a CPOE system through the HIS as vancomycin order set ( Figure 2). Pharmacy practice resident conducted educational sessions for surgical residents on how to use the vancomycin order set.

Third phase (post-implementation phase over 5 months)
Similarly to first phase, a report of patients who received vancomycin was generated on daily basis and vancomycin orders met eligibility criteria were included. Subsequently, the pharmacy practice resident used the data collection sheet for documentation of study data and observed all vancomycin orders submitted via the order set in the HIS. Appropriate sampling time means that vancomycin trough level was taken within 1 h before the 4 th dose. Appropriate vancomycin use based on indication refers to vancomycin orders which started empirically (duration less than 3 days) or given to treat gram positive cultures which are susceptible to vancomycin.

Statistical Analysis
A sample of 256 trough levels was estimated (128 before the implementation of the vancomycin order set and 128 after the implementation) to provide a 90% power to detect 20% difference for the correct sampling time before and after the implementation of the vancomycin order set with a type I error of alpha 0.05 [7].
1. Descriptive statistics are used to report baseline characteristics as Mean ± SD for continuous variables and proportions for binary and categorical data.
2. Chi-square test is used to test for the proportion of appropriate vancomycin orders based on indication, sampling time and initial dosing.
3. Survival analysis is used to assess the median time to reach vancomycin target trough level and log rank test to compare median time to reach vancomycin level between preimplementation phase and post implementation phase.
4. Data analysis was conducted using STATA version 14.

Results
Data were collected prospectively over 11 months, starting from July 2014 to May 2015. Thirty three patients (11 patients in the preimplementation phase and 22 patients in the post-implementation phase) were enrolled for a total of 272 vancomycin orders (Figure 3).
In the implementation phase, two educational sessions on use of vancomycin order set integrated into CPOE were conducted to 28 surgical residents out of 58 invited residents (48%) over one month period. Baseline characteristics of the study patients are shown in Table 1.

Ethics/IRB Approval
This study received an approval from the hospital ethical committee on July, 2014.
A research amendment was approved by IRB to change the effect sample size from 13% difference to 20 % with a power of 90%, and total sample size from 462 levels to 256 levels (128 before the implementation and 128 after). This was done due to low number of vancomycin levels on surgical wards and the goal was to improve clinical significance by increasing the effect size to 20%.

Secondary objectives results
Appropriate vancomycin initial dosing: We have 2 out of 11 patients (18%) in the pre-phase had appropriate initial vancomycin orders compared to 11 out of 22 patients (50% ) with a p-value of 0.078. There is no statistically significance difference between pre and post in time to reach steady state concentration using log rank test (P value=0.9725) ( Figure 5 for demonstration of Kaplan-Meier curves for survival estimates between pre and post-phase).

Discussion
The primary objective of the study showed a 9.6% difference in reduction of inappropriate vancomycin sampling time ordered between the post-implementation phase (62.5%) and the pre-phase (52.9%), however, the results were not statistically significant (p=0.478). Our findings are comparable to Traugott et al. study [7], which had a larger sample size with 545 vancomycin serum levels in 200 patients (310 serum levels in 100 patients in the pre-implementation group and 235 serum levels in 100 patients in the post-implementation group). This study evaluated the effects of Therapeutic Drug Monitoring (TDM) criteria in a CPOE system on the appropriateness of orders for vancomycin serum levels, and the percentage of appropriate orders for vancomycin levels significantly increased after criteria implementation (58% vs. 68% in the pre-implementation and post-implementation groups, respectively; p=0.02) [7]. This study included all adult patients who met the study eligibility criteria in all hospital wards [7], compared to our study which only focused on adult surgical wards.
Our secondary objective was to evaluate the appropriateness of vancomycin initial dosing, showed an improvement in the post-phase from 2/11 patients (18%) to 11/22 patients (50%) but the findings were not significant with p=0.078. This could be attributed to small number of participants in our study which included only 33 patients and showed an improvement in the initial vancomycin dosing by 32%. These findings are consistent with McCluggage et al. [6], who evaluated the implementation of a vancomycin nomogram a CPOE system Duration of vancomycin treatment (days) (mean ± SD) 28 ± 22 23 ± 18   on 522 vancomycin orders, and showed a statistically significant difference in the percentage of initial vancomycin orders that met the study nomogram recommendations in the post-implementation group (n=243) compared to the pre-implementation group (n=279) (36% vs. 24%, p=0.0028) [6].
Our study evaluated the appropriate vancomycin use based on indication, and no statistically significant difference between the pre and post implementation groups were noted (P=0.6). Another study objective was to evaluate the time to reach target therapeutic vancomycin level, which did not show any statistically significant difference between in time to reach steady state concentration, the prephase and post-phase, p-value (0.9725).
Our study had several limitations; first, we did not assess the adherence of surgical resident to the use of vancomycin order set. Second, there were insufficient number of residents attended the educational sessions, total of 58 residents were invited and only 28 residents (48%) attended the sessions.
Additionally, we conducted educational sessions within one month, which might had an impact on negative findings, as it may be considered a short time for the residents to adapt on using the order set. On contrary to Hall et al. [9], who retrospectively studied vancomycin order set implementation through CPOE in emergency department, a period of 2-years between the pre-implementation and post-implementation groups was chosen to allow for implementation, physician education, and to acquire familiarity with the order set. The study resulted in a 21.9% increase in appropriate dosing in post-CPOE group compared to the pre-CPOE group (p<0.05) [9].
Third, we had a small number of vancomycin patients enrolled during the study period as we accounted for vancomycin orders in our sample size estimation and not number of patients. A larger sample might detect smaller differences between pre and post intervention. Despite we were aiming for a clinically meaningful difference of 20% between the study groups, this might have contributed to the negative findings of our study with detection of only 9.6% Fourth, the quasiexperimental study design using one point before and after for different population, has many limitations that might impact on internal validity such as lack of blinding, maturation, instrumentation, regression to the mean, selection bias and also difficulty to know if the intervention was only responsible for any observed changes [10].
Fifth, the intervention which was only focused on education about using the order set, which considered one of the least effective methods to reduce medication errors. Methods such as forced function or system alerts or reminders will be more effective [11].
And finally, the study has limited generalizability to surgical residents who may need more in depth knowledge in vancomycin dosing and monitoring.
Our study strengths included the quasi-experimental study design, actual implementation of an order set in a CPOE system, and although the findings are not statistically significant, they may provide clinically insights on role of pharmacist in initial assessment of vancomycin orders, future studies should assess the utility of forced function and clinical decision support system to enhance adherence and utilization of order sets in clinical practice to improve desired therapeutic outcomes.

Conclusion
The implementation of an institutional vancomycin order set and incorporated it in the hospital CPOE system did not result in a statistically significant change to appropriate vancomycin initial dosing and trough level sampling time in our hospital adult surgical wards however, a quasi-experimental, study with larger number of patients, use of forced functions to maximize adherence are highly recommended.