Received date: February 21, 2017; Accepted date: March 21, 2017; Published date: March 28, 2017
Citation: Farah JR, Lee C, Kantorovich S, Smith GA, Meshkin B, et al. (2017) Evaluation of a Predictive Algorithm that Detects Aberrant Use of Opioids in an Addiction Treatment Centre. J Addict Res Ther 8:312. doi:10.4172/2155-6105.1000312
Copyright: © 2017 Farah JR, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Addiction Research & Therapy
Introduction: Physicians prescribing opioids are at the front lines of the opioid abuse epidemic, battling to tip the scale between rising abuse rates and adequate pain control. This study evaluates the performance of an algorithm that incorporates genetic and non-genetic risk factors in accurately predicting patients at risk of Opioid Use Disorder (OUD). Materials and methods: In this study, we evaluated the ability of the Proove Opioid Risk (POR) algorithm to correctly identify OUD in patients at an addiction treatment facility versus healthy, non-addicted controls. The algorithm was applied to 186 participants: 94 patients at an addiction treatment facility who had documented opioid abuse and 92 healthy patients with no history of opioid use. OUD cases were diagnosed by an expert addictionologist using a predetermined set of criteria, including demonstrated tolerance to an opioid, dependence on an opioid for at least one year, and history of self-administration of an opioid on a daily basis. The performance of the POR using sensitivity, specificity, positive and negative predictive values, and area under the curve (AUC) measures was assessed in OUD cases versus the healthy controls. Results: The average POR score of patients with diagnosed OUD was significantly greater than those of the controls. The receiver operator characteristic (ROC) curve of the POR had an area under the curve (AUC) of 0.967, indicating the algorithm correctly categorizes those with OUD nearly 97% of the time. The sensitivity of the algorithm was 98% and the specificity was 100%, demonstrating that the POR is very unlikely to misclassify true positives and true negatives in this study. Conclusion: The POR reliably identified OUD in patients who were addicted to opioids, while classifying healthy controls as low risk. This can be used clinically to predict patients at high risk of OUD before prescribing opioid pain medications.
Opioid use disorder; Predictive analytics; Precision medicine; Addiction; Personalized medicine; Predictive algorithm; Clinical validity
Opioid Use Disorder (OUD) is a public health crisis that has skyrocketed over the past two decades. In 2015, 12.5 million Americans reported using prescription painkillers for non-medical reasons in the past year, and 2 million suffered from a substance use disorder involving prescription painkillers . Sales of prescription painkillers quadrupled between 1999 and 2010 and overdose death rates due to prescription opioid abuse increased at the same rate between 1999 and 2008 . Today, opioids are involved in more drug overdose deaths than any other drug, both legal and illegal .
Prescriptions for opioids began increasing in the late 1990’s when doctors were urged to improve their treatment of patients suffering from pain . More than 30% of Americans have some type of acute or [5-7]. Pain management and meeting patient needs are of paramount importance to physicians; hence opioids are prescribed frequently to treat acute and chronic pain. In fact, opioids are the most commonly prescribed drug class in the United States . Most opioid prescriptions are for short-term use, but about 10 million Americans are prescribed opioids for longer than three weeks . A fraction of these patients will develop OUD, become dependent, or abuse the drugs.
This over-prescription of opioids has resulted in an OUD epidemic that has inflicted an economic burden on society, imposing direct and indirect costs on OUD sufferers, their caregivers (spouses and dependents), the health care system, the judicial system, and on workplaces because of lost productivity. Florence et al. estimated that the total societal costs of OUD in 2013 were $78.5 billion . Healthcare costs (excess medical and drug costs, and substance abuse treatment and prevention) accounted for 36% of that total. Substance abuse treatment alone costs federal, state, and local governments $2.8 billion annually .
Physicians in clinical settings have several ways of assessing the risk of their patients developing OUD. They can conduct clinical chronic pain and persistent pain is estimated to cost $560-$635 billion annually due to healthcare costs, lost productivity, and lower wages assessments with written questionnaires, like the Opioid Risk Tool (ORT) and the Screener and Opioid Assessment for Pain Patients (SOAPP-R) [11,12]. These rely on self-reporting from the patient about their own behavior, and personal and family history. Because the risk of OUD is so highly influenced by environmental factors, such as previous substance abuse issues, exposure to illicit substances, age, stress, and family situations, these clinical screening tools are useful for identifying red flags .
These screening measures, however, ignore the role of genetics in addiction. The American Society of Medicine calculates that genetics can account for half of a person’s likelihood to develop OUD . From studies of twins, we know that genetics contributes to up to 44% of the variance in opioid abuse . A commercially available test that combines genetic risk factors with phenotypic risk factors, such as the Proove Opioid Risk Profile (POR) can be of great clinical utility to physicians. The POR is a clinically validated algorithm that combines phenotypic risk factors with genotypic markers to stratify patients into low-, moderate- and high-risk of OUD with high sensitivity and specificity [16-19]. Moreover, the POR has been shown to detect OUD risk with higher specificity than either ORT or SOAPP-R . Beyond its established clinical validity, however, two clinical utility studies demonstrated that physicians use the results of the POR to guide treatment decisions and found it useful for both clinical decision making and patient clinical improvements [20,21].
This study seeks to build upon previous studies demonstrating the predictive accuracy of the POR. However, whereas previous studies [16-19] used ICD coding to identify patients with OUD, the OUD status of patients in this study was evaluated and identified by an expert addictionologist.
This multi-center, observational study (protocols 2016/07/26 and 2016/08/17) was reviewed, approved, and overseen by Solutions IRB, an institutional review board licensed by the United States Department of Health and Human Services, Office for Human Research Protections. All participants signed informed consent forms prior to data collection (Table 1).
|Population||n Total||n Female||n Caucasian||Mean Age|
|OUD cases||94||53 (56%)||89 (95%)||38|
|Healthy controls||92||55 (60%)||58 (62%)||37|
Table 1: Subject demographics, In total, 186 patients were enrolled in the study; there were no significant differences in age (p=0.94) or gender (p=0.77) between cases and controls
OUD cases (n=94) were enrolled from the Phoenix Health Center in Hagerstown, MD, a clinic specializing in addiction and rehabilitation from drug abuse. In this study, OUD is the diagnostic term for opioid substance use disorder, rather than the physiological state of opioid dependence alone. In order to be diagnosed, OUD cases were required to meet all of the following criteria: (1) “Does the patient demonstrate evidence of tolerance to an opioid?” (2) “Does the patient present with current physical dependence with onset at least one year prior to admission?” (3) “Does the patient present with a history of multiple self-administrations of an opioid on a daily basis?” In addition, inclusion criteria for OUD patients involved confirmation of present experience of chronic non-cancer pain, consumption of opioid medication as part of a pain-management plan, and fluency in English. POR test results were not considered when diagnosing OUD. The control group consisted of individuals in good health, with no personal or family history of mental illness, no pain, no personal history of substance abuse, and were non-smokers; they were enrolled offsite and were matched for age and sex.
Genomic DNA was isolated from buccal swabs obtained from each patient using a proprietary DNA isolation technique and DNA isolation kit (Macherey Nagel GmbH & Co, KG; Germany), according to the manufacturer’s instructions. Genotyping was performed using pre-designed TaqMan® assays (Applied Biosystems; Foster City, CA). Allele-specific fluorescence signals were distinguished by measuring endpoint 6-FAM or VIC fluorescence intensities at 508 nm and 560 nm, respectively, and genotypes were generated using Genotyper® Software V 1.3 (Applied Biosystems; Foster City, CA). The DNA Elution Buffer was used as a negative control, and K562 Cell Line DNA (Promega Corporation; Madison, WI), was included in each batch of samples tested as positive control.
Phenotypic information was also collected, including whether patients had a personal history of alcoholism, personal history of illegal drug abuse, personal history of prescription drug abuse, family history of alcoholism, family history of illegal drug abuse, family history of prescription drug abuse, mental health disorders and/or depression and whether or not they were 16-45 years old. This information was collected in a paper questionnaire that asked patients to give yes or no answers to the phenotypic factors indicated above.
The proove opioid risk (POR) algorithm
A POR score and its associated risk stratification were calculated for each subject. The POR algorithm is a patent-protected, validated measure of opioid risk described elsewhere [16-19]. In short, it combines phenotypic and genotype information to calculate a risk score that correlates to high, moderate or low risk stratifications of opioid dependence [16-19], such that a score of 1-11 is associated with low risk, 12-23 with moderate risk and ≥ 24 with high risk of OUD. Low risk denotes the subject is at low risk of OUD and the clinician may proceed with prescription opioid therapy; moderate risk suggests the clinician should proceed with caution and may consider more routine urine drug testing and possibly limit the duration of opioid therapy; high risk suggests the physician may consider prescribing an alternative analgesic to improve patient outcomes, implementing more routine urine drug testing, limiting the duration of opioid therapy, titrating the patient off opioid therapy, maintaining vigilant awareness of patient outcomes, and/or possibly considering medically-assisted treatment for detoxification.
A Mann-Whitney U test was used to determine the statistical significance of the difference between the POR scores of the OUD group and control group, as well as if there was any confounding due to age. Fisher’s exact test was performed to determine any differences in gender. A cross-tab analysis was performed to assess the diagnostic performance of the POR as a comprehensive algorithm for the evaluation of OUD risk.
|Protein Name||Gene||SNP Marker||Associated Neuro-Psychiatric Disorders|
|Catechol-O-Methyltransferase||COMT||rs4680||Alcohol and Drug Abuse [25,26]|
|Dopamine Beta-Hydroxylase||DBH||rs1611115||Cocaine Addiction [29,30]|
|Dopamine D1 Receptor||DRD1||rs4532||Depression ]|
|Heroin Addiction |
|Ankyrin Repeat and Kinase Domain Containing 1/Dopamine Receptor D2||ANKK1/DRD2||rs1800497||Alcohol and Cocaine Dependence |
|Dopamine D4 Receptor||DRD4||rs3758653||Anxiety [36,37]|
|Dopamine Transporter SLC6A3||DAT||rs27072||Methamphetamine Addiction |
|Gamma Aminobutyric Acid Receptor A, gamma2 subunit||GABRG2||rs211014||Alcohol Abuse |
|Opioid Receptor, Kappa1||OPRK1||rs1051660||Mood Disorders |
|Alcohol Dependence |
|Methylenetetrahydrofolate Reductase||MTHFR||rs1801133||Bipolar Disorder, Depression |
|Opioid Receptor, Mu 1||OPRM1||rs1799971||Heroin Addiction |
|Serotonin Receptor 2A||HTR2A||rs7997012||Drug Abuse  Depression |
Table 2: PROOVE opioid risk test panel markers.
Measures of the POR’s performance included (1) sensitivity: the percentage of OUD patients correctly identified by POR scores, (2) specificity: the percentage of control patients correctly identified by POR scores, (3) positive predictive value (PPV): the percentage of patients who were identified as OUD by the POR and were actually OUD cases, (4) negative predictive value (NPV): the percentage of patients who were identified as healthy by the POR and were actually healthy controls. Logistic regression was used to assess the predictability of the OUD cases and controls given a POR score. A pseudo-R2 was calculated and reported as a measure of model goodness-of-fit . For the risk stratification portion of the analysis, POR scores of OUD patients and controls were divided into three predicted OUD risk categories: low (POR score <12), moderate (POR score 12-23) and high (POR ≥ 24). All statistical analyses were implemented in R v3.2.5 (Table 2).
Distribution of POR scores
Higher POR scores are indicative of higher risk of OUD. Among the OUD cases (n =94), the mean POR score for patients with diagnosed OUD was 29.4 (std. dev=7.39). Healthy controls (n=92) had a mean POR score of 11.6 (std. dev=3.14), with a maximum score of 18. Overall, POR scores were significantly higher in those diagnosed with OUD (p=6.15 × 10-29; Figure 1).
Figure 1: Patients with OUD had significantly higher POR scores than healthy; controls. OUD patients had an average POR score 17.8 points higher than that of the control group (29.4 (median=29.5) compared to 11.6 (median=12; p=6.15 × 10-29), respectively). POR scores ranged from 7-46 in the OUD group, and from 2-18 in the control group.
POR algorithm performance
POR scores were highly predictive of the occurrence of OUD cases. Figure 2 shows that when the proportion of OUD cases was regressed with the POR score alone, the predicted OUD cases with any given POR score matched closely with what was observed in the data (pseudo-R2=0.89, p=1.21 × 10-8). Overall, as POR scores increased, so did the likelihood of identifying OUD cases.
Indicators of diagnostic performance such as sensitivity measures how well the POR is able to identify true positives or those diagnosed with OUD, and specificity measures how well the POR is able to identify true negatives, or those who are healthy controls. At a POR score of ≥ 12 to predict OUD in this cohort, the sensitivity of the POR was high at 97.9%, while the specificity was 41.8%. At a POR score of ≥ 24, the sensitivity decreased to 84.0% but the specificity increased to 100% as no healthy individuals scored above 18. Sensitivity and specificity are both measures of accuracy. PPV and NPV, on the other hand, are measures of precision. At a POR score of ≥ 12, the PPV, or the percent of OUD patients who scored ≥ 12, was 63.4%, and the NPV or the percent of healthy patients who scored <12, was 95.4%. At a POR score of ≥ 4, the PPV was 100% and the NPV was 86.3% (Table 3).
Figure 2: POR scores are predictive of OUD. Higher POR scores correlated with higher occurrence of OUD cases. When the proportion of OUD cases was regressed with the POR score alone, the predicted OUD cases with any given POR score matched closely with what was observed in the data (pseudo-R2=0.85, p=4.44 × 10-10). At POR score ≥ 18, 100% of patients were OUD cases. Dot size is proportional to total number of patients with the POR score.
|POR Score ≥||Sens. (%)||Spec. (%)||PPV (%)||NPV (%)|
Table 3: Performance of the POR in identifying cases and controls, the POR exhibits strong sensitivity in both moderate and high risk categories, supporting its utility as a tool for identifying patients at risk for further evaluation, especially those being considered for chronic opioid therapy. Excellent specificity and PPV for high risk cases is also necessary, to avoid mislabelling patients who may benefit from opioid use for legitimate purposes. POR scores of ≥ 12 correspond to predicted moderate to high risk of a patient being diagnosed with OUD. POR scores ≥ 24 correspond to a predicted high risk of a patient being diagnosed with OUD. *Maximum POR score of controls was 18, thus all patients who scored higher were OUD cases; Sens: Sensitivity; Spec: Specificity; PPV: Positive Predictive Value; NPV: Negative Predictive Value.
To predict how well the POR may extrapolate to other samples of OUD and healthy individuals, the area under the curve (AUC) was calculated for the receiver operating characteristic (ROC) curve. This additionally provides information about the accuracy of the test, where an AUC of 1 is equal to 100% accuracy and an AUC of 0.5 is equal to random chance. Though sample size was small, the AUC of the ROC curve was high at 96.7% (Figure 3).
Figure 3:ROC curve for POR-predicted OUD diagnosis. A POR score of ≥ 12, which corresponds to moderate risk of OUD, has a sensitivity of 97.9% and specificity of 41.8%. This cut-off is used for ruling out patients unlikely to exhibit aberrant behavior with opioids. At a POR score of ≥ 24, which corresponds to high risk, the sensitivity decreases to 84.0% and the specificity increases to 100% (no healthy individuals scored above 18). This cut-off is used for ruling in patients for conservative treatment protocols and more regimented monitoring while on opioids. The AUC of the ROC curve is 0.967 (95% confidence interval: 0.935, 0.999), indicating the POR algorithm is able to correctly diagnose OUD >96% of the time. Sensitivities and specificities for all POR scores are shown in Supplementary Table 1.
The POR algorithm combines phenotypic and genotypic data to predict a person’s risk of developing OUD. It has been shown previously to predict the risk of OUD with high specificity [16-19], greater than other available screening tools, such as the ORT and SOAPP-R . While previous studies demonstrated the ability of the POR to identify chronic pain patients diagnosed with OUD, this study supports the validity of the POR in identifying fully characterized, OUD patients presenting to addiction treatment facilities. The average POR score for those diagnosed with OUD is nearly three times higher than those of subjects who have not been diagnosed with OUD. The ROC curve demonstrates that POR is an excellent model for OUD, correctly identifying those with OUD nearly 97% of the time. Our study demonstrates that POR can be used to stratify patients by risk of OUD with high accuracy.
The POR algorithm showed increasing specificity as the risk of OUD increased; the specificity for those at moderate risk was 98%, and those at high risk were 100%. Furthermore, there were no healthy, nonaddicted subjects who had POR scores greater than 18, indicating that none of the controls fell into the high risk classification. Due to the high sensitivity and specificity of the algorithm, the POR is unlikely to misclassify true positives and true negatives for moderate and highrisk patients. Because the algorithm is especially sensitive to both the moderate and high risk categories, the POR is useful for discerning between patients that should be monitored for potential problems and those that should be labelled high risk and should, thus, avoid opioids altogether.
As OUD prevalence continues to rise, so does the cost to care for and treat OUD patients. Both privately- and publicly-insured OUD patients have more hospital stays, emergency room visits and doctor visits than patients without OUD, and a patient with OUD has excess annual medical costs of $15,000-$20,000, depending on insurance type [10,23]. Physicians can prevent or reduce these excess costs by implementing a comprehensive approach to patient treatment that includes screening their patients before prescribing opioids or when reevaluating a treatment program. The POR is a clinical tool that physicians can use to improve outcomes that benefit both patient and doctor and it can reduce medical costs by guiding physicians to prescribe opioids safely to patients that are low risk, and provide alternative treatment options for patients that the POR identifies as the most at risk for developing OUD .
The limitations of this study include the small size of the cohorts and the different racial distributions of the cohorts. Future studies are needed to confirm the results presented herein.
Our study provides further evidence that the POR algorithm is a highly accurate tool for identifying patients who are at high risk of developing OUD. Physicians often struggle with providing effective treatment to their patients while minimizing risk. The POR can be used by physicians to identify patients at high risk of developing OUD and modify treatment options appropriately. Identifying those at high risk allows physicians to prevent OUD and, thus, the considerable physical, emotional and economic burdens associated with addiction. Clinical use of the POR may result in the prevention of nearly 97% of OUD cases through limiting initial opioid exposure.
Make the best use of Scientific Research and information from our 700 + peer reviewed, Open Access Journals