Estimated Effects of Whole-system Naturopathic Medicine in Select Chronic Disease Conditions: A Systematic Review

Background: Naturopathic medicine (NM) is a holistic approach to primary care that almost always employs multi-modal interventions, i.e. nutrition and lifestyle change recommendations plus dietary supplements. While evidence supports individual elements of NM, the whole practice is often critiqued for its lack of evidence. Methods: We systematically searched PubMed/MEDLINE, EMBASE, CINAHL, Cochrane Library and AMED from inception to April, 2012 as well as conducted hand searches of existing grey literature. For inclusion, studies had to report results from multi-modal treatment delivered by North American naturopathic doctors. The effect size for each study was calculated; no pooled analysis was undertaken. Risk of bias was assessed using the Cochrane risk of bias as well as Downs and Black tools. Results: Fifteen studies met inclusion criteria, investigating a range of chronic diseases of public health significance. Studies were of good quality and had low to medium risk of bias including acknowledged limitations of pragmatic trials. Effect sizes (Cohen's d) for the primary medical outcomes varied and were statistically significant (p<0.05) in 10 out of 13 studies. A quality of life metric was included in all of the randomized controlled trials with medium effect size and statistical significance in some subscales. Conclusions: Previous reports about the lack of evidence or benefit of NM are inaccurate; a small but compelling body of research exists. Further investigation is warranted into the effectiveness of whole practice NM across a range of health conditions. [PROSPERO 2012:CRD42012002176]


Introduction
The burden of chronic disease on the healthcare system is well established. Estimates are that 44% of Americans (133 million people) suffer from at least one chronic disease, while 13% (63 million) suffer from three or more [1,2]. These numbers are rising, and it is estimated that by 2020, 157 million Americans will have at least one chronic disease, while a staggering 81 million are expected to have multiple chronic conditions, with the most common being hypertension, hyperlipidemia, diabetes, heart failure, arthritis, and cancer [1,3]. To take the example of type 2 diabetes, coined as the "the epidemic of the 21st century" [4], approximately 24 million Americans are currently affected, and this number is expected to double in the next 25 years [1]. According to the American Diabetes Association, "people diagnosed with diabetes incur average medical expenditures of about $13,700 per year, of which $7,900 is attributable to diabetes" [5].
A second disease of epidemic proportions, cardiovascular disease (CVD) currently affects 83.6 million Americans (greater than one in three people): 77.9 million have hypertension, and 15.4 million have coronary heart disease [6]. By 2030, over 40% of the American population is expected to have some form of CVD [6]. In 2009, 1/6th of all hospital stays in the US (six million) were attributable to CVD; the direct cost associated with this was $71.2 billion dollars [6]. Direct and indirect costs of CVD were $312 billion, and this is expected to rise to $1.48 trillion dollars by 2030 [6]. When considering the sum of healthcare expenditure, 75% of all healthcare costs in the US (equivalent to $1.7 trillion dollars annually) are attributable to chronic disease [1,3]. It is clear that new solutions are needed, including a greater emphasis on multidisciplinary, prevention oriented care [2].
Between 2002 and 2005, the Naturopathic Medical Research Agenda (NMRA) formalized the focus of a research mission for naturopathic medicine (NM), as practiced in North America. The NMRA envisioned a pathway for the development of research focused on evaluating outcomes from naturopathic practice assessed by applying appropriate "whole practice" methodologies within pragmatic clinical settings [7]. Parallel to establishing the NMRA, in 2005, the Institute of Medicine (IOM) report "Complementary and Alternative Medicine in the United States" identified important gaps in the knowledge of CAM effectiveness and utilization [8].One of the IOM's recommendations to address these gaps was to conduct outcomes research on routine care delivery.
By linking the care people get to the outcomes they experience, clinical outcomes research has become a key to developing better ways to monitor and improve the quality of care, naturopathic and otherwise. The limitations of classical medical research designs, such as the single agent placebo-controlled randomized clinical trial, have become well-appreciated, especially as they relate to CAM research [9][10][11][12]. Alternative methods, such as patient-centered outcomes research and the inclusion of metrics that assess quality of life (QOL) have become increasingly prioritized in research, as exemplified by program announcements from the Patient-Centered Outcomes Research Institute, created under Section 6301 of the Affordable Care Act of 2010 [13]. Thus, "whole practice" research has emerged as a strong, valid and relevant methodology to investigate naturopathic medicine, as well as other multi-modal practices [9].
Standard research methodology designed to test the null hypothesis often does not specifically examine the magnitude of the effect, nor the precision around the magnitude of the effect. For clinical questions, this is critical information since clinicians must be able to evaluate the relative importance or impact a given therapeutic approach may have on their patient population, not simply the statistical significance. In whole systems research, the magnitude of the effect, or the effect size, of the multi-component intervention may be most appropriate. Metaanalysis of effect sizes allows for an examination of the clinical significance of both specific and non-specific effects of a treatment in comparison to control. Here we report effect size, calculated as a standard mean difference (Cohen's d), for all studies in which data were available (two were missing data required to calculate an effect size). The standardized mean difference provides a less heterogeneous metric of the magnitude of the effect of whole-practice naturopathic medicine and the precision around the range of probable effect size of the multi-modal naturopathic approach in a given condition or disease.
The research base describing practice outcomes from NM in health and disease is typically assumed to be limited [14]. However, in the domain of patient-centered research, the field of NM has been in the vanguard, defining its priorities in patient-reported outcomes research before well-established nomenclature (and avenues for funding) came into fashion. The purpose of this review was to compile and consolidate research that has investigated the whole practice of naturopathic medicine as it is practiced in community settings in order to better assess the quantity and quality of the research, and clinical effect, if any.

Definition of naturopathic medicine
Naturopathic medicine is a distinct primary health care profession, emphasizing prevention, treatment and optimal health through the use of therapeutic methods prescribed according to a therapeutic order which encourage the inherent self-healing process of the body [15]. Clinical practice focuses on the patient as a whole person, and addresses physical, mental, emotional, spiritual, and environmental dimensions of health concurrently to promote healing. Naturopathic doctors (NDs) in Canada and the United States receive training to a common standard as defined by the Council on Naturopathic Medical Education (CNME) [15]. Graduates from accredited schools are eligible for licensing in 17 states, two US territories, the District of Columbia, and five provinces in Canada. Scope of practice and training standards differ from practitioners called naturopaths in Australia, New Zealand, and European and other countries; thus, for purposes of this review, we limited research to investigations of North American naturopathic practice.

Identification of eligible studies
In April 2012, a comprehensive search was conducted in the PubMed/MEDLINE database  to locate studies pertaining to North American licensed naturopathic medicine with patientreported, clinical, and/or cost outcomes. The search included both controlled vocabulary (MeSH) terms, e.g. Naturopathy and Treatment Outcome, and natural language words and phrases, e.g. naturopathic medicine and whole practice. The PubMed/MEDLINE search query was then "translated" and the following electronic databases were searched according to their specific conventions: EMBASE , CINAHL with Full Text (1982-2011), the Cochrane Library , and AMED . A language filter was applied limiting citations to English, French and Spanish to reflect the emphasis on naturopathic medicine as practiced in North America. No other filters were used, in order to increase search sensitivity. Additionally, conference proceedings of the American Association of Naturopathic Physicians (AANP) and Canadian Association of Naturopathic Doctors (CAND) were scanned for relevant abstracts.

Study selection
In accordance with PRISMA guidelines [17], two reviewers independently reviewed the search results to select studies that met inclusion criteria. To be included, studies had to be longitudinal clinical studies reporting longitudinal outcomes, in which North American licensed naturopathic doctors managed clinical conditions employing their full scope of practice. Observational and randomized clinical trials (RCTs) were included if they were consistent with whole practice (i.e. the naturopathic intervention consisted of at least two different treatment modalities).

Data abstraction
Data abstraction was conducted in duplicate (CC, DS), using piloted data extraction sheets. The following data items were extracted: study setting; study population, participant demographics and baseline characteristics; details of the intervention and control conditions; study methodology: recruitment and study completion rates; outcomes and times of measurement; indicators of acceptability to users; suggested mechanisms of intervention action; information of assessment of the risk of bias; information on or to calculate effect size. In the event that data was missing or unclear, a request for information was sent to the original investigators. Author assessment took place independently with disagreements being resolved by consensus.

Methodological assessment and quality rating
Two authors (KC, JG) independently assessed each randomized controlled trial for risk of bias using the Cochrane risk of bias tool [18]. The use of the risk of bias tool is explained in detail elsewhere [19] and includes assessment of the following domains: sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessors, incomplete outcome data, selective outcome reporting, and other sources of bias (e.g. distribution of baseline characteristics, industry initiation and funding). The same two authors also used the Downs & Black checklist as a second critical appraisal tool for methodological quality of the studies included in the review [20]. This 28-item checklist is appropriate for both randomized and non-randomized studies, quantifying an overall score (out of 28; higher scores indicate greater methodological rigor) as well as domain-specific scores for reporting (11), external validity (3), internal validity (bias) (7), internal validity (confounding) (6) and power (1). Author assessment took place independently with disagreements being resolved by consensus.

Assessment of clinical outcomes and effect size
Primary outcome measures, as defined by the investigators of the original research, as well as quality of life metrics, such as the SF-36, were abstracted from reported results with the intention to compare and pool effect sizes using forest plots and standard methods for metaanalyses. When necessary, communication with authors provided missing data such as standard deviations. The heterogeneity of the studies we reviewed warranted use of a statistic that could be applied to summarize results of various study designs. Standardized mean differences were calculated for each study for the between group differences in the author-specified primary outcome [19,21]. For measures of quality of life, standardized mean differences were calculated for summary scores in physical, mental, and general health when available.
Using pooled variances for the standard deviations in point estimates of active and control group outcomes, Cohen's d effect sizes were computed. When interpreting Cohen's d values, an effect size of 0.2 to 0.3 is considered a "small" effect, whereas around 0.5 is considered a "medium" effect and 0.8 or greater is considered a "large" effect [22].
Though other measures such as Glass's ∆ or Hedge's g can be used to calculate effect size, Cohen's d was selected based on popular use and that the calculation accounts for variance in non-specific effects seen in both the intervention and control group. This calculation results in a conservative effect size, weighted to maximize the impact of the control group and minimize the impact of the intervention group. Study heterogeneity (I2) was calculated and due to significantly different treatment populations and a large initial I2, the studies were not pooled for analysis.

Results
The PubMed/MEDLINE search string provided 1,257 total citations before removing duplicates. After 329 duplicates were removed, 931 citations were reviewed for inclusion. The final selection yielded 12 qualifying clinical studies, one of which had a cost-effectiveness analysis associated with the primary trial results. Results from hand searching added two additional studies: a more recently published randomized controlled trial (RCT) & companion cost-effectiveness analysis (CEA). In total, 15 studies of whole practice naturopathic medicine were included [23][24][25][26][27][28][29][30][31][32][33][34][35][36][37]. Figure 1 shows the literature flowchart. Of the five retrospective studies, one included a control group [35]. Two studies were prospective observational cohort studies, of which one was comparative to an electronically created usual care control group [24], and one made comparisons to other forms of CAM [34]. Six of the studies were randomized controlled trials (RCTs) with usual care comparators [23,[27][28][29][30]32], although one made comparisons to other forms of CAM as well [30].

Study quality and risk of bias
In general, the RCTs included in this review were assessed as having a low risk of bias on most domains and were clearly reported. However, none of the included trials were blinded, a recognized impracticality in whole systems research [10].
Because of this limitation, as well as missing outcome data in some trials, the assessment of the overall risk of a biased effect estimate was high in all but one trial. Methodological rigor, as quantified by the Downs and Black checklist, varied among the studies, with higher rigor demonstrated by randomized controlled trials and in the more recent studies; see  Cardiometabolic disease (Cardiovascular disease, diabetes, hypertension, metabolic syndrome): Cardiometabolic disease represents one of the leading public health concerns in the United States, and the majority of studies of whole practice naturopathic medicine focused on these conditions. Bradley et al. conducted three observational studies in diabetes and one in hypertension [24,25,28,33] and Seely et al. conducted a pragmatic RCT and economic evaluation [23,37]. In the first retrospective evaluation of diabetes care, glycemic control and other risk factors improved during receipt of naturopathic care [33].

Primary clinical outcomes
Studies reported results in comparison to usual care controls or other CAM modalities, or reported differences between treatment groups, typically reporting mean changes and p values. To address the heterogeneity amongst the studies and the need to assess the magnitude of the observed effects systematically, we calculated effect sizes for each study, which are presented in Figure 4 for primary outcomes. No adverse events were reported.
Rates of health promotion delivery were substantially higher than national primary care averages. Evidence-based therapeutic lifestyle change recommendations (dietary) were made for 100% of patients, and 94% of patients were prescribed exercise. 69% of patients received counseling regarding stress reduction techniques. In the second retrospective evaluation of diabetes care, observations of clinical outcomes following a mean 27 month duration of care demonstrated the following significant mean changes in clinical risk factors: -0.65% for HbA1c (p=0.046), -45 mg/dL for triglyceride (TAG) (p=0.037), -7 mm Hg in Systolic Blood Pressure (SBP) (p=0.02), and -5 mm Hg in Diastolic Blood Pressure (DBP) (p=0.003) [28]. Mean changes in total cholesterol did not reach statistical significance. Three unique dichotomous outcomes were reported for this study: the percentage of patients who achieved new control (i.e. had not achieved control or within normal limits after diagnosis but prior to the study), had clinically significant risk factor improvements, or had any improvement. For HbA1c, the percent reaching new control was 26%, 42% had clinically significant improvements in risk factors, and 68% had any improvement. Respectively, percent of patients achieving these outcomes were LDL,7%, 28%, and 62% for LDL; 0%, 25%, and 39% for HDL; 14%, 38%, and 52% for TAG; 16%, 51%, and 86% for SBP; and 27%, 54%, and 70% for DBP. In the most recent study, a prospective observational cohort study conducted within a large health maintenance organization, a mean HbA1c decrease of -0.90% (P=0.02) was observed [24]. This was a -0.51% mean difference compared to usual care (P=0.07), following six-months of care. Significant improvements were found in most patient-reported measures, including glucose testing (P=0.001), healthy dietary behaviors (P=0.001), increased physical activity (P=0.02), improved mood (P=0.001), increased self-efficacy (P=0.0001) and increased motivation to change lifestyle (P=0.003).
In the 2011 retrospective evaluation of hypertension (HTN) outcomes, Bradley et al. found that patients with both stage 1 and stage 2 HTN had blood pressure reductions during receipt of naturopathic care, with stage 2 patients achieving mean reductions of 26 mmHg (P<0.0001) and 11 mmHg (P<0.0001) in systolic BP (SBP) and diastolic BP (DBP), respectively. The proportion of patients with resolution of hypertension, defined as BP less than <140/90 mmHg, was 33% (P<0.033) [25].
In 2013, Seely et al. reported results of an RCT comparing the addition of naturopathic treatment to community based usual care by a general practice medical doctor (GP) alone [23]. Composite reduction in cardiovascular disease risk as measured by Framingham risk scores demonstrated that the group receiving naturopathic care in addition to usual care experienced a 3.6% favorable difference in risk compared to usual care alone (NNT=30 for 10 year event risk). This absolute risk reduction was the equivalent to seeing a risk reduction of becoming 5.5 years "younger in CV years." A 27% absolute reduction in the prevalence of metabolic syndrome was also observed in the naturopathic care arm. A cost effectiveness analysis also accompanied this trial [37]. Of the 246 employees who consented to the trial, twothirds gave consent to make available their claims and sick leave data. There were no statistically-significant differences between those who did and did not give this consent across baseline characteristics, outcomes, or tendency to miss study visits. The measured risk reductions came with an average net savings of $1138 in societal costs and $1187 in employer costs. There was no change in quality-adjusted life-years across the study year.

Pain
The studies summarizing naturopathic treatment of pain syndromes cover a wide range of conditions. Pain is among the top five reasons patients seek medical attention [38]. In the study by Secor et al., data obtained from an electronic outcome measures data management system at an integrative pain clinic found significant improvements in pain scores and quality of life measures [34]. Patients who received naturopathic care experienced the greatest pain reductions (60%, p ≤ 0.0001); however, the chiropractic and acupuncture treatment groups also experienced statistically significant pain reductions of 34% and 52% respectively.
A three-armed randomized controlled trial of naturopathy (NM), traditional Chinese medicine (TCM), or standard care (SC) for temporal mandibular jaw pain was undertaken in a large health maintenance organization in Portland [30]. TCM and NM both demonstrated significantly greater in-treatment reductions for worst facial pain compared to SC; reductions for NM versus SC were greatest (p=0.019).
In the RCT by Szczurko et al. investigating rotator cuff tendinitis, pain scores decreased by 54.5% (P<0.0001) in the NM group and by 18% (P=0.0241) in the comparison group which received physical therapy exercise recommendations [26]. The same team's 2007 RCT of naturopathic vs. standard physiotherapy for chronic low back pain also found significant benefits in the naturopathic treatment group (-6.89, 95% CI. -9.23 to -3.54, p=<0.0001) as measured by the Oswestry questionnaire. An additional finding of this trial included a 0.5 kg/m2 reduction in Body Mass Index (BMI) (p=0.01).

Other chronic conditions
The remaining studies investigated other chronic conditions: multiple sclerosis, hepatitis C, anxiety, and menopausal symptoms. The 2008 RCT by Shinto et al. investigated outcomes in multiple sclerosis [29]. Three intervention arms were defined: usual care, naturopathic medicine plus usual care, and MS education plus usual care. While there were no significant differences between groups on any outcome measure, there was a trend favoring the naturopathic group in the General Health subscale of the SF-36 (p=0.11), Timed Walk (p=0.11), and neurologic impairment (EDSS) (p=0.07). In 2009, Cooley et al. randomized people with anxiety to receive naturopathic care or psychotherapy over a six week period [27]. Beck Anxiety Inventory scores decreased significantly in the naturopathic group compared to the psychotherapy group (p=0.003). Significant differences between groups were also observed in mental health, concentration, fatigue, social functioning, and vitality. In a retrospective study of menopausal symptoms, management by both naturopathic and conventional primary care physicians in a community health clinic was compared [35]. Multivariate analyses found that patients treated with NM were approximately seven times more likely than conventionally treated patients to report improvements for insomnia ( [36]. A statistically significant reduction in alanine transaminase (ALT), the primary liver enzyme, was reported.

Quality of life outcomes
Amongst the RCTs and prospective observational studies evaluated, all but one included a quality of life outcome measure as well as a disease-centered metric [23][24][25][26][27][28][29][32][33][34][35][36]. The outlier did assess the impact of the disease state (TMJ) on social function and activities of daily living [30]. Effect size estimates for quality life outcomes are summarized in Figure 5. The SF-36 was the most common quality of life measure used (five RCTs), however the SF-12, PHQ-8 (a depression score), and a rating of the impact of facial pain on social functioning were also used. All eight studies using these instruments showed improvements with NM, with a majority demonstrating a medium effect size and statistical significance in some scales. In the evaluation of cardiometabolic risk reduction by Seely et al., improvements between groups were seen in all domains of the SF-36, but these did not reach statistical significance and the effect size was relatively small (<0.2) [23]. Bradley et al. found improved mood (as measured by PHQ-8) in the NM group at 6 and 12 months as compared to usual care (p=0.001, and 0.005, respectively) [24]. In their study of rotator cuff tendonitis, Szczurko et al. found statistically significant between-group differences in all SF-36 subscales; the aggregate mental and physical components demonstrated a 5.73 and 5.71 point difference respectively, favoring the NM group (p=0.01, p=0.0004 respectively) [26]. Effect size estimates were medium to small (0.39 to 0.28). In the same team's low back pain trial, QOL improved 7.0 (mental) and 8.47 (physical) points between groups (p-0.004 and <0.0001, respectively) [32]. Effect sizes were medium (0. 33

Cost outcomes
Two cost-effectiveness analyses (CEAs) were performed in conjunction with two of the clinical trials [31,37]. Both trials recruited from a population of Canadian postal workers and had access to the same employment data. Both CEAs revealed societal and employer savings associated with use of NM. In one study, naturopathic treatment of LBP showed participant savings and significant benefit in quality-adjusted life years [31]. The cost-effectiveness analysis found that patients randomized to the naturopathic group improved their health status by the equivalent to 9.4 "perfect health" days over the three-month study period. In addition, naturopathic care significantly reduced societal costs by $1212 per participant. From the perspective of the employer, the intervention cost $154 per absentee day avoided. This compares favorably to employer cost of lost productivity of $172 per day, translating into a return on investment of 7.9% under the healthcare coverage limits set by this employer, assuming the employer paid the full cost of naturopathic care. Participants experienced savings related to lower expenditures on additional care of $1096 per participant [31].

Discussion
While the sample sizes of many of the studies are small, the results of this meta-analysis indicate that receiving whole-system naturopathic medicine as practiced in North America is associated with improved health outcomes, as well as improved quality of life, in patients with or at risk for chronic conditions, including cardiovascular disease, diabetes, chronic pain, anxiety, multiple sclerosis, hepatitis C, as well as menopausal symptoms. Although costeffectiveness data are limited on naturopathic medicine in the context of chronic disease, the data thus far suggest the use of NM results in cost savings to employers as well as reductions in societal costs. This is especially relevant in the current era when healthcare resources are increasingly consumed by the management of chronic diseases and associated complications. Additional demonstrations and evaluations of licensed North American naturopathic doctors are warranted to enhance current strategies and improve health outcomes and should focus on diverse healthcare environments in generalizable populations.

Primary outcomes, patient-centered outcomes
The primary outcomes that were assessed and that showed improvement with the short-term use of NM were: reduced prevalence of metabolic syndrome by 16.9% compared to usual care [23]; improvements in glycemic control (reduced HbA1c), reductions in blood pressure (systolic and diastolic blood pressure reductions up to 26 and 11 mmHg respectively) [24,25,28]; improvement in anxiety (>50% reduction compared to active psychotherapy controls) [27]; reduced pain severity (>50% reductions in chronic low back pain and rotator cuff tendonitis compared to active physiotherapy controls) [26,32], and improvement in the pain of temporomandibular syndrome (TMJ) and other body pain compared to controls receiving other complementary modalities [30,34]. Although not statistically significant, point estimates of effect also suggest improvement in fatigue, neurologic, and cognitive impairment associated with multiple sclerosis compared to enhanced usual care [29], menopausal symptoms when compared to usual care [35], as well as ALT levels in hepatitis C patients [36]. For quality of life, patients treated with multimodal interventions by naturopathic doctors showed small to moderate improvements in overall QOL scores as well as for physical function and mental function [26,32], and specific symptoms such as psychosocial interference [30], mental health, concentration, fatigue, social functioning, and vitality [27]. No adverse effects were reported in any of the studies. When considered collectively, the results of our study suggest that the care delivered by naturopathic doctors improves the health outcomes of patients affected by chronic disease, as measured by both clinical and patient-reported outcomes, as well as reduces direct and indirect costs.

Characterizing the NM intervention
Naturopathic doctors employ a diverse toolkit of treatment modalities which are selected based on the therapeutic order and emphasize healthy lifestyle choices and minimally invasive treatments. Details of the specific modalities used in each intervention can be found in the original studies and summarized in Table 1. In summary, the naturopathic interventions included diet counseling and nutritional recommendations, specific home exercises and physical activity recommendations, deep breathing techniques or other stress reduction strategies, dietary supplements including vitamins, hydrotherapy, soft-tissue manual techniques, electrical muscle stimulation, and botanical medicines. None of the studies included pharmaceutical agents even though these are sometimes part of naturopathic practice.

Impact of comparators
As a multi-modal system, naturopathic medicine is inherently difficult to evaluate using the single-agent placebo-controlled model. In the real world of clinical practice (independent of discipline), interventions are typically utilized in combination and patients are not blinded, are unlikely to be representative of a narrowly defined disease group, often present with co-morbidities, and typically receive other forms of care. No randomized, placebo-controlled trial is able to recruit these heterogeneous participant samples, nor mimic real world clinical settings. Therefore, objective evaluations of "whole practice" outcomes and the conduct of pragmatic trials provides the best true estimates of "real world" expectations for the effects of any new clinical service, including assessments of naturopathic medicine. However, even in pragmatic research, appropriate comparators remain necessary to assess the incremental value of an experimental health service. For evaluations of naturopathic medicine, rather than comparison to placebo or overly contrived time-attention control groups mimicking services that do not exist, the appropriate comparator is the usual care practices within the same clinical setting, i.e., the status quo.
Three of the included RCTs employed usual care/enhanced usual care comparators [23,29,30]. but one included an active control arm as well [30]. The other RCTs used active comparators [26,27,32]. Of the five retrospective studies, one included an electronically created usual care control group matched from medical records [35]. Two studies were prospective observational cohort studies, of which one was compared to an electronically created usual care control group [24]. The other made comparisons to other forms of CAM [34]. Use of enhanced usual care and active controls (acupuncture, chiropractic, education, psychotherapy) provides a useful picture of real-world effect. When most patients seek care from a naturopathic doctor, they are motivated to engage in something more and might be looking for other additions to their usual care. While heterogenous, the different comparators provide a real-world perspective about what can be expected at the population level, especially in our era of increasing valuation of patient-preferences and autonomy.
Similarly, for evaluations of non-pharmacological therapies, appropriate comparators were chosen based on customary usual care practices, such as standardized physical exercises for rotator cuff tendonitis and standardized psychotherapy for anxiety [26,27,32]. The studies analyzed here are all examples of evaluations of actual practice outcomes, i.e., pragmatic research, which most accurately estimates the effectiveness in a real world setting, and thus the results have greater external validity than could be achieved from any placebocontrolled clinical trial [43][44][45].

Impact of study design
Evidence now shows that unblinded studies are not inherently inferior to double blinded, placebo controlled trials with respect to internal validity, bias, and methodological rigor. In fact, contrary to common perception, evidence favors non-blinded studies utilizing objective measures [46]. Wood et al., have shown that unlike trials with subjective outcomes (which may exaggerate treatment effects when there is inadequate allocation concealment or lack of blinding) unblended trials with objective outcomes do not exaggerate treatment effects [46]. Savovic et al. conducted an analysis of 234 meta-analyses containing 1973 trials and reported similar findings, with little evidence of bias in non-randomized or non-blinded trials utilizing objective and mortality outcomes [47]. Therefore despite the lack of comparator for several of the observational studies, the emphasis on objective outcome measures of clinical risk and health status strengthens the results, and limits the bias inherent in observational studies.
The observational studies of whole practice naturopathic medicine themselves are of methodologic interest. Observational studies have been recognized as an important method for the evaluation of healthcare delivery, in part due to their generally larger size and longer duration, as well as including a more representative patient population than RCTs [48]. Research has also confirmed that well-designed observational studies can capture outcomes data that is comparable in accuracy to that reported by RCTs, without systematic overestimation of the magnitude of treatment effects [49,50]. Large analyses published in the New England Journal of Medicine have found that observational and randomized studies conducted in the same conditions and using similar outcome measures report similar estimations of treatment effects [49,50]. Yet, observational studies remain limited in their ability to control for confounders that may influence outcomes, e.g., other concurrent care services, and thus interpretation should remain conservative regarding attribution of cause and effect. Despite these limitations, the outcomes reported demonstrate no evidence of harm or worsening clinical status secondary to exposure to naturopathic medicine, and in fact suggest clinical benefit in all occasions.

Limitations
Despite finding beneficial effects on recognized predictors of cardiovascular events, the studies included in our review were not able to assess the effect of naturopathic medicine on the incidence of cardiovascular events or mortality from cardiometabolic disease. Second, due to heterogeneity among studies, we were not able to pool primary outcomes or quality of life data. It is hoped that with the growth of evidence in this area, a more rigorous synthesis of data may be possible in future. It should be noted that although the studies included here were pragmatic in nature, the delivery of care within the context of a research study may not always reflect that in the clinical field. The studies included here examined the practice of North American naturopathic medicine only, so these findings are not necessarily generalizable to the European or Australian practice of NM and also may not reflect the entirety of evidence relating to whole practice naturopathic medicine, or therapies used by naturopathic doctors in the treatment of chronic conditions. Finally, a considerable amount of time has elapsed from the date of the earliest published studies, during which naturopathic medicine has evolved to its current form.

Future Directions
Despite the results presented here in select conditions, research on the practice and outcomes of naturopathic medicine is in its infancy, and more research is required to further assess its contributions to health care. Because of the difficulties inherent in evaluating the multimodal, non-pharmaceutical-dependent, interventions used by emerging disciplines, we recommend all future evaluations of whole practice naturopathic medicine include objective outcomes, provided they are clinically relevant and important to the patient. Additionally, extreme care should be taken to avoid (or quantify and carefully describe) participant withdrawals, losses to follow-up, and other causes of missing outcome data. Trials with large amounts of missing outcome data may benefit from sensitivity analyses especially extreme plausible analyses [51,52]. In addition to design considerations, future research on naturopathic medicine should assess its potential on additional high morbidity, high mortality chronic diseases for which usual practices results in disappointing clinical outcomes (e.g., depression, chronic kidney disease, advanced cardiovascular disease, asthma, irritable bowel syndrome, inflammatory bowel disease, fibromyalgia, chronic fatigue syndrome) as well as include larger replication studies for diabetes, CVD prevention, pain conditions and mental health.

Conclusion
When examined as a body of literature, the universally positive trends in favor of naturopathic medicine warrant increased funding to continue and expand research efforts. The quality of research assessing clinically relevant disease-oriented and patient-centered outcomes of whole practice naturopathic medicine is generally good. Naturopathic medicine appears to be a system of medicine with potentially positive public health implications for a wide variety of chronic health conditions.