Journal of Addiction Research & Therapy
Open Access

Addiction disorders; Substance abuse; mHealth interventions; mHealth recruitment strategies; Retention

Background

Addiction disorders and problematic substance use are significant public health problems, requiring a cross-disciplinary and multi-level action approach to effective interventions. The prevalence of addictive disorders and problematic substance use varies by the substance or Behaviour of concern and across population groups. For example, international standardized prevalence rates of gambling disorders range from 0.5% to 7.6%, with an average rate across all countries of 2.3% [1]. While the global prevalence of alcohol use disorders is estimated at 8.6% (95% Confidence Interval (CI) 8.1-9.1) in men and 1.7% (95% CI 1.6-1.9) in women [2 ].

Age-standardized prevalence of dependent cannabis use is 3.5% [3] and an estimated 22% of the global population smoke tobacco daily and 0.77% use amphetamines daily [4]. The misuse and abuse of drugs contribute significantly to the global burden of disease. For example, 4.2% (3.7-4.6) of disability-adjusted life years (DALYs) are attributable to alcohol use. Despite the disease burden and individual harms experienced, and irrespective of the disorder, many people with gambling or substance use disorders are not receiving treatment. Access is a major barrier to care, especially when treatment interventions are primarily delivered face-to-face [5 ,6]. Financial issues can also be a barrier to treatment; for example, a USA survey of 9000 people with mental health and substance use disorders reported that 15% of respondents did not seek help at all and 17% left treatment early due to financial costs [7] Geographic location has also been found to be a barrier where people living in rural locations have less service provision or fewer choices than their urban counterparts [8]. Other barriers to seeking and receiving help include reports of a feeling of shame or stigma as well as a fear of government agencies [9]. Cultural appropriateness of services delivered and innate racial bias have also been reported as barriers [10]. As a result, significant proportions are without treatment or flexible treatment options [11 ,12].

mHealth interventions have been suggested as an alternative to overcome many barriers that deter individuals from seeking help. mHealth interventions are typically shorter and found to be more costeffective, enable immediate treatment access, and have a greater and more diverse reach than analogue interventions [13] Thus, they have the potential to reach a more significant number of those in need of help than traditional intervention models.

mHealth tools

mHealth is a catchall term that encompasses and refers to the many different capabilities of mobile phone technology, such as talking,texting, on and offline internet content and sensors within or tethered to mobile phones and applies them to health across the continuum [14].

Smartphone applications (apps) are among the more common mHealth tools developed for health interventions. Many have been designed and developed to support self-management and behaviour change for smoking cessation [15 ] cardiac rehabilitation, 16 healthy lifestyles [16 ], Diabetes [17 ], Human Immunodeficiency Virus (HIV) [18] nutrition [19 ] mental illness [20] and even youth driving [21] mHealth interventions designed and developed using good evidence collectively show promise [22], and mHealth and app use is growing in other health domains, including other forms of addiction and problematic substance use [23] However, relatively few mHealth interventions regarding problem, disordered or harmful gambling have been developed. In contrast, in alcohol and substance misuse, many of the apps that have been developed are not based on empirically supported interventions, have not been empirically tested and are often not readily available [24] This lack of evidence can lead to unintended negative consequences, such as delaying help-seeking or promoting information inconsistent with current health advice [25] However, apps that have been evaluated, such as those for alcohol Step Away A-CHESS [26] Promillekol [27] and SMS programmer with and without web-based support or feedback, [28-30] have been found effective in reducing substance use long-term. SMS aftercare programmes have also been evaluated in adults discharged from rehabilitation facilities [31] and to help adults reduce marijuana use [32].

Recruiting hard-to-reach populations

mHealth tools can potentially reach more hard-to-reach populations, such as those with comorbid mental health or substance use disorder and marginalized groups. In smoking cessation trials, retention of participants with mental health or substance use disorders or problematic substance use can be poor compared to other population groups [33,34] with some trials losing more than twothirds of their participants at follow-up. In general, people of colour, people from minority groups, and those from lower socioeconomic groups are underrepresented as research participants in clinical trials despite often having an increased disease burden due to socioeconomic determinants of health [35,36].

In 2019 it was estimated that 5 billion people worldwide had mobile devices, over half of which were smartphone [37]. In parallel, mobile technology has been proliferating. New capabilities such as GPS, augmented and virtual reality, wearable and implantable sensors, and biometric authentication are difficult to ignore. These capabilities have highlighted the role mobile devices such as smartphones can play in the addiction intervention space.

Rationale

This systematic review aimed to identify published peer-reviewed literature on the recruitment and retention of participants in studies of mHealth interventions for people with addiction and/or problematic substance use and to identify successful recruitment strategies. This systematic review focuses on different types of addictive disorders or problematic substance use, such as gambling, tobacco, problematic drinking and the use of these addictive substances.

Methods

We conducted a systematic review following the Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA) guidelines [38] the review protocol was prospectively registered with PROSPERO (CRD 42021279724).

Search strategy

We conducted an electronic search of Medline, Embase, PsychINFO, and Cochrane Central Register of Controlled Trials (CENTRAL). Search terms combined addiction-related terms with mHealth, treatment terms, participant recruitment and retention (see Appendix 1). An electronic search using Google Scholar to check for missed publications (limited to the first 200 results) was also conducted.

We limited the search to peer review literature and English language abstracts or text. Our search was limited to publications after January 1998, when text messaging became mainstream. All searches were conducted up until 02 August 2023.

In addition to our database search, we hand-searched the references of eligible publications for additional references.

Screening and data extraction

All search results were exported to Rayyan.ai, and duplicates were removed automatically. Titles and abstracts were screened independently by two authors (BK and JCM) against the screening criteria for potential relevance (Table 1). Only results papers were included, although protocol papers, lessons learned, and formative papers could be included as sources of additional information if the results paper was also included. Any disagreements between reviewers were resolved by a third reviewer (GH).

Table 1:Study inclusion criteria.

We retrieved the full-text articles of all relevant articles for further screening by both reviewers. All articles that were not excluded were imported to NVivo for data extraction.

Study quality

For all RCTs, the ROB-2 assessment [39] was conducted by one reviewer (BK or AO) (Table 2). For all non-randomized and quasirandomized studies, ROBINs-I [40] was used to evaluate quality (Table 3). A second reviewer (JM) assessed 20% of included studies. Any disagreements in the scoring between reviewers were resolved by discussion. If disagreements were unresolved by discussion, they were arbitrated (GH).

Table 2: Summary of ROB-2 quality assessment for included RCTs.

Table 3: Summary of ROBIN-S quality assessment for included non-RCTs.

Data synthesis

We summarised study information and conducted a narrative review and qualitative literature synthesis to summarize the findings across studies. We used descriptive statistics to summarize the study and participant characteristics where appropriate.

The following outcomes were evaluated where possible:

· Types of recruitment strategies used

· Effectiveness of different recruitment strategies

· Population groups targeted

· Effectiveness of recruitment strategies for different

population groups and different disorders

· Explanation for differential recruitment

We also compared different recruitment methods on recruitment and retention numbers and how effective strategies were with different population groups.

Results

Study selection

The electronic search results in 2135 papers. After duplicates were removed, 1654 papers were screened for eligibility. After initial screening by title and abstract, 1550 papers were excluded, and 104 were included, including eight additional results papers identified from protocol papers meeting criteria. After full-text screening, there were 60 relevant papers (Figure 1). Of these papers, 53 were primary analyses, five secondary analyses and two protocols (Table 4). Study designs for the primary analyses included 45 randomized controlled trials; one pseudo-randomized trial, five non-randomized trials, and one cluster randomized controlled trial.

Figure 1: PRISMA Flow diagram.

Table 4: Overview of included studies, characteristics and recruitment

Study characteristics

The included studies primarily comprised primary analyses (N=53) [15,28,41-91], secondary analyses (N=5), [92-96] and study protocols (N=2) [97,98] The primary analyses were mostly RTCs (N=45) [15,28 ,41-47,50-56,58 -84], [89,90] pseudo-RCTs (N=1), [91] cluster RCTs (N=2) [49,97] and non-RCTs (N=5) [ 48,85-88] Secondary analyses were all based on RCTs, and the study protocols were based on an RCT and cluster RCT design each.

Most of the included studies were concerned with smoking cessation (N=33) [15,41-43], [45-49], [51-54], [58,59], [61,65,70,71,74 ,77-80,83,85-87,89,91,92,97,98] of which 30 were primary analyses, one secondary analysis, and two study protocols. Nine studies were concerned with problematic substance use (i.e., heroin, cannabis) [62,63, 67, 72, 75, 82, 93-95] of which six were primary analyses and three secondary analyses. Twelve studies were concerned with alcohol use [28, 44, 50, 56, 60] [64, 66, 68, 69, 84, 88, 96] of which 11 were primary analyses and one a secondary analysis. Two studies were concerned with co-occurring alcohol use and smoking, [57,81] all of which were primary analyses. Two primary analyses were found of interventions for problem gambling.73 75 one study was concerned with vaping cessation [55] as the primary analysis.

More than half of the included studies were from the United States (US) (n=32) [41,42,45-47,50-52 ,54-56,58,59,61,62,66- 69,71,72,74,75,77,78,81,83,88,92-94,96] Twelve studies were from Europe (including two from the United Kingdom (UK)) [28,43,57,64,70,79,85-87,98] ten from Asia [48,49,63,65]. [76,82,84,95,97 89] and one from South America [53 ] Two studies were from Australia [60,73] one from New Zealand (NZ) [80 ] and one from Canada.[44] The remaining study was a multi-site study that included participants from the USA, Australia, Singapore, and the UK [44] The most common mHealth intervention was text message programs (n=20) [28,41,42,44,50-52,55,61,62 ,64,66,67,69,70,72,73,76,93,94] and mobile applications (n=25) [15,46,47,53,56,58-60,63,65,68,74,75,77- 79,81,85,86,88,95,96 89-91 ] followed by social media groups (Facebook, Twitter, Whatsapp, and WeChat) (n=5) [48,49,71,84,97] web and mobile applications (n=3) [43,82,98] web and text messaging program (n=2) [54,57] video-based text message program (n=1) [80], text message and text peer support program (n=-1) [83] web application (n=1)87 and cell-phone based counseling sessions (n=2) [45,92] The number of participants ranged from 5 to 2806. Twenty-one studies had less than 100 participants, twelve were 100-199 participants, and fourteen had 200-499 participants. Eleven studies had more than 500 participants, of which six had more than 1000 participants (Table 4).

Risk of bias

The quality and risk of bias assessment for the RCTs showed that the majority of papers were deemed as ‘low risk’ or ‘some concerns’ for overall bias. The main areas of concern were limited information on the selection of the reported result and potential deviations for intended interventions. The few RCTs deemed ‘high risk’ were mainly for providing little information on the selection of the reported result, randomization process, or concealment process.

For the non-RCTs, the quality and risk of bias assessment revealed the papers as ‘low’ for overall bias except for Vilaplana [42] deemed as ‘moderate’. Within each criteria domain, each criterion was mainly deemed as ‘low’ for the papers except for the missing data, measurement of outcomes, confounding, and deviations from intended intervention domains. The moderate rating for Vilaplana was due to little information on missing data, concealment among outcome assessors, selection of the reported result, and the administration of the study instruments [42]

Study findings

Only 17 studies reported on their recruitment targets, of which 2 were protocols (Table 4). Eleven studies met their recruitment target [47,64,65,70,71,76-79,86,97] and six did not meet their target. [44,52,55,56,76,86] The overall retention rates as follow-up according to the timeline reported from the included studies (23.1%-100%) (n=32) are summarised in Table 5. According to Dettori (2011) [100] retention rates of 95% or greater represent little bias (in green), 80- 95% show some bias (in yellow), and less than 80% represent serious bias (in red). Only three studies retained recruitment above 95% at their longest follow-up period [67,75,78] Two studies retained 95% at the first follow-up but fell below 95% at subsequent follow-up.[60,94] Thirteen studies [41,42,49],[53,55,56,58,60,80,94] retained less than 80% of participants at their final follow-up, including four studies that retained less than half of their participants at follow-up (Table 6).

Table 5: Overview of type of problem, sample size target and actual numbers and recruitment method.

Table 6: Follow-up rates from the largest to smallest final sample size.

In studies with high retention of participants (at least one followup rate ≥95% after Day 1), researchers appeared to use a combination of in-person (e.g., coordinating with healthcare clinics), traditional (e.g. print flyers and radio advertising) and online recruitment (e.g., digital signs, electronic health records) methods, rather than relying solely on one type of recruitment method [60,67,75,78] These studies also predominately involved populations already actively engaged in an existing service (e.g., treatment, health services) or easily accessible to the researchers (e.g., university students). As universities commonly conduct this research, these populations are usually easier to recruit and engage.

Follow-up methods used by studies with high retention included screening assessments conducted in person rather than online screening methods. [60,67,75,78] Using in-person screening methods could result in the screening assessment being conducted more rigorously and capturing participants more likely to adhere to a mHealth intervention. In studies with the lowest retention (at least one follow-up rate ≥70% after Day 1), researchers appeared to predominately rely on one primary recruitment method, explicitly using in-person recruitment through healthcare services or treatment programs [53,58,73,95] The mHealth interventions for these studies commonly include applications with mobile phone-based ecological momentary assessment (EMA) features. Authors of one of the studies have suggested that combining pre-intervention education with the EMA technology could increase adherence to the intervention among participants [95]. Screening among the studies with low retention reported that they conducted the screening online, which may have resulted in participants who were less likely to adhere to the study being included compared to in-person screening.

Problematic substance use (excluding alcohol)

Nine studies evaluated digital interventions for problematic substance use, including cannabis, opioids, and stimulants. [62,63,67,72,75,82,93-95] Recruitment methods included traditional media, such as flyers, radio, and print media, as well as online media, such as Craigslist, dating apps, and social media advertising. Five studies also conducted recruitment through treatment providers. The studies generally relied on multiple recruitment methods, primarily when recruiting hard-to-reach populations (i.e., Men Who Have Sex with Men (MSM) using methamphetamine vs. college students) [72,94] these studies included participants recruited from college student participant pools, clinical settings, and public settings. None of these studies reported a recruitment target. The actual recruitment numbers ranged from 5 participants to 401 participants. Four of the studies only included participants who met the Diagnostic Statistical Manual of Mental Disorders (DSM) criteria for substance use disorder [67,72,75,82,94,95] two studies included participants who had used substances in the previous 24 hours or 30 days, and one used the addiction severity index [63]. Two studies described the relationship of the recruiter to participants; one described it as generally by research staff attending treatment agencies with patients75, the other hired social workers to conduct trainings [82]

Participant retention was relatively high across all seven studies,with more than 80% of participants retained in all seven studies. Reasons for drop-out included arrests and withdrawal from the study. In three studies, participants were compensated for completing followup measures [67, 72, 94] Compensation ranged from USD 25 to USD 50 per follow-up and up to USD 150 over the course of a study. Retention was also maintained through community outreach [72,77] and treatment providers. [63] Follow-up ranged from 4 weeks (i.e., following intervention) to 9 months.

Alcohol

Thirteen studies evaluated digital interventions for alcohol use, including two interventions that combined alcohol and tobacco interventions [28,44,50,56,57,60,64,66,68,69,81,84,88,96] Seven interventions targeted participants meeting formal diagnostic criteria (i.e., Diagnostic and Statistical Manual of Mental Disorders), including three for alcohol use disorder. Ten studies recruited participants through treatment providers, including hospitals, outpatient clinics, and rehabilitation facilities [28,44,50,56,64,68,84,96,90,91] and one study recruited participants from schools [57] The remaining studies recruited through traditional and online media [60,69,81] including one study that recruited through online help-seeking resources [69] Three studies reported meeting their recruitment targets [44,56,64] one study [64] met their target while two did not, although they missed achieving their target by one participant.[44,56] Recruitment numbers ranged from 15 to 1471 participants. Participants were recruited over five months to 33 months. Of the included papers discussing the recruiter role, recruiters were typically study therapists, research assistants or project coordinators attending the primary setting (health clinic or school, for example) of their potential participants to discuss and enable them to take part [50,56,57,64].

Most of the studies from clinical settings reported high retention of participants (i.e., follow-up for more than 90% of participants [28,50,60,66,84] four studies reported retention under 90% at 89%,[81] 75%,[88] 73% [44] and 78%.56 Rete ntion of 76% was reported for the school setting [57] The remaining studies reported retention equal to or above 90%. [28,50,60,66,84]. Follow-up ranged from 4 weeks to 12 months. In four studies, participants were compensated for completing surveys, [57,69,81,88] which ranged from 10 Swiss Francs (USD$ 10.23) to up to USD 168 for completing the study.

Gambling

Two of the included studies were digital interventions for problem gambling [73,76] Both were text-based interventions using SMS or messenger apps as the mode of delivery. The two studies used different recruitment methods: one recruited through traditional methods via a gambling helpline, while the other used online advertisement. The recruitment duration was 5 and 12 months. Only one of the studies indicated a recruitment target; the target of 198 participants was exceeded, although only 197 participants were analyzed [76]. Both studies recruited more than 200 participants. The two studies differed in the stringency of the eligibility criteria. In Scott [76], participants were eligible if their problem gambling severity index (PGSI) score of three or greater. They were excluded if they were receiving faceto- face support for their gambling problems. In contrast, participants were included in Rodda [73] if they were engaged with at least one service offered by the Gambling Helpline. Retention rates also varied between the two studies: Rodda [73] reported 50% drop-out while Scott reported post-intervention assessment for 91-97% of participants analyzed; however, the study follow-up period was only four weeks compared to 12 weeks.

Smoking cessation: Hard-to-reach populations

Only six studies recruited specific population groups or were targeted for a specific population group, including smokers with HIVpositive status [45] pregnant women [42,52] smokers with a diagnosis of PTSD, schizophrenia, schizoaffective disorder, or mood disorders [59,78] and young Māori smokers.80 Eligibility criteria included a minimum number of cigarettes per day for most studies; however, in the trials targeting pregnant women, any cigarette consumption was sufficient to meet eligibility. Three of the studies recruited participants through clinical settings [45,52,78] with two of those using participants already receiving care at the associating clinic [45,52] One study recruited participants solely through online media41, and the last study used traditional and online media [80] One study did not provide recruitment information.59 Of the three studies that reported recruitment targets [52,78,80] only one met the recruitment target;78 the target sample size ranged from 60 to 1300 participants. Participants recruited ranged from 11 to 595 participants. One study disclosed that participants were engaged with the primary care provider or study physician to facilitate recruitment [59]

Retention rates varied considerably across these studies. One study of adult smokers with mental health conditions reported retention of 98% at 8 weeks [78] The study excluded participants with poor adherence to treatment, and study compensation included USD 110 and a smartphone used. Three studies reported retention in the 70- 85% range for follow-up between 4 weeks and six months [42, 59, 80] One study reported high lost-to-follow-up rates, with 39% of pregnant women lost to follow-up at 6-12 weeks. [52] Four studies reported compensation or incentives for participating in the study, which was reported as ranging between USD 110 and USD 530 [41,59,78,80].

Smoking cessation: General population

The largest group of studies was smoking cessation interventions for the general population [15,42,43,46- 49,51,53,54,58,61,65,70,71,74,77,79,83,85-87,92,97,98 89,91] of the twenty-seven smoking Cessation studies targeting the general population, online recruitment methods (such as social media and online advertisements) were used exclusively in twelve studies [15, 42,43,47,48,54,61,71,74,77,83,98] and offline recruitment methods (such as flyers and clinic visits) were used exclusively in eight studies [49,58,65,70,79,87,89-91] the remaining three studies used a combination of methods [46,53,86] and four did not provide information on their recruitment methods [51,85,92,97] Six studies recruited participants through clinical settings or smoking cessation services [49,58,65,70,78,79] One study, Graham et al., was a quality improvement study, meaning that participants were automatically enrolled in the study when they signed up for an online smoking cessation program. The number of participants ranged from 8 participants to 2,806 participants. Of the studies that discussed the relationship between the recruiter and participants, trained smoking cessation ambassadors (being college/university students and volunteers from non-governmental organizations) approached smokers at recruitment sites [97 ] and case managers liaised with participants in another study [74]. Eleven studies reported a recruitment target, reflecting that the studies targeting smoking cessation in the general population were generally clinical trials [43,46,47,65,70,71,77,79,86,97 ,98] the recruitment target ranged from 120 to 3000 participants. The recruitment target was met in almost all studies except Affret et al. 2020 [43] recruiting 2806 of the 3000 participants, and Rajani et al., 2021 [86] recruiting 116 of the 140 participants required.

Three studies reported high rates of loss-to-follow-up in one or both conditions [43,53] In Affret et al., almost two-thirds of participants in each group were lost to follow-up in six months [43]. In Goldenhersche et al., 80% of participants in the control group dropped out of the study after 90 days [53] Likewise, in Garcia-Pazo et al., 61% of the intervention group dropped out before the end of the treatment [91] Retention was also poor in Graham et al., where only 54% of participants completed the programme; however, the opt-out design of the study may have contributed to the high drop-out rate [54]. Most of the other studies reported retention between 70 and 90%, with five studies reporting retention above 90% [15,58,70,74,77] Twelve studies reported the use of incentives provided at follow-up, which ranged from around USD 10-15 per survey to USD 150 for completing 90% of ecological momentary assessments (EMA) [41,46,49,57,58,65,74,79,86]

Lessons learned

Several key lessons were highlighted in the included papers related to recruitment and retention. The authors reflected on their methods and insights into recruitment and retention that are likely useful for other researchers working in mHealth interventions for addiction and problematic substance use. These are summarised in Figure 2 and described below (Figure 2).

Figure 2: Key lessons learned for recruitment and retention.

Recruitment

Authors noted the need for proactive and responsive recruitment strategies, such as recruitment at large public settings, including schools [57] although these were not always effective [80] Timing of recruitment pushes; for example, around New Year, appeared to contribute to increases participant recruitment [55]. Provision of novel technology interventions, such as Virtual Reality (VR) and the use of a mobile Carbon Monoxide (CO) checker, also were effective in attracting participants [53,65] Concerns about privacy, especially for interventions for problematic substance use, were a potential deterrent to participation. These concerns were generally found in older rather than younger participants [95]

Internet advertisements for recruitment should be approached with caution as these methods result in large amounts of people registering interest but being subsequently disqualified [41] Researchers also noted the importance of knowing whether grants used to fund the research allow for the duration of recruitment to be extended, especially given the hard-to-reach nature of populations in the addictions field [52]

Retention

Potential facilitators of retention included participant verification, incentives, and technology. One study included safeguards to verify participant information, enabling follow-up at the intervention’s end [41] Incentives for completing follow-up tasks also appeared to enhance the retention of participants [55] Studies also noted how technology facilitated follow-ups, such as push notifications to notify participants of follow-up and the appearance of SMS on the user’s screen automatically [15, 69].

Two studies noted that the nature of the intervention, that is, passive versus active interventions, may have contributed to poor retention and utilization [58,88] Text-based interventions were discussed as active interventions with good reach and greater effectiveness compared to more passive interventions such as mobile applications which mean the user has to engage directly with the applications themselves [68- 70]. For example, tips or treatment content that are unlocked over time as an active mode of delivering the intervention content to participants [58].

Participant engagement to increase retention was also discussed. Han et al. [95] highlighted that the participants with problematic substance use preferred face-to-face rather than online engagements as most were isolated from society [95] Furthermore, Han also stated the importance of pre-intervention education for participants before engaging with mHealth interventions. This provides the knowledge needed to engage and install a study mobile application, which could help increase their engagement with the intervention and subsequent retention. [77,95] Cost barriers such as mobile phone data charges are also essential to address for participants to retain them in mHealth studies [80].

Discussion

Our review of recruitment and retention in mHealth interventions highlights the need to use various recruitment methods, especially when recruiting hard-to-reach communities. While some studies point to crucial techniques that have improved recruitment, such as partnering with community services with strong links to the target population, our analysis demonstrates that researchers should rely on more than just one method of recruitment or source when recruiting participants to mHealth interventions.

Our review also highlights the potential trade-off between recruitment and retention. In some studies, there was a high bar to participation in the study, either through safeguards (such as verified email address in Abroms et al. [41,42] or eligibility criteria [78] that excluded participants unlikely to engage with the intervention, which excluded high numbers of potential participants but enabled better retention of participants. In contrast, a low entry bar to participation may result in poorer retention of participants [73], with high lossto- follow-up affecting the analysis and validity of findings. Several studies also showed variability in retention across groups. For example, retention rates in Goldenhersch et al. [53] nearly halved between Day 1 and Day 90, which the study authors attributed to high loss to followup in the control group (97% Day 1 vs. 57% Day 90). This highlights the importance of ensuring strong recruitment and retention techniques across all study groups.

Differential retention across control and intervention conditions suggests that alternative study designs may be needed in the digital intervention space. Clinical trial methodologies, specifically RCTs, have been explicitly designed to evaluate pharmacological interventions rather than behavioral or mHealth interventions and may not be suitable for evaluating these kinds of interventions. Other reviews have highlighted the practical issues in evaluating mHealth apps using RCTs, including limitations in keeping up with technological changes [100]. There may also be ethical issues, such as the appropriateness of inactive controls when mHealth interventions digitize existing interventions and the risk of crossover. There is a need to identify trial designs to improve retention in mHealth interventions and adapt to evolving technology.

Technology was identified in some studies as a critical component in both recruitment and retention. Technology features such as active apps, text message messages and support for notification were seen as enhancing retention and building into existing platforms (such as WeChat) and ease of use.

Strengths and Limitations

Our review is limited by the exclusion of unpublished and grey literature. Studies that fail to meet the recruitment targets may go unpublished, particularly where recruitment has been significantly below expected. The failure to meet recruitment targets in such studies provides valuable lessons about recruiting hard-to-reach populations without publishing negative outcomes, including lessons that there is a risk of repeating the same mistakes.

Strength of our review is the inclusion of different types of addiction and problematic substance use to compare how the recruitment and retention problem has been addressed across different domains. However, a limitation of our review was that most of the studies involved smoking cessation and primarily involved the general population with a hard to reach subgroup, which may not have generalizability to other forms of addiction and problematic substance use amongst hard-toreach populations. In particular, we found few mHealth interventions to address problem gambling.

Our review was also limited by the amount of information provided in studies, with some describing very little information about the retention and recruitment methods used or publishing this detail in protocol papers. The low proportion of papers documenting their recruitment and retention efforts in this review demonstrates the need for journal publications to request comprehensive reporting on their recruitment and retention efforts in the methods and discussion sections of their papers if these are not published elsewhere, for example, in a published study protocol or in a clinical trial registration platform. Ensuring this documentation is available enables researchers, practitioners and the public to understand the barriers and enables recruitment and retention efforts in mHealth interventions for addiction and problematic use to learn from and develop new strategies to engage better those with addiction and problematic use patterns that we aim to help.

Implications

Researchers need to consider the trade-off between recruiting high numbers with low retention against low numbers with high attrition when defining the eligibility and exclusion criteria for participation. For example, more stringent criteria for participants, such as specific diagnosis or adherence to treatment – can improve participant retention but limit the participants that may take part in the study, as well as limiting the ecological validity of the study as treatment adherence is likely lower outside of clinical trial settings.

Technological features that improved retention, such as text messages, should be incorporated into mHealth intervention programmes whenever feasible. However, those with limited access to such services, e.g., rural and low socioeconomic communities, may need other modalities accommodated within the mHealth space. Often, the populations with limited access to such interventions will benefit most from these interventions. Therefore, future studies should reflect on novel ways to minimize the effect of different settings and population groups to ensure that barriers are not designed into mHealth interventions.

Alternative study designs might also be required in the digital intervention space, given that the RCT was not designed for digital and/ or psychological interventions and does not appear fit for purpose in these areas. As such, questions must be asked regarding designing trials to retain participants effectively. What designs can mitigate retention/ recruitment issues? How do we take into account participant preference when designing trials? Perhaps there is a need to move towards steppedwedge designs or adaptive trials, while the interventions might need greater personalization, regular feedback, and more engaging content and gamification, depending on the delivery method.

Conclusion

This review has shown the importance of using various recruitment strategies for mHealth intervention studies targeting addictive disorders and substance use. Specifically, our review has shown that partnering with community services with a solid link to the target populations is a core foundation of all recruitment strategies. A failure to effectively recruit and retain study participants affects our ability as researchers to evaluate the effectiveness of new interventions. It, therefore, brings into question the validity and generalizability of study findings, which may not be a valid outcome. However, the consequences of this outcome can be far-reaching in that funding for research into the effectiveness of mHealth interventions for those living with problem addictions may become more difficult to obtain.

Acknowledgements

Funding

This was an investigator-initiated study funded by a grant from the Health Research Council (reference 18-237).

Declarations of interest

The authors have no conflicts of interest to declare.

Author contributions

GH conceived of the study and developed the study design. BK, and JMcC contributed to the study design and were involved in data collection. KG and AO’S were involved in data collection. DN contributed to drafting and critical revisions of the manuscript. All authors contributed to revisions and final manuscript approval.

1. Volberg RA, Williams R (2012) Developing a Short Form of the PGSI: Report to the Gambling Commission. Birmingham, England.

Google Scholar

2. World Health Organisation (2018). Global status report on alcohol and health. Report no. 978-92-4-156563- 9 World Health Organisation.

Google Scholar

3. Degenhardt L, Charlson F, Ferrari A (2018) The global burden of disease attributable to alcohol and drug use in 195 countries and territories 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016. The Lancet Psychiatry 5: 987-1012.

Indexed at, Google Scholar, Crossref

4. Peacock A, Leung J, Larney S (2018) Global statistics on alcohol, tobacco and illicit drug use: 2017 status report. Addiction (Abingdon, England) 113: 1905-1926.

Google Scholar, Crossref

5. Ministry of Health. Preventing and Minimising Gambling Harm: Practitioner’s Guide. 2019. Wellington: Ministry of Health.

Google Scholar

6. Galea-Singer S, Newcombe D, Farnsworth-Grodd V (2020) Challenges of virtual talking therapies for substance misuse in New Zealand during the COVID-19 pandemic: an opinion piece. The New Zealand Med J 133: 104.

Indexed at, Google Scholar

7. Mojtabai R, Olfson M, Sampson NA (2011) Barriers to mental health treatment: results from the NationalComorbidity Survey Replication. Psychol Med 41: 1751-1761.



Indexed at , Google Scholar, Crossref

8. Pullen E and Oser C (2014) Barriers to substance abuse treatment in rural and urban communities: counselor perspectives. Substance use & misuse 49: 891-901.

Indexed at , Crossref

9. Rizzo D, Mu T, Cotroneo S (2022) Barriers to Accessing Addiction Treatment for Women at Risk of Homelessness. Frontiers in Global Women’s Health 3.

Indexed at, Google Scholar, Crossref

10. Rockville MD (2021) Center for Behavioral Health Statistics and Quality cR MD: Substance Abuse and Mental Health Services Administration. Racial/ethnic differences in substance use, substance use disorders, and substance use treatment utilization among people aged 12 or older (2015-2019) Substance Abuse and Mental Health Services Administration.

Google Scholar

11. Suurvali H, Cordingley J, Hodgins DC (2009) Barriers to Seeking Help for Gambling Problems: A Review of the Empirical Literature. J Gambl Stud 25: 407-424.

Indexed at, Google Scholar, Crossref

12. Cumming C, Troeung L, Young JT (2016) Barriers to accessing methamphetamine treatment: A systematic review and meta-analysis. Drug and Alcohol Depend 168: 263-273.

Indexed at, Google Scholar, Crossref

13. Schweitzer J, Synowiec C (2012) the Economics of Health and Health. J Health Commun17: 73-81.

Indexed at, Google Scholar, Crossref

14. Cameron JD, Ramaprasad A, Syn T (2017) ontology of and roadmap for mHealth research. Int J Med Inform; 100: 16-25.

Indexed at, Google Scholar, Crossref

15. Bindhim NF, McGeehan K, Trevena L (2018) Smartphone Smoking Cessation Application (SSC App) trial: a multicounty double-blind automated randomized controlled trial of a smoking cessation decision-aid ‘app’ 8: e017105.

Indexed at, Google Scholar, Crossref

16. Pfaeffli L, Maddison R, Whittaker R (2012) A mHealth cardiac rehabilitation exercise intervention: findings from content development studies. BMC cardio dis 12: 36.

Indexed at, Google Scholar

17. Gimbel R, Shi L, Williams JE (2017) Enhancing mHealth Technology in the Patient-Centered Medical Home Environment to Activate Patients with Type 2 Diabetes: A Multisite Feasibility Study Protocol. JMIR Res Proto 6: e38.

Indexed at, Google Scholar, Crossref

18. Muessig KE, Nekkanti M, Bauermeister J (2015) A Systematic Review of Recent Smartphone, Internet and Web 2.0 Interventions to Address the HIV Continuum of Care. Current HIV/AIDS Rep 12: 173- 190.

Indexed at, Google Scholar, Crossref

19. Volkova E, Li N, Dunford E (2016) Smart RCTs: Development of a Smartphone App for Fully Automated Nutrition-Labeling Intervention Trials. JMIR mHealth uHealth 4: e23.

Google Scholar

20. Manji H, Saxena S (2019).The power of digital tools to transform mental healthcare.

Indexed at

21. Warren I, Meads A, Whittaker R (2018) Behavior Change for Youth Drivers: Design and Development of a Smartphone-Based App. JMIR Formativ Res2: e25.

Indexed at, Google Scholar, Crossref

22. Payne HE, Lister C, West JH (2015) Behavioral functionality of mobile apps in health interventions: a systematic review of the literature. J Med Internet Res3.

Indexed at , Google Scholar, Crossref

23. Martínez PB, Torre-Díez I, López CM (2013) Mobile health applications for the most prevalent conditions by the world health organization: Review and analysis. JMIR 15.

Indexed at, Google Scholar, Crossref

24. Tofighi B, Chemi C, Ruiz VJ (2019) Smartphone Apps Targeting Alcohol and Illicit Substance Use: Systematic Search in in Commercial App Stores and Critical Content Analysis. JMIR Mhealth Uhealth7: e11831.

Indexed at , Google Scholar, Crossref

25. Cohn AM, Hunter-Reel D, Hagman BT (2011) Promoting Behavior Change from Alcohol Use Through Mobile Technology: The Future of Ecological Momentary Assessment. Alcohol Clin Exp Res 35: 2209-2215.

Indexed at, Google Scholar, Crossref

26. Gustafson DH, McTavish FM, Chih M-Y (2014) a Smartphone Application to Support Recovery from Alcoholism: A Randomized Clinical Trial. JAMA Psychiatry 71: 566-572.

Indexed at, Google Scholar, Crossref

27. Gajecki M, Berman AH, Sinadinovic K (2014) Mobile phone brief intervention applications for risky alcohol use among university students: a randomized controlled study. Addict Sci Clin Pract 9: 11.

Indexed at, Google Scholar, Crossref

28. Agyapong VIO, Ahern S, McLoughlin DM (2012) Supportive text messaging for depression and comorbid alcohol use disorder: single-blind randomized trial. J Affect Disord 141: 168-176.

Indexed at , Google Scholar, Crossref

29. Haug S, Schaub MP, Venzin V (2013) A pre-post study on the appropriateness and effectiveness of a Web- and text messaging-based intervention to reduce problem drinking in emerging adults. J Med Internet Res 15: 196.

Indexed at, Google Scholar, Crossref

30. Suffoletto B, Callaway C, Kristan J (2012) Text-Message-Based Drinking Assessments and Brief Interventions for Young Adults Discharged from the Emergency Department. Alcohol Clin Exp Res 36: 552-560.

Indexed at , Google Scholar, Crossref

31. Lucht MJ, Hoffman L, Haug S (2014) A Surveillance Tool Using Mobile Phone Short Message Service to Reduce Alcohol Consumption Among Alcohol-Dependent Patients. Alcohol Clin Exp Res 38: 1728-1736.

Indexed at, Google Scholar, Crossref

32. Shrier LA, Rhoads A, Burke P (2014) Real-time, contextual intervention using mobile technology to reduce marijuana use among youth: A pilot study. Addict Behav 39: 173-180.

Indexed at, Google Scholar, Crossref

33. Apollonio D, Philipps R, Bero L (2016) Interventions for tobacco use cessation in people in treatment for or recovery from substance use disorders. CDSR.

Indexed at , Google Scholar, Crossref

34. Van der Meer RM, Willemsen MC, Smit F, (2013) Smoking cessation interventions for smokers with current or past depression. CDSR.

Indexed at, Google Scholar, Crossref

35. Gilmore-Bykovskyi AL, Jin Y, Gleason C (2019) Recruitment and retention of underrepresented populations in Alzheimer’s disease research: A systematic review. TRCI 5: 751-770.

Indexed at, Google Scholar, Crossref

36. Heller C, Balls-Berry JE, Nery JD (2014) Strategies addressing barriers to clinical trial enrollment of underrepresented populations: A systematic review. Contemp Clin Trials 39: 169-182.

Indexed at, Google Scholar, Crossref

37. Silver L (2019) Smartphone Ownership Is Growing Rapidly Around the World, but Not Always Equally. Pew Research Center.

Indexed at, Google Scholar, Crossref

38. Page MJ, McKenzie JE, Bossuyt PM (2021) The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ 372: 71.

Indexed at, Google Scholar, Crossref

39. Guyatt GH, Oxman AD, Vist GE (2008) GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 336: 924.

Indexed at, Google Scholar, Crossref

40. Sterne JA, Hernán MA, Reeves BC (2016) ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ 355.

Indexed at, Google Scholar, Crossref

41. Abroms LC, Boal AL, Simmens SJ (2014) A randomized trial of Text2Quit: A text messaging program for smoking cessation. Am J Prev Med 47: 242-250.

Indexed at, Google Scholar, Crossref

42. Abroms LC, Johnson PR, Leavitt LE (2017) A Randomized Trial of Text Messaging for Smoking Cessation in Pregnant Women. Am J Prev Med 53: 781-790.

Indexed at, Google Scholar, Crossref

43. Affret A, Luc A, Baumann C, (2020) Effectiveness of the e-Tabac Info Service application for smoking cessation: A pragmatic randomized controlled trial. BMJ Open 10.

Indexed at, Google Scholar, Crossref

44. Agyapong VIO, Juhás M, Mrklas K, (2018) Randomized controlled pilot trial of supportive text messaging for alcohol use disorder patients. J Subst Abuse Treat 94: 74-80.

Indexed at, Google Scholar, Crossref

45. Aigner CJ, Gritz ER, Tamí-Maury I (2017) the role of pain in quitting among human immunodeficiency virus (HIV)–positive smokers enrolled in a smoking cessation trial. Substance Abuse 38: 249-252.

Indexed at, Google Scholar, Crossref

46. Bricker JB, Mull KE, Kientz JA (2014) Randomized, controlled pilot trial of a smartphone app for smoking cessation using acceptance and commitment therapy. Drug and Alcohol Dependence 143: 87-94.

Indexed at, Google Scholar, Crossref

47. Bricker JB, Watson NL, Mull KE (2020) Efficacy of Smartphone Applications for Smoking Cessation: A Randomized Clinical Trial. JAMA Intern Med 180: 1472-1480.

Indexed at, Google Scholar, Crossref

48. Chen J, Ho E, Jiang Y (2020) Mobile social network-based smoking cessation intervention for Chinese male smokers: Pilot randomized controlled trial. JMIR mHealth uHealth 8.

Indexed at, Google Scholar, Crossref

49. Derek Cheung YT, Helen Chan CH, Lai CKJ (2015) Using Whatsapp and Facebook online social groups for smoking relapse prevention for recent quitters: A pilot pragmatic cluster randomized controlled trial. J Med Internet Res 17.

Indexed at, Google Scholar, Crossref

50. DeMartini KS, Schilsky ML, Palmer A, (2018) Text Messaging to Reduce Alcohol Relapse in Prelisting Liver Transplant Candidates: A Pilot Feasibility Study. Alcohol Clin Exp Res 42: 761-769.

Indexed at, Google Scholar, Crossref

51. DeStasio KL, Hill AP and Berkman ET (2018) Efficacy of an SMS-Based Smoking Intervention Using Message Self- Authorship: A Pilot Study. J Smok Cessat 13: 55-58.

Indexed at, Google Scholar, Crossref

52. Forinash AB, Yancey A, Chamness D, (2018) Smoking Cessation Following Text Message Intervention in Pregnant Women. Ann Pharmacother 52: 1109-1116.

Indexed at, Google Scholar, Crossref

53. Goldenhersch E, Thrul J, Ungaretti J (2020) Virtual reality smartphone-based intervention for smoking cessation: Pilot randomized controlled trial on initial clinical efficacy and adherence. J Medical Internet Res 22.

Indexed at, Google Scholar, Crossref

54. Graham AL, Papandonatos GD, Jacobs MA (2020) Optimizing Text Messages to Promote Engagement With Internet Smoking Cessation Treatment: Results From a Factorial Screening Experiment. J Med Internet Res 22: e17734.

Indexed at, Google Scholar, Crossref

55. Graham AL, Amato MS, Cha S (2021) Effectiveness of a Vaping Cessation Text Message Program among Young Adult e-Cigarette Users: A Randomized Clinical Trial. JAMA Intern Med 181: 923-930.

Indexed at, Google Scholar, Crossref

56. Gustafson DH, McTavish FM, Chih MY (2014) A smartphone application to support recovery from alcoholism a randomized clinical trial. JAMA Psychiatry 71: 566-572.

Indexed at, Google Scholar, Crossref

57. Haug S, Paz Castro R, Kowatsch T (2017) Efficacy of a technology-based, integrated smoking cessation and alcohol intervention for smoking cessation in adolescents: Results of a cluster-randomized controlled trial. J Subst Abuse Treat 82: 55-66.

Indexed at, Google Scholar, Crossref

58. Hébert ET, Ra CK, Alexander AC (2020) a mobile just-in-time adaptive intervention for smoking cessation: Pilot randomized controlled trial. J. Medical Internet Res 22.

Indexed at, Google Scholar, Crossref

59. Hicks TA, Thomas SP, Wilson SM (2017) A Preliminary Investigation of a Relapse Prevention Mobile Application to Maintain Smoking Abstinence Among Individuals With Posttraumatic Stress Disorder. J Dual Diagn 13: 15-20.

Indexed at, Google Scholar, Crossref

60. Hides L, Quinn C, Cockshaw W (2018) Efficacy and outcomes of a mobile app targeting alcohol use in young people. Addictive Behaviors 77: 89-95.

Indexed at, Google Scholar, Crossref

61. Hoeppner BB, Hoeppner SS, Abroms LC (2017) how do text-messaging smoking cessation interventions confer benefit? A multiple mediation analysis of Text2 Quit. Addiction; 112: 673-682.

Indexed at, Google Scholar, Crossref

62. Keoleian V, Alex Stalcup S, Polcin DL A (2013) Cognitive Behavioral Therapy-Based Text Messaging Intervention for Methamphetamine Dependence. J Psych Drugs 45: 434-442.

Indexed at, Google Scholar, Crossref

63. Liang D, Han H, Du J (2018) a pilot studies of a smartphone application supporting recovery from drug addiction. J Sub Abu Treat 88: 51-58.

Indexed at, Google Scholar, Crossref

64. Lucht M, Quellmalz A, Mende M (2021) Effect of a 1-year short message service in detoxified alcohol-dependent patients: a multi-center, open-label randomized controlled trial. Addiction 116: 1431-1442.

Indexed at, Google Scholar, Crossref

65. Masaki K, Tateno H, Nomura A (2020) A randomized controlled trial of a smoking cessation smartphone application with a carbon monoxide checkers. npj Digital Med 3.

Indexed at, Google Scholar

66. Mason M, Benotsch EG, Way T (2014) Text messaging to increase readiness to change alcohol use in college students. J Primary Prev 35: 47-52.

Indexed at, Google Scholar, Crossref

67. Mason MJ, Zaharakis NM, Moore M (2018) who responds best to text-delivered cannabis use disorder treatment? A randomized clinical trial with young adults. Psychology of addictive behaviors: J Soc Psych Add Beh 32: 699-709.

Google Scholar

68. McTavish FM, Chih MY, Shah D (2012) How patients recovering from alcoholism use a smartphone intervention. J Dual Diag 8: 294-304.

Google Scholar, Crossref

69. Muench F, Van Stolk-Cooke K, Kuerbis A (2017) A randomized controlled pilot trial of different mobile messaging interventions for problem drinking compared to weekly drink tracking 12.

Indexed at, Google Scholar, Crossref

70. Müssener U, Bendtsen M, Karlsson N (2015) Effectiveness of short message service text-based smoking cessation intervention among university students a randomized clinical trial. JAMA Int Med 176: 321-328.

Indexed at, Google Scholar, Crossref

71. Pechmann C, Delucchi K, Lakon CM (2017) Randomized controlled trial evaluation of tweet2quit: A social network quit-smoking intervention. Tobacco Control 26: 188-194.

Indexed at, Google Scholar, Crossref

72. Reback CJ, Fletcher JB, Swendeman DA (2019) Theory-Based Text-Messaging to Reduce Methamphetamine Use and HIV Sexual Risk Behaviors among Men Who Have Sex with Men: Automated Unidirectional Delivery Outperforms Bidirectional Peer Interactive Delivery. AIDS and Behavior; 23: 37-47.

Indexed at, Google Scholar, Crossref

73. Rodda SN, Dowling NA, Knaebe B (2018) Does SMS improve gambling outcomes over and above access to other e-mental health supports? A feasibility study. Int Gambl Stud; 18: 343-357. Article.

Indexed at, Google Scholar, Crossref

74. Schlam TR, Baker TB (2020) Playing Around with Quitting Smoking: A Randomized Pilot Trial of Mobile Games as a Craving Response Strategy. Games Health J 9: 64-70.

Indexed at, Google Scholar, Crossref

75. Scott CK, Dennis ML, Johnson KA (2020) A randomized clinical trial of smartphone self-managed recovery support services. J Substance Abuse Treat 117.

Indexed at, Google Scholar, Crossref

76. So R, Furukawa TA, Matsushita S (2020) Unguided Chatbot-Delivered Cognitive Behavioural Intervention for Problem Gamblers Through Messaging App: A Randomized Controlled Trial. J Gambling Stud 36: 1391-1407.

Indexed at, Google Scholar, Crossref

77. Sridharan V, Shoda Y, Heffner J (2019) a pilot randomized controlled trial of a web-based growth mindset intervention to enhance the effectiveness of a smartphone app for smoking cessation. JMIR mHealth and uHealth 7.

Indexed at, Google Scholar, Crossref

78. Vilardaga R, Rizo J, Palenski PE (2020) Pilot Randomized Controlled Trial of a Novel Smoking Cessation App Designed for Individuals With Co-Occurring Tobacco Use Disorder and Serious Mental Illness. Nicotine & tobacco research: official J Soc Res Nicotine and Tobacco 22: 1533-1542.

Google Scholar

79. Webb J, Peerbux S, Smittenaar P (2020) Preliminary outcomes of a digital therapeutic intervention for smoking cessation in adult smokers: Randomized controlled trial. JMIR Mental Health 7.

Indexed at, Google Scholar, Crossref

80. Whittaker R, Dorey E, Bramley D (2011) a theory-based video messaging mobile phone intervention for smoking cessation: Randomized controlled trial. J Med Int Res13.

Indexed at, Google Scholar, Crossref


81. Witkiewitz K, Desai SA, Bowen S (2014) Development and evaluation of a mobile intervention for heavy drinking and smoking among college students. Psychology of Addictive Behaviors 28: 639-650.



Indexed at, Google Scholar, Crossref

82. Xu X, Chen S, Chen J (2021) Feasibility and Preliminary Efficacy of a Community-Based Addiction Rehabilitation Electronic System in Substance Use Disorder: Pilot Randomized Controlled Trial. JMIR Mhealth Uhealth 9: e21087.

Indexed at, Google Scholar, Crossref

83. Ybarra ML, Holtrop JS, Prescott TL (2013) Pilot RCT results of stop my smoking USA: A text messaging-based smoking cessation program for young adults. Nicotine Tob Res 15: 1388-1399.

Indexed at, Google Scholar, Crossref

84. Chen J, Qian M, Sun C (2019) Clinical effectiveness of cognitive behavioral therapy on alcohol-dependent patients: An observation with the We Chat platform. General Psychiatry 32.

Indexed at, Google Scholar, Crossref

85. Eiler TJ, Haßler B, Grünewald A (2020) Swipe up to smoke less cigarettes! Introducing a mobile Approach- Avoidance Task Application to fight Smoking. Current Directions Bio Eng 6.

Google Scholar, Crossref

86. Rajani NB, Mastellos N and Filippidis FT (2021) Self-efficacy and motivation to quit of smokers seeking to quit: Quantitative assessment of smoking cessation mobile apps. JMIR mHealth and uHealth 9.

Google Scholar

87. Vilaplana J, Solsona F, Abella F (2014) S-PC: An e-treatment application for management of smoke-quitting patients. Computer Methods and Programs in Biomedicine 115: 33-45.

Indexed at, Google Scholar, Crossref

88. Gonzalez VM and Dulin PL (2015) Comparison of a smartphone app for alcohol use disorders with an internet- based intervention plus bibliotherapy: A pilot study. J Consul Clin Psych 83: 335-345.

Indexed at, Google Scholar, Crossref

89. Asayut N, Olson PS, Kanjanasilp J (2022) a community pharmacist-led smoking cessation intervention using a smartphone app (PharmQuit): A randomized controlled trial. 17: e0265483.

Indexed at, Google Scholar, Crossref

90. Farren C, Farrell A, Hagerty A (2022) A 6-Month Randomized Trial of a Smartphone Application U Control Drink, in Aiding Recovery in Alcohol Use Disorder. Eur Addict Res 28: 122-133. 20211119.

Indexed at, Google Scholar, Crossref

91. Garcia-Pazo P, Sese A, Llabres J (2021) No Fumo A Clinical Trial of an mHealth for Smoking Cessation with Hospitalized Patients. Int J Environ Res Public Health; 18: 20211006.

Indexed at, Google Scholar, Crossref

92. Tamí-Maury I, Vidrine DJ, Fletcher FE (2013) Poly-tobacco use among HIV-positive smokers: Implications for smoking cessation efforts. Nicot Tobacco Res 15: 2100-2106.

Indexed at, Google Scholar, Crossref

93. Mason MJ (2020) Depressive symptoms moderate cannabis use for young adults in a Text-Delivered randomized clinical trial for cannabis use disorder. Addict Behaviors 104.

Indexed at, Google Scholar, Crossref

94. Mason MJ, Zaharakis NM, Russell M A (2018) pilot trial of text-delivered peer network counseling to treat young adults with cannabis use disorder. J Sub Abuse Treat 89: 1-10.

Indexed at, Google Scholar, Crossref

95. Han H, Zhang JY, Hser YI (2018) Feasibility of a mobile phone app to support recovery from addiction in China: Secondary analysis of a pilot study. JMIR mHealth and uHealth 6.

Indexed at, Google Scholar, Crossref

96. Glass JE, McKay JR, Gustafson DH (2017) Treatment seeking as a mechanism of change in a randomized controlled trial of a mobile health intervention to support recovery from alcohol use disorders. J Sub Abuse Treat 77: 57-66.

Indexed at, Google Scholar, Crossref

97. Luk TT, Li WHC, Cheung DYT (2019) Chat-based instant messaging support combined with brief smoking cessation interventions for Chinese community smokers in Hong Kong: Rationale and study protocol for a pragmatic, cluster-randomized controlled trial. Contemporary Clinical Trials 77: 70-75.

Indexed at, Google Scholar, Crossref

98. Cambon L, Bergman P, Le Faou A (2017) Study protocol for a pragmatic randomized controlled trial evaluating efficacy of a smoking cessation Tabac Info Service ee-TIS trial. Open 7: 13604-13604.

Google Scholar, Crossref

99. Keoleian V, Alex Stalcup S, Polcin DL A Cognitive Behavioral Therapy-Based Text Messaging Intervention for Methamphetamine Dependence.

Google Scholar, Crossref

100. Dettori JR (2011) Loss to follow-up. Evid Based Spine Care J 2: 7-10.

Google Scholar

101. Pham Q, Wiljer D, Cafazzo JA (2016) Beyond the Randomized Controlled Trial: A Review of Alternatives in mHealth Clinical Trial Methods. JMIR mHealth and uHealth 4: e107-e107.

Google Scholar