Predictors of Sustained Implementation of Low-Literacy Health Education Programs

Bergen B Nelson^1,2,*, Carol Teutsch³, Paul J Chung^1,2,4, and Ariella Herman³

¹Department of Pediatrics, Mattel Children’s Hospital and David Geffen School of Medicine at UCLA, USA

²UCLA Children’s Discovery and Innovation Institute, USA

³Health Care Institute, Anderson School of Management, UCLA, USA

⁴Department of Health Management and Policy, Fielding School of Public Health, UCLA 5RAND Corporation, Santa Monica, CA, USA

*Corresponding Author:

Bergen B Nelson
Division of General Pediatrics
UCLA, 10833 Le Conte Ave
Los Angeles, CA, USA
Tel: (310) 794-8833
Fax: (310) 206-4855
E-mail: bnelson@mednet.ucla.edu

Received date August 12, 2014; Accepted date November 19, 2014; Published date November 21, 2014

Citation: Nelson BB, Teutsch C, Chung PJ, Herman A (2014) Predictors of Sustained Implementation of Low-Literacy Health Education Programs. J Community Med Health Educ 4:314. doi:10.4172/2161-0711.1000314

Copyright: © 2014 Nelson BB, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Visit for more related articles at Journal of Community Medicine & Health Education

Abstract

Sustained implementation of health education interventions is important to optimize the impact of initial investments in program development and training. From 2002-2012, the Health Care Institute (HCI) at UCLA trained 192 Head Start grantees, serving low-income preschool children and families in the U.S., on how to implement lowliteracy health education programs, using a train-the-trainer (TTT) model. The purpose of this study was to investigate what key factors of implementation are associated with sustainability of HCI programs, based on an online survey of Head Start grantees who have participated in the HCI TTT. A 51% response rate was achieved, with 96 surveys returned from the 188 sent to agencies that are still open. Of the grantees responding to the survey, 84% reported that they continue to implement HCI programs and 71% continue to implement them annually. Key predictors of annual implementation included: engagement of stakeholders at all levels; continuing to provide incentives for families; putting HCI into the annual training and technical assistance budget; sending additional staff to the TTT; seeing an improvement in overall program performance; and adapting HCI topics to meet local Head Start grantee needs. Qualitative responses described these same factors in additional detail. These results are consistent with previous research on program sustainability and suggest which elements of implementation may be most important as initial and ongoing investments, if sustainability is a program goal.

Keywords

Head start program; Health education; Program implementation; Sustainability

Introduction

Achieving sustained implementation of health-related projects beyond initial funding periods is an important feature of nearly all successful public health and policy investments. Despite increasing interest in implementation science and a growing literature on program sustainability [1-5], few previous studies have empirically measured the specific factors associated with sustainability. The purpose of this study was to examine key predictors of sustained implementation of low-literacy health education programs in Head Start, a federally-funded preschool program in the United States that serves approximately one million low-income families each year. Because of the disproportionate health risks faced by families living in poverty [6-8], especially when also challenged by poor health literacy, finding effective and sustainable approaches to health promotion is especially important. This study analyzes survey data from Head Start grantees trained from 2002-2012 to deliver health education interventions, using an approach developed by the Health Care Institute (HCI) at the Anderson School of Management at the University of California, Los Angeles (UCLA).

The HCI training approach has been described previously [9] and is based upon a 2-day multi-agency train-the-trainer (TTT) event, which provides teams from each Head Start grantee agency with the skills for project planning; targeted marketing approaches for staff, parents, and community; and tools for staff and family engagement. The health topic showcased during the TTT demonstrates how to engage and teach parents of young children to manage common childhood illnesses, emphasizing key basic skills such as how to take a child’s temperature and when to contact a health care provider. Prior research has shown a reduction in reported emergency room visits among families after participating in this health education module [10]. Additional HCI modules available to the Head Start grantees participating in this study included oral health, healthy nutrition and physical activity, prenatal care, vaccination, over-the-counter medications and sun safety. The healthy nutrition and physical activity module (“Eat Healthy, Stay Active!”) has previously been associated with statistically significant reductions in body mass index (BMI) among Head Start students, parents and staff [11]. Once grantees have participated in the full 2-day HCI TTT program, they may choose to continue to implement additional HCI health promotion modules for families using the same approach, with local adaptations as needed, without needing to attend another live training event.

The purpose of this study was to document to what extent Head Start grantees trained in the HCI approach continued to implement HCI health education modules, and to explore the factors associated with sustained implementation.

A long-standing conceptual model for sustainability comes from Robert Yin’s (1979, 1981) ‘routinization’ framework, developed using case studies of how innovations introduced into local governments become standard practice [12,13]. Yin argued that new practices become routine when 12 processes take place, which can be grouped into 5 domains: 1) budget (program becomes supported with hard money and survives budget cycles); 2) personnel (program activities become part of job description, survive staff turnover, are spread to all users in an agency, and program champions are promoted); 3) supply and maintenance (required materials are provided by agency and survive equipment turnover); 4) training (there are many training cycles and skills become standard part of job) and 5) organizational governance (program is recognized in agency procedures and regulations and considered permanent within agency). Although subsequent scholars have argued that assessment of sustainability should be included in any program evaluation, there are few empirical studies that have examined the factors related to sustainability [14], and few studies have looked back at how long programs continue after initial funding has ended.

In this survey we asked participants, all of whom were Head Start grantees who had attended at least one HCI TTT, whether they continued to implement HCI health education modules for their families. We probed for details of their implementation practices and local adaptations as a way to elucidate predictors of sustainability.

Methods

Survey

An online survey using Qualtrics was sent by email in 2013 to the directors of all of the Head Start grantee agencies (N=192) that participated in the HCI TTT program between 2002 and 2012. Each survey response represented one unique Head Start agency. Primary questions on the survey included, “Do you still conduct HCI trainings with your families?” and “If you do conduct HCI trainings with your families, do you conduct them annually?” Further questions included: the numbers of trainings conducted and numbers of families trained; which health modules had been used; any adaptations made; barriers faced; whether they continued to give incentives for families and/or staff; whether HCI trainings were part of their annual training and technical assistance budgets; whether they were able to sustain engagement of parents, staff, organizational leadership and/or the local community; whether they felt HCI made an impact on families, staff, the local health community or community at large; and whether they saw improvements in their program’s health training, other training or general performance as a result of the HCI training. In addition to questions to elicit yes/no responses, several open-ended questions were posed which yielded qualitative responses.

Analyses

Statistical analyses on survey responses were conducted using STATA. Descriptive statistics were generated to reveal the percentage of respondents reporting sustained implementation of HCI modules, average numbers of health education sessions conducted and families participating, and the percentage of agency directors providing an affirmative response for each yes/no question on the survey. We grouped qualitative responses into themes, coded them and then generated descriptive statistics for each theme. Other qualitative data were analyzed using inductive thematic analysis, a qualitative analysis method in which the investigator reads qualitative text and looks for patterns, or themes, which emerge based on similar statements made by multiple respondents. For each theme which emerged, we present a quotation which seemed typical of the sentiments expressed by several respondents, to serve as an example of that theme.

In order to explore associations between sustainability and potential predictors, we conducted a series of bivariate logistic regressions using annual sustained implementation as a dichotomous outcome and testing each predictor individually. We also conducted linear regression analyses using the numbers of health education sessions conducted and total number of participating families at each agency, as continuous dependent variables. For these linear regressions we controlled for both the TTT session attended and the number of years since they first started implementation of HCI programs, but were only able to test one other predictor at a time due to the small sample size. Therefore, the majority of results presented are bivariate associations testing predictors of sustainability.

Results

Of the 192 surveys distributed, 188 were to Head Start programs that continue to operate and ninety-six (N=96) of these surveys were completed, yielding a response rate of 51%. Eighty-six percent (N=83) of the grantees had attended one HCI TTT session and 14% (N=13) attended two. Grantees attended their initial TTT sessions between 2 and 10 years prior to the survey, with an average time of 5 years since the initial training. The average number of parent sessions conducted by an agency after the initial year was 7, with a range of 0-75, and the average number of families reached by an agency was 413, ranging from 24-2700. On average, agencies conducted 2 parent sessions per year, reaching an average of 104 families annually. In addition to the management of common childhood illness module, other health promotion modules implemented with families included: oral health (51%); “Eat Healthy, Stay Active!” (24%); over-the-counter medications (15%); prenatal care (14%); and sun safety (4%). Forty-five percent of respondents (N=43) reported using just the initial health module, while 20% implemented two different modules, another 20% of agencies used three different modules, and 15% utilized 4 or more.

Responses to each survey question are summarized in Table 1. A large majority (84%) of grantees reported that they continue to implement HCI programs, with 71% implementing them annually. Of 22 respondents reporting that they do not continue to implement HCI programs, the most often cited reasons were funding constraints (41%) and staff turnover (36%). While 56% of grantees reported that they continue to conduct sessions with large groups of parents, a substantial percentage reported doing them in small groups (36%) and/or in home visits (20%). When asked about how they funded ongoing implementation, 45% of respondents reported that they put the HCI programs into their annual training and technical assistance (T&TA) budgets; of the 30 respondents who sustained the program but did not put HCI into their T&TA budgets, 93% reported that they used grants or donations to maintain their implementation of the HCI programs. A majority (65%) of grantees reported that they continued to provide incentives to parents, while only 31% reported providing continued staff incentives.

Table 1: Survey Responses from Head Start Directors (N=96) Regarding Implementation of HCI Health Education Programs.

Providing incentives for both staff and families is fundamental to the HCI approach, as a tool for engagement and motivation. Incentives for families may be related to the health topic being presented (such as thermometers, dosage spoons, or fitness materials) and/or designed to reduce barriers to participation (food, child care, and transportation). More respondents (56%) reported continuing to provide topic-specific incentives than general ones (56% vs. 22%). While most of the grantees reported being able to continue to engage their families, staff, and management/leadership (68%, 71% and 70%, respectively), only 43% of respondents reported being able to continue to engage their community partners. In terms of a perceived impact of HCI training on their Head Start programs, 80% said they have seen an impact on parents, while 51% reported seeing an impact on staff; 29% on their local health community, and 25% on their communities at large.

Results of the bivariate associations with annual sustained implementation are summarized in Table 2. Several key factors showed strong and statistically significant associations with whether a grantee continued to conduct HCI trainings annually. These variables included: engaging families, staff, program leaders and community partners; providing parent incentives, particularly incentives specific to the health topic being covered; putting the program in the annual T&TA budget; sending additional staff to the HCI TTT; and adapting the HCI trainings to meet the specific needs of individual Head Start programs. There was a statistically significant lower odds of annual sustainability when respondents reported continuing HCI only as home visits (OR=0.27, p=0.02). These results were similar when we looked at the less stringent outcome of sustaining the program at all, even non-annually. There were no statistically significant differences in either outcome among grantees attending different waves of the HCI TTT program, likely reflecting the consistency of the training with a stable faculty.

Table 2: Predictors of Sustained Implementation—Bivariate Associations with Annual Implementation of HCI Programs using Logistic Regressions.

Beyond simple sustainability, investment of resources was associated with volume of sessions offered and clients served. In multivariate analyses, providing parent incentives showed a statistically significant positive association with the number of sessions held, and putting the program in the annual T&TA budget had a statistically significant positive association with the number of families reached, controlling for both training wave and number of years of implementation in both models.

Qualitative analysis of the open-ended survey responses yielded several themes, which are summarized in Table 3. Head Start directors responding to the survey described several ways in which they have been able or unable to sustain implementation of HCI programs. One theme to emerge was about how directors have been able to engage and motivate stakeholders at all levels of the program—from agencies’ leadership to staff, families, and community partners—by communicating the importance of the health topics and sharing the results of the program, so that “participant feedback and data collection” are tools for engagement. For example, one director reported that she engages her staff by “sharing the results. They understand that the [HCI program] is making a difference and are committed to supporting it.” Another theme that emerged was about ways that directors made the HCI trainings part of the routine operations of their programs, from commitment of funds through their annual training and technical assistance (T&TA) budgets, to making it “part of their job requirements.” Conversely, respondents who have not been able to sustain the programs described how failure was due to financial limitations and time restrictions, including overall burden for Head Start staff who already have many responsibilities. Finally, respondents described the numerous ways they see the HCI trainings as having an impact on their staff (“The staff involved have a much better understanding of how to empower parents in their child’s care.”); families (“They use the book! They feel empowered!”); and communities (“More aware of the health needs of our low-income families.”). Respondents noted that they believe the program has led to a decrease in absences because parents “are treating their children at home instead of the doctor’s [office] or ER;” and that the parent engagement leads to increased involvement in their children’s education even beyond Head Start: “…parents who have moved to state preschool and/or elementary school continue to be involved in their new schools.”

Table 3: Qualitative Responses.

Discussion

This study, which documents high sustainability of low-literacy health education programs in Head Start organizations after participation in the HCI TTT, is unique in that it elucidates several key predictors of sustained implementation. These predictors include the ability to engage and motivate stakeholders at all levels, to integrate implementation into the Head Start program by including it in the T&TA budget, sending additional staff to the TTT to keep a core of trained staff who can lead and spread the method, and adapting the HCI modules to meet the needs and designs of their Head Start programs and local communities. Interestingly, there were some factors that appeared not to correlate with sustained implementation. One of these, providing incentives to staff, may reflect the sentiment of some of the directors that the trainings become part of their usual job descriptions. Also, Head Start programs that reported implementing HCI modules during home visits, rather than in groups, had statistically significant lower odds of sustained implementation, suggesting that the benefits of holding group events in terms of social learning and engagement may justify the costs and logistics involved in terms of scheduling, space and family participation.

Our finding that health education programs can be sustained is consistent with previous studies of sustainability. Scheirer, Hartling & Hagerman (2008) conducted a similar online survey grantees from the New Jersey Health Initiative, a program funded by the Robert Wood Johnson Foundation, and found that approximately 75% of grantees continued their projects beyond the initial grant period, although often in modified forms [15]. Other previous research has demonstrated that sustainability of health promotion and health education programs is possible [16], although few key factors have been measured empirically. Several of the predictors identified in this study are consistent with the routinization framework by Yin (1979, 1981) and a systematic review of empirical studies by Scheirer (2005); namely that systems must be put into place to support the work financially and survive budget cycles, that the project activities become part of the routine operations of a program and staff job descriptions, and that additional training on the project is provided to new staff so that the activities continue despite staff turnover [12-14]. As far as we know, this is the first study to investigate sustainability of health programs in Head Start, but the predictors we identified are consistent with those studies that have looked at programs in other settings.

This study has several limitations. Given the 51% response rate, the responses reported here may not represent all of the grantees trained through the HCI TTT. The modest sample size limited our ability to conduct multivariate statistical analyses. Although well aligned with the HCI approach and previous research by others, the factors identified here which are associated with sustained implementation may be confounded by unmeasured variables; for example, directors who are especially enthusiastic about the HCI program are likely the same directors who are able to engage their staff, continue to provide family incentives, and put the HCI program into their T&TA budgets. Based on this study we can only report factors that are associated with sustainability, not causative relationships, and we cannot know with certainty the direction of the relationships.

Despite these limitations, there are a number of important implications for policy and future research we can identify from this study. Head Start is an important national platform to deliver health programs for low-income families with young children. In addition to identification of programs with proven effectiveness, sustained implementation of effective programs should be a priority. This study suggests that keeping stakeholders engaged, finding stable financial support, protecting against the loss of the program with staff turnover, and allowing adaptation to meet local program needs, may all be key factors in sustaining a health promotion program. The HCI training model has recently been incorporated into the Head Start National Center on Health and spread to approximately 150 additional grantees over the past three years. This study confirms that it can also be sustainable. Future studies should more rigorously examine the effectiveness of specific aspects of the HCI training approach and implementation of specific health education modules in Head Start, as well as the effects of adaptations that would allow expansion to other early child care and education settings.

References

1. Ackerlund KM (2000) Prevention Program Sustainability: The State’s Perspective. Journal of Community Psychology 28: 353-362
2. Backer TE (2000) The Failure of Success: Challenges of Disseminating Effective Substance Abuse Prevention Programs. Journal of Community Psychology 28: 363-373
3. Durlak JA, DuPre EP (2008) Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation.  Am J Community Psychol 41: 327-350.
4. Johnson K, Hays C, Center H, Daley C (2004). Building Capacity and Sustainable Prevention Innovations: A Sustainability Planning Model. Evaluation and Program Planning 27: 135-149
5. Scheirer MA, Dearing JW (2011) An agenda for research on the sustainability of public health programs.  Am J Public Health 101: 2059-2067.
6. Zagorsky JL (1999). Health and the Working Poor. Eastern Economic Journal, 25: 169-189.
7. Currie J (2005) Health disparities and gaps in school readiness.  Future Child 15: 117-138.
8. Larson K, Russ SA, Crall JJ, Halfon N (2008) Influence of multiple social risks on children's health. Pediatrics 121: 337-344.
9. Herman A, Nelson BB, Teutsch C, Chung PJ (2013) A Structured Management Approach to Implementation of Health Promotion Interventions in Head Start. Preventing Chronic Disease 10: 130015
10. Herman A, Jackson P (2010) Empowering low-income parents with skills to reduce excess pediatric emergency room and clinic visits through a tailored low-literacy training intervention. Journal of Health Communication 15: 895-910
11. HermanA, NelsonBB, TeutschC, ChungPJ (2012) Eat Healthy, Stay Active!: A coordinated intervention to improve nutrition and physical activity among Head Start parents, staff and children. American Journal of Health Promotion, 27: 27-36
12. Yin RK (1979) Changing urban bureaucracies: How new practices become routinized. Lexington, MA: Lexington Books.
13. Yin RK (1981) Life histories of innovations: How new practices become routinized. Public Administration Review41: 21-28
14. Scheirer MA (2005) Is Sustainability Possible? A Review and Commentary on Empirical Studies of Program Sustainability. American Journal of Evaluation 26: 320-347
15. Scheirer MA, Hartling G, Hagerman D (2008) Defining sustainability outcomes of health programs: Illustrations from an on-line survey. Evaluation and Program Planning 31: 335-346
16. Hoelscher DM, Feldman HA, Johnson CC, Lytle LA, Osganian SK, et al. (2004) School-based health education programs can be maintained over time: results from the CATCH Institutionalization study. Preventive Medicine 38: 594-606

--