Journal of Community Medicine & Health Education
Open Access

With the goal of addressing well documented racial, ethnic, and socio-economic health disparities [1], medical training programs must train community responsive physicians who can meet the needs of patients with diverse backgrounds. Unfortunately, residents’ knowledge and understanding of underserved populations is low [2,3]. This deficit contributes to clinical barriers to health care delivery which impacts the socioeconomic, racial, and ethnic health disparities that exist today [4]. One strategy to address this problem has been to include “cultural competence” training in medical education. Many Accreditation Council for Graduate Medical Education (ACGME) competencies and Residency Milestones address trainees’ ability to practice effectively with underserved populations, but are difficult to teach [5]. These competencies and milestones include the ability to empower patients to participate in their care, enable shared decision-making, overcome barriers to disease prevention and health promotion, understand health inequities and social determinants of health, and communicate effectively across a broad range of socioeconomic and cultural backgrounds.

Community engagement for medical trainees holds potential to promote these competencies and milestones. Although community engagement and service learning have been integrated into medical education in a variety of ways, few studies evaluate and quantify this potential [6]. Stewart and Wubbena’s systematic review of servicelearning in medical education from 1998-2012 suggests that community engagement in medical education could play a role in improving communication skills, understanding of diverse patient populations, understanding of the needs of socioeconomically disadvantaged communities, and understanding of health disparities [7]. Stewart and Wubbena conclude that “increased rigor in servicelearning research in general is necessary.” Additionally, research suggests that exposure to patients from diverse backgrounds can improve residents' preparedness to provide cross-cultural care [8], but little is known about how much exposure is needed to have an impact.

The objectives of this study are to assess i) whether medical trainees perceive participation in a community engagement program such as the Stanford Youth Diabetes Coaches Program (SYDCP) [9] directly contributes to acquisition of competencies and milestones developed to address health disparities; and ii) whether the amount of time spent in service-learning affects the perceived acquisition of competencies.

Project setting

The SYDCP facilitates partnerships between medical training programs and high schools serving youth from socioeconomically disadvantaged, primarily underrepresented minority communities. The program utilizes a “train-the-trainer” model in which medical trainees teach healthy youth to become self-management coaches for family members with diabetes. The program content was developed using feedback from medical trainees, youth coaches, and family members being coached. The tightly scripted curriculum incorporates evidencebased approaches to chronic disease management [10], while also encouraging discussion and interaction between instructors and youth. Medical trainees go to participating high schools and present program curriculum once a week for one hour for eight weeks. Youth (grades 9-12) attend the program classes where they are trained in coaching skills as well as basic diabetes and health knowledge. The youth coaches then meet weekly outside of school with a family member with diabetes to complete a coaching assignment. The coaching assignments are structured such that the family members share their experiences and challenges managing their chronic illnesses with their youth coaches. In turn, as part of structured check-ins and discussions, the youth coaches share these realities with the medical trainees in class.

Following a successful pilot in 2012-13 [9], SYDCP was expanded from 2014-16 to include five medical training programs and five high schools around the United States. Training programs learned about the SYDCP through presentations given by the research team at national meetings, as well as through word of mouth discussions. The local training program was located in San Jose, CA, while the remote training programs were located in Philadelphia, PA, Cincinnati, OH, Ann Arbor MI and Huntsville, AL. Participating trainees responded to online post-participation surveys to provide feedback about their experience with the SYDCP.

Sample size and description

Over a two-year period, 60 medical trainees including 43 Family Medicine residents, 1 attending, 6 medical students, and 10 pharmacy students participated in the SYDCP at five high schools with students from socioeconomically disadvantaged, ethnic minority communities. The trainee group consisted of 44 females and 16 males; 33 trainees identified themselves as white, and 27 trainees belonged to other ethnicities including 19 Asian, 6 African-American, and 1 American Indian. Two trainees identified themselves as Hispanic. 28 trainees taught at the local site in San Jose, California, while 32 trainees taught at remote sites in Philadelphia, PA; Cincinnati, OH; Ann Arbor, MI; and Huntsville, AL. Table 1 describes all trainee demographics, as well as demographics based on number of classes taught. On average, participants taught 3 classes; 20 participants taught only one class; while 40 taught two or more classes.

Table 1: Demographics of participant trainees in Stanford Youth Diabetes Coaches Program (SYDCP) 2014-2016, San Jose, CA, Philadelphia, PA, Cincinnati, OH, Ann Arbor, MI, Huntsville, AL, N=60.

Participant selection

Recruitment of medical trainees to participate in the SYDCP varied by site. At some sites (San Jose and Huntsville), participation was mandatory as part of the training program and at others (Philadelphia, Ann Arbor, and Cincinnati), participation was voluntary and often depended on trainees’ availability, interest, and awareness of the program. For example, in San Jose, medical residents participated as part of the required Community Health rotation in their residency program. In Philadelphia, the program director reported that several residents wanted to participate more than once but were unable due to scheduling conflicts.

Data collection and measures

Online surveys were developed building on the 2012-13 pilot study [11] of medical trainees. Surveys incorporated specific questions developed to assess the acquisition of competencies and milestones pertaining to cultural competence. Specifically, questions were formulated from eight general physician competencies as adapted by Englander et al. from the ACGME [12] and eight Family Practice residency milestones [13] (Table 2).

Table 2: Survey Questions and Corresponding Competency and Milestone Evaluated, Stanford Youth Diabetes Coaches Program, 2014-15.

Trainees completed post-participation surveys including five retrospective post-then-pre questions [14] designed to assess program participation’s impact on trainees’ confidence in meeting competencies related to patient care, knowledge of practice, interpersonal and communication skills and professionalism. Competencies related to practice based learning and improvement, interpersonal and communication skills, and professionalism were assessed based on analysis of four Likert style questions. Further, open-ended feedback questions were included in the survey to gain better understanding of what trainees perceived as the most rewarding and most challenging aspects of teaching high school students.

Data analysis

Objective 1: Data was analyzed for all 60 trainees combined to assess competency acquisition based on responses to the retrospective postthen- pre questions; responses to Likert-style questions; and answers to open-ended questions. For retrospective post-then-pre questions, we performed paired-sample t-tests using SPSS version 23 to test for significance in difference of pre and post-participation scores. For Likert-style questions, percentages were calculated to determine the extent to which trainees perceived program participation influenced their acquisition of competencies. Open and axial coding methods were used to analyze qualitative data [15].

Objective 2: To explore whether the number of classes taught had an impact on perceived acquisition of competencies, we stratified data according to number of classes taught. We organized data into three subgroups: trainees who taught only one class, trainees who taught two classes and trainees who taught three or more classes. Our analysis showed no difference in scores when trainees taught two classes or three or more classes, but there was a significant difference if trainees taught only one class. We then stratified the analysis into two subgroups based on number of classes taught by trainees one subgroup including 20 trainees that taught only one class, and another group of 40 trainees who taught two or more classes. Finally, we ran linear regression models with difference in pre and posttests as the dependent variable and number of classes taught as the independent variable, while controlling for gender, race and location of training programs as independent variables using SAS Enterprise Guide version 6.1. We corroborated our findings by conducting chi square tests to compare differences between levels (i.e., strongly agree vs. agree) of the Likert style questions for the two subgroups. Further, two independent researchers analyzed the responses of open-ended questions to examine whether the trainees’ qualitative responses in the two subgroups were in concordance with results obtained by quantitative analysis.

In all, 60 trainees participated in the program and completed online surveys that were included in the analysis.

Objective 1

Analysis of all participants’ responses demonstrated that program participation directly contributes to the acquisition of competencies and achievement of milestones. Analysis of the self-reflective retrospective post-then-pre questions demonstrated statistically significant improvement in trainees’ acquisition of competencies. Specifically, after participation, medical trainees reported statistically significant improvements in their: confidence to provide sensitive and culturally responsive care to diverse patient populations (p<0.001); understanding of the barriers to care that diverse patient populations face (p<0.001); ability to communicate effectively with youth (p<0.001); ability to empower patients and their families to participate in their care (p=0.002); and plans to use action plans (a validated self management tool) with patients in a clinical setting (p<.001). Analysis of all participants’ responses to Likert-style questions demonstrated that the vast majority agreed or strongly agreed that program participation facilitated their achievement of milestones. 93% reported that program participation allowed them to contribute to the health of the community outside the hospital and clinic setting; 90% that it gave them an opportunity to practice interpersonal and communication skills; 95% that it gave them an opportunity to learn more about the communities they serve as physicians; 92% that it gave them an opportunity to educate students and their families about preventing health problems; 78% that it improved their ability to communicate with teens; and 88% that it gave them an opportunity to serve as a role model. Qualitative analysis of the question: “What was the most rewarding part of the experience?” yielded six major themes: communicating with teens; empowering youth and their family members to improve their health; acting as a role model; increasing health knowledge; making a community health impact; and learning about the community (Table 3).

Table 3: Samples of Theme Characteristic Quotes from Medical Trainee Survey Question “What was the most rewarding part of the program?”; SYDCP 2014-2016, San Jose, CA, Philadelphia, PA, Cincinnati, OH, Ann Arbor MI, Huntsville, AL, N=60.

Objective 2

To explore whether amount of time spent in service-learning activities affected the perceived acquisition of competencies, we analyzed the data by subgroups based on whether trainees taught only one class or two or more classes. As described in Figure 1, after controlling for gender, race and location for training program in our linear regression model, medical trainees who taught two or more classes (N=40) reported statistically significant improvements in i) confidence to provide sensitive and culturally responsive care to diverse patient populations (p=.007); ii) understanding of barriers to care faced by diverse patient populations (p=.004); iii) ability to communicate effectively with youth (p=.0001); iv) ability to empower patients and families to participate in their care (p=.03); and v) intention to use action plans in a clinical setting (p=.05). Trainees who taught only one session (N=20) did not report significant improvements in competency acquisition, except for intention to use action plans. We did not find any statistical differences in results when data was analyzed by type of medical trainee (i.e., whether trainee was medical student, resident or pharmacy student).

Figure 1: Percent change in mean scores of trainees’ self-ratings before and after program participation (retrospective post-then-pre questions); SYDCP 2014-2016, San Jose, CA, Philadelphia, PA, Cincinnati, OH, Ann Arbor MI, Huntsville, AL, N=60.

To further validate our findings, analysis of Likert style questions by subgroups based on number of classes taught demonstrated that compared to trainees who taught only one class, trainees who taught two or more classes were more likely to strongly agree that program participation contributed to community health (67.5% vs. 40%); provided an opportunity to practice interpersonal and communication skills (67.5% vs. 40%); provided an opportunity to learn more about the community they serve (67.5% vs. 50%); and improved their ability to communicate with teens (52.5% vs. 30%). As seen in Figure 2, based on results of chi square tests, trainees who taught two or more sessions were significantly more likely to “strongly agree” that program participation contributed to health of the community (p=0.024) and gave them opportunity to practice interpersonal and communication skills (p=0.037). This illustrates that while participation in even one class of the program improved trainee perception of acquired competencies, the trainees who visited the classrooms two or more times were more likely to show significant improvements in perceived competency acquisition.

Figure 2:Percent distribution of trainees’ responses to questions about competency enhancement with program participation (Likert style questions); SYDCP 2014-16, San Jose, CA, Philadelphia, PA, Cincinnati, OH, Ann Arbor MI, Huntsville, AL, N=60.

Results from qualitative analysis also demonstrated that trainees who taught two or more classes were more likely to express they were acting as role models, contributing to community health, and empowering patients and families. Despite not being asked about the relationship between the number of classes taught and their experience, 18 of the 60 residents addressed the topic in their responses to open-ended questions. Of the 20 residents who taught only one session, four wrote that only teaching one session was a barrier to receiving benefit from the experience.

Some examples include:

• I don't feel that I participated in enough sessions to really reliably evaluate the program--I was only a part of one session.

• If I had [taught] more than 1 session, I may have different feelings, been more invested in the class, but it went well overall.

Of the 40 of the residents who taught 2 or more classes, 14 made a statement about the benefits they received from teaching more than one class.

Some examples include:

• I really enjoyed being able to see the students multiple times so they were more comfortable asking questions and had time to think of things as they came up.

• Teaching not just diabetes management but life skills to these youngsters. At first I didn't think they really cared or were interested, but I was really impressed at the end how much they had retained. Forming connections with the students was also pretty awesome.

Medical trainees also reported that teaching and interacting with students was a rewarding experience that gave them an opportunity to learn more about the challenges faced by the diverse communities they serve. Additionally, they also expressed their satisfaction witnessing when students gained medical knowledge and family members made lifestyle changes based on interactions with their student coaches. Medical trainees noted that challenges of teaching included keeping students engaged and attentive in class, and managing the allotted time to cover all instructional materials.

The impetus to evaluate the impact of the SYDCP on medical trainees stemmed initially from residents’ unsolicited feedback about how the opportunity was providing needed experience to enhance their medical training. Results from the retrospective post-then-pre design, Likert style, and open-ended feedback questions all suggest participation in the SYDCP has potential to significantly increase medical trainees’ acquisition of ACGME competencies and facilitate the achievement of residency milestones in domains designed to address health disparities. Utilizing three methods of evaluation (retrospective post-then-pre, Likert, and open-ended qualitative questions) validates the results demonstrating that more involvement is associated with greater acquisition of competencies and milestones and suggests medical trainees participate in at least two servicelearning sessions to maximize benefit.

Ultimately, enhancing medical trainees’ understanding of health disparities–including their ability to practice effectively with diverse patient populations-holds promise to advance health equity [16]. While it is widely recognized that medical trainees should be developing cultural competency skills, many training programs have yet to incorporate meaningful opportunities to develop or assess these skills [17]. One way some medical training institutions are addressing this need is through incorporating discussions of social determinants of health as a means for understanding health disparities [18], and there is a growing understanding that trainees should learn about medicine more from a framework of the patient rather than the disease [19]. Additionally, e-learning courses such as Quality Interactions [20] are available to increase cultural competency of medical trainees. It is also understood, however, that the utility of cultural competency knowledge is heightened when it translates to building effective skills with patients and the community [21].

We suggest that an effective method for developing practical skills in cultural competence could be service-learning opportunities where medical trainees are immersed in a community setting in which the social determinants of health are evident; and in which they have in- person interactions where they can apply the skills required to practice effectively with diverse patient populations. Sparse research suggests that service-learning can play an important role in developing cultural competency skills and may encourage future engagement in the community [22], but more research is needed to validate this methodology and determine which types of service-learning are most effective [23].

Additional program benefits

In addition to the acquisition of competencies and achievement of milestones related to health disparities, medical trainees’ feedback suggests program participation may have additional benefits. For example, program curriculum teaches methods for nutrition and physical activity counseling and promotion which is a noted area of deficit in physician training [24]. Program curriculum also emphasizes the fundamental importance of having health goals come directly from the person with diabetes which is a key feature of personalized care planning, a tool frequently touted for its benefits in helping primary care physicians manage patients with chronic disease [25]. Lastly, medical trainees have noted that being present at the high school energizes them and seems to create community relationships and trust that was not present before. One medical trainee noted: I think that it is a great program and that its lessons go beyond discussing diabetes and more into closing the gap (the distrust) between urban physicians and their patients. Many residents describe their experience as a great way to get some perspective and “recharge their batteries.” This opportunity fulfills the well-documented desire of physicians to be active participants and advocates for health in their communities [26].

The retrospective design of this study has several limitations. Although the retrospective post-then-pre design is shown to reduce response shift bias associated with pretest overestimation or underestimation [14], the design is subject to biases in self-appraisal and recall. We cannot rule out the bias that exists with all self-report studies. It is possible, for example, that medical trainees may have been able to guess the hypothesis of this study and formulated their answers to conform with this hypothesis.

Our participant sample also had limitations. Our program was implemented among five residency programs with which we had established partnerships. Participants were selected based on their availability to teach classes given their training schedules and in some cases, on their willingness to participate. Randomization was not feasible given the resource and scheduling limitations of this multi-site study. The sample differs from an ideal sample that would be randomly selected in that it only occurred at sites where a training program director felt the program was worthwhile which likely lead to an overrepresentation of trainees who are interested in community health and outreach. Trainees in programs that do not emphasize community health and outreach were left out of this sample. Additionally, because some of the participating trainees elected to participate, we recognize the potential for self-selection bias. Thus, this non-probability sample is limited in that it may not provide a representative sample of all residents or medical trainees, and we can’t make generalizations pertaining to all residency programs in the US.

Understanding that limitation, however, we believe that for trainees not represented in our sample, participation may have yielded greater gains in competencies, as those trainees not represented in our sample may have begun the process with less confidence and skill in the competencies we were measuring which would create more room for growth.

Additionally, when we compared residency programs where participation in the intervention was mandatory for all residents to programs where participation was voluntary, we found that the 38 residents who were mandated by their programs to participate in the intervention were as likely to report significant positive improvements as the 22 residents who participated on a voluntary basis.

Additionally, in pursuing our second objective, we were limited by lack of randomization to allocate medical trainees into one or two or more sessions, which could also contribute to self-selection bias. Trainees who taught only one class included about the same percentage of males and females, less African American participants (1 out of 20, versus 5 out of 40), a higher percentage of Asian (45%, versus 25% participating in 2 or more), and lower percentage of Whites (45%, versus 60% participating in two or more sessions). To address this issue, we plan to replicate the program making it mandatory for all participants to provide a specific number of sessions. Additionally, we only surveyed trainees after participation using retrospective postthen- pre design to assess perceived competencies prior to and after participating. To ensure this methodology accurately captures differences in perceived competency, we intend to add preparticipation surveys.

Service-learning opportunities for medical trainees have potential to address the problem of medical trainees’ lack of knowledge and understanding of underserved populations. Participation in such programs can directly contribute to the acquisition of competencies and milestones developed to address health disparities, and continuity of participation appears to be important to maximize benefit.

The authors wish to thank Dr. Kelley Skeff (Professor of Medicine, Primary Care and Population Health; Co-Director Stanford Faculty Development Center for Medical Teachers, Stanford University School of Medicine) and Dr. Lee Sanders (Associate Professor, Stanford Department of Pediatrics; Co-Director Center for Policy, Outcomes, and Prevention, Stanford University School of Medicine) for their guidance and support of this project. The authors also appreciate continued guidance from Dr. Kate Lorig (Professor Emeritus, Department of Medicine, Stanford University School of Medicine). Additionally, the authors appreciate our residency and high school partners in this work. Specifically, we would like to thank Dr. Andrew Schectman (O’Connor Stanford Family Medicine Residency Program, San Jose, CA); Dr. Andrew Berta (Chestnut Hill Family Medicine Residency Program, Philadelphia, PA); Dr. Philip Diller (University of Cincinnati Family Medicine Residency Program, Cincinnati, OH); Dr. Margaret Riley, University of Michigan, Family Medicine Residency Program, Ann Arbor, MI), and Ms. Meredith Lewis (UAB Huntsville, Family Medicine Center; Huntsville, AL).

Funding for this work was given by anonymous private donors and a grant from Goldman Sachs Gives, at the request of R. Martin Chavez and Collin Gage.