Received Date: April 12, 2014; Accepted Date: May 15, 2014; Published Date: May 20, 2014
Citation: Tran BX, Nguyen QLT, Nong VM, Maher RM, Nguyen AT, et al. (2014) Student-Centred Outcomes of an E-learning Course on Public Health in Hanoi and New York. J Community Med Health Educ 4:288. doi:
Copyright: © 2014 Tran BX, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Community Medicine & Health Education
Background: We evaluated the satisfaction and improvement in learning outcomes of students taking a distance course in Public Health facilitated by the Institute for Preventive Medicine and Public Health at the Hanoi Medical University and the State University of New York at Albany.
Methods: A total of 36 students participated in pre- and post- course surveys online. We developed the Vietnamese E-Learning Student Outcomes Profile (VESOP) to evaluate the impact of distance learning methods on students’ learning outcomes.
Factor analysis was applied to construct the measure items into five domains: Academic Skills, Intellectual Abilities, Interactions, Responsibility and Educational Engagement. Multivariate regression models, selected using stepwise approach, determined factors associated with changes in students’ learning outcomes. Student satisfaction with the course was assessed by determining their willingness to participate in and pay for future distance learning courses.
Results: The students showed moderate improvement in “Interactions” and large improvements in the other four domains. Factors associated with improvements included previous academic performance, English proficiency, ease of access to the internet and computers, and time spent studying. All students passed the course, all would be willing to take another distance learning course, and 26 students would be willing to pay for such as course with certification.
Conclusions: Findings of this study highlight the potential of distance learning as a complementary educational method that could greatly benefit Public Health students. It encourages and supports the development of such training programs for Public Health students in Vietnam.
Distance learning; Academic performance; Capability; Public health; Vietnam
E-learning has rapidly emerged over the past few decades as an effective alternative to traditional classroom-based education. Because it transcends geographical and time boundaries, e-learning has the potential to reach a much wider, even global, audience. It is also a very learner-centred form of education, and given its inherent need to archive teaching material, can be a useful platform for storing and sharing information . Finally, despite the large initial start-up costs for e-learning, it has the potential for enormous cost-savings in the long term.
However, despite the immense promise of e-learning, there are several concerns and documented disadvantages of the e-learning format. The mostly passive and isolated format of e-learning courses can lead to boredom and disengagement from the material, in turn leading to a lack of motivation to learn [1,2]. Indeed, e-learning has been found to have higher dropout rates than traditional education programs . Without having the constant presence of other students and instant feedback from professors, e-learning can be limited in terms of opportunities for interaction and collaboration [1-4]. Finally, many people have raised the concern that e-learning is only good for knowledge acquisition, and not effective at training people in more abstract concepts and hands-on skills.
Given these concerns, our purpose with this study was to assess the ability of an e-learning course to teach students academic and professional skills, rather than just scientific knowledge, and to provide a satisfying educational experience. This is particularly important when considering the use of e-learning for public health. E-learning for healthcare has been strongly developed in other countries, as it can easily provide continuing medical education , but in Vietnam, it is only just beginning to be explored. In a previous needs assessment of healthcare workers, we identified a strong need for improved training in management skills and intellectual abilities [6-8]. Additionally, despite the healthcare professionals’ expression of interest in lifelong continued professional education, we found that high workloads and lack of public appreciation towards public health and preventive medicine lead to a high rate of turnover and lack of motivation and desire to continue learning among healthcare workers. Therefore, our purpose with this study was to explore the ability of an e-learning course to deliver a highly motivating and satisfying educational experience that teaches academic and professional skills in addition to knowledge acquisition.
This study evaluated students’ experience and achievement of learning outcomes during an e-learning course in “Introduction to Public Health” that was facilitated in the fall term of 2011 by the Institute for Preventive Medicine and Public Health at Hanoi Medical University (HMU) and State University of New York (SUNY) at Albany. The course was funded by the Vietnam Educational Foundation with the intent of strengthening ties between the US and Vietnam and opening Vietnamese students to the benefit of international expertise and education. By evaluating the students’ experience with this course, we hope to illuminate the areas in which e-learning can improve to best deliver a satisfying and engaging course capable of training students in necessary academic and professional skills.
Study design and setting
We designed a pre- and post- evaluation to assess students’ changes in learning outcomes through participation in the distance course on “Introduction to Public Health” at HMU from October 2011 to March 2012 .
Course Facilitation of “Introduction to Public Health”
The course was facilitated by a primary instructor at SUNY and assistants at HMU through a technology platform including simultaneous videoconferencing of the lecture, online archival of recorded lectures, and Skype sessions. The course included both American and Vietnamese students, which allowed for the American students to assist in the development of conversational English, the language of the course, and allowed the students to gain a better understanding of the public health issues in both countries. The primary instructor facilitated communication sessions between American and Vietnamese students, SUNY and HMU teachers through traditional videoconferencing, Skype sessions and an online component. A group of faculty at HMU was formed to coordinate students’ discussion throughout the course and to support them through face-to-face meetings.
The goals of the course included: 1) to help students understand what public health is and its importance; 2) to illustrate how public health functions by discussing issues currently confronting the nation and the global community; and 3) to encourage students to consider public health dimensions in their medical careers. The course was taught in 7 sections, including, 1) What is Public Health? 2) Epidemiology: The Basic Science of Public Health, 3) Biostatistics: Calculating the Population’s Health, 4) The Biomedical Basis of Public Health, 5) The Environment and Its Effects on Health, 6) Social, Cultural and Behavioral Factors of Health, 7) Comparative Medical Care Systems. The course was graded based on three short research papers, class participation, a final paper, a midterm exam, and a final exam, and all students who completed the course with a passing grade received a certificate.
There were 36 students including 22 undergraduates, 4 Public Health graduate students, and 10 medical students. Any student who signed up for the class and knew English was allowed to participate. All participants took part in an orientation session prior to the course and were invited to participate in the study on a voluntary basis, and with informed consent.
We conducted 2 cross-sectional surveys using a web-based selfadministered questionnaire in Google Docs. An invitation to complete the survey was sent to all students prior to and after completing the course. Anyone who experienced difficulties in answering the questionnaire was instructed and supported via phone or in person.
The baseline assessment, carried out in October, 2011, collected information about access to required technology, attitudes of learners towards distance learning, and certain characteristics, including academic performance, capabilities, skills and competence prior to participating in the course. The post-evaluation, carried out in March 2012, measured the extent to which these characteristics changed over the course
Measures and instruments
Our survey was designed to explore various individual and learning characteristics of participants, including academic performance, access to e-learning technologies, and knowledge about and attitudes towards distance education. Much of the literature on e-learning explores the question of whether the e-learning format is suitable for teaching “soft skills,” in addition to knowledge acquisition [2,4]. Therefore, not only were we interested in assessing students’ cognitive learning outcomes, which deal with “recall or recognition of knowledge and the development of intellectual abilities and skills,” but we were also particularly interested in their affective learning outcomes, defined as students’ interest towards learning and satisfaction with the learning environment . While cognitive learning outcomes can be assessed through exams and class assignments, affective learning outcomes can be more difficult to measure.
In order to assess the affective learning outcomes of students throughout our e-learning course, we organized a team of one faculty group and one student group to develop a measure, the Vietnam E-learning Students Outcomes Profile (VESOP). By including students throughout the entire process, we ensure that the learning outcomes we assess are student-centred, in order to evaluate whether the e-learning course lives up to its potential of being a learner-centred learning environment . Items to be included in the measure were pooled from various sources. Literature searches provided us with many skills to include, such as motivation and desire to learn [1,2] and interaction and collaboration skills [1-4]. We identified further specific skills through a review of our previous study on necessary skills for preventive medicine workers, which included management skills and intellectual abilities, as well as expected training outcomes for accredited public health and preventive medicine programs in Vietnam [6,7].
After our initial consolidation of skills from a variety of external sources, we had both the faculty and student groups participate in focus group discussions to explore if there were additional factors that should be included from the perspective of public health and preventive medicine learners and faculty members. From this long list of skills, the two groups limited it to the most important by ranking all items based on their importance and relevance to public health education and short-listing them for use in the evaluation. Through this process, we arrived at a list of 46 specific skills to include in the VESOP measure.
We asked students to rate their current levels on each measure item prior to attending the course. The response options included five levels in a Likert scale format. These baseline data were used for exploratory factor analysis to reclassify measure items into measure domains. These domains included Academic Skills, Intellectual Abilities, Interactions, Responsibility, and Educational Engagement. In the post-evaluation survey, we asked students to rate their perceived levels of improvements on each of the measure items. For example, for the measure item about teamwork skills, we asked students: “How would you rate your changes in teamwork skills after completing the course?” The responses included: “1- No improvement; 2-Little improvement; 3- Moderate improvement; 4- Large improvement; 5- Very large improvement”. Domain scores were estimated by averaging the scores of all domain items. Higher scores indicated higher levels of improvements. In addition to the students public health knowledge acquisition – which is indicated by the grades they received - these changes represented the learning outcomes attained by the students.
Descriptive statistical analysis was used to describe the characteristics of respondents. Chi-square and Student’s t-test were used to examine the variation in these results. Exploratory factor analysis was applied to examine the construct validity of the VESOP. Five factors were extracted by the principle component analysis at an eigenvalue of 1.95, which was the threshold, defined using the screen test, where the eigenvalue curve flattened out. Orthogonal Varimax rotation with Kaisers’ normalization was used to reclassify the measure items in order to increase the interpretability of these factors. The cut off point for factor loadings was set at 0.40. For those items, which had cross-loadings, we assigned them into one domain based on the nature of the questions justified by expert’s opinions. Internal consistency reliability of measurement using the VESOP was estimated using Cronbach’s alpha.
We used multivariable linear regression to determine factors associated with improvements in learning outcomes. The candidate predictors of the model included students’ previous academic performance, Information Technology (IT) conditions, use of computer and internet, time dedicated to the distance learning course, and their knowledge about distance learning. We treated Likert scale responses as continuous variables. The self-reported English proficiency (score range of [4; 20]) was assessed by summing students’ perceived levels of four English skills from 1- Poor to 5-Excellent. We applied a stepwise forward model building strategy which selected variables based on the log-likelihood ratio test at a p-value <0.1, and excluded variables at p-values>0.2 . The significance level was set at p-value <0.05.
Characteristics of participants
The majority of students (80%) were in their senior year of the preventive medicine program at HMU (Table 1). Most learners had good previous academic performance, with an average cumulative GPA of 7.22/ 10 (SD=0.63). Their self-reported ability in 4 English language skills was good, with an average score of 3.5, SD=0.83. Students reported that they had easy access to the internet, and used it for an average of 5.3 hours per day. Most students were used to studying with computers and the internet, for an average of 3 hours a day.
|2. Current academic year|
|1st - 2nd||2||5.6|
|3rd - 4th||20||55.6|
|3. High School|
|4. University academic performance||Mean||SD|
|The last semester’s GPA||7.7||0.6|
|The latest grade in English||7.3||1.6|
|5. Self-reported English proficiency*|
|6. IT conditions|
|6.1. Having Desktop vs. Laptop||N||Percent|
|6.2. Access to computer & internet**||Mean||SD|
|Ease of computer access||4.1||0.9|
|Ease of internet access||4.0||0.8|
|7. Computer and internet use|
|Computer use (hr/day)||5.6||3.4|
|Internet use (hr/day)||5.3||3.4|
|Frequency of daily computer use***||4.3||0.6|
|Frequency of daily internet use***||4.3||0.6|
|Daily study time using computer and internet (hr)||3.0||2.0|
|Daily study time using computer and internet (hr)||3.0||2.0|
|8. Time spent for distance learning (hr/day)||2.7||1.7|
|9. Ever taken a distance training course||N
Table 1: Characteristics and learning condition of participants.
Student experience with course
Of the 36 initial students in the course, 31 (86.11%) completed the course with certification. All students received a grade of at least C+, and 38.71% earned an A. These students’ success suggests a high level of participation, since students were highly encouraged to participate, with class participation – along with three short research papers – accounting for 15% of the final grade.
Students also reported a high level of satisfaction with the course. All of the students expressed some desire to participate in a distance learning course in the future, and 29 (85.29%) of them expressed a moderate or extreme desire to do so. Additionally, 15 of these students would even be willing to pay for such a course, a number that increases to 26 if the students are provided with certification (Table 5).
Psychometric properties of the measurement
Table 2 shows the construct validity and reliability of the cognitive and affective learning outcomes measured using VESOP. In factor analysis, five main factors were selected, which accounted for 56.5% of the variance. The “Academic Skills” factor accounted for 31.3% of the variance, and all major factors had at least 3 items. Measure items were re-classified into five corresponding domains. Cronbach’s alpha was good to excellent across domains, ranging at (0.69; 0.94) (Table 2). Prior to the distance course, students reported an average score range of (3.1; 3.7), equivalent to (Moderate; Good).
|Original domains||Measure Items||Factors structure|
|Responsibility||Punctually attend in all lectures|
|Punctually submit homework||0.57|
|Participated actively in the discussion group||0.75|
|Participated actively in the lecture||0.56|
|Preparation before class||0.62|
|Find information from external sources||0.75|
|Interactions||Discussing with friends||0.47|
|Discussing with teacher||0.47|
|Sharing specialized references||0.54|
|Listen and effectively apply ideas of members in work team||0.48|
|Help others in learning||0.56|
|Working under pressure||0.56|
|Acquisition of new knowledge||0.42|
|Explore innovative idea||0.50|
|Applying new idea and knowledge||0.50|
|Academic Skills||Team work skill||0.68|
|Searching information from internet||0.80|
|Searching medical information from international Journal, database||0.83|
|Writing essay skill||0.84|
|English writing skill||0.84|
|Using distance learning tools (webinar, Skype...)||0.83|
|Using software (MS WORD)||0.60|
|Using references management software (endnote)||0.58|
|Using social network (Facebook...)||0.74|
|Analyzing- synthetizing- evaluating a problem||0.55|
|Communication skill – community connection skill||0.44|
|Writing and presenting a report||0.76|
|Exploring and understanding your self||0.56|
|Listening to others||0.78|
|Passion for learning ( desire to learn)||0.65|
|Actively doing homework||0.46|
|Actively reviewing lesson||0.53|
|Baseline domain score (mean)||3.1
Table 2: Factor loading and reliability of the measurement.
Changes in students’ learning outcomes
Students’ average domain scores saw large improvements in Academic Skills, Intellectual Abilities, Responsibility, and Educational Engagement, and a moderate improvement in Interactions. The proportion of students reporting large and very large improvements was more than 90% in Academic Skills and Educational Engagement, 83-86% in Intellectual Abilities and Responsibility, and 46% in Interactions (Table 3).
|Measures||Level of improvement||Distribution (%)|
|Mean||(95% CI)||1- None||2- Small||3- Moderate||4- Large||5- Very large|
Table 3: Improvements in domains scores of the learning outcomes.
When analysing the factors associated with improvements in four VESOP domains, we dropped knowledge of distance learning prior to the course as a variable in the reduced model. We found that English proficiency positively predicted improvements in Responsibility throughout the distance course. Easier access to computers was significantly associated with higher improvements in Academic Abilities and Educational Engagement. Similarly, frequency of computer use predicted positive changes Responsibility, and longer time spent studying using the computer and internet positively predicted larger increases in Responsibility and Interaction. Conversely, more frequent internet use was actually associated with lower improvements in Academic Skills, Intellectual Abilities and Responsibility (Table 4).
|Previous academic performance|
|Most recent GPA||0.25|
|Most recent English Grade||0.09||0.22|
|(-0.02; 0.19)||(0.12; 0.31)**|
|Self reported English proficiency
|No. distance courses taken||0.18|
|Having desktop PC vs. Laptop||0.30||0.34|
|(-0.17; 0.78)||(-0.16; 0.84)|
|Ease of computer access
|(-0.01; 0.51)||(0.20; 0.70)**||(0.16; 0.78)**|
|Ease of internet access
|Computer and internet use|
|Frequency of daily computer use
(1-None; 5-Very often)
|(0.01; 0.56)*||(-0.02; 1.46)|
|Computer use (hr/day)||0.17||0.23|
|(-0.03; 0.37)||(-0.03; 0.49)|
|Frequency of daily internet use
(1-None; 5-Very often)
|Internet use (hr/day)||-0.23||-0.30||-0.10||-0.05|
|(-0.44; -0.03)*||(-0.55; 0.04)*||(-0.15; -0.05)**||(-0.12; 0.02)|
|Daily study time using computer and internet (hr)||-0.09||0.16||-0.09||0.13|
|(-0.21; 0.03)||(0.07; 0.25)**||(-0.22; 0.04)||(0.02; 0.24)*|
|Time for distance learning (hr/day)||0.06||0.08|
|(-0.02; 0.15)||(-0.04; 0.21)|
|(1.70; 6.27)**||(1.62; 3.33)**||(-0.63; 3.39)||(0.83; 4.12)**||(2.23; 3.04)**|
|95% confidence intervals in parentheses|
|* significant at 5%; ** significant at 1%|
Table 4: Factors associated with improvements in learning outcomes.
|Total||sex||cycle||The latest grade in English||Cumulative GPA||
|Male||female||p value||Pre-||Post-||p value||<7||>=7||p value||<7||>=7||p value|
|Desire to take part in distance courses(n,%)||moderate||5 (14.71)||3 (27.27)||2 (8.7)||3 (13.64)||2(16.67)||1 (20)||4 (14.29)||2 (22.22)||3 (12)|
|very much||21 (61.76)||7 (63.64)||14 (60.87)||12(54.55)||9 (75)||2 (40)||18 (64.29)||4 (44.44)||17 (68)|
|extremely||8 (23.53)||1 (9.09)||7 (30.43)||7 (31.82)||1 (8.33)||2 (40)||6 (21.43)||3 (33.33)||5 (20)|
|Total||34 (100)||11 (100)||23 (100)||0.2a||22 (100)||12 (100)||0.34a||5 (100)||28 (100)||0.5a||9 (100)||25 (100)||0.47a|
|Students who are willing to pay for distance learning courses without certification but new knowledge (n,%)||no||18 (51.43)||3 (27.27)||15 (62.5)||12 (52.17)||6 (50)||4 (80)||13 (44.83)||6 (66.67)||12 (46.15)|
|yes||17 (48.57)||8 (72.73)||9 (37.5)||11 (47.83)||6 (50)||1 (20)||16 (55.17)||3 (33.33)||14 (53.85)|
|Total||35 (100)||11 (100)||24 (100)||0.053a1||23 (100)||12 (100)||0.903a1||5 (100)||29 (100)||0.2a||9 (100)||26 (100)||0.25a|
|Students who are willing to pay for distance learning courses without certification but new knowledge (n,%)||no||14 (48.28)||2 (25)||12 (57.14)||10 (52.63||4 (40)||3 (75)||10 (41.67)||4 (57.14)||10 (45.45)|
|yes||15 (51.72)||6 (75)||9 (42.86)||9 (47.37||6 (60)||1 (25)||14 (58.33)||3 (42.86)||12 (54.55)|
|Total||29 (100)||8 (100)||21 (100)||0.1a||19 (100)||10 (100)||0.4a||4 (100)||24 (100)||0.2a||7 (100)||22 (100)||0.46a|
|Students who are willing to pay for distance learning courses with certification (n,%)||no||6 (17.14)||1 (9.09)||5 (20.83)||3 (13.04)||3 (25)||1 (20)||5 (17.24)||3 (33.33)||3 (11.54)|
|yes||29 (82.86)||10 (90.91)||19 (79.17)||20 (86.96)||9 (75)||4 (80)||24 (82.76)||6 (66.67)||23 (88.46)|
|Total||35 (100)||11 (100)||24 (100)||0.4a||23 (100)||12 (100)||0.329a||5 (100)||29 (100)||0.6a||9 (100)||26 (100)||0.16a|
|Students who want to partipate and is willing to pay for the distance learning course with certification (n,%)||no||3 (10.34)||0 (0)||3 (14.29)||2 (10.53)||1 (10)||0 (0)||3 (12.5)||1 (14.29)||2 (9.09)|
|yes||26 (89.66)||8 (100)||18 (85.71)||17 (89.47)||9 (90)||4 (100)||21 (87.5)||6 (85.71)||20 (90.91)|
|Total||29 (100)||8 (100)||21 (100)||0.4a||19 (100)||10 (100)||0.733a||4 (100)||24 (100)||0.6a||7 (100)||22 (100)||0.58a|
Table 5: The willingness of students to participate and pay for distance learning courses.
In this study, we evaluated students’ learning outcomes and experience with an e-learning course, “Introduction to Public Health.” By using technology to bridge the gap between University at Albany SUNY and Hanoi Medical University, students were able to learn from an international team of experts in the field of public health. In addition to teaching the students about the basic concepts of public health, we also hoped to equip them with a lifelong desire for learning, and the academic and professional skills that they will need in their careers.
This study found that, in general, the students did have a positive experience with the course. 86% of the students completed the course and the students’ high degree of knowledge acquisition is well indicated by their success in the course, in which all passed and over a third earned an A. But contrary to concerns that e-learning is only effective at knowledge acquisition, we also saw large increases in intellectual abilities - such as independent working skills, memory, and time management - and academic skills – such as analysing, presenting, and searching for information. Additionally, despite the fact that e-learning is often considered to be a boring and de-motivating learning format, we also saw large increases in students’ educational engagement and desire to learn.
The only domain that showed only a moderate increase was the interaction domain. While this is consistent with the literature which describes the lack of interaction as a major drawback of e-learning, it is also a little concerning, given that one of the identified needs for public health workers is inter-sectoral cooperation development [1-4,6]. The fact that there was some improvement indicates that certain elements of our course design were successful. For example, research has established social presence as a prerequisite for increased participation, and our use of videoconferencing and simultaneous communication built a lot of social presence into the course . Additionally, attendance and participation was mandatory for this course. However, if e-learning is to be implemented for public health education, it is important to see further improvements in interactions in the future. One study found that perceived ease of use of technology improved social presence, so perhaps implementing user-friendly technology and familiarizing students with the technology may help improve interactions . Additionally, we found that while the simultaneous videoconferencing helped create a strong social presence, it was also difficult to coordinate simultaneous communication given the time difference between New York and Hanoi. It may be beneficial to keep these geographic factors in mind, and to include an online forum for more, albeit not immediate, communication.
Our analysis illuminated some additional potential limitations with the e-learning format, as well. We characterized IT conditions and access among public health students, and while we found that students generally had high access to computers and internet, we also found that ease of access was correlated with better improvements in learning outcomes. While this was not a significant issue in this study, it highlights the fact that IT infrastructure may be a significant limiting factor when trying to provide distance learning education to students in more rural locations [8,13,14]. Finally, since the course was facilitated in English, we found that higher English proficiency was correlated with greater improvements in learning outcomes. Thus, while technology can bridge the gap between countries, the success of e-learning is also dependent on the students’ skills.
These limitations highlight certain areas for future improvements and innovations in the field of e-learning in Vietnam. Since students with greater ability to use and access technology are clearly more successful at e-learning courses, as these programs are developed, careful attention must be paid to IT issues. In the long-term, IT infrastructure should be improved in Vietnam, but in the mean-time, e-learning courses should factor these problems into their design. For example, in our course, we began the class with an introductory session to familiarize students with the technology, and provided access to a computer and internet for all students at the university. Finally, ensuring that students have the necessary skills - including IT skills and English proficiency - for accessing these courses is absolutely essential to ensure their success.
While this study was the first in Vietnam to assess the effectiveness of e-learning for public health, it did have some limitations that should be acknowledged. Firstly, the lack of a control group confined our evaluation of the impact of the distance learning method to causal inference. In addition, our small sample size might limit the capability of the regression model to estimate and in factor analysis. Finally, the small sample size, consisting of all extremely successful students, may have skewed our results, given that e-learning is most effective among highly self-motivated and mature students . Nevertheless, since this is the first experience in facilitating a distance course in public health with international experts, findings of this study would be helpful for the development of such training programs in Vietnam.
This e-learning course in public health not only provided important knowledge to the participating students, but also improved their academic and intellectual skills and inspired a greater desire for learning. This study demonstrated that such international partnerships as the one forged between HMU and SUNY can be greatly beneficial to Vietnamese students. Therefore, international collaboration and e-learning courses should continue to be developed in Vietnam, and continue to be improved in the areas that this study highlighted.
BXT, QLTN, VMN, RMM, ATN, HAN, CTN, HPD, HTL were involved in the design and writing of this study. BXT, QLTN, VMN, ATN, HAN, and CTN collected and analyzed the data.
We would like to thank the Institute of Preventive Medicine and Public Health, Hanoi Medical University, Vietnam for substantial support throughout the study.