Received date: May 07, 2014; Accepted date: June 18, 2014; Published date: June 20, 2014
Citation: Garoushi SK, Taher SM, Al-Tawaty AI (2014) Dental Students Perception and Attitude Towards their First Exit Objective Structured Clinical Examination. Dentistry 4:243. doi: 10.4172/2161-1122.1000243
Copyright: © 2014 Garoushi SK, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Dentistry
Aim was to assess students’ perceptions and attitudes towards having their first summative (OSCE) at the dental school of the Libyan International Medical University, Libya. A 14 item questionnaire used in previous publications was adapted to assess 5th year students’ responses. Twenty four students filled the questionnaire immediately after sitting OSCE examination. The questionnaire was administered to the same group of students on 3 study blocks. A 5 point Likert scale was used for rating. The OSCE consisted of total 72 stations. The responses were analyzed using descriptive statistics. The Cronbach’s alpha index of the questionnaire was 0.92 and the mean score of all items was 3.03 (SD 0.09) with a range from 1 to 4. The mean of scores assessed by students regarding the exam easiness was low (2.5). Students score for the adequacy of the stations number and the stations time was 3.3. Based on the evaluations of questionnaires, it could be concluded that our students neutrally evaluated their first experience with OSCE. However, most students perceived it as a difficult method of assessment. Writing stations induced less anxiety than immediately scored ones.
OSCE; Undergraduate dental students; Assessment of competencies
The objective structured clinical examination (OSCE) uses a series of test stations to test clinical competencies. Since 1975, the OSCE has been widely used for testing competencies in medical education , and around 1997, the OSCE was also introduced in dental education [2-4]. The OSCE is a clinical competency test where the student rotates between 10 to 20 test stations with tasks of mostly 5 or 10 minutes in duration. At each station, the student's performance is observed and assessed by an examiner using a multi-item criteria checklist . OSCE has been used to evaluate student performance both at undergraduate and postgraduate levels [5,6]. It is important for dental practitioners and specialists in dentistry to attain the necessary competencies, to continue education and to keep up-to-date with the latest developments in order to ensure the provision of best treatment and to maintain high standards of care for patients. It is equally crucial that appropriate methods of assessment of clinical competencies are developed so that it is possible to detect a fall in standards below an acceptable level. OSCE was intended to serve several purposes: to improve alignment with the goals of clinical training; to facilitate structured, objective assessment ; and to reveal students' strength and weaknesses, thereby enhancing the potential for learning [8,9].
All dental schools in Libya adopted the performance of a clinical dental procedure on a patient as a method of evaluating and testing clinical skills of dental students in the final two years of their study. For example, the students’ performance on caries removal and placement of a filling in a tooth is tested by a committee of three examiners. This method is not without inherent drawbacks and disadvantages, which include the burden of selecting a patient and ensuring his or her attendance to the exam, the lack of uniformity in selected cases, stress associated with such clinical exams, and interexaminer differences in evaluation. In addition, it assesses the student’s ability to manage one clinical case, leaving behind a multitude of clinical scenarios the students should be able to manage in order to be judged as competent and clinically satisfactory. Furthermore, subjectivity and bias may play a role in this method of evaluation .
One of the main strengths of the OSCE examination is its inherent objectivity whereby the aim is to remove patient and examiner variation so that the only variable being examined is the ability of the candidate. Other advantages of the OSCE system include the flexibility and versatility made possible by the multiple station design. This means that it is possible to examine a range of skills and disciplines and even to incorporate more than one skill or discipline simultaneously in the design of a particular station .
Moreover, OSCE formats are more likely to measure other qualities such as problem-solving ability, critical thinking, and communication skills . Overall, it has been found that the OSCE can be a reasonably reliable, valid, and objective method of assessment [6,10,11].
The introduction of an OSCE in the dental school of Libyan International Medical University is a new experience for both staff and students. Thus, the aim of this study was to assess dental students’ perceptions and attitudes towards having their first summative OSCE examination.
Materials and Methods
This study has been approved by our faculty review board. The OSCE was conducted as final clinical exam for the 24 dental students in their 5th year (males and females were equal in number). It was carried out in September 2013 at the School of Dentistry in Libyan International Medical University, Benghazi, Libya. The education unit in the University had the task of educating staff members regarding the nature of the exam and its various aspects. The students were informed of the procedures and the assessment criteria a few weeks before the OSCE. The teachers (examiners) of the clinical disciplines were asked by the faculty's dean to present as an example one or two OSCE scenarios for the students. All teachers were asked to participate in the preparation of the OSCE stations. They were specifically requested to design clinically oriented questions that would neutrally assess students’ clinical competence or judgment. The OSCE committee reviewed all the prepared questions, confirmed their clinical orientation and validity, and ensured that questions could be answered in the given time.
The OSCE (seventy-two stations in total) was divided into three study blocks (Oral medicine, Surgery and Diagnosis block; Restorative dentistry and periodontology block; and Child dental care, orthodontics and community dentistry block). The OSCE for each block was conducted on a separate day. Each block consisted of 24 stations which were divided into short (5 min) and long (10 min) tasks. These stations were settled into two places (simulation laboratory and dental clinic) and each consisted of approximately 12 stations. Three faculty members were appointed to supervise each exam area. Their responsibilities included monitoring to prevent tampering with exam items, to ensure a smooth flow of students, and to collect examination booklets at the end of the exercise.
Most stations included props such as dental models, simulated patients, photos, or radiographs. These stations were shared equally between the clinical disciplines, and domains of competencies tested were in alignment with the faculty-adopted competencies. Stations were numbered, and arrows were provided to direct the students through the exam. Task details were labeled beside or in front of each station. The students were divided into two groups (n=12) in accordance with their university registration number on the attendance sheet. Alternately, one group started short time OSCE stations and the other started the long time OSCE stations. For long time OSCE stations, examiners with checklists were evaluating each student in the station (scoring station). While in short time OSCE stations, students had to write their answers in an examination booklet, which was given to them at the starting point (writing station). A staff member was given a stopwatch and asked to adjust it every 5 or 10 minutes, and students were asked to change stations when they heard the alarm.
Immediately at the end of the OSCE, the questionnaire was distributed to the students, and they were asked to anonymously and voluntarily participate by completing the questionnaire and then returning it to a delegated member of the staff in the area.
The questionnaire was adapted from ones reported in literature for similar purposes [6,10,11]. Although we did not make many changes to the original version of the questionnaire, we had to review its reliability. A Crohnbach's alpha was calculated for this purpose. A 5 points Likert scale was used for rating the first 13 items (1 Strongly disagree; 2 Disagree; 3 Neither Agree nor Disagree; 4 Agree; 5 Strongly agree). The questionnaire contained 14 items divided into two main themes (only the last item was scoring yes or no). The first theme concerned the format of the exam. Its items investigated students’ perceptions of the exam’s easiness and meaningfulness, the awareness of OSCE style, general performance of the OSCE as an assessment of clinical skills, general perception of the OSCE in comparison to the traditional clinical exam, general student performance, identification of student weaknesses, and relaxation during the OSCE. The second theme concerned the logistics of the exam, e.g., number of stations, organization, and station’s time. In addition, the level of student relaxation during the OSCE was also investigated.
The students were encouraged to record useful comments or observations on how we can improve the OSCE for next time. It was intended that the form could be easily and quickly completed by students.
The collected data was fed into SPSS software (SPSS Inc. 18, Chicago, IL, USA) and was statistically analyzed using descriptive statistics.
The results of the statistical analysis are shown in the table 1. Cronbach’s alpha reliability of the questionnaire was high (0.92) and the mean score of the 13 items was 3.03 (SD 0.09) with a range from 1 to 4. The mean score of students’ perceptions of OSCE easiness was 2.5 (SD 0.72), which means that students perceived the exam as a relatively difficult one.
|Degree of OSCE easiness||2.51 (0.72)|
|Awareness of OSCE style||2.91 (1.12)|
|Meaningfulness of OSCE||3.26 (1.14)|
|Exam organization||3.16 (1.22)|
|Number of stations||3.30 (1.04)|
|Stations time||3.33 (1.11)|
|Testing clinical judgment and skills||3.22 (1.01)|
|Identification of personal weaknesses||3.09 (1.05)|
|Performance consistency throughout stations||2.96 (1.06)|
|Relaxation during the exam||2.39 (1.25)|
|OSCE preference||3.09 (1.17)|
|Overall performance||3.11 (1.04)|
Table 1: Students’ feedback questionnaire regarding OSCE
Regarding the meaningfulness of the exam, the mean score was 3.2 (SD 1.14) and this same score was recorded with regards to the ability of the exam to test clinical judgment and skills. The mean of scores 3 (SD 1.05) was reported regarding students ability through the OSCE to identify their personal weakness and the same mean of scores 3 was found regarding the students’ preference for this type of test rather than traditional patient based exams.
The mean score of students’ perceptions of their overall performance in OSCE was 3.1 (SD 1.04). The majority of students indicated a low level of relaxation (mean 2.3) during OSCE. The stress level during the scoring type exam was (65%) while in the writing style exam it was (35%). However, the success rate of students in scoring and writing stations was similar (74.2% and 71.3% respectively).
Concerning the logistics of the OSCE, a score of 3.1 (SD 1.22) was reported for exam organization, whereas 3.3 (SD 1.11) was reported for the number of stations and station time. The overall percentage of student success in the OSCE was 91% and only 2 students failed (male).
It is believed that it is crucial to use reliable and objective assessment methods in education in order to properly assess student performance. OSCE has certain advantages in the sense of being standardized and reasonably objective. The questionnaire we used was used by others [6,10,11]. We modified it locally to suit the local situation. However, we can confirm that only a few changes were made to the original version of the questionnaire. The internal consistency of the items in the questionnaire was proven by a Cronbach’s alpha of 0.92. Questionnaires with a reliability index of over 0.7 are generally considered reliable ones . The validity of an exam is defined as the extent to which an exam reflects the taught material . Though we think that the validity of this exam is reasonable, we cannot claim that it is a reliable one because it has been conducted only once. Administering the test to the same group of students at some time in the future might shed light on this. This is quite important because our dental faculty is new and there is a need to evaluate questions used in the examination to review the possibility of reusing them in the future.
In our study, most of the attending students responded to the questionnaire, which was completed anonymously by students immediately after the OSCE sessions. This procedure was in agreement with previous studies [10, 14]. This questionnaire tests level one of Kirkpatrick’s model . Although it relies on students’ perceptions and does not assess the impact of OSCE in more objective way, using such a level is still useful in acquiring insight into how students perceived their first experience with OSCE exam. The mean score of the students’ perceptions of their ability to identify their personal weaknesses was 3 (SD 1.05). This value indicates that students are not sure about their ability to identify their weaknesses through OSCE. The administration of the questionnaire immediately after finishing the examination, when students were still stressed, might have contributed to this.
The mean score of the students’ perceptions of OSCE easiness was 2.5. This contradicts with Hammad et al., which showed that the majority of their students scored OSCE as an easy or not difficult exam . It could be speculated that the degree of easiness of the OSCE is a key factor and, as a result, has an influence on students’ opinions and attitudes towards it. Since this was the first OSCE in our school, there were no experienced peers to inform the students about what to expect. Apparently, the lecture given on the methods of the OSCE and the information on the internet were not sufficient to make students reasonably aware of OSCE style (mean 2.9). This reflects the importance of student training before conducting OSCE for first time. In the long-term, the students’ knowledge of the content of OSCEs can be used in order to direct their learning. Hence, it has been reported that frequent reuse of a station on a central subject resulted in a marked increase in scores at this station .
Although a timed examination can provoke more anxiety, it has been shown that more time per station did not improve student performance . In accordance, the majority of our students indicated a low level of relaxation (mean 2.3) during OSCE. Sixty five percent of students were more stressed in stations where examiners were scoring them during the exam compared to only 35% when scoring was made on students’ written answers. Most students thought that they could express themselves better in writing and had doubts in their ability to pass the clinical tasks at test stations under the examiners’ immediate assessments. However, it seemed that this had no effect on actual student achievement since the pass rate was almost the same.
The planning phase of the OSCE was very time-consuming, as the dental school has not previously performed this kind of assessment. The creation of exam questions, key, and checklist sheets took place over several weeks. This aspect has also been recognized by former organizers of dental as well as medical OSCEs [1,3]. The main issues to consider were the content and the organization of the OSCE. However, the experiences gained now can, to a large extent, be used when performing another OSCE, thus reducing the time needed for preparation. Our checklist items are scored as correct, partially correct, or incorrect and we assign each a specific point value. For example, an item may be worth a total of five points. If a student scores the item totally correctly, he or she receives five points toward the total score. If the faculty judge determines the answer is partially correct, the student earns two and half points. If the student answered the item incorrectly, he or she receives no points for that item. Students’ total raw scores are collected and then converted to a traditional grade on a letter scale. One advantage of this method is that difficult items that test higher order skills or items inherent to the skill can be assigned higher point values than the easier items. This is known as “weighting” of the exam questions .
There was no immediate feedback given directly after each of the three sets of OSCE stations. Immediate feedback can be highly influential on students’ learning and performance; as Hodder noted , it can improve the students’ competency in the performance of reasoning-based tasks. Still, it is possible that the general feedback given after the exam had a positive influence and improved their competency overall (Table 1).
Concerning the logistics of the OSCE, a score of 3.1 was reported for exam organization, whereas 3.3 were reported for number of stations and station time.
It is clear from these figures that our students were satisfied with the organization of the exam. However, a mean value of just above 3 for the exam organization, adequacy of number of stations and station time invites us to consider how we should further improve the organization of the exam. The OSCE study of Schoonheim et al. showed that, for a reliable decision, at least 12 stations are needed in OSCE; in our exam, we used 24 stations for each study block . Three categories of stations have been identified: 1) Interactive, nonproctored stations where the student examines materials and provides a written answer to the question; 2) Observed stations where the student performs a task in front of a passive observer who judges his or her performance; and 3) Proctored, interactive stations where the student interacts with a proctor trained to act as a standardized patient or parent and also serve as a judge of the student’s performance. Space availability, time allotted to the exam, class size and available staff members are factors controlling the numbers, lengths and types of stations .
The student failure rate was very low 9% (2 students). As all students in this study were in the final year of the dental school, this low failure rate is not surprising and, indeed, was expected. On other hand, OSCE, like other evaluation methods, has limitations like not assessing some patient-related factors such as soft tissue management, moisture control, gagging reflex, effectiveness of local anesthesia and patient restlessness or anxiety. However, these important factors are usually assessed throughout the students’ clinical logbooks.
Another limitation of this study is that we did not perform a pilot study before designing the questionnaire although the results showed high value of the questionnaire’s reliability (0.92).
In conclusion, the OSCE was neutrally received by our students. Because of the interdisciplinary set-up and the students’ more realistic self-assessment in the test, the OSCE is considered as a suitable tool to promote reflective thinking in the students.
Authors wish to thank Shafaq Malik, Hadil Naser and Ausama Gargum for their kind help in this study.