Journal of Health & Medical Informatics

Back to the Future: Electronic Marking of Objective Structured Clinical Examinations and Admission Interviews Using an Online Management Information System in Schools of Health Sciences

Abstract

Thomas JB Kropmans *,Liam Griffin ,David Cunningham ,Domhnall Walsh ,Winny Setyonugroho ,Catherine Anne Field ,Eva Flynn ,Kieran M Kennedy

Abstract Background: The Objective Structured Clinical Examination (OSCE) and Multi Mini Interviews (MMI) are established tools in the repertoire of clinical assessment methods in Schools of Medicine and Health Sciences worldwide. The use of OSCEs facilitates the assessment of psychomotor skills as well as knowledge and attitudes. Identified benefits of OSCE assessment include development of students’ confidence in their clinical skills and preparation for clinical practice. However, a number of challenges exist with the traditional paper methodology, including documentation errors and inadequate student feedback, electronic assessment is therefore new future. Objectives: To explore electronic OSCE delivery and evaluate the benefits of using an electronic OSCE management system. Design: A pilot study was conducted using electronic software in the management of a five station OSCE assessment with a cohort of first year undergraduate medical students delivered over two consecutive years (n=383) in one higher education institution in Ireland. Methods: All OSCE documentation was converted to electronic format. Assessors were trained in the use of the OSCE management software package and laptops were procured to facilitate electronic management of the OSCE assessment. Following the OSCE assessment, assessors were invited to evaluate the experience. Results: Electronic software facilitated the storage and analysis of overall group and individual results thereby offering considerable time savings. Submission of electronic forms was allowed only when fully completed thus removing the potential for missing data. Conclusions: Analysis of results highlights issues around inter-rater reliability and validity of measurement tools. Regression analysis, as a standard setting method, increases fairness of result calculations as compared to static cutoff scores.