3Consultant Anaesthetist, Department of Anaesthesia, Nottingham University Hospitals NHS Trust, Division of Anaesthesia and Intensive Care, University of Nottingham, Queen’s Medical Centre, Nottingham, UK
Received date: June 02, 2014; Accepted date: June 17, 2014; Published date: July 24, 2014
Citation: Daunt M, Flack J, Baxendale B, McCahon RA (2014) Evaluation of Human Factors in Airway Management Course. J Anesth Clin Res 5:416. doi: 10.4172/2155-6148.1000416
Copyright: © 2014 Daunt M, et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Visit for more related articles at Journal of Anesthesia & Clinical Research
The Human Factors in Airway Management course was designed to emphasise the importance of human factors and non-technical skills in difficult airway management scenarios, whilst teaching the practical skills of using specialised airway devices. A validated version of the Operating Room Management Attitudes Questionnaire (ORMAQ) was used to assess changes in twelve delegates’ attitudes as a result of attending the course. This measures attitudes to leadership, communication, teamwork, stress and fatigue, work values, error, and error management. Pre-course attitudes to leadership hierarchy, stress and fatigue, and information sharing were more positive than those reported in previous ORMAQ surveys of anaesthetists. Eight weeks following the course, there was a preference for a reduced authority gradient within the operating theatre team, increased assertion in the face of seniority, and an improvement in attitude to multidisciplinary team-working. By demonstrating these changes in attitude to patient safety, we believe the impact of non-technical skills training can translate into positive changes in clinical practice.
Human factors; Airway management; ORMAQ, Nontechnical skills
Effective difficult airway management is dependent on the specialist knowledge and practical expertise of a capable clinician who has access to appropriate equipment and assistance. Recent reports have emphasised the importance of specialists also having an awareness of how human factors influence clinical performance in the workplace [1,2]. This highlights the relevance of developing and evaluating high quality education and training to address the key attributes required by clinicians for the complexities of current professional practice .
In 2010, the Helsinki Declaration for Patient Safety in Anesthesiology stated that effective education is essential to the improvement of patient safety . Despite the clear benefits of human factors training described in other high-reliability organisations, healthcare has only recently seen the introduction of these concepts within undergraduate or postgraduate curricula. Evidence of associated improvements in patient safety and evaluation of how best to teach these topics remains scarce, although the use of simulation is promoted as advantageous in terms of the requisite technical and non-technical skills [5,6].
We developed a one day airway skills training course (‘Human Factors in Airway Management’) comprising small group teaching, bench model practice, and high fidelity simulation with debriefing of scenario-based performance. The course programme was designed to incorporate materials to raise awareness of human factors in clinical practice, and emphasised the importance of non technical skills in the context of difficult airway management alongside learning about the technical and practical features of using specialised airway devices. We aimed to evaluate the course by using a validated questionnaire, the Operating Room Management Attitudes Questionnaire (ORMAQ). Our hypothesis was that there would be no change in the delegates’ attitudes toward teamwork and patient safety, as measured by the ORMAQ, following their attendance at the course .
Using the Operating Room Management Attitudes Questionnaire (ORMAQ), we evaluated the impact of a blended simulationbased educational intervention on the attitudes of a single cohort of anaesthetic trainees and qualified ODPs toward patient safety and teamwork. The local NHS Research & Innovation Department waived formal review of the evaluation, and Ethics Committee approval was not required . The Trust Caldicott Guardian determined that the evaluation did not present any information governance issues as all data was anonymised.
Fifteen delegates attended the Human Factors in Airway Management course, of whom 12 were anaesthetic specialty trainees and three were qualified Operating Department Practitioners (ODPs). The course structure is described in more detail below. At registration, delegates were asked to complete the Operating Room Management Attitudes Questionnaire (ORMAQ) which is an adaptation of the Cockpit and Flight Management Attitudes Questionnaire [7,9-12]. The ORMAQ consists of 4 sections and has previously been described and reported within the context of UK anaesthetic practice . We chose to use section 1 (54 items) which measures attitudes to leadership, communication, teamwork, stress and fatigue, work values, and error, and section 3 (5 items) which explores attitudes toward error management. Following an arbitrary period of eight weeks after the course, we asked delegates to repeat the ORMAQ in order to determine if there were any sustained changes in delegate attitudes to patient safety and teamwork.
The Human Factors in Airway Management course was developed by a collaboration of experts in human factors, simulation, medical education and anaesthetic practice in relation to difficult airway management. Its content was designed to address an educational gap in the local delivery of the clinical and non-technical aspects of the Airway Management and Human Factors sections of the 2010 RCA Curriculum . The aims and learning objectives of the course are summarised in Tables 1 and 2.
|•To improve knowledge and understanding of airway related morbidity.|
|•To develop an appreciation of the key role of human factors in high stakes situations and how this can affect performance.|
|•To become familiar with the use of difficult airway equipment within the context of the Difficult Airway Society guidelines|
Table 1: Human Factors in Airway Management course aims.
|Technical skills learning objectives|
|•To use a mental model to help plan airway management.|
|•List the key causes of serious airway morbidity.|
|•To practice using the following airway skills on manikins/airway bench models within the context of DAS guidelines:
•Needle cricothyroidotomy&jet ventilator (Manujet)
•Video laryngoscopy using the Airtraq& C-Trach
•Low skill fibreoptic intubation using the Aintree intubation catheter
|Non-technical skills learning objectives|
|•To identify unhelpful thought processes and behaviours associated with poor performance under stress.|
|•To define non-technical and non-clinical skills|
|•To relate non-clinical skills to clinical skill and knowledge|
|•To identify the major Human Factors issues|
|•To correlate risk and performance|
|•To establish a basic appreciation of team skills|
|•To explore three fundamental non-clinical themes:
•Effective professional communication
|•To consider how to choose behaviour|
Table 2: Human Factors in Airway Management learning objectives.
Pre-course materials were prepared and made available via a dedicated website http://trag.snappages.com which provided delegates with an overview of the technical airway management skills and nontechnical skills to be covered on the course day itself. The website learning resources included video presentations on specific aspects of clinical human factors as well as demonstrations of advanced airway techniques.
The course ran over one day at the Trent Simulation & Clinical Skills Centre. In the first session, the delegates had small-group teaching focussing on the prediction of airway difficulty and planning, airway morbidity and mortality, and an introduction to human factors. Following this, the delegates rotated around five 25-minute workshops in groups of three (Table 3). The content and techniques demonstrated in these practical skills workshops were all mapped to current Difficult Airway Society guidelines . At the cricothyroidotomy workshop, delegates were reminded of the importance of early, clear decisionmaking and they were encouraged to reflect upon the impact of acute personal psychological stress on their own decision-making processes.
|Indirect laryngoscopy||Demonstration and practice on manikin using: •Airtraq video laryngoscope
|Low-skill fibreoptic intubation||Demonstration and practice on manikin with the Aintree Intubation catheter|
|Needle cricothyroidotomy& jet ventilation||Demonstration and practice on manikin and sheep larynx using:
•Ravussincricothyroidotomy needle &Manujet
•Quiktrach needle cricothyroidotomy
|Surgical cricothyroidotomy||Demonstration and practice on sheep larynx|
|Human factors||Small-group session|
Table 3: Workshop activity.
In the afternoon, delegates were exposed to three separate high fidelity simulation scenarios requiring application of difficult airway management knowledge and skills. Each scenario was designed to explore the use of relevant non-technical skills and provide opportunity to discuss this during subsequent debriefing. Six anaesthetists and three ODPs took part in the scenarios, whilst the remainder of the delegates observed and participated in the debriefing process. In order to make the observation an active and more focused exercise, delegates were briefed in using the Anaesthetist Non Technical Skills (ANTS) framework to assist them in identifying specific behaviours which they felt worthy of subsequent discussion . The debrief sessions were facilitated by experts in human factors and medical simulation.
Data collection and statistical considerations
Demographic data was collected for each delegate. ORMAQ data was grouped into eight attitudinal themes for subsequent analysis  leadership-structure, confidence-assertion, information sharing, stress and fatigue, teamwork, work values, error, and error management. Each question in ORMAQ required a response based on a five point Likert-scale ranging from strong disagreement (1) to strong agreement (5) with the statement provided. Flin and colleagues previously treated ORMAQ data as normally distributed, but due to our small sample size we calculated the median pre- and post-course scores for the response to each question, and applied the Wilcoxon signed rank test to test for a significant difference in the paired medians for each question . We considered differences statistically significant at p<0.05. Statistical analysis was performed using Minitab 15 (Minitab Inc., USA) with the pwilcoxon.mac MACRO.
Fifteen delegates attended the course, of which 14 completed precourse ORMAQs and 12 post-course ORMAQs. Nine of these 12 were specialty trainees in anaesthesia and three were operating department practitioners (ODPs). Respondents had worked within an operating theatre environment for 3-20 years, and currently all were employed at hospitals on the East Midlands Deanery (North) School of Anaesthesia rotation.
The changes in attitudes from pre-course to eight weeks postcourse are shown in Figures 1-4. A median change in attitude greater than zero on the ORMAQ scale is highlighted in dark grey for each figure.
Leadership-structure and confidence-assertion
Eight weeks post-course, there was increased preference for a reduced authority gradient within theatre teams (Figure 1A; item 50). Regarding assertiveness, respondents stated that they would be more likely to challenge a senior colleague before patient safety was compromised (Figure 1B; item 38), and that they were more likely to ask questions if they did not understand (Figure 1B; item 60) (Figure 1A and B).
Information sharing and teamwork
Respondents felt more encouraged to report incidents which they observed after the course (Figure 2A; item 16). There was also greater agreement with statements relating to the importance of discussing differences and having open feedback between team members (Figure 2B; items 31, 44, 48, 56). There was greater agreement that feedback should be multi-disciplinary and that teamwork works (Figure 2B; items 17, 54) (Figure 2A and B).
Error and error management
Following the course, there was increased agreement that it was rare for a team member(s) to be unable to deal with error appropriately (Figure 3B; item 67). Respondents also disagreed more that they made errors in theatre (Figure 3B; item 69) (Figure 3A and B).
Stress and fatigue
There was increased agreement that team members should be obliged to declare their own psychological and physical stresses before and during a shift (Figure 4A; item 51). There was greater disagreement with the statement that tiredness does not affect performance (Figure 4A; item 4), and with the statement “personal problems can adversely affect my performance” (Figure 4A; item 55). Respondents felt as able to make good decisions in emergencies as they were in elective situations (Figure 4A; item 11).
There was greater agreement that senior staff deserve extra benefits, that competence should be acknowledged, that tardiness in the operating theatre was insulting, and that respect from other members of the operating team was deserved (Figure 4B; items 6, 20, 40, 52). There was greater disagreement that one works best when left alone, and with the statement “it bothers me when others do not respect my professional capabilities” (Figure 4B; items 7, 9).
Evidence to demonstrate the educational impact of non-technical and human factors training in healthcare professionals is difficult to collate due to the complex nature of the workplace as well as the variety of educational approaches available, which Ringsted described as the ‘Learning Ecology’ . This has resulted in an absence of clear published data to help identify ‘best educational practice’ in this field. The results of the ORMAQ surveys in our study suggest that attitudinal changes occurred following attendance at the Human Factors in Airway Management course, and that they were maintained for at least eight weeks after the course. In particular, the delegates indicated a preference for a reduced authority gradient within the operating theatre team, increased assertion in the face of seniority and an improvement in attitudes to multidisciplinary team-working. These attitudinal changes are especially relevant in light of well publicised individual cases . Increased information sharing, increased tendency to report incidents, and greater recognition of one’s limits of performance were also seen.
Kirkpatrick has described a ‘four level’ model of training programme evaluation . Level one describes the ‘reaction’ from the participants, levels two and three describe ‘learning’ and ‘behavioural change’ respectively, and level 4 describes any changes resultant on the behavioural changes, e.g. patient outcomes. Specifically, Kirkpatrick defined learning as the extent to which participants change attitudes, increase knowledge, and/or increase skill as a result of attending an educational programme. Currently, many educational courses gain feedback which corresponds to the first level of evaluation by addressing whether candidates ‘enjoyed’ the experience whilst describing any changes in their self-perceived confidence in applying new knowledge or skills in the clinical workplace. We believe that by demonstrating attitudinal changes we have gone beyond simply determining whether the delegates enjoyed the course, and that we have satisfied the criteria for Kirkpatrick’s second level of programme evaluation which describes learning. However, we acknowledge that a weakness of this course evaluation is that we have not attempted to assess delegates’ behavioural changes in their clinical practice following the course, i.e. Kirkpatrick’s third level
The pre-course attitudes were somewhat different to those of an anaesthetic cohort of 2003 . In particular, there appears to be a healthy shift in attitudes to leadership hierarchy, stress and fatigue, and information sharing. In our sample, this baseline shift may represent previous training in human factors and non-technical skills, or engagement with the pre-course learning material on the website. In addition, it is likely that it is due to the increased emphasis on patient safety which the WHO checklist first heralded in 2008 .
Participants developed an increased awareness of personal limitations, and an increasing number would let others know if their workload was excessive. Interestingly, more participants felt after the course that personal problems would not adversely affect performance. We believe that this may represent recognition and adoption of Reason’s three-bucket model by delegates as a strategy to reduce the probability of error in the workplace . However, the continued perception of invulnerability to stress and fatigue is something that needs to be addressed, as the adverse consequences are well known [19,20].
We recognise that there are obvious limitations to the data presented. The internal consistency (Cronbach’s alpha) of the survey items within each domain are low and have previously been reported as 0.18-0.54 . Therefore, any changes in the ORMAQ are indicative, rather than definitive, of attitudinal changes . However, the ORMAQ is well-cited in the literature, and has face validity as a tool to measure and describe attitudinal changes to teamwork and safety in the healthcare setting [7,21,22]. A second limitation is that no firm conclusions can be made based on the small sample described; any change in attitudes in the eight weeks following the course may have been coincidental. Despite this, we do believe that the attitudinal changes did not occur simply as a result of chance as the respondents acted as their own controls, and that the described attitudinal changes indicate achievement of the course’s learning objectives related to highlighting the importance of non-technical skills in difficult airway management.
Despite these limitations, we feel that the course provides a useful and valid model for the creation, quality assurance and development of simulation-based training in human factors and non-technical skills. McGaghie et al. identified 12 evidence-based features which improve the educational impact of simulation-based medical education (SBME) . We believe that we have incorporated several of these best-practice features into our course, i.e. feedback, a valid outcome measure, an appropriate match between learning objectives and simulation fidelity, and an educational and professional context. Indeed, these features reinforce our assertion that the measured attitudinal changes were not merely the result of chance. By using these best-practice features, we have also met several of the recommendations set out in the recently published Framework for Technology Enhanced Learning, specifically .
Patient-centred and service driven
Recommendation 3-Learning should meet a clearly defined patient and service need. The findings of the 4th National Audit Project in Anaesthesia (NAP4) highlighted the need for human factors and nontechnical skills training for the anaesthetic team .
Recommendation 5c-Simulation should be mapped to specific learning outcomes in identified areas of the curriculum. The 2010 edition of the curriculum for a CCT in Anaesthesia pre-dates NAP4, but recognised the importance and need for the development of nontechnical skills throughout training in anaesthesia .
Innovative and evidence-based
Recommendation 6-There should be an aspiration to educational excellence by encouraging innovation, evaluation, and the dissemination and adoption of evidence-based, good practice. At the time of course development and delivery, there was no other course nationally available that incorporated high-fidelity, simulation-based human factors training within the context of airway management. Subsequently, a number of such courses have become available, but as yet there are no published data on their educational impact with respect to non-technical skills learning.
It is reassuring that the Human Factors in Airway Management course is associated with positive changes in personal attitudes to factors which impact on individual and team performance. The next logical steps are to examine whether improved attitudes to teamwork and safety actually translate into positive changes in practice. Ultimately, the bottom line is an improvement in patient safety and thus, patient outcomes. This will be difficult to measure in the short to medium-term. However, in the meantime we must at least ensure that we are delivering effective education, so that improvements in patient safety may be realised.