Simulation Training and Skill Assessment in EMS


Introduction

Emergency Medical Services (EMS) is the system tasked with providing rapid medical care and transport in the pre-hospital setting. Different forms of EMS have existed throughout history from the Knights Hospitaller during the middle ages to the first ambulance service in the United States in Cincinnati in 1865. The current organization of EMS in the US began after a report titled “Accidental Death and Disability” was published in 1966 by the National Academy of Sciences detailing the state of EMS in the United States and areas for improvement. The National Highway Safety Bureau, now the National Highway Traffic Safety Administration, was created in 1970 as a response. Among its first actions was to create the National Registry of Emergency Medical Technicians to establish training and education standards for the industry.

Simulation in medical education is defined as the imitation of clinical experience. It can be used both as an educational tool and a skill assessment tool. As an educational tool, there are multiple modalities available, both high- and low-fidelity; these include the human patient simulators like the electronic adult-sized mannequins, the task trainers such as the IV arm trainer, standardized patients, or virtual or augmented reality.[1]

Skill assessment is the evaluation of an individual to gauge their ability to perform a task. In EMS, the ultimate goal is to assess a trainee’s ability to perform a skill in the field on patients as the need arises. There are numerous methods in use to evaluate skills, including simulation. Other popular assessments include the multiple-choice question test & field observation.[2]

Function

This brief review seeks to summarize the current literature on the use of simulation in EMS education and in skill assessment. Simulation is widely used by EMS education programs. Its origins can arguably be traced back to the collaboration between the toy company Laerdal and Dr. Safar, who developed mouth-to-mouth resuscitation. This partnership led to the production of the first Resusci-Anne training simulator.[3] Today, nearly all EMS training programs have access to at least one simulation modality.[4]

Issues of Concern

Early exposure to ALS patients during EMS training has been identified as a strategy utilized among highly effective EMS training programs, however, a barrier to training then, is the infrequency of exposure to out-of-hospital cardiac arrest or other events such as mass-casualty incidents even among working medics.[5][6][3] A 2017 survey of ten Australian & New Zealand EMS employers identify simulation-based training in resuscitation skill refresher courses as a possible way to remedy this lack of direct exposure to important clinical scenarios.[6]

Recent events have also demonstrated simulation as an effective means to coordinate the responses of various agencies during a mass casualty incident (MCI). This specialized kind of simulation training was touted as one of the reasons for an integrated local response to the Pulse Nightclub shooting in Orlando in 2016, and it is likely to be used more heavily in future EMS training programs.[3][7]

A survey of 389 paramedic programs found all programs have access to some kind of simulation tool (and very often to a wide variety of such tools), meaning simulation is universally available in EMS education. Unfortunately, despite this availability, several barriers prevent the appropriate use of the resources at hand. In approximately one-third of surveyed EMS programs, some portion of available simulation tools were found to sit idly without regular use, most commonly because of a lack of educator familiarity and training in the use of these tools. In programs using advanced mannequins, 19% of teaching faculty had no formal training with the equipment. Addressing this neglect may lead to improved efficacy of simulation training down the road.[4]

Curriculum Development

The NREMT is the governing body accountable for certifying trainees in EMS. To become certified, candidates must pass a 2-hour computer-adaptive multiple-choice question exam. At this time, there is no other skill assessment involved. Training programs that consistently had higher than a 75% first-time pass rate were interviewed with the intent of identifying the keys to the success of their graduates. The researchers identified 12 strategies common among these high-performing programs. Although simulation-based training was not explicitly mentioned as one of these strategies, most interviewed directors revealed the regular use of simulation as part of their curricula.[5][4]

When developing a strong curriculum, directors should consider many different theories of adult learning. Miller’s pyramid is one model that references simulation as an essential component of the learning process. In this model, a learner progresses from Knowledge (e.g., gathering facts) to Competence (i.e., knowing how), to Performance (i.e., showing how), and finally to Action (e.g., integration of new skills into actual practice).[8] Simulation falls into the Competence level of the pyramid, allowing trainees to practice procedures (especially rare ones) without risk of patient harm before they progress to performing the procedure on an actual patient.

Not just any simulation will allow learners to successfully move from Miller’s third level to the final one. A quality simulation-based training session consists of several important components. First, the topic to be simulated, as well as the goals of learning, should be well-defined. The topic can be anything from a pediatric MCI to a single-patient dyspnea call, while the learning goals can be successful triage or successfully recognizing the need for and completing intubation. Second, the type of simulation tools to be used to achieve the learning goals should be identified. Third, checklist-based evaluation tools for learners and educators should be available.[9] Finally, there should be a plan in place for a debriefing session following the simulation activity.[1]

Procedural Skills Assessment

Simulation is also widely used as either a supplement or replacement for clinical skills assessment in EMS education. For example, a paper on the state of EMS training and skills in under-resourced areas like Kenya used observation of simulation scenarios as part of its assessment of the skills of EMS personnel.[10] Another study on the effect of teamwork on medical errors used simulation as the sole method of data collection.[11] One other review demonstrates that paramedics who were taught cricothyrotomy on a task trainer were proficient in the procedure on the same device 12 weeks later. More research is needed to determine if those results accurately reflect the trainees’ skills in the field.[12]

A statistical analysis examining the validity of simulation as an accurate assessment of clinical competence in the field found that in five of seven dimensions considered essential paramedic skills (situation awareness, history gathering, patient assessment, decision making, and communication), trainees scored comparably when observed in simulation and the field.  In the remaining two dimensions (resource utilization and procedural skill), trainee performance scores during simulation did not significantly correlate with scores obtained in the field. This study is critical to understanding the use of simulation in the assessment of EMS skills. Evaluation of clinical skills by simulation is overall a reliable means of evaluating clinical competence but does not guarantee skill level in the field or replace assessment in the field altogether.[2]

Clinical Significance

Simulation, the imitation of clinical experience, is being used both to train and test EMS personnel. Quality simulation experiences are beneficial when teaching and evaluating skills and scenarios that rarely happen in the field.[6][3][7] These experiences should consist of defined learning goals, identifiable simulation tools, a clear checklist for evaluation, and a debriefing session.[9][1] There are some significant barriers to simulation as an educational tool, however, including the lack of training for educators in the available equipment. It is also unclear if procedural competence during simulated scenarios translates into competence in the field.[2] Despite these shortcomings, simulation is still considered a valuable tool in EMS training programs and will continue to be so as technology advances.

Enhancing Healthcare Team Outcomes

The International Network for Simulation-Based Pediatric Innovation, Research & Education (INSPIRE), a research network focused on the transferability of simulation skills to clinical practice, is an organization to watch in the future as the application of simulation evolves and its prevalence in EMS education expands.[13] It is also likely that telesimulation will become a more popular educational tool as technology continues to advance. A recent feasibility study analyzing telesimulation using Google Glass concluded that it is possible to conduct an intercontinental MCI simulation and that participants were favorable to the process.[14]

Goals going forward within the EMS simulation community include increasing educator familiarity with simulation tools to encourage utilization, as well as further investigation into how simulation training translates to real-world skills.[4]


Details

Author

Andrea Green

Editor:

Mark Hug

Updated:

5/1/2023 6:57:03 PM

References


[1]

Huang GC, McSparron JI, Balk EM, Richards JB, Smith CC, Whelan JS, Newman LR, Smetana GW. Procedural instruction in invasive bedside procedures: a systematic review and meta-analysis of effective teaching approaches. BMJ quality & safety. 2016 Apr:25(4):281-94. doi: 10.1136/bmjqs-2014-003518. Epub 2015 Nov 5     [PubMed PMID: 26543067]

Level 2 (mid-level) evidence

[2]

Tavares W, LeBlanc VR, Mausz J, Sun V, Eva KW. Simulation-based assessment of paramedics and performance in real clinical contexts. Prehospital emergency care. 2014 Jan-Mar:18(1):116-22. doi: 10.3109/10903127.2013.818178. Epub 2013 Aug 20     [PubMed PMID: 23961742]


[3]

Ganss PE, McCain C. Task training and simulation in the pre-hospital environment. Missouri medicine. 2013 Mar-Apr:110(2):125-8     [PubMed PMID: 23724483]


[4]

McKenna KD, Carhart E, Bercher D, Spain A, Todaro J, Freel J. Simulation Use in Paramedic Education Research (SUPER): A Descriptive Study. Prehospital emergency care. 2015 Jul-Sep:19(3):432-40. doi: 10.3109/10903127.2014.995845. Epub 2015 Feb 9     [PubMed PMID: 25664774]


[5]

Margolis GS, Romero GA, Fernandez AR, Studnek JR. Strategies of high-performing paramedic educational programs. Prehospital emergency care. 2009 Oct-Dec:13(4):505-11. doi: 10.1080/10903120902993396. Epub     [PubMed PMID: 19731164]


[6]

Dyson K, Bray JE, Smith K, Bernard S, Straney L, Finn J. Paramedic resuscitation competency: A survey of Australian and New Zealand emergency medical services. Emergency medicine Australasia : EMA. 2017 Apr:29(2):217-222. doi: 10.1111/1742-6723.12715. Epub 2017 Jan 16     [PubMed PMID: 28093867]

Level 3 (low-level) evidence

[7]

Albert E, Bullard T. Training, Drills Pivotal in Mounting Response to Orlando Shooting. ED management : the monthly update on emergency department management. 2016 Aug:28(8):85-9     [PubMed PMID: 29211414]


[8]

Taylor DC, Hamdy H. Adult learning theories: implications for learning and teaching in medical education: AMEE Guide No. 83. Medical teacher. 2013 Nov:35(11):e1561-72. doi: 10.3109/0142159X.2013.828153. Epub 2013 Sep 4     [PubMed PMID: 24004029]


[9]

Cicero MX, Brown L, Overly F, Yarzebski J, Meckler G, Fuchs S, Tomassoni A, Aghababian R, Chung S, Garrett A, Fagbuyi D, Adelgais K, Goldman R, Parker J, Auerbach M, Riera A, Cone D, Baum CR. Creation and Delphi-method refinement of pediatric disaster triage simulations. Prehospital emergency care. 2014 Apr-Jun:18(2):282-9. doi: 10.3109/10903127.2013.856505. Epub 2014 Jan 8     [PubMed PMID: 24401167]


[10]

Nicholson B, McCollough C, Wachira B, Mould-Millman NK. Emergency medical services (EMS) training in Kenya: Findings and recommendations from an educational assessment. African journal of emergency medicine : Revue africaine de la medecine d'urgence. 2017 Dec:7(4):157-159. doi: 10.1016/j.afjem.2017.06.002. Epub 2017 Aug 8     [PubMed PMID: 30456131]


[11]

Herzberg S, Hansen M, Schoonover A, Skarica B, McNulty J, Harrod T, Snowden JM, Lambert W, Guise JM. Association between measured teamwork and medical errors: an observational study of prehospital care in the USA. BMJ open. 2019 Oct 31:9(10):e025314. doi: 10.1136/bmjopen-2018-025314. Epub 2019 Oct 31     [PubMed PMID: 31676639]

Level 2 (mid-level) evidence

[12]

March JA, Kiemeney MJ, De Guzman J, Ferguson JD. Retention of cricothyrotomy skills by paramedics using a wire guided technique. The American journal of emergency medicine. 2019 Mar:37(3):407-410. doi: 10.1016/j.ajem.2018.05.073. Epub 2018 May 31     [PubMed PMID: 29891124]


[13]

Calhoun AW, Nadkarni V, Venegas-Borsellino C, White ML, Kurrek M. Concepts for the Simulation Community: Development of the International Simulation Data Registry. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2018 Dec:13(6):427-434. doi: 10.1097/SIH.0000000000000311. Epub     [PubMed PMID: 29672467]


[14]

McCoy CE, Alrabah R, Weichmann W, Langdorf MI, Ricks C, Chakravarthy B, Anderson C, Lotfipour S. Feasibility of Telesimulation and Google Glass for Mass Casualty Triage Education and Training. The western journal of emergency medicine. 2019 May:20(3):512-519. doi: 10.5811/westjem.2019.3.40805. Epub 2019 Apr 26     [PubMed PMID: 31123554]

Level 2 (mid-level) evidence