Standardized Patient Assessment Of Learners In Medical Simulation


Introduction

Performance-based assessment is consistent with outcomes-based education,[1] whereby learners can demonstrate the performance of tasks, approach to tasks, and professionalism. Specifically, standardized patient-based performance assessment has advanced to include undergraduate and graduate medical education and is commonly used to evaluate both the technical and nontechnical skills necessary for the safe and effective practice of medicine.[2][3][4] A standardized, objective, and structured method of assessment is critical for quality and accountability in medical education and transition to clinical practice. Miller’s prism of clinical competence provides a framework for simulation-based performance assessment of cognition and behaviors that demonstrate knowledge, skills, and attitudes on the continuum from novice to expert medical professional.[5] Similarly, the Kirkpatrick model provides an adaptable framework to evaluate learners acquisition of knowledge, skills, and attitudes in the simulation lab and the subsequent transfer to clinical practice and actual patient outcomes.[6] Thus, medical learners are commonly required to demonstrate acquisition of skills and competence via simulation before integration into clinical practice.[7][8] Specifically, standardized patient methodology applied to performance-based assessment has been shown to provide a means of valid standardized objective assessment of learning and clinical skills before clinical practice.[9][10][4][11][12]

Function

Over the past two decades, simulation in general and standardized patient methodology specifically, have been utilized as techniques in medical education to teach and assess competent patient-centered care.[13] Standardized patients (SPs) provide an opportunity for the medical learner to immerse and interact in patient-care scenarios that evoke or replicate substantial aspects of actual clinical practice with planned attention to the psychological and physical safety of learners and SPs and without risk of harm to an actual patient.[14][9] Two exam formats have emerged that utilize a standardized patient methodology to assess medical learners' clinical skills, competence, and performance - the Objective Structured Clinical Exam (OSCE) and the Clinical Skills Assessment (CSA).  An OSCE is comprised of multiple focused stations to assess discrete clinical skills, including data and image interpretation, technical skills, communication skills, or physical exam skills.[15][16][17][18]  An OSCE station may include a standardized patient if the objective is best accomplished via a simulated patient encounter, e.g., communication or counseling skills. Similarly, the CSA is an OSCE-like multi-station exam designed to assess a broad sample of clinical skills and competencies. However, unlike the focused OSCE, a CSA station has extended time and breadth and importantly, includes an SP in each station. The CSA assesses the medical learner's ability to integrate and apply multiple skills in each station, e.g., communication, physical exam, diagnostic, and professionalism.[17] Notably, OSCEs and CSAs may be formative or summative assessments of medical learners' clinical skills and competency. Formative assessments aim at providing feedback to the learner, and summative assessments focus on ensuring proficiency and accountability.[19]

Standardized or simulated patients have been part of the assessment of medical learners since the 1960s when neurologist Howard Barrows introduced this educational modality to evaluate clinical skills of 3rd-year clerkship students [2].  An SP is a person who has received coaching to accurately portray a patient and present consistent verbal and nonverbal communication, personality characteristics, emotions, and physical findings.[2] After two decades of research and pilots, in 2004, the National Board of Medical Examiners introduced the Step 2 Clinical Skills exam (Step 2 CS) to assess medical student diagnostic, clinical reasoning, and patient-physician communication skills as part of the United States Medical Licensure Exam. The USMLE Step 2 CS is a high stake summative CSA that utilizes SPs to assess performance competency of examinees.[20][21][22] Formulation of these SP-based clinical skills performance assessments was through the widespread adoption of focused objective structured clinical examination (OSCE) stations that often included an SP as part of formative or summative assessments. Thus, the SPs' role in the assessment of medical learners in the US became institutionalized at the undergraduate medical education (UGME) level via the federal medical licensing exam's adoption of CSAs and via medical schools adoption of OSCEs and CSAs. In addition to high stakes summative CSAs for medical student licensure, standardized patients have participated in OSCEs as a just-in-time formative assessment for the Core Entrustable Professional Activities for Entering Residents (CEPAERs) to provide baseline data and curricular feedback to medical school faculty and residency directors.[23] Similarly, SPs have performed standardized direct observation of medical residents' communication and professionalism milestones to satisfy the Accreditation Council for Graduate Medical Education (ACGME) requirements.[24]

Curriculum Development

Importantly, widespread adoption and uptake of standardized patients in summative and formative CSAs and OSCEs at the undergraduate medical education (UGME) and graduate medical education (GME) level behoove medical educators to adhere to standards of best practice in SP methodology in curriculum development. The Association of Standardized Patient Educators (ASPE) was created in 2001 to promote standards of best practice related to SP methodology in medical education and beyond. From 2013 to 2017, to ensure quality, safety, and accountability, standards were developed by SP educator experts in the field, and five domains were identified to guide the application of this methodology validly and reliably. The ASPE SOBP design was for application in collaboration with the International Nursing Association for Clinical Simulation and Learning (INACSL) Standards of Best Practice: Simulation.[25] The five ASPE SOBP domains include safe work environment; case development; training SPs; program management; and professional development.[9] These domains are germane to innovative curriculum development for both formative training and summative assessment of medical and health professions learners. Domain 1, safe work environment, ensures a safe physical and psychological work environment for learners, faculty, and simulated patients in service to humane and ethical patient care and safety and includes the role of pre-briefing and debriefing both learners and standardized patients.[26][27][28][29] Domain 2, case development, ensures that cases are designed based on sound education theory in instructional curriculum design and include goals and objectives; related performance measures and evaluation instruments; a pilot process and time to revise case elements; and content experts in the health professions domains, simulation and evaluation and measurement.[22][30][31][32] Domain 3, SP training, ensures the following of proven principles and practices such that SPs are trained to accurately and consistently portray a role, provide feedback, and assess learners.[33][21][34][32] Domains 4 & 5 address SP program management and SP educator professional development to drive continuous quality improvement, scholarship, and innovation in the field of simulated patient methodology.[28][35]   

Clinical Significance

The clinical significance of standardized patient assessment of medical learners is via the application of both formative and summative assessments.  Formative assessments, including CSAs and OSCEs, provide developmentally appropriate teaching of clinical skills and competency in the delivery of safe and effective patient-centered care. Summative assessments, such as Step 2 Clinical Skills, are valid and reliable high stakes exams.[36][37] Such high stakes opportunities allow learners to demonstrate clinical competence necessary for matriculation, promotion, and advancement within the medical education curriculum and from UGME to GME. Indeed, the clinical significance of standardized patients assessment of clinical competency appears to apply broadly to healthcare professionals, including nursing, pharmacy, physiotherapy, occupational therapy, and dentistry.[38][39]

Enhancing Healthcare Team Outcomes

Interprofessional education became a mandated Liaison Committee on Medical Education (LCME) accreditation standard in 2013 and opportunities exist in UGME to apply innovative simulation educational strategies to enhance knowledge, skills, attitudes, and patient outcomes.[40][41] [Level V],[Level II] Notably, simulated patient methodology presents an important educational strategy to teach and provide feedback to medical learners and practitioners regarding team-based interprofessional communication, patient-centered care, and patient safety to enhance health outcomes.[42][43][44][45][46] [Level V]

Details

Author

Holly A. Gerzina

Editor:

Erica Stovsky

Updated:

7/24/2023 9:52:45 PM

Feedback:

Send Us Your Comments

References

[1]

Harden RM, Crosby JR, Davis MH, Friedman M. AMEE Guide No. 14: Outcome-based education: Part 5-From competency to meta-competency: a model for the specification of learning outcomes. Medical teacher. 1999:21(6):546-52. doi: 10.1080/01421599978951. Epub     [PubMed PMID: 21281173]

[2]

Barrows HS. An overview of the uses of standardized patients for teaching and evaluating clinical skills. AAMC. Academic medicine : journal of the Association of American Medical Colleges. 1993 Jun:68(6):443-51; discussion 451-3     [PubMed PMID: 8507309]

Level 3 (low-level) evidence

[3]

Winkel AF, Gillespie C, Uquillas K, Zabar S, Szyld D. Assessment of Developmental Progress Using an Objective Structured Clinical Examination-Simulation Hybrid Examination for Obstetrics and Gynecology Residents. Journal of surgical education. 2016 Mar-Apr:73(2):230-7. doi: 10.1016/j.jsurg.2015.10.004. Epub     [PubMed PMID: 26868313]

[4]

Park YS, Hyderi A, Heine N, May W, Nevins A, Lee M, Bordage G, Yudkowsky R. Validity Evidence and Scoring Guidelines for Standardized Patient Encounters and Patient Notes From a Multisite Study of Clinical Performance Examinations in Seven Medical Schools. Academic medicine : journal of the Association of American Medical Colleges. 2017 Nov:92(11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 56th Annual Research in Medical Education Sessions):S12-S20. doi: 10.1097/ACM.0000000000001918. Epub     [PubMed PMID: 29065018]

[5]

Williams BW, Byrne PD, Welindt D, Williams MV. Miller's Pyramid and Core Competency Assessment: A Study in Relationship Construct Validity. The Journal of continuing education in the health professions. 2016 Fall:36(4):295-299. doi: 10.1097/CEH.0000000000000117. Epub     [PubMed PMID: 28350312]

[6]

Boet S, Bould MD, Fung L, Qosa H, Perrier L, Tavares W, Reeves S, Tricco AC. Transfer of learning and patient outcome in simulated crisis resource management: a systematic review. Canadian journal of anaesthesia = Journal canadien d'anesthesie. 2014 Jun:61(6):571-82. doi: 10.1007/s12630-014-0143-8. Epub 2014 Mar 25     [PubMed PMID: 24664414]

Level 1 (high-level) evidence

[7]

Dearani JA, Gold M, Leibovich BC, Ericsson KA, Khabbaz KR, Foley TA, Julsrud PR, Matsumoto JM, Daly RC. The role of imaging, deliberate practice, structure, and improvisation in approaching surgical perfection. The Journal of thoracic and cardiovascular surgery. 2017 Oct:154(4):1329-1336. doi: 10.1016/j.jtcvs.2017.04.045. Epub 2017 Apr 27     [PubMed PMID: 28554678]

[8]

Barsuk JH, McGaghie WC, Cohen ER, O'Leary KJ, Wayne DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Critical care medicine. 2009 Oct:37(10):2697-701     [PubMed PMID: 19885989]

[9]

Lewis KL, Bohnert CA, Gammon WL, Hölzer H, Lyman L, Smith C, Thompson TM, Wallace A, Gliva-McConvey G. The Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP). Advances in simulation (London, England). 2017:2():10. doi: 10.1186/s41077-017-0043-4. Epub 2017 Jun 27     [PubMed PMID: 29450011]

Level 3 (low-level) evidence

[10]

Vu NV, Barrows HS, Marcy ML, Verhulst SJ, Colliver JA, Travis T. Six years of comprehensive, clinical, performance-based assessment using standardized patients at the Southern Illinois University School of Medicine. Academic medicine : journal of the Association of American Medical Colleges. 1992 Jan:67(1):42-50     [PubMed PMID: 1729994]

[11]

Stillman PL, Ruggill JS, Rutala PJ, Sabers DL. Patient instructors as teachers and evaluators. Journal of medical education. 1980 Mar:55(3):186-93     [PubMed PMID: 7359543]

[12]

Boulet JR, De Champlain AF, McKinley DW. Setting defensible performance standards on OSCEs and standardized patient examinations. Medical teacher. 2003 May:25(3):245-9     [PubMed PMID: 12881044]

[13]

Adamo G. Simulated and standardized patients in OSCEs: achievements and challenges 1992-2003. Medical teacher. 2003 May:25(3):262-70     [PubMed PMID: 12881047]

[14]

Gaba DM. The future vision of simulation in health care. Quality & safety in health care. 2004 Oct:13 Suppl 1(Suppl 1):i2-10     [PubMed PMID: 15465951]

Level 2 (mid-level) evidence

[15]

Hamann C, Volkan K, Fishman MB, Silvestri RC, Simon SR, Fletcher SW. How well do second-year students learn physical diagnosis? Observational study of an Objective Structured Clinical Examination (OSCE). BMC medical education. 2002:2():1     [PubMed PMID: 11888484]

Level 2 (mid-level) evidence

[16]

Zayyan M. Objective structured clinical examination: the assessment of choice. Oman medical journal. 2011 Jul:26(4):219-22. doi: 10.5001/omj.2011.55. Epub     [PubMed PMID: 22043423]

[17]

Tsai TC. Using children as standardised patients for assessing clinical competence in paediatrics. Archives of disease in childhood. 2004 Dec:89(12):1117-20     [PubMed PMID: 15557044]

[18]

Harden RM, Stevenson M, Downie WW, Wilson GM. Assessment of clinical competence using objective structured examination. British medical journal. 1975 Feb 22:1(5955):447-51     [PubMed PMID: 1115966]

[19]

Chisnall B, Vince T, Hall S, Tribe R. Evaluation of outcomes of a formative objective structured clinical examination for second-year UK medical students. International journal of medical education. 2015 Jun 21:6():76-83. doi: 10.5116/ijme.5572.a534. Epub 2015 Jun 21     [PubMed PMID: 26094249]

[20]

Whelan GP, Boulet JR, McKinley DW, Norcini JJ, van Zanten M, Hambleton RK, Burdick WP, Peitzman SJ. Scoring standardized patient examinations: lessons learned from the development and administration of the ECFMG Clinical Skills Assessment (CSA). Medical teacher. 2005 May:27(3):200-6     [PubMed PMID: 16011942]

[21]

Hoppe RB, King AM, Mazor KM, Furman GE, Wick-Garcia P, Corcoran-Ponisciak H, Katsufrakis PJ. Enhancement of the assessment of physician-patient communication skills in the United States Medical Licensing Examination. Academic medicine : journal of the Association of American Medical Colleges. 2013 Nov:88(11):1670-5. doi: 10.1097/ACM.0b013e3182a7f75a. Epub     [PubMed PMID: 24072122]

[22]

Colliver JA, Williams RG. Technical issues: test application. AAMC. Academic medicine : journal of the Association of American Medical Colleges. 1993 Jun:68(6):454-60; discussion 461-3     [PubMed PMID: 8507310]

[23]

CarlLee S, Rowat J, Suneja M. Assessing Entrustable Professional Activities Using an Orientation OSCE: Identifying the Gaps. Journal of graduate medical education. 2019 Apr:11(2):214-220. doi: 10.4300/JGME-D-18-00601.2. Epub     [PubMed PMID: 31024656]

[24]

Vora S, Lineberry M, Dobiesz VA. Standardized Patients to Assess Resident Interpersonal Communication Skills and Professional Values Milestones. The western journal of emergency medicine. 2018 Nov:19(6):1019-1023. doi: 10.5811/westjem.2018.8.37204. Epub 2018 Oct 18     [PubMed PMID: 30429936]

[25]

Sittner BJ, Aebersold ML, Paige JB, Graham LL, Schram AP, Decker SI, Lioce L. INACSL Standards of Best Practice for Simulation: Past, Present, and Future. Nursing education perspectives. 2015 Sep-Oct:36(5):294-8     [PubMed PMID: 26521497]

Level 3 (low-level) evidence

[26]

Raemer D, Hannenberg A, Mullen A. Simulation Safety First: An Imperative. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2018 Dec:13(6):373-375. doi: 10.1097/SIH.0000000000000341. Epub     [PubMed PMID: 30499869]

[27]

Gamble A, Bearman M, Nestel D. A systematic review: Children & Adolescents as simulated patients in health professional education. Advances in simulation (London, England). 2016:1():1. doi: 10.1186/s41077-015-0003-9. Epub 2016 Jan 11     [PubMed PMID: 29449970]

Level 1 (high-level) evidence

[28]

Pritchard SA, Blackstock FC, Keating JL, Nestel D. The pillars of well-constructed simulated patient programs: A qualitative study with experienced educators. Medical teacher. 2017 Nov:39(11):1159-1167. doi: 10.1080/0142159X.2017.1369015. Epub 2017 Aug 28     [PubMed PMID: 28845722]

Level 2 (mid-level) evidence

[29]

Gerzina HA, Porfeli EJ. Mindfulness as a predictor of positive reappraisal and burnout in standardized patients. Teaching and learning in medicine. 2012:24(4):309-14. doi: 10.1080/10401334.2012.715255. Epub     [PubMed PMID: 23035997]

[30]

Anderson JM, Aylor ME, Leonard DT. Instructional design dogma: creating planned learning experiences in simulation. Journal of critical care. 2008 Dec:23(4):595-602. doi: 10.1016/j.jcrc.2008.03.003. Epub 2008 May 14     [PubMed PMID: 19056028]

[31]

Nestel D, Tierney T. Role-play for medical students learning about communication: guidelines for maximising benefits. BMC medical education. 2007 Mar 2:7():3     [PubMed PMID: 17335561]

[32]

Serwint JR. The use of standardized patients in pediatric residency training in palliative care: anatomy of a standardized patient case scenario. Journal of palliative medicine. 2002 Feb:5(1):146-53     [PubMed PMID: 11839238]

Level 3 (low-level) evidence

[33]

Furman GE. The role of standardized patient and trainer training in quality assurance for a high-stakes clinical skills examination. The Kaohsiung journal of medical sciences. 2008 Dec:24(12):651-5. doi: 10.1016/S1607-551X(09)70031-2. Epub     [PubMed PMID: 19251561]

Level 2 (mid-level) evidence

[34]

Kassab ES, King D, Hull LM, Arora S, Sevdalis N, Kneebone RL, Nestel D. Actor training for surgical team simulations. Medical teacher. 2010:32(3):256-8. doi: 10.3109/01421590903514648. Epub     [PubMed PMID: 20218842]

[35]

Bickel J. The role of professional societies in career development in academic medicine. Academic psychiatry : the journal of the American Association of Directors of Psychiatric Residency Training and the Association for Academic Psychiatry. 2007 Mar-Apr:31(2):91-4     [PubMed PMID: 17344436]

[36]

Park YS, Lineberry M, Hyderi A, Bordage G, Xing K, Yudkowsky R. Differential Weighting for Subcomponent Measures of Integrated Clinical Encounter Scores Based on the USMLE Step 2 CS Examination: Effects on Composite Score Reliability and Pass-Fail Decisions. Academic medicine : journal of the Association of American Medical Colleges. 2016 Nov:91(11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 55th Annual Research in Medical Education Sessions):S24-S30     [PubMed PMID: 27779506]

[37]

Park YS, Hyderi A, Bordage G, Xing K, Yudkowsky R. Inter-rater reliability and generalizability of patient note scores using a scoring rubric based on the USMLE Step-2 CS format. Advances in health sciences education : theory and practice. 2016 Oct:21(4):761-73. doi: 10.1007/s10459-015-9664-3. Epub 2016 Jan 12     [PubMed PMID: 26757931]

Level 3 (low-level) evidence

[38]

Williams B, Song JJY. Are simulated patients effective in facilitating development of clinical competence for healthcare students? A scoping review. Advances in simulation (London, England). 2016:1():6. doi: 10.1186/s41077-016-0006-1. Epub 2016 Feb 16     [PubMed PMID: 29449975]

Level 2 (mid-level) evidence

[39]

Gibbons SW, Adamo G, Padden D, Ricciardi R, Graziano M, Levine E, Hawkins R. Clinical evaluation in advanced practice nursing education: using standardized patients in Health Assessment. The Journal of nursing education. 2002 May:41(5):215-21     [PubMed PMID: 12025865]

[40]

Alexandraki I, Hernandez CA, Torre DM, Chretien KC. Interprofessional Education in the Internal Medicine Clerkship Post-LCME Standard Issuance: Results of a National Survey. Journal of general internal medicine. 2017 Aug:32(8):871-876. doi: 10.1007/s11606-017-4004-3. Epub 2017 Mar 10     [PubMed PMID: 28284014]

Level 3 (low-level) evidence

[41]

Lempicki KA, Holland CS. Web-based versus face-to-face interprofessional team encounters with standardized patients. Currents in pharmacy teaching & learning. 2018 Mar:10(3):344-351. doi: 10.1016/j.cptl.2017.11.014. Epub 2017 Dec 18     [PubMed PMID: 29764639]

[42]

Lee WJ, Clark L, Wortmann K, Taylor LA, Pock AR. Interprofessional Healthcare Student Training in the Care of Sexual Assault Patients Utilizing Standardized Patient Methodology. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2019 Feb:14(1):10-17. doi: 10.1097/SIH.0000000000000331. Epub     [PubMed PMID: 30407955]

[43]

Bradway C, Cotter VT, Darrah NJ, Gibbs VD, Hadley D, Kim EH, LaMarra D, Packel L, Westcott AM. An Interprofessional Education Simulation Workshop: Health Professions Learning Palliative Care Communication. The Journal of nursing education. 2018 Aug 1:57(8):493-497. doi: 10.3928/01484834-20180720-08. Epub     [PubMed PMID: 30070675]

[44]

Lu WH, Goolsarran N, Hamo CE, Frawley SM, Rowe C, Lane S. Teaching Patient Safety Using an Interprofessional Team-Based Learning Simulation Model in Residency Training. MedEdPORTAL : the journal of teaching and learning resources. 2016 Jun 3:12():10409. doi: 10.15766/mep_2374-8265.10409. Epub 2016 Jun 3     [PubMed PMID: 31008189]

[45]

Goolsarran N, Hamo CE, Lane S, Frawley S, Lu WH. Effectiveness of an interprofessional patient safety team-based learning simulation experience on healthcare professional trainees. BMC medical education. 2018 Aug 8:18(1):192. doi: 10.1186/s12909-018-1301-4. Epub 2018 Aug 8     [PubMed PMID: 30089502]

[46]

Liaw SY, Zhou WT, Lau TC, Siau C, Chan SW. An interprofessional communication training using simulation to enhance safe care for a deteriorating patient. Nurse education today. 2014 Feb:34(2):259-64. doi: 10.1016/j.nedt.2013.02.019. Epub 2013 Mar 19     [PubMed PMID: 23518067]