Set Up and Execution of an Effective Standardized Patient Program in Medical Simulation

Hillary Moss; Jonathan Weil; Pinaki Mukherji

Set Up and Execution of an Effective Standardized Patient Program in Medical Simulation

Free Review Questions

Introduction

Standardized or simulated patients (SPs) have become an essential aspect of medical education. They date back to the 1960s when Dr. Howard Barrows of the University of Southern California first utilized them to simulate multiple sclerosis patients and trained them to evaluate learners as well.[1] Dr. Paula Stillman of the University of Arizona is identified as another early user of SPs, training actors in the 1970s to portray mothers of child patients to assist students with acquiring appropriate histories. She is also cited as one of the first to use ‘standardized actors’ to teach physical exam and direct students on how to perform aspects of the physical correctly.[2] Building upon these successes, SPs have slowly been embraced in the medical education community, especially at the undergraduate level, where they are utilized for formative assessment in the form of the Objective Structured Clinical Exam (OSCE) and the summative evaluation in the form of the USMLE Step 2 CS.[3][4] SPs have also been utilized in graduate medical education in more formative roles and have been applied to other medical disciplines, including nursing, physical therapy, and respiratory therapy.[5] The flexibility of SPs in their ability to be utilized in multiple disciplines and multiple education levels and its superiority in the development of learner interpersonal skills all serve to emphasize the importance of having a strong standardized patient program.[6]

Curriculum Development

The methodology of SP training has been described since 1964, but only recently has there been interest in studying and standardizing the methodology of training.[7] Outside of case content, SPs require training in the specific areas of role portrayal, feedback, and use of assessment instruments. Ongoing SP programs are encouraged to incorporate trainees' reflections on the training process. The accuracy and validity of standardized patients ought to be analyzed before using them to make assessments on an individual or systems level. While SPs are well established as quality raters in the sense of interrater reliability, findings may not correlate with real patient experience. Some report that SPs tend to give physicians lower ratings compared to actual patients. This variation from true clinical experience does not undermine the utility of SPs. Rather, it indicates the need for internal validation before deployment, perhaps with a focus not on SP ratings but on whether or not these assessments lead to demonstrable changes between pre- and post-intervention clinical measures.

Role portrayal seeks to ensure that performance between SPs is both consistent and accurate. Clinician involvement is required to review case content, but SPs are also called upon to display both physical and emotional vulnerability. Ensuring a safe psychological environment is a crucial component of training in accurate role portrayal.[8]

Educators often receive training in providing feedback, but SPs fill a unique role. Being able to give feedback on emotional connection, trust, and communication, SPs offer a rare perspective to trainees before their clinical exposure. SP educators may provide specific models of feedback used by institutions or use scripting of oral or written feedback.

SPs may offer formative or summative assessments and are expected to use assessment tools, rubrics, and narrative feedback. Trainees and SPs should have transparency regarding the assessment tools used and should be encouraged to reflect on the process to improve the training for both groups.[9]

Safety and Ethics of Standardized Patient Deployment

The safety and consent of all participants is the first concern in any standardized patient deployment. SPs should be counseled on the expectations of their role and allowed to ask questions and explore the problem space on their own terms. They must have the opportunity to refuse participation or disengage at any time; this is essential even for casual SPs, such as students and residents in "role-playing" scenarios.[10] The rigor of such briefings varies with the potential for psychological or physical compromise of the actors during the simulation. For instance, a simple intervention for conveying bad news may carry little or no risk. By contrast, portraying a patient with agitated delirium requires that both SP and clinical participants have a clear understanding of ground rules, limits on acceptable actions, and key phrases for terminating the scenario if needed.

Additional considerations abound for children, who are uniquely appealing when they can portray rare pathologies but are uniquely vulnerable. Some have suggested that very young children should not be used at all, though the exact age cutoff is debatable.[11][12][13] Projects involving children of elementary school age have reported substantial parent and child satisfaction.[14] At a minimum, parental consent is required in all cases. Parents should be present in the room or at least able to observe the encounter from a nearby area for younger children, though this requirement can be waived for adolescents. The right to refuse participation should be respected in any child old enough to understand the event and express their disinterest. Facilitators must attend to the faster onset of fatigue and stress experienced by children, and limit the duration and number of interactions accordingly.[11][12][13][15][16]

Clinical Significance

SPs have become an integral aspect of healthcare education, and in fact, are a required part of licensure in the United States. Step 2 CS, half of the second licensing exam, is a summative assessment, with medical students performing histories and physicals on multiple SPs. These SPs grade the learners on their interpersonal skills and appropriate questioning and physical examination.[4] This test is meant to confirm a minimum standard that medical students must meet to be able to continue in their education. While the SP experience in USMLE licensing is a visible sign of their significance, their utilization in OSCEs is arguably more important.

Studies have demonstrated that OSCEs are more reliable in regards to assessing learners than traditional multiple-choice question tests.[17] OSCE success is ultimately determined by reliable and consistent standardized patients who are trained to be able to give the same performance to each student and be able to improvise where appropriate to provide learners an even playing field. Appropriately trained SPs are known to be excellent raters/graders of students, with good inter-rater reliability, and often are better than faculty who are explicitly trained to rate students.[18] Studies have also demonstrated that learners acquire more developed and sophisticated emotional and communication skills when learning through SP encounters.[6] Given the multitude of evidence that supports the superiority of SPs in medical education, facilities must develop excellent SP training programs.

Enhancing Healthcare Team Outcomes

Patient Communication

Certain communication domains are either too logistically challenging or emotionally fraught to execute in situ. Most prominently, practicing the disclosure of medical errors and bad news is uniquely accessible to the SP approach. To our knowledge, however, it has not been described outside of a controlled environment, reflecting the complicated psychological challenges associated with “surprise” deployments.[10][19][20] Nonetheless, it is one of the better-studied and essential opportunities afforded by SPs. The tradition for this practice in undergraduate medical education dates back decades and is addressed elsewhere, and instead now the focus on its use amongst practicing clinicians.[21][22][23][24] As an assessment tool, SPs can identify deficits in how physicians acknowledge and take responsibility for medical errors[25], while simultaneously encouraging accurate self-assessment of disclosure competence.[26] Relatively simple sessions facilitate rapid improvement in both self-assessed skills and independent ratings.[27][28][29][30] The same benefits are found for the communication of bad news through a multitude of interventions involving SPs.[23][31][32][33]

A common theme that arises from exploring these reports is the multimodal integration of SPs with traditional didactic methods rather than the strict replacement of the former for the latter. For instance, one should strongly consider a classroom teaching session on communication best practices before having students engage with simulations. Asynchronous study before the scheduled session is also expected to have comparable benefits. Within the SP segment itself, feedback can be obtained from the SP themself, an independent observer, or participant self-reflection, often with similar accuracy objectively while providing valuable variance in subjective perspectives.

Clinical

Perhaps the most ambitious but irreplaceable application of standardized patients is the announced simulated patient, where an SP arrives in a clinic under the full pretense of a routine visit. In such cases, there is no pre-brief or other announcement that the encounter is an educational or assessment tool. The benefit is that the entire medical staff is no longer subject to the Hawthorne effect -- the changes in behavior thought to arise from knowing one is being observed.[34] It has been suggested that such medical performance during overt assessments differs substantially from true competence, a term which refers to actual clinical practice.[35][36] In situ, unannounced SPs are perhaps the only means by which a health system can accurately assess the latter on an individual basis.

Incognito SPs (ISPs), as they are sometimes called, have proven remarkably effective for identifying deficits in care across numerous types of providers and expansive geographic ranges. An assessment of the literature yields examples of ISPs in Western nations, Kenya, India, and China, to name a few, consistently demonstrating discrepancies between physician behavior and national guidelines or specialty best-practices, as well as shortcomings in physical exam skills.[37][38][39][40][41] Interestingly, these studies, at times, reinforce the concerns raised by Rethans, where the same cohort of clinicians demonstrates a good knowledge of traditional assessments, but accuracy drops precipitously in an incognito clinical setting.[38] Critically, an SP can assess every element of a practice environment. There is no reason to limit their utility to just a single provider type. Zabar et al. describe how their ISPs simultaneously identified issues with hand hygiene and screening questions performed by medical assistants, in addition to physician shortcomings.[42]

Special logistical considerations apply to the use of unannounced SPs. Health care personnel should be informed ahead of time that such patients will be deployed. This step is as much to ensure buy-in with the educational goals as it is about providing informed consent. People are unlikely to respond to feedback well if completely unaware of the project.[35][43][44][45] Despite this forewarning, however, SP detection rates are generally very low.[46] A washout period between the time of announcing the program and introducing simulated patients may help as well. That said, it may be useful for someone in the backend of the clinical practice, such as an office manager, to remain apprised of the visits, which speaks mainly to the challenges of fabricating an identity in a digital era, where frontend personnel will expect automatic confirmation of insurance information and test results. It is especially important to plan for the former, a ubiquitous feature in any setting. Workarounds include a patient presenting as self-pay or coordinating with an office manager to ensure the patient’s “insurance” is accepted. On the patient end, a detailed backstory is essential to ensuring success. The SP should have a manufactured address consistent with the geography of the clinic, a real mailing address if such correspondence is expected, and a passable photo ID for their invented identity. The actor should also consider the details of their “life,” such as information about their neighborhood, family, and hobbies, to make small talk appear natural.

The adoption of a rigorous SP program requires substantial resources, both upfront and on an ongoing basis. This likely explains why they are prevalent in the literature in small studies, but it is exceedingly difficult to find descriptions of sustained projects within a health system. This fact may relate to the absence of evidence demonstrating long-term benefits in outcome-oriented or financial metrics. However, this calculus may change as reimbursement increasingly ties to guideline adherence and patient satisfaction, not to mention the fact that patient satisfaction and doctor-patient communication are the key elements in preventing lawsuits. One must wonder how much more evidence is necessary to warrant programs that are known to identify shortcomings in all these domains, especially as large health systems become the norm in the medical landscape and have both the resources and institutional motivation for ongoing quality checks.

Details

References

[1]

May W. Training standardized patients for a high-stakes Clinical Performance Examination in the California Consortium for the Assessment of Clinical Competence. The Kaohsiung journal of medical sciences. 2008 Dec:24(12):640-5. doi: 10.1016/S1607-551X(09)70029-4. Epub [PubMed PMID: 19251559]

[2]

Wallace P. Following the threads of an innovation: the history of standardized patients in medical education. Caduceus (Springfield, Ill.). 1997 Autumn:13(2):5-28 [PubMed PMID: 9509634]

[3]

Adamo G. Simulated and standardized patients in OSCEs: achievements and challenges 1992-2003. Medical teacher. 2003 May:25(3):262-70 [PubMed PMID: 12881047]

[4]

Cuddy MM, Winward ML, Johnston MM, Lipner RS, Clauser BE. Evaluating Validity Evidence for USMLE Step 2 Clinical Skills Data Gathering and Data Interpretation Scores: Does Performance Predict History-Taking and Physical Examination Ratings for First-Year Internal Medicine Residents? Academic medicine : journal of the Association of American Medical Colleges. 2016 Jan:91(1):133-9. doi: 10.1097/ACM.0000000000000908. Epub [PubMed PMID: 26397703]

[5]

Swift MC, Stosberg T. Interprofessional Simulation and Education: Physical Therapy, Nursing, and Theatre Faculty Work Together to Develop a Standardized Patient Program. Nursing education perspectives. 2015 Nov-Dec:36(6):412-3 [PubMed PMID: 26753308]

Level 3 (low-level) evidence

[6]

Peabody JW, Luck J, Glassman P, Dresselhaus TR, Lee M. Comparison of vignettes, standardized patients, and chart abstraction: a prospective validation study of 3 methods for measuring quality. JAMA. 2000 Apr 5:283(13):1715-22 [PubMed PMID: 10755498]

Level 2 (mid-level) evidence

[7]

Pritchard SA, Blackstock FC, Keating JL, Nestel D. The pillars of well-constructed simulated patient programs: A qualitative study with experienced educators. Medical teacher. 2017 Nov:39(11):1159-1167. doi: 10.1080/0142159X.2017.1369015. Epub 2017 Aug 28 [PubMed PMID: 28845722]

Level 2 (mid-level) evidence

[8]

Pritchard SA, Denning T, Keating JL, Blackstock FC, Nestel D. "It's Not an Acting Job … Don't Underestimate What a Simulated Patient Does": A Qualitative Study Exploring the Perspectives of Simulated Patients in Health Professions Education. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2020 Feb:15(1):21-29. doi: 10.1097/SIH.0000000000000400. Epub [PubMed PMID: 31743313]

[9]

Miller C, Toy S, Schwengel D, Isaac G, Schiavi A. Development of a Simulated Objective Structured Clinical Exam for the APPLIED Certification Exam in Anesthesiology: A Two-Year Experience Informed by Feedback from Exam Candidates. The journal of education in perioperative medicine : JEPM. 2019 Oct-Dec:21(4):E633 [PubMed PMID: 32123698]

[10]

Lewis KL, Bohnert CA, Gammon WL, Hölzer H, Lyman L, Smith C, Thompson TM, Wallace A, Gliva-McConvey G. The Association of Standardized Patient Educators (ASPE) Standards of Best Practice (SOBP). Advances in simulation (London, England). 2017:2():10. doi: 10.1186/s41077-017-0043-4. Epub 2017 Jun 27 [PubMed PMID: 29450011]

Level 3 (low-level) evidence

[11]

Khoo EJ, Schremmer RD, Diekema DS, Lantos JD. Ethical Concerns When Minors Act as Standardized Patients. Pediatrics. 2017 Mar:139(3):. pii: e20162795. doi: 10.1542/peds.2016-2795. Epub 2017 Feb 7 [PubMed PMID: 28174202]

[12]

Hilliard R, Fernandez C, Tsai E. Ethical participation of children and youth in medical education. Paediatrics & child health. 2011 Apr:16(4):223-32 [PubMed PMID: 22468127]

[13]

Tsai TC. Using children as standardised patients for assessing clinical competence in paediatrics. Archives of disease in childhood. 2004 Dec:89(12):1117-20 [PubMed PMID: 15557044]

[14]

Darling JC, Bardgett RJ. Primary school children in a large-scale OSCE: recipe for disaster or formula for success? Medical teacher. 2013 Oct:35(10):858-61. doi: 10.3109/0142159X.2013.806790. Epub 2013 Jul 12 [PubMed PMID: 23848302]

[15]

Hanson M, Tiberius R, Hodges B, MacKay S, McNaughton N, Dickens S, Regehr G. Adolescent standardized patients: method of selection and assessment of benefits and risks. Teaching and learning in medicine. 2002 Spring:14(2):104-13 [PubMed PMID: 12058545]

[16]

Gamble A, Bearman M, Nestel D. A systematic review: Children & Adolescents as simulated patients in health professional education. Advances in simulation (London, England). 2016:1():1. doi: 10.1186/s41077-015-0003-9. Epub 2016 Jan 11 [PubMed PMID: 29449970]

Level 1 (high-level) evidence

[17]

Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002 Jan 9:287(2):226-35 [PubMed PMID: 11779266]

[18]

Dickter DN, Stielstra S, Lineberry M. Interrater Reliability of Standardized Actors Versus Nonactors in a Simulation Based Assessment of Interprofessional Collaboration. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2015 Aug:10(4):249-55. doi: 10.1097/SIH.0000000000000094. Epub [PubMed PMID: 26098494]

[19]

Gaba DM. Simulations that are challenging to the psyche of participants: how much should we worry and about what? Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2013 Feb:8(1):4-7. doi: 10.1097/SIH.0b013e3182845a6f. Epub [PubMed PMID: 23380693]

[20]

Rudolph JW, Raemer DB, Simon R. Establishing a safe container for learning in simulation: the role of the presimulation briefing. Simulation in healthcare : journal of the Society for Simulation in Healthcare. 2014 Dec:9(6):339-49. doi: 10.1097/SIH.0000000000000047. Epub [PubMed PMID: 25188485]

[21]

Kiluk JV, Dessureault S, Quinn G. Teaching medical students how to break bad news with standardized patients. Journal of cancer education : the official journal of the American Association for Cancer Education. 2012 Jun:27(2):277-80. doi: 10.1007/s13187-012-0312-9. Epub [PubMed PMID: 22314793]

[22]

Garg A, Buckman R, Kason Y. Teaching medical students how to break bad news. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 1997 Apr 15:156(8):1159-64 [PubMed PMID: 9141988]

[23]

Rosenbaum ME, Kreiter C. Teaching delivery of bad news using experiential sessions with standardized patients. Teaching and learning in medicine. 2002 Summer:14(3):144-9 [PubMed PMID: 12189633]

[24]

Rosenbaum ME, Ferguson KJ, Lobas JG. Teaching medical students and residents skills for delivering bad news: a review of strategies. Academic medicine : journal of the Association of American Medical Colleges. 2004 Feb:79(2):107-17 [PubMed PMID: 14744709]

[25]

Chan DK, Gallagher TH, Reznick R, Levinson W. How surgeons disclose medical errors to patients: a study using standardized patients. Surgery. 2005 Nov:138(5):851-8 [PubMed PMID: 16291385]

[26]

Stroud L, McIlroy J, Levinson W. Skills of internal medicine residents in disclosing medical errors: a study using standardized patients. Academic medicine : journal of the Association of American Medical Colleges. 2009 Dec:84(12):1803-8. doi: 10.1097/ACM.0b013e3181bf9fef. Epub [PubMed PMID: 19940591]

[27]

Sukalich S, Elliott JO, Ruffner G. Teaching medical error disclosure to residents using patient-centered simulation training. Academic medicine : journal of the Association of American Medical Colleges. 2014 Jan:89(1):136-43. doi: 10.1097/ACM.0000000000000046. Epub [PubMed PMID: 24280843]

[28]

Bonnema RA, Gosman GG, Arnold RM. Teaching error disclosure to residents: a curricular innovation and pilot study. Journal of graduate medical education. 2009 Sep:1(1):114-8. doi: 10.4300/01.01.0019. Epub [PubMed PMID: 21975717]

Level 3 (low-level) evidence

[29]

Raemer DB, Locke S, Walzer TB, Gardner R, Baer L, Simon R. Rapid Learning of Adverse Medical Event Disclosure and Apology. Journal of patient safety. 2016 Sep:12(3):140-7. doi: 10.1097/PTS.0000000000000080. Epub [PubMed PMID: 24583959]

[30]

Spalding CN, Rudinsky SL. Preparing Emergency Medicine Residents to Disclose Medical Error Using Standardized Patients. The western journal of emergency medicine. 2018 Jan:19(1):211-215. doi: 10.5811/westjem.2017.11.35309. Epub 2017 Dec 14 [PubMed PMID: 29383083]

[31]

Amiel GE, Ungar L, Alperin M, Baharier Z, Cohen R, Reis S. Ability of primary care physician's to break bad news: a performance based assessment of an educational intervention. Patient education and counseling. 2006 Jan:60(1):10-5 [PubMed PMID: 16122897]

[32]

Greenberg LW, Ochsenschlager D, O'Donnell R, Mastruserio J, Cohen GJ. Communicating bad news: a pediatric department's evaluation of a simulated intervention. Pediatrics. 1999 Jun:103(6 Pt 1):1210-7 [PubMed PMID: 10353931]

[33]

Bowyer MW, Hanson JL, Pimentel EA, Flanagan AK, Rawn LM, Rizzo AG, Ritter EM, Lopreiato JO. Teaching breaking bad news using mixed reality simulation. The Journal of surgical research. 2010 Mar:159(1):462-7. doi: 10.1016/j.jss.2009.04.032. Epub 2009 May 20 [PubMed PMID: 19665731]

[34]

McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. Journal of clinical epidemiology. 2014 Mar:67(3):267-77. doi: 10.1016/j.jclinepi.2013.08.015. Epub 2013 Nov 22 [PubMed PMID: 24275499]

Level 1 (high-level) evidence

[35]

Rethans JJ, Gorter S, Bokken L, Morrison L. Unannounced standardised patients in real practice: a systematic literature review. Medical education. 2007 Jun:41(6):537-49 [PubMed PMID: 17518833]

Level 1 (high-level) evidence

[36]

Rethans JJ, Norcini JJ, Barón-Maldonado M, Blackmore D, Jolly BC, LaDuca T, Lew S, Page GG, Southgate LH. The relationship between competence and performance: implications for assessing practice performance. Medical education. 2002 Oct:36(10):901-9 [PubMed PMID: 12390456]

[37]

Hutchison B, Woodward CA, Norman GR, Abelson J, Brown JA. Provision of preventive care to unannounced standardized patients. CMAJ : Canadian Medical Association journal = journal de l'Association medicale canadienne. 1998 Jan 27:158(2):185-93 [PubMed PMID: 9469139]

[38]

Das J, Kwan A, Daniels B, Satyanarayana S, Subbaraman R, Bergkvist S, Das RK, Das V, Pai M. Use of standardised patients to assess quality of tuberculosis care: a pilot, cross-sectional study. The Lancet. Infectious diseases. 2015 Nov:15(11):1305-13. doi: 10.1016/S1473-3099(15)00077-8. Epub 2015 Aug 9 [PubMed PMID: 26268690]

Level 2 (mid-level) evidence

[39]

Sylvia S, Shi Y, Xue H, Tian X, Wang H, Liu Q, Medina A, Rozelle S. Survey using incognito standardized patients shows poor quality care in China's rural clinics. Health policy and planning. 2015 Apr:30(3):322-33. doi: 10.1093/heapol/czu014. Epub 2014 Mar 20 [PubMed PMID: 24653216]

Level 2 (mid-level) evidence

[40]

Daniels B, Dolinger A, Bedoya G, Rogo K, Goicoechea A, Coarasa J, Wafula F, Mwaura N, Kimeu R, Das J. Use of standardised patients to assess quality of healthcare in Nairobi, Kenya: a pilot, cross-sectional study with international comparisons. BMJ global health. 2017:2(2):e000333. doi: 10.1136/bmjgh-2017-000333. Epub 2017 Jun 10 [PubMed PMID: 29225937]

Level 2 (mid-level) evidence

[41]

Borrell-Carrió F, Poveda BF, Seco EM, Castillejo JA, González MP, Rodríguez EP. Family physicians' ability to detect a physical sign (hepatomegaly) from an unannounced standardized patient (incognito SP). The European journal of general practice. 2011 Jun:17(2):95-102. doi: 10.3109/13814788.2010.549223. Epub 2011 Jan 12 [PubMed PMID: 21226545]

[42]

Zabar S, Hanley K, Stevens D, Murphy J, Burgess A, Kalet A, Gillespie C. Unannounced standardized patients: a promising method of assessing patient-centered care in your health care system. BMC health services research. 2014 Apr 5:14():157. doi: 10.1186/1472-6963-14-157. Epub 2014 Apr 5 [PubMed PMID: 24708683]

[43]

Rethans JJ, Drop R, Sturmans F, van der Vleuten C. A method for introducing standardized (simulated) patients into general practice consultations. The British journal of general practice : the journal of the Royal College of General Practitioners. 1991 Mar:41(344):94-6 [PubMed PMID: 2031766]

[44]

Siminoff LA, Rogers HL, Waller AC, Harris-Haywood S, Esptein RM, Carrio FB, Gliva-McConvey G, Longo DR. The advantages and challenges of unannounced standardized patient methodology to assess healthcare communication. Patient education and counseling. 2011 Mar:82(3):318-24. doi: 10.1016/j.pec.2011.01.021. Epub [PubMed PMID: 21316182]

[45]

Rosenhan DL. On being sane in insane places. Science (New York, N.Y.). 1973 Jan 19:179(4070):250-8 [PubMed PMID: 4683124]

[46]

Franz CE, Epstein R, Miller KN, Brown A, Song J, Feldman M, Franks P, Kelly-Reif S, Kravitz RL. Caught in the act? Prevalence, predictors, and consequences of physician detection of unannounced standardized patients. Health services research. 2006 Dec:41(6):2290-302 [PubMed PMID: 17116121]