Volume 21, Issue 5 by Western Journal of Emergency Medicine

Western Journal of Emergency Medicine

Volume 21, Number 5, September 2020

West

Open Access at WestJEM.com

ISSN 1936-900X

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health Indexed in MEDLINE

Injury Prevention 1036 The Bullets He Carried Hargarten S

VOLUME 21, NUMBER 5, September 2020

Health Policy Analysis 1037 United States Congressional COVID-19 Legislation: Recent Laws and Future Topics Dowling MK, Terry AT, Kirilichin NL, Lee JS, Blanchard JC Technology in Emergency Medicine 1042 Point-of-Care Ultrasound for Intubation Confirmation of COVID-19 Patients Gottlieb M, Alerhand S, Long B Endemic Infections 1046 COVID-19: Implications for Advanced Care Planning and End of Life Care Reja M, Naik J, Parikh P

1048

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2 Karb R, Samuels E, Vanjani R, Trimbur C, Napoli A

1054

Development and Usability Testing of a Web-based COVID-19 Self-triage Platform Schrager JD, Schuler K, Isakov AP, Wright DW, Yaffee AQ, Jacobson KL, Parker RM, Goolsby C

1059

The Impact of COVID-19 on Healthcare Worker Wellness: A Scoping Review Shreffler J, Petrey J, Huecker M

1067 1076

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation Kerpel A, Apter S, Nissan N, Houri-Levi E, Klug M, Amit S, Konen E, Marom EM MIST Bundle (Modified Intubating Sequence for Transmissibility) for Infectious Diseases with Aerosol Hazard Balakrishnan JM, Sanjan A, Wilson W, Sujir SN, Bha Rt, Vandana KE

PAGES 1036-1287

Contents continued on page iii

A Peer-Reviewed, International Professional Journal

ACOEP stands with all emergency physicians and providers on the front line. We thank you for your tireless work and effort.

www.acoep.org

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded Andrew W. Phillips, MD, Associate Editor University of North Carolina-Chapel Hill, North Carolina Chadd Kraus, DO, DrPH, MPH, Associate Editor Geisinger Health System- Danville, Pennsylvania Edward Michelson, MD, Associate Editor Texas Tech University- El Paso, Texas Gavin Budhram, MD, Associate Editor Tufts University- Medford, Massachusetts James R. Langabeer II, MBA, EMT, PhD, Associate Editor University of Texas Medical School-Austin, Texas

Resident Editors AAEM/RSA

John J. Campo, MD Resident/Fellow Section Editor Harbor-University of California, Los Angeles Medical Center Tehreem Rehman, MD Resident/Fellow Section Editor Advocate Christ Medical Center

ACOEP

Justina Truong, DO Resident Editor Kingman Regional Medical Center

Section Editors Behavioral Emergencies

Erin Dehon, PhD University of Mississippi Medical Center Leslie Zun, MD, MBA Chicago Medical School

Mark I. Langdorf, MD, MHPE, Editor-in-Chief University of California, Irvine School of MedicineIrvine, California

Shadi Lahham, MD, MS, Deputy Editor University of California, Irvine School of MedicineIrvine, California

Margaret Samuels-Kalow, MD, Associate Editor Massachusetts General Hospital- Boston, Massachusetts Michael Gottlieb, MD, Associate Editor Rush Medical Center-Chicago, Illinois

Shahram Lotfipour, MD, MPH, Managing Associate Editor University of California, Irvine School of MedicineIrvine, California

Niels K. Rathlev, MD, Associate Editor Tufts University School of Medicine-Boston, Massachusetts

Susan R. Wilcox, MD, Associate Editor Massachusetts General Hospital- Boston, Massachusetts

Rick A. McPheeters, DO, Associate Editor Kern Medical- Bakersfield, California

Douglas Ander, MD Emory University

Emergency Medicine University of California, Davis

Cristina M. Zeretzke-Bien, MD University of Florida

Jeffrey Druck, MD University of Colorado

Stephen Liang, MD, MPHS Washington University School of Medicine

Public Health

John Burkhardt, MD, MA University of Michigan Medical School

Injury Prevention

Michael Epter, DO Maricopa Medical Center

ED Administration, Quality, Safety David C. Lee, MD Northshore University Hospital

Clinical Practice

Cortlyn W. Brown, MD Carolinas Medical Center

Jacob Manteuffel, MD Henry Ford Hospital

Wirachin Hoonpongsimanont, MD, MSBATS Eisenhower Medical Center

John Ashurst, DO Lehigh Valley Health Network

International Medicine

Tony Zitek, MD Kendall Regional Medical Center Trevor Mills, MD, MPH Northern California VA Health Care

Rolando Valenzuela, MD University of Southern California

Brian J. Yun, MD, MBA, MPH Harvard Medical School

Erik S. Anderson, MD Alameda Health System-Highland Hospital

Shada Rouhani, MD Brigham and Women’s Hospital

William Fernandez, MD, MPH University of Texas Health-San Antonio

Technology in Emergency Medicine

Legal Medicine

Nikhil Goyal, MD Henry Ford Hospital

Greg P. Moore, MD, JD Madigan Army Medical Center

Emergency Cardiac Care

Marc L. Martel, MD Hennepin County Medical Center

Mark Faul, PhD, MA Centers for Disease Control and Prevention

Chris Mills, MD, MPH Santa Clara Valley Medical Center

Gary Johnson, MD Upstate Medical University

Michael Kurz, MD University of Alabama at Birmingham

Phillips Perera, MD Stanford University Medical Center

Statistics and Methodology

Elizabeth Burner, MD, MPH University of Southern California

Semhar Z. Tewelde, MD University of Maryland School of Medicine

Robert L. Rogers, MD University of Kentuky

Shu B. Chan MD, MS Resurrection Medical Center

Emergency Medical Services

Jeremy Hess, MD, MPH University of Washington Medical Center

Trauma

Stormy M. Morales Monks, PhD, MPH Texas Tech Health Science University

David Peak, MD Massachusetts General Hospital/Havard Medical School

Joshua B. Gaither, MD University of Arizona, Tuscon

Dan Mayer, MD American College of Emergency Physicians

Patrick Joseph Maher, MD Icahn School of Medicine at Mount Sinai

David Thompson, MD University of California, San Francisco

Shira A. Schlesinger, MD, MPH Harbor-UCLA Medical Center

Soheil Saadat, MD, MPH, PhD University of California, Irvine

Kenneth S. Whitlow, DO Kaweah Delta Medical Center

Daniel Joseph, MD Yale University

Musculoskeletal

Pierre Borczuk, MD Massachusetts General Hospital/Havard Medical School

Patrick Meloy, MD Emory University

Derek Cooney, MD State University of New York Upstate Medical University, New York

Eric Snoey, MD Alameda County Medical Center

Critical Care

Christopher “Kit” Tainter, MD University of California, San Diego Gabriel Wardi, MD University of California, San Diego Joseph Shiber, MD University of Florida-College of Medicine Matt Prekker MD, MPH Hennepin County Medical Center Todd Slesinger, MD Aventura Hospital and Medical Center

Disaster Medicine

Christopher Kang, MD Madigan Army Medical Center Gentry Wilkerson, MD University of Maryland

Education

Danya Khoujah, MBBS University of Maryland School of Medicine

Juan F. Acosta DO, MS Pacific Northwest University

William Paolo, MD SUNY Upstate

Neurosciences

Toxicology

Emily C. Manchanda, MD, MPH Boston University School of Medicine

Edward P. Sloan, MD, MPH Emeritus

Jeffrey R. Suchard, MD University of California, Irvine

Mandy J. Hill, DrPH, MPH UT Health McGovern Medical School

Kori S. Zachrison, MD Harvard Medical Center

Ultrasound

K. Tom Xu, MD, PhD Texas Tech University Health Sciences Center

Rick Lucarelli, MD Medical City Dallas Hospital

Infectious Disease

William D. Whetstone, MD University of California, San Francisco

Geriatrics

Teresita M. Hogan, MD University of Chicago

Antonio Siniscalchi, MD Annunziata Hospital

Health Equity

Elissa Schechter-Perkins, MD, MPH Boston University School of Medicine

Brandon Wills, DO, MS Virginia Commonwealth University

J. Matthew Fields, MD Thomas Jefferson University

Pediatric Emergency Medicine

Laleh Gharahbaghian, MD Stanford University Shane Summers, MD Brooke Army Medical Center

Paul Walsh, MD, MSc University of California, Davis

Ioannis Koutroulis, MD, MBA, PhD Drexel University College of Medicine Kevin Lunney, MD, MHS, PhD University of Maryland School of Medicine

Muhammad Waseem, MD Lincoln Medical & Mental Health Center

Robert Derlet, MD Founding Editor, California Journal of

Donna Mendez, MD, EdD University of Texas-Houston/McGovern Medical School

Official Journal of the California Chapter of the American College of Emergency Physicians, the America College of Osteopathic Emergency Physicians, and the California Chapter of the American Academy of Emergency Medicine

Available in MEDLINE, PubMed, PubMed Central, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 333 City Blvd, West, Rt 128-01, Orange, CA 92868, USA Office: 1-714-456-6389; Email: Editor@westjem.org

Volume 21, no. 5: September 2020

Western Journal of Emergency Medicine

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Editorial Board Amin A. Kazzi, MD, MAAEM The American University of Beirut, Beirut, Lebanon Anwar Al-Awadhi, MD Mubarak Al-Kabeer Hospital, Jabriya, Kuwait Arif A. Cevik, MD United Arab Emirates University College of Medicine and Health Sciences, Al Ain, United Arab Emirates Abhinandan A.Desai, MD University of Bombay Grant Medical College, Bombay, India Bandr Mzahim, MD King Fahad Medical City, Riyadh, Saudi Arabia

Francesco Dellacorte, MD Azienda Ospedaliera Universitaria “Maggiore della Carità,” Novara, Italy

Niels K. Rathlev, MD Tufts University School of Medicine

Gayle Galleta, MD Sørlandet Sykehus HF, Akershus Universitetssykehus, Lorenskog, Norway

Pablo Aguilera Fuenzalida, MD Pontificia Universidad Catolica de Chile, Región Metropolitana, Chile

Hjalti Björnsson, MD Icelandic Society of Emergency Medicine

Peter A. Bell, DO, MBA Liberty University College of Osteopathic Medicine

Jacob (Kobi) Peleg, PhD, MPH Tel-Aviv University, Tel-Aviv, Israel Jonathan Olshaker, MD Boston University

Daniel J. Dire, MD University of Texas Health Sciences Center San Antonio

Katsuhiro Kanemaru, MD University of Miyazaki Hospital, Miyazaki, Japan

David F.M. Brown, MD Massachusetts General Hospital/ Harvard Medical School

Khrongwong Musikatavorn, MD King Chulalongkorn Memorial Hospital, Chulalongkorn University, Bangkok, Thailand

Nadeem Qureshi, MD St. Louis University, USA Emirates Society of Emergency Medicine, United Arab Emirates

Francis Counselman, MD Eastern Virginia Medical School

Brent King, MD, MMM University of Texas, Houston

Edward Michelson, MD Texas Tech University

Linda S. Murphy, MLIS University of California, Irvine School of Medicine Librarian

Edward Panacek, MD, MPH University of South Alabama

Peter Sokolove, MD University of California, San Francisco Robert M. Rodriguez, MD University of California, San Francisco

Adam Levy Interim Executive Director, American College of Osteopathic Emergency Physicians John B. Christensen, MD California Chapter Division of AAEM Lori Winston, MD California ACEP American College of Emergency Physicians Kaweah Delta Healthcare District Mark I. Langdorf, MD, MHPE UC Irvine Health School of Medicine

Scott Zeller, MD University of California, Riverside Steven H. Lim, MD Changi General Hospital, Simei, Singapore Terry Mulligan, DO, MPH, FIFEM ACEP Ambassador to the Netherlands Society of Emergency Physicians Vijay Gautam, MBBS University of London, London, England Wirachin Hoonpongsimanont, MD, MSBATS Siriraj Hospital, Mahidol University, Bangkok, Thailand

Robert W. Derlet, MD University of California, Davis Rosidah Ibrahim, MD Hospital Serdang, Selangor, Malaysia

Advisory Board

Elena Lopez-Gusman, JD California ACEP American College of Emergency Physicians

Scott Rudkin, MD, MBA University of California, Irvine

Robert Suter, DO, MHA UT Southwestern Medical Center

Leslie Zun, MD, MBA Chicago Medical School

Amal Khalil, MBA UC Irvine Health School of Medicine

Samuel J. Stratton, MD, MPH Orange County, CA, EMS Agency

Editorial Staff

Nicholas T. Sawyer, MD, MBA California ACEP American College of Emergency Physicians University of California, Davis Peter A. Bell, DO, MBA American College of Osteopathic Emergency Physicians Liberty University, College of Osteopathic Medicine

Leila Danishgar, BS Executive Editorial Director

Christine Louis, BS WestJEM Publishing Director

Maria Nguyen, BS WestJEM Editorial Director

Cassandra Saucedo, BS CPC-EM Publishing Director

Nicholas Gossett, BS Isabelle Nepomuceno, BS CPC-EM Associate Editorial Director WestJEM Associate Publishing Director Associate Marketing Director Associate Marketing Director

Robert Suter, DO, MHA American College of Osteopathic Emergency Physicians UT Southwestern Medical Center

June Casey, BA Copy Editor

Ishan Shah, BS CPC-EM Associate Publishing Director Associate Marketing Director Alissa Fiorentino, BA WestJEM Staff Liaison

Shahram Lotfipour, MD, MPH UC Irvine Health School of Medicine Trevor Mills, MD, MPH California Chapter Division of AAEM Northern California VA Health Care

Available in MEDLINE, PubMed, PubMed Central, Europe PubMed Central, PubMed Central Canada, CINAHL, SCOPUS, Google Scholar, eScholarship, Melvyl, DOAJ, EBSCO, EMBASE, Medscape, HINARI, and MDLinx Emergency Med. Members of OASPA. Editorial and Publishing Office: WestJEM/Depatment of Emergency Medicine, UC Irvine Health, 333 City Blvd, West, Rt 128-01, Orange, CA 92866, USA Office: 1-714-456-6389; Email: Editor@westjem.org

Western Journal of Emergency Medicine

Volume 21, no. 5: September 2020

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

JOURNAL FOCUS Emergency medicine is a specialty which closely reflects societal challenges and consequences of public policy decisions. The emergency department specifically deals with social injustice, health and economic disparities, violence, substance abuse, and disaster preparedness and response. This journal focuses on how emergency care affects the health of the community and population, and conversely, how these societal challenges affect the composition of the patient population who seek care in the emergency department. The development of better systems to provide emergency care, including technology solutions, is critical to enhancing population health.

Table of Contents Endemic Infections 1080 Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario: A Crossover Study Laack TA, Pollok F, Sandefur BJ, Mullan AF, Russi CS, Yalamuri SM 1089

Mastery Learning Ensures Correct Personal Protective Equipment Use in Simulated Clinical Encounters of COVID-19 Pokrajac N, Schertzer K, Poffenberger CM, Alvarez A, Marin-Nevarez P, Winstead-Derlega C, Gisondi MA

1095

Streamlining Care in Crisis: Rapid Creation of a Digital Support Tool for COVID-19 Stark N, Kerrissey M, Grade M, Berrean B, Peabody C

1156

Triage in The Time of Diphtheria Hart J, Tracy R, Johnston M, Brown S, Stephenson C, Kegg J, Waymack

1283

COVID-19: An Emerging Threat to Antibiotic Stewardship in the Emergency Department Pulia MS, Wolf I, Schulz LT, Pop-Vicas A, Schwei RJ, Lindenauer PK

Behavioral Health 1102 Impact of Social Distancing on Individuals Who Use Drugs: Considerations for Emergency Department Providers LeSaint KT, Snyder HR 1175

Buprenorphine for Opioid Use Disorder in the Emergency Department: A Retrospective Chart Review LeSaint KT, MD, Klapthor B, MD, Wang RC, Geier C

1182

Post-traumatic Stress Disorder in Family Witnessed Resuscitation of Emergency Department Patients Erogul M, Likourezos A, Meddy J, Terentiev V, Davydkina D, Monfort R, Pushcar I, Vu T, Achalla M, Fromm C, Marshall J

1188

Point-of-sale Naloxone: Novel Community-based Research to Identify Naloxone Availability Olives TD, Willhite LA, Lee SC, Evans DK, Jensen A, Regelman HT, McGillis ES

1195

The Utility of Serum Creatinine Phosphokinase in Emergency Department Patients with Possible Substance Use Related Condition Alzahri M

Education 1105 COVID-19: A Driver for Disruptive Innovation of the Emergency Medicine Residency Application Process Pelletier-Bui A, Franzen D, Smith L, Hopson L, Lutfy-Clayton L, Parekh K, Olaf M, Morrissey T, Gordon D, McDonough E, Schnapp B, Kiemeney M 1258

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy Miller DT, Zaidi HQ, Sista P, Dhake SS, Pirotte MJ, Fant AL, Salzman DH

Policies for peer review, author instructions, conflicts of interest and human and animal subjects protections can be found online at www.westjem.com. Volume 21, no. 5: September 2020

iii

Western Journal of Emergency Medicine

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Table of Contents continued 1266

Effect of Resident Physicians in a Supervisory Role on Efficiency in the Emergency Department Kraut AS, Sheehy L, Schnapp B, Patterson B

Research Methodology 1114 Leveraging Remote Research Associates during a Pandemic Cronin AO, Carlile MA, Dameff CJ, Coyne CJ, Castillo EM 1118

Resident Research in Emergency Medicine: An Introduction and Primer Paxton JH, Messman AM, Harrison NE, Malik AM, Burke RJ, Levy PD

Injury Prevention 1123 Counseling on Access to Lethal Means-Emergency Department (CALM-ED): A Quality Improvement Program for Firearm Injury Prevention Mueller KL, Naganathan S, Griffey RT Provider Workforce 1131 Intention to Leave Emergency Medicine: Mid-Career Women are at Increased Risk Lall MD, Perman SM, Garg N, Kohn N, Whyte K, Gips A, Madsen T, Baren JM, Linden J Population Health 1140 Using an Online Vaccination Registry to Confirm Tetanus Status in Children with Tetanus-Prone Wounds McCall J, Zeretzke-Bien C, Wylie T, Chowdhury MAB, Balakrishnan M, Hendry P, Kalynych C, Chung HJ Healthcare Utilization 1147 A Method for Grouping Emergency Department Visits by Severity and Complexity Theiling BJ, Kennedy K, Limkakeng AT, Manandhar P, Erkanli A, Pitts S Health Equity 1160 Academic Emergency Medicine Faculty Experiences with Racial and Sexual Orientation Discrimination Lu DW, Pierce A, Jauregui J, Heron S, Lall MD, Mitzman J, McCarthy DM, Hartman ND, Strout TD 1170

Improving Understanding of Screening Questions for Social Risk and Social Need Among Emergency Department Patients Ciccolo G, Curt A, Camargo C, Samuels-Kalow M

Emergency Department Operations 1201 Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact Hwang MI, Bond WF, Powell ES 1211 Dumpster Diving in the Emergency Department: Quantity and Characteristics of Waste at a Level 1 Trauma Center Hsu S, Thiel C, Mello MJ, Slutzman JE 1218

Patient Characteristics and Clinical Process Predictors of Patients Leaving Without Being Seen from the Emergency Department Rathlev N, Visintainer P, Schmidt J, Hettler J, Albert V, Li H

Emergency Medical Services 1227 Development and Implementation of a Community Paramedicine Program in Rural United States Myers LA, Carlson PN, Krantz PW, Johnson H, Will MD, Bjork TM, Dirkes M, Bowe JE, Gunderson KA, Russi CS 1234

Paramedic Pain Management Practice with Introduction of a Non-opiate Treatment Protocol Oâ&#x20AC;&#x2122;Connor L, Dugas J, Brady J, Kamilaris A, Shiba SK, Kue RC, Broach JP

Volume 21, no. 5: September 2020

Western Journal of Emergency Medicine

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded

Table of Contents continued Health Outcomes 1242 Insurance Does Not Affect Adverse Events While Awaiting Surgery for Ankle Trauma in One System Dobbins AB, Krumme J, Gaddis M, Park SH, Varghese M, Brancato MR, Shaw CM, Wambach K 1249 Age-Adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the Emergency Department Miller R, Cannon CM, Grow KL, Purcell S, Nazir N Geriatrics 1270 Feasibility of Health Literacy Tools for Older Patients in the Emergency Department McGuinness MJ, Bucher J, Karz J, McCoy J, Pardee C, Patti L, Ohman-Strickland P 1275

Two-Item Fall Screening Tool Identifies Older Adults at Increased Risk of Falling After Emergency Department Visit Solie CJ, Swanson MB, Harland K, Blum C, Kin K, Mohr N

Erratum 1287 This Article Corrects: â&#x20AC;&#x153;Conference Didactic Planning and Structure: An Evidence-Based Guide to Best Practices from the Council of Emergency Medicine Residency Directorsâ&#x20AC;? Wood DB, Jordan J, Cooney R, Goldflam K, Bright L, Gottlieb M

VITAL STATISTICS

Western Journal of Emergency Medicine (WestJEM): Integrating Emergency Care with Population Health (WestJEM) is the premier open-access Medline-indexed EM journal in the world. As the official journal of the California Chapter of the American College of Emergency Physicians (ACEP), American College of Osteopathic Emergency Physicians (ACOEP), and the California chapter division of the American Academy of Emergency Medicine (AAEM), the journal focuses on how emergency care affects health and health disparities in communities and populations. Additionally, WestJEM focuses on how social conditions impact the composition of patients seeking care in emergency departments worldwide. WestJEM is distributed electronically to 19,000 emergency medicine scholars and 2,800 in print. This includes 83 academic department of emergency medicine subscribers and 7 AAEM State Chapters. Volume 21, no. 5: September 2020

Western Journal of Emergency Medicine

Western Journal of Emergency Medicine: Integrating Emergency Care with Population Health

Indexed in MEDLINE, PubMed, and Clarivate Web of Science, Science Citation Index Expanded This open access publication would not be possible without the generous and continual financial support of our society sponsors, department and chapter subscribers.

Professional Society Sponsors American College of Osteopathic Emergency Physicians California ACEP

California Chapter Division of American Academy of Emergency Medicine

Academic Department of Emergency Medicine Subscriber Albany Medical College Albany, NY

INTEGRIS Health Oklahoma City, OK

Oregon Health and Science University Portland, OR

University of Illinois at Chicago Chicago, IL

Allegheny Health Network Pittsburgh, PA

Kaiser Permenante Medical Center San Diego, CA

Penn State Milton S. Hershey Medical Center Hershey, PA

University of Iowa Iowa City, IA

American University of Beirut Beirut, Lebanon

Kaweah Delta Health Care District Visalia, CA

University of Louisville Louisville, KY

AMITA Health Resurrection Medical Center Chicago, IL

Kennedy University Hospitals Turnersville, NJ

Regions Hospital Emergency Medicine Residency Program St. Paul, MN

Arrowhead Regional Medical Center Colton, CA

Kent Emergency Medical Residency Program Warwick, RI

Baystate Medical Center Springfield, MA

Kern Medical Bakersfield, CA

Bellevue Hospital Center New York, NY

Lakeland HealthCare St. Joseph, MI

Beth Israel Deaconess Medical Center Boston, MA

Lehigh Valley Hospital and Health Network Allentown, PA

Boston Medical Center Boston, MA

Loma Linda University Medical Center Loma Linda, CA

Brigham and Womenâ&#x20AC;&#x2122;s Hospital Boston, MA

Louisiana State University Health Sciences Center New Orleans, LA

Brown University Providence, RI

Madigan Army Medical Center Tacoma, WA

Carl R. Darnall Army Medical Center Fort Hood, TX

Maimonides Medical Center Brooklyn, NY

Conemaugh Memorial Medical Center Johnstown, PA

Maine Medical Center Portland, ME

Crozer-Chester Medical Center Upland, PA

Massachusetts General Hospital Boston, MA

Desert Regional Medical Center Palm Springs, CA

Mayo Clinic College of Medicine Rochester, MN

Eastern Virginia Medical School Norfolk, VA

Mercy Health - Hackley Campus Muskegon, MI

Einstein Healthcare Network Philadelphia, PA

Merit Health Wesley Hattiesburg, MS

Eisenhower Medical Center Rancho Mirage, CA

Midwestern University Glendale, AZ

Emory University Atlanta, GA

New York University Langone Health New York, NY

Geisinger Medical Center Danville, PA

North Shore University Hospital Manhasset, NY

Grand State Medical Center Allendale, MI

Northwestern Medical Group Chicago, IL

Hennepin County Medical Center Minneapolis, MN

Ohio State University Medical Center Columbus, OH

Henry Ford Medical Center Detroit, MI

Ohio Valley Medical Center Wheeling, WV

State Chapter Subscriber

Arizona Chapter Division of the American Academy of Emergency Medicine California Chapter Division of the American Academy of Emergency Medicine Florida Chapter Division of the American Academy of Emergency Medicine

Rhode Island Hospital Providence, RI

University of Massachusetts Amherst, MA

Robert Wood Johnson University Hospital New Brunswick, NJ

University of Michigan Ann Arbor, MI

Rush University Medical Center Chicago, IL

University of Missouri, Columbia Columbia, MO

St. Lukeâ&#x20AC;&#x2122;s University Health Network Bethlehem, PA

University of Nebraska Medical Center Omaha, NE

Spectrum Health Lakeland St. Joseph, MI

University of Nevada, Las Vegas Las Vegas, NV

Stanford Stanford, CA

University of Southern Alabama Mobile, AL

SUNY Upstate Medical University Syracuse, NY

University of Southern California Los Angeles, CA

Temple University Philadelphia, PA Texas Tech University Health Sciences Center El Paso, TX University of Alabama, Birmingham Birmingham, AL University of California, Davis Medical Center Sacramento, CA University of California, Irvine Orange, CA University of California, Los Angeles Los Angeles, CA University of California, San Diego La Jolla, CA University of California, San Francisco San Francisco, CA UCSF Fresno Center Fresno, CA University of Chicago Chicago, IL University of Florida Gainesville, FL University of Florida, Jacksonville Jacksonville, FL

Great Lakes Chapter Division of the American Academy of Emergency Medicine Tennessee Chapter Division of the American Academy of Emergency Medicine

University of Maryland Baltimore, MD

University of Tennessee, Memphis Memphis, TN University of Texas, Houston Houston, TX University of Washington Seattle, WA University of Wisconsin Hospitals and Clinics Madison, WI Virginia Commonwealth University Medical Center Richmond, VA Wake Forest University Winston-Salem, NC Wake Technical Community College Raleigh, NC Wayne State Detroit, MI Wright State University Dayton, OH York Hosptial York, ME

Uniformed Services Chapter Division of the American Academy of Emergency Medicine Virginia Chapter Division of the American Academy of Emergency Medicine

International Society Partners Emergency Medicine Association of Turkey Lebanese Academy of Emergency Medicine Mediterranean Academy of Emergency Medicine

Norwegian Society for Emergency Medicine Sociedad Argentina de Emergencias

Sociedad Chileno Medicina Urgencia Thai Association for Emergency Medicine

To become a WestJEM departmental sponsor, waive article processing fee, receive print and copies for all faculty and electronic for faculty/residents, and free CME and faculty/fellow position advertisement space, please go to http://westjem.com/subscribe or contact: Alissa Fiorentino WestJEM Staff Liaison Phone: 1-800-884-2236 Email: sales@westjem.org

Western Journal of Emergency Medicine

Volume 21, no. 5: September 2020

Integrating Emergency Care with Population Health

CALL FOR PAPERS Special Issue on Gun Violence Research Submit your articles to WestJEM today! Deadline: September 30, 2020 at 11:59PM PST

Call for Section Editors • • • • •

Behavioral Emergencies • Trauma Care Emergency Cardiac Care • Ultrasound International Medicine Pediatric Emergencies Public Health

Send CV and letter of interest to Editor@WestJEM.org

NEW: HEALTH EQUITY SECTION Call for Reviewers and Manuscripts Send CV and letter of interest to Editor@WestJEM.org

CALL FOR PAPERS

WestJEM and CPC-EM to publish COVID-19 Related Manuscripts Submit your articles to WestJEM or CPC-EM today!

Editorial

The Bullets He Carried Stephen W. Hargarten, MD, MPH

Medical College of Wisconsin, Comprehensive Injury Center, Milwaukee, Wisconsin

Section Editor: Mark I. Langdorf, MD, MHPE Submission history: Submitted May 14, 2020; Accepted May 14, 2020 Electronically published August 7, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.48216

[West J Emerg Med. 2020;21(5)1036.]

The Sandy Hook Elementary School mass shooting on December 14, 2012, killed 26 people including 20 young children ages six to seven. The Sandy Hook shooter fired 154 bullets in less than four minutes, or about 38 bullets per minute from a semiautomatic rifle. When the bullet leaves a Bushmaster rifle, it travels over 2000 feet per second. This velocity gives this bullet its devastating wounding potential. As this rifle bullet penetrates a human body, the energy of the bullet tears and shreds through tissue and bone, resulting in fractures, ruptured livers, and swollen brains, leading to hemorrhage, shock, and death. As an emergency physician, I have cared for hundreds of patients injured by bullets. I have had to tell parents that their teenager has died. Even those who survive are forever maimed and suffering. As a physician, I am interested in better understanding this pathogen of gun violence: the bullet and the guns that carry them.1 Recently, my colleagues and I at the Medical College of Wisconsin’s Comprehensive Injury Center focused our attention on the bullet and its energy. This energy is a measure of the potential for causing wounds. Other factors play a role in wounding including the mass of the bullet and the direct tearing of tissues. But understanding the energy of a bullet and its wounding potential can help develop better treatment of the wounds. Using the latest in high-speed video cameras, we discharged bullets through gelatin, which is commonly used to mimic human tissue. We measured the kinetic energy release of a modern, high-speed rifle bullet, and of a musket ball similar to those used in the 1780s (https://www.mcw.edu/departments/comprehensiveinjury-center/research). Note the dramatic difference in speed, cavitation, wave propagation, and resultant tissue damage of the rifle bullet vs the musket ball. We found that the rifle bullet’s energy release was over nine times greater than the musket ball because of the rifle bullet’s significantly greater velocity compared to the musket ball’s velocity. In 1789, when the Second Amendment was passed by Congress, the average number of musket balls that could be fired by a member of the militia was about two per minute. Using this number-of-bullets-released-per-minute comparison, the Sandy Hook mass shooter represented the equivalent of 19 militiamen storming the elementary school. Even worse, the energy of the

Western Journal of Emergency Medicine

rifle bullet released by the Sandy Hook mass shooter was in turn at least nine times greater per bullet than the energy released by the musket balls shot by the militia. Using this energy-release-perminute calculation, and its accompanying wounding potential, the number of bullets and their energy fired by the Sandy Hook shooter equaled an estimated 171 militiamen storming the school. The rifle and bullet technology of 2020 far exceeds that available 230 years ago. When Congress passed the Second Amendment, they could not have anticipated that, in 2012, a single man in Connecticut would use a weapon with the killing power of an army of 171 members of the Connecticut militia. Understanding and addressing today’s bullets, their energy, their wounding potential, and the weapons that carry them are essential elements in any comprehensive solution to gun violence. It is of critical importance that all sectors of civil society understand this energy focus when discussing policies about these bullets and the guns that carry them.

Address for Correspondence: Stephen W. Hargarten, MD, MPH, Medical College of Wisconsin, Department of Emergency Medicine, 8701 W. Watertown Plank Rd, Milwaukee, WI 53226. Email: Hargart@mcw.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Hargarten. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1. Hargarten SW, Lerner EB, Gorelick M, et al. Gun Violence: A

1036

Biopsychosocial Disease. West J Emerg Med. 2018;19(6):1024-7.

Volume 21, no. 5: September 2020

Commentary

United States Congressional COVID-19 Legislation: Recent Laws and Future Topics George Washington University Medical Faculty Associates, Department of Emergency Marisa K. Dowling, MD, MPP Medicine, Washington, District of Columbia Aisha T. Terry, MD, MPH Natalie L. Kirilichin, MD, MPH Jennifer S. Lee, MD Janice C. Blanchard, MD, PhD, MPH Section Editor: Dan Mayer, MD Submission history: Submitted June 26, 2020; Revision received July 24, 2020; Accepted July 20, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48891 [West J Emerg Med. 2020;21(5)1037-1041.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. INTRODUCTION Nothing is normal now, least of all the United States Congress. As the novel coronavirus (COVID-19) pandemic devastates Americans’ health and livelihoods, Congress has passed sweeping legislation to address the nation’s parallel medical and economic crises. These legislative interventions have important implications for emergency physicians—as frontline workers, family members, and advocates. This article summarizes the new laws’ most relevant provisions for emergency physicians. LEGISLATION TO DATE To date, the US Congress has passed four coronavirus relief bills (Table 1). First Law On March 6, 2020, Congress passed the first coronavirus relief law (Coronavirus Preparedness and Response Supplemental Appropriations Act, Public Law 116-123). At a cost of $8.3 billion, the law focuses on immediate pandemic response efforts, including funding to create viral test kits, vaccine and drug development, and aid for state and local health departments. Second Law At a price tag of $192 billion, Congress enacted the Families First Coronavirus Response Act (P.L. 116-127) on March 18, 2020. The law provides significant aid to individuals and families suffering from the economic effects Volume 21, no. 5: September 2020

of COVID-19 related shutdowns, including expanded unemployment benefits and emergency paid sick leave for eligible workers. Of note, “frontliners” (such as emergency physicians) were excluded from sick leave expansions. This exclusion was intentional due to concerns for potential healthcare staffing shortages if sick leave expansions included essential medical workers. Third Law On March 27, 2020, Congress passed the largest ($1.7 trillion) stimulus law in US history, the Coronavirus Aid, Relief, and Economic Security (CARES) Act (P.L. 116–136).1 The CARES Act dramatically escalated Congress’ response to the virus’ staggering economic impact through direct stimulus cash payments to most Americans, expanded unemployment benefits, and aid to businesses. Most notably for healthcare providers, the CARES Act created the Provider Relief Fund (PRF) with $100 billion in aid for healthcare organizations and clinicians of all types to assist with lost revenues and COVID-19 preparedness expenses. Congress gave the Department of Health and Human Services (HHS) considerable discretion in the distribution of PRF monies. HHS subsequently faced an onslaught of funding appeals from various provider groups. To date, PRF disbursements have included the following: • $30 billion based on a provider’s share of 2019 Medicare fee-for-service reimbursements. 320,000 providers received funds through this mechanism. • $20 billion based on a provider’s share of net patient revenue; 15,000 providers received funds through this disbursement. • $15 billion for providers serving high numbers of Medicaid and Children’s Health Insurance Program (CHIP) patients. Data on the number of recipients is pending.

1037

Western Journal of Emergency Medicine

Dowling et al.

US Congressional COVID-19 Legislation: Recent Laws and Future Topics •

•

$13 billion for hospitals with high numbers of low-income and uninsured patients based on disproportionate share hospital (DSH) funding. Information on the number of recipients is pending. $22 billion for hospitals with high numbers of COVID-19 patients. • $12 billion was first distributed to 395 hospitals with 100 + COVID-19 patients before April 10, 2020, which averaged to $76,975 per eligible admission. • A second round of payments totaling $10 billion started July 20, 2020, for hospitals that had more than 161 COVID-19 admissions (ie, averaging one COVID-19 admission per day) between January 1–June

•

10, 2020, and/or had a high intensity of COVID-19 admissions (exceeded the average ratio of COVID-19 admissions/bed). One thousand hospitals are expected to benefit from these payments, which average out to $50,000 per eligible admission. HHS will consider hospitals’ funding from the first round when allocating the second round of payments. HHS has stated that it plans to evaluate and provide additional relief funds to future COVID-19 hotspot hospitals as monies allow. $11 billion for over 4000 rural hospitals (including critical access hospitals), rural health clinics, and rural health centers. Payments included a minimum

Table 1. Key provisions of COVID-19 relief laws. Law

Date

Cost

Coronavirus Preparedness and Response Supplemental Appropriations Act (Public Law 116-123)

March 6

$8.3 billion

Families First Coronavirus Response Act (P.L. 116-127)

March 18

$192 billion

Key Healthcare Provisions •

Other Notable Provisions

Immediate pandemic response $6.7 billion for test kits, vaccine and drug development, and state and local health departments • $100 million in grants to rural/ underserved communities • Health and Human Services Secretary given the authority to loosen Medicare telehealth restrictions • •

•

• •

Free COVID-19 testing for the insured • Requires all insurers to cover COVID-19 treatment, though cost-sharing • requirements (co-pays, deductible, etc.) remain in effect. Increases federal matching funds (Federal • Medical Assistance Percentages, FMAP) for Medicaid by 6.2%

$20 million in small business loans $1.6 billion in international COVID-19 response aid

$8 billion for nutrition assistance programs Expanded unemployment insurance benefits Emergency paid sick leave for eligible workers (“frontliners” excluded)

Coronavirus Aid, Relief, and Economic Security (CARES) Act (P.L. 116–136)

March 27

$1.7 trillion

Payments • • $100 billion Provider Relief Fund (PRF) to assist with pandemic response, lost revenues. • $34 billion in advance Medicare payments to assist provider cash flow • • Delayed planned disproportionate share hospital (DSH) cuts • Testing & Supplies • $11 billion for state and local testing • Funding for personal protective equipment (PPE) procurement and supply chain improvements. • Requires any future COVID-19 vaccine to be free for insured patients

$349 billion for Small Business Administration’s Paycheck Protection Program (PPP) $25 billion for nutrition assistance programs Federal student loan debt assistance

Paycheck Protection Program and Health Care Enhancement Act (P.L. 116–139)

April 24

$396 billion

• •

$321 billion more for the PPP

Western Journal of Emergency Medicine

$75 billion more towards PRF $1 billion for COVID-19 testing for the uninsured

1038

•

Volume 21, no. 5: September 2020

Dowling et al.

US Congressional COVID-19 Legislation: Recent Laws and Future Topics

base payment ($100,000 for clinics, $1 million for hospitals) plus a percent of the site’s annual expenses. • $4.9 billion for skilled nursing facilities (SNF). So far, 13,000 SNFs have benefited from such funding. • $500 million for the Indian Health Service. HHS also reserved $12 billion for reimbursing providers caring for uninsured COVID-19 patients. Of note, for all the above funds, HHS requires that providers complete an online application (which includes questions about the entity’s finances) by certain deadline(s) and accept terms and conditions (which include a prohibition against balance billing COVID-19 patients). Overall, how and whether these funds will trickle down to individual emergency physicians—many of whom have seen their hours cut or faced furlough— remains to be seen. Given that funds are largely disbursed through billing mechanisms, employed and group practice physicians will likely not receive direct payments from the relief fund. Rather, the vast majority of PRF funding has gone to hospitals or other large care organizations, rather than to individual clinicians.2 Solo practitioners and/or independent contractors who manage their own billing, however, can receive funds directly from the PRF via their tax identification number. Fourth Law Following this whirlwind of COVID-19 related legislation, Congress entered a legislative stalemate for about a month. Ultimately when funding for small business loans lapsed, Congressional leaders compromised, and on April 24 passed the Paycheck Protection Program and Health Care Enhancement Act (P.L. 116–139). At a cost of $396 billion, the law limits itself to supplemental funding for small business loans and the PRF ($75 billion). Next Bill Congress sank into a period of political gridlock after the fourth law’s passage. In an attempt to spur negotiations, on May 15 the House of Representatives passed the Health and Economic Recovery Omnibus Emergency Solutions (HEROES) Act (H.R. 6800). The bill represents House Democrats’ ideal version of the next COVID-19 relief package, which they hope will set the terms of the coming debate. The HEROES Act comprises a wide range of provisions from significant aid to state and local governments to direct cash payments to most Americans. Provisions that are most likely to affect emergency physicians include the following: • Hazard Pay – Calls for a “Heroes Fund” of $200 billion in “premium” pay for essential workers, such as health professionals, sanitation personnel, and grocery store employees. Workers earning less than $200,000 would be eligible to receive up to $10,000 in hazard pay. Workers earning more than $200,000 would be capped at $5,000. In order for employees to get any of this money, their employer would need to Volume 21, no. 5: September 2020

apply to the federal government for a “Heroes Fund” grant. The employer would then distribute the grant money to eligible workers in the form of a supplement to the workers’ hourly wage ($13/hour) for work done during the public health emergency (PHE) up to the worker’s maximum eligibility ($10,000 or $5,000). Workers could not apply for funds directly. • Personal Protective Equipment (PPE) Standards – The Occupational Safety and Health Administration (OSHA) previously issued guidance on what qualifies as proper PPE for health care workers; this includes gloves, gowns, goggles/face shield, and National Institutes of Safety and Healthcertified, disposable N-95 filter facepiece respirators or higher.3 The HEROES Act tasks OSHA with strictly enforcing these PPE standards for infection control. Moreover, the law would prohibit employers from retaliating against workers who report infection control problems and protect employees who wish to use their personally owned, more protective PPE at work, if not provided by the employer. • Student Loans – Grants up to $10,000 in federal and $10,000 in private student loan forgiveness to eligible borrowers who are struggling financially. It also extends the pause on student loan payments until September 2021 for nearly all federal loan types.4 The CARES Act had already automatically paused federal student loan payments, set interest rates to 0%, and decreed that any “non-payments” through September 2020 will still qualify toward student loan forgiveness program payment obligations. • Provider Funding – Adds $100 billion to the CARES Act PRF, bringing the total across all bills to $275 billion. Whether these provisions become law hinges on future Senate deliberations. LOOKING FORWARD While the ground is constantly shifting as Congressional negotiations proceed, most observers believe Congress will pass a bill this summer given the nation’s ongoing economic crisis and the continued rise in COVID-19 cases. Yet some commentators feel the next bill will be the last “definite” COVID-19 legislation passed before Congress succumbs to its traditional election year-related doldrums. Liability Reforms While not included in the HEROES Act, liability reform may be central to the next COVID-19 package. Senate Majority Leader Mitch McConnell has insisted that the next coronavirus package include broad liability protections for medical professionals and businesses to stop a “second pandemic” of “lawsuits against doctors, nurses, hospitals, and brave business

1039

Western Journal of Emergency Medicine

Dowling et al.

US Congressional COVID-19 Legislation: Recent Laws and Future Topics people who are opening up” covering the period from December 2019–December 2024.5,6 In the meantime, 26 states previously had or recently enacted some type of civil liability immunities and/or Good Samaritan protections for physicians during the public health emergency (PHE).7

Even so, many questions (reimbursement, coverage, access, security, privacy, and inter-state medical licensure) remain over how to implement telehealth services going forward.

Telehealth In response to the pandemic, the Centers for Medicare & Medicaid Services (CMS) significantly relaxed a number of previous telehealth regulations in order to reduce the spread of the virus and make it easier for patients to receive needed medical care. To date, CMS has waived rules regarding the following: • Geographic Limits – Now patients can use telehealth anywhere in the US (urban or rural), rather than only certain qualifying rural areas. • Site of Care – CMS removed “originating site” requirements. As a result, patients can now use telehealth at home, rather than having to go to certain health facilities to use the technology. • Privacy & Security – Providers can now use common, unsecured, non-HIPPA compliant applications such as Zoom, Skype, and Facetime for telehealth. • Technology – Audio-only phone calls, in addition to audio-visual telecommunications, now qualify as telehealth. This especially helps those unfamiliar with newer devices/technology and those who lack access to broadband Internet (such as rural areas). • Medical Care – Physicians may treat nearly any condition via telehealth with no prior in-person patient-doctor relationship required. Of note for emergency medicine, permitted telehealth services include the Emergency Medical Treatment and Labor Act-required medical screening exams and tele-triage. • Payments – Medicare now pays equally for in-person and telehealth visits. Many state Medicaid programs and private insurers have similarly expanded their telehealth offerings and increased payments. As a result of these regulatory changes, telehealth usage has grown significantly nationwide. Specifically, Medicare telehealth visits jumped from 13,000 per week prior to the pandemic to 1.7 million per week by late April.8 Analysts predict that telehealth visits across all specialties and payers could top one billion visits this year alone. 9 However, the clock is ticking on the telehealth expansion. All of CMS’ relaxed rules and increased payments are set to expire when the HHS Secretary (in consultation with other public health experts) determines that the PHE is over. By law, PHE declarations last 90 days and can be renewed in 90-day increments as long as the HHS Secretary determines it is needed. Notably, the PHE had been set to expire on July 25, 2020, but HHS officially extended the PHE for an additional 90 days to a new end date of October 23, 2020. Ultimately, industry insiders hope that Congress will move to make a number of the new telehealth rules permanent after the PHE. Western Journal of Emergency Medicine

CONCLUSION Emergency physicians have faced unprecedented challenges during this pandemic. As Congress attempts to mitigate the ongoing COVID-19 crisis, continued advocacy from emergency physicians is needed to ensure that the needs of our patients, communities, and profession remain prioritized. Consider reaching out to your local, state, and federal government representatives regarding your frontline experiences and the need for their support on the issues most critical to our specialty and society. Address for Correspondence: Marisa K. Dowling, MD, MPP, George Washington University Medical Faculty Associates, Department of Emergency Medicine, 2120 L Street, NW, Suite 450, Washington, DC 20037. Email: mdowling@mfa.gwu.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Dowling et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/.

REFERENCES 1.

Congressional Budget Office. H.R. 748, CARES Act, Public Law 116136. 2020. Available at: https://www.cbo.gov/publication/56334. Accessed June 9, 2020.

Health Resources & Services Administration. HHS Provider Relief Fund. 2020. Available at: https://data.cdc.gov/Administrative/HHS-ProviderRelief-Fund/kh8y-3es6. Accessed June 9, 2020.

Occupational Safety and Health Administration. COVID-19 - Control and Prevention- Healthcare Workers and Employers. Available at: https://www.osha.gov/SLTC/covid-19/healthcare-workers.html. Accessed July 19, 2020.

Minsky AS. House Passes HEROES Act With Limits On Student Loan Relief – What’s Next? 2020. Available at: https://www.forbes.com/sites/ adamminsky/2020/05/15/house-passes-heroes-act-with-limits-on-student-loan-relief--whats-next/. Accessed June 10, 2020.

1040

Segers G. McConnell says next round of coronavirus relief could include business liability protections. 2020. Available at: https://www.cbsnews. com/news/mitch-mcconnell-coronavirus-relief-business-liability-protections/. Accessed July 17, 2020.

Volume 21, no. 5: September 2020

Dowling et al. 6.

US Congressional COVID-19 Legislation: Recent Laws and Future Topics

Raju M, Byrd H, Zaslav A. McConnell: No recovery bill without lawsuit

protections for â&#x20AC;&#x153;everyone related to the coronavirus.â&#x20AC;? 2020. Available

COVID-19. 2020. Available at: https://www.healthaffairs.org/do/10.1377/

at: https://www.cnn.com/2020/07/13/politics/covid-aid-congress-mitch-

hblog20200715.454789/full/. Accessed July 17, 2020.

mcconnell/index.html. Accessed July 17, 2020. 7.

Verma S. Early Impact of CMS Expansion of Medicare Telehealth During

Coombs B. Telehealth visits are booming as doctors and patients

Medical Professional Liability Association. COVID-19 Information Center.

embrace distancing amid the coronavirus crisis. 2020. Available at:

2020. Available at: https://www.mplassociation.org/Web/Government_

https://www.cnbc.com/2020/04/03/telehealth-visits-could-top-1-billion-in-

Relations/COVID-19_Information_Center.aspx. Accessed June 9, 2020.

2020-amid-the-coronavirus-crisis.html. Accessed June 9, 2020.

Volume 21, no. 5: September 2020

1041

Western Journal of Emergency Medicine

Commentary

Point-of-Care Ultrasound for Intubation Confirmation of COVID-19 Patients Michael Gottlieb, MD* Stephen Alerhand, MD† Brit Long, MD‡

* Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois † Rutgers New Jersey Medical School, Department of Emergency Medicine, Newark, New Jersey ‡ Brooke Army Medical Center, Department of Emergency Medicine, San Antonio, Texas

Section Editor: R. Gentry Wilkerson, MD Submission history: Submitted June 10, 2020; Revision received July 16, 2020; Accepted July 17, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48657

The novel coronavirus disease of 2019 (COVID-19) is associated with significant morbidity and mortality, as well as large numbers of patients requiring endotracheal intubation. While much of the literature has focused on the intubation technique, there is scant discussion of intubation confirmation. Herein, we discuss the limitations of traditional confirmatory approaches, summarize the literature supporting a role for point-of-care ultrasound in this application, and propose an algorithm for intubation confirmation among COVID-19 patients. [West J Emerg Med. 2020;21(5)1042-1045.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. BACKGROUND Novel coronavirus disease of 2019 (COVID-19) was first identified in Wuhan, China, beginning in December 2019.1,2 Since then, the virus has spread rapidly, infecting over 13.3 million people worldwide and resulting in nearly 580,000 deaths.2 Hypoxemic respiratory failure requiring intubation may occur in up to 19% of all COVID-19 hospitalized patients and 70% of those admitted to the intensive care unit.3-5 COVID-19 is rapidly transmissible and, while the most common means of transmission is droplet, airborne transmission may also occur during aerosol-generating procedures such as intubation and subsequent bag-valve ventilation.6 While much of the transmission conversation has revolved around intubation itself,7 the discussion of risk associated with post-intubation endotracheal tube (ETT) confirmation is more limited. This commentary will highlight the limitations associated with current intubation confirmation techniques in light of COVID-19 and propose an alternate approach using point-of-care ultrasound (POCUS). Western Journal of Emergency Medicine

LIMITATIONS OF TRADITIONAL CONFIRMATORY METHODS Traditional methods of intubation confirmation (eg, auscultation for bilateral breath sounds, condensation in the ETT) are insufficiently accurate in isolation.8,9 Visualization of ETT passage may be limited by difficult laryngoscopic views and the use of personal protective equipment (PPE). Auscultation can also be challenging in a loud room and may not be possible with some forms of PPE. Moreover, in light of the surface stability of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), auscultation with a stethoscope increases the potential risk for transmission via fomite exposure, while also requiring clinicians to be much closer to the patient which can increase their risk of infection.10,11 Other devices, such as end-tidal carbon dioxide (CO2) detectors and colorimetric capnometry, require at least five breaths for confirmation. This can lead to gastric distension and an increased risk of aspiration if the ETT is incorrectly placed in the esophagus, as well as increased risk of particle aerosolization to providers from the positive pressure ventilations.8,12 Additionally, capnography may be less reliable in patients where there is a paucity of CO2 produced (eg, cardiac arrest, pulmonary embolism), with studies suggesting that quantitative capnography may be only 60-65% sensitive during cardiac arrest.13,14 When assessing for mainstem (ie, endobronchial) intubation, auscultation is equally problematic, with studies demonstrating

1042

Volume 21, no. 5: September 2020

Gottlieb et al.

POCUS for Intubation Confirnation of COVID-19 Patients

that auscultation alone may misidentify mainstem intubation in 35-60% of patients.15-17 While radiographs are typically the gold standard for assessing ETT depth, they can be significantly delayed due to the PPE necessary to perform this task and limited departmental resources, which may lead to significant lung barotrauma for unrecognized mainstem intubations in this population with limited oxygen reserve. ROLE OF POINT-OF-CARE ULTRASOUND FOR INTUBATION CONFIRMATION POCUS has been increasingly recognized as a valuable tool for intubation confirmation. One approach for this is the transtracheal technique, wherein a clinician places the transducer across the patientâ&#x20AC;&#x2122;s neck post-intubation to visualize the ETT within the trachea or esophagus. This can be facilitated by gently twisting the ETT to create a motion artifact.18,19 A recent systematic review and meta-analysis found that transtracheal ultrasound was 99% sensitive and 97% specific for confirming ETT location among adult patients.20 A similar review among pediatric patients found that POCUS was 92-100% sensitive and 100% specific.21 Studies have also demonstrated that the accuracy remains consistent regardless of ETT size or transducer type.22,23 Additionally, the learning curve for identifying ETT placement with transtracheal POCUS has been suggested to be relatively short.24 Importantly, this modality offers the unique benefit that it does not require positive pressure ventilation, thereby minimizing additional exposure to staff. Other studies have suggested using indirect signs, such as bilateral lung sliding or diaphragmatic elevation for intubation confirmation with a high degree of accuracy.25 Two studies demonstrated that the combination of lung sliding with transtracheal POCUS further increased the diagnostic accuracy over either in isolation.26,27 ROLE OF POINT-OF-CARE ULTRASOUND FOR DETECTING MAINSTEM INTUBATION Mainstem intubation can be detected through the following three sonographic assessments: lung sliding; diaphragmatic excursion; or the presence of lung pulse. In a mainstem intubation there is no air flow through the contralateral lung, resulting in the absence of the lung sliding (ie, motion artifact visualized between the visceral and parietal layers of the pleura) on that side. Studies of both cadaveric models and emergency department patients have demonstrated that unilateral right lung sliding was 69-92% sensitive and 55.6-100% specific for detecting right mainstem intubation.28,29 When compared with auscultation, this technique has outperformed auscultation in both adult and pediatric patients.30,31 Sonographic assessment of hemidiaphragmatic movement can also be used as a surrogate for ventilation of that lung. When a lung is ventilated by air, the diaphragm will move inferiorly, allowing for direct visualization of lung expansion. Studies have found that this technique is 91-100% sensitive and 50-100% specific, with near-perfect inter-rater reliability.32,33 Volume 21, no. 5: September 2020

Finally, lung pulse is the visualization of the rhythmic movement of the visceral pleura against the stationary parietal pleura resulting from cardiac pulsations through an airless and motionless left lung due to right mainstem intubation.34,35 This technique was found to be 93% sensitive and 100% specific for detecting right mainstem intubation.34 The lung pulse may be particularly valuable for differentiating a mainstem intubation from a pneumothorax, as both would demonstrate unilateral absence of lung sliding. PROPOSED ALGORITHM We propose a rapid POCUS algorithm for confirming intubation in COVID-19 patients (Figure). First, transtracheal POCUS can be used to identify endotracheal vs esophageal

Figure. POCUS algorithm for confirming intubation in COVID-19 patients.

1043

Western Journal of Emergency Medicine

Gottlieb et al.

POCUS for Intubation Confirnation of COVID-19 Patients intubation. If there is concern with regard to location, secondary findings (eg, lung sliding) can be used. After confirming the endotracheal location, bilateral lung sliding or diaphragmatic excursion should be used to identify whether a mainstem intubation has occurred. If there is ambiguity regarding this, lung pulse can be used to differentiate unilateral lung sliding from a pneumothorax vs a mainstem intubation. If a mainstem intubation is suggested, the clinician should slowly withdraw the ETT while visualizing the contralateral lung for the reappearance of lung sliding. This algorithm has not been prospectively validated and future studies should assess the accuracy and safety of this approach.

4. Wu Z, McGoogan JM. [Ahead of Print]. Characteristics of and important lessons from the coronavirus disease 2019 (COVID-19) outbreak in China: summary of a report of 72314 cases from the Chinese Center for Disease Control and Prevention. JAMA. 2020. In press. 5. Arentz M, Yim E, Klaff L, et al. Characteristics and outcomes of 21 critically ill patients with COVID-19 in Washington state. JAMA. 2020;323(16):1612-4. 6. Centers for Disease Control and Prevention. Interim Infection Prevention and Control Recommendations for Patients with Suspected or Confirmed Coronavirus Disease 2019 (COVID-19) in Healthcare Settings. Available at: https://www.cdc.gov/ coronavirus/2019-ncov/hcp/infection-control-recommendations.html. Accessed July 16, 2020.

CONCLUSION Post-intubation ETT confirmation of COVID-19 patients presents a significant risk of exposure to providers and may be more limited by PPE. We propose the integration of POCUS into the intubation confirmation pathway and present a novel algorithm. Future studies should assess the impact of this on provider safety and the diagnostic accuracy of the protocol compared with current methods.

7. Canelli R, Connor CW, Gonzalez M, et al. Barrier enclosure during endotracheal intubation. N Engl J Med. 2020;382:(1857-8). 8. Takeda T, Tanigawa K, Tanaka H, et al. The assessment of three methods to verify tracheal tube placement in the emergency setting. Resuscitation. 2003;56(2003):153-7. 9. Kelly JJ, Eynon C, Kaplan JL, et al. Use of tube condensation as an indicator of endotracheal tube placement. Ann Emerg Med. 1998;31(5):575-8. 10. van Doremalen N, Bushmaker T, Morris DH, et al. Aerosol and surface stability of SARS-CoV-2 as compared with SARS-CoV-1. N Engl J Med. 2020;382(16):1564-7.

Address for Correspondence: Michael Gottlieb, MD, Rush University Medical Center, Department of Emergency Medicine, 1750 West Harrison Street, 108 Kellogg, Chicago, IL 60612. Email: michaelgottliebmd@gmail.com.

11. Buonsenso D, Pata D, Chiaretti A. COVID-19 outbreak: less stethoscope, more ultrasound. Lancet Respir Med. 2020;8(5):e27. 12. MacLeod BA, Heller MB, Gerard J, et al. Verification of endotracheal

tube placement with colorimetric end-tidal CO2 detection. Ann Emerg Med. 1991;20(3):267-70. 13. Tanigawa K, Takeda T, Goto E, et al. Accuracy and reliability of the self-inflating bulb to verify tracheal intubation in out-of-hospital cardiac arrest patients. Anesthesiology. 2000;93(6):1432-6. 14. Tanigawa K, Takeda T, Goto E, et al. The efficacy of esophageal detector devices in verifying tracheal tube placement: a randomized cross-over study of out-of-hospital cardiac arrest patients. Anesth Analg. 2001;92(2):375-8. 15. Brunel W, Coleman DL, Schwartz DE, et al. Assessment of routine chest roentgenograms and the physical examination to confirm endotracheal tube position. Chest. 1989;96(5):1043-5. 16. Sitzwohl C, Langheinrich A, Schober A, et al. Endobronchial intubation detected by insertion depth of endotracheal tube, bilateral auscultation, or observation of chest movements: randomised trial.

REFERENCES 1. Huang C, Wang Y, Li X, et al. Clinical features of patients

BMJ. 2010;341:c5943. 17. Geisser W, Maybauer DM, Wolff H, et al. Radiological validation

infected with 2019 novel coronavirus in Wuhan, China. Lancet.

of tracheal tube insertion depth in out-of-hospital and in-hospital

2020;395(10223):497-506. 2. World Health Organization. Situation Report 178. Available at:

emergency patients. Anaesthesia. 2009;64(9):973-7. 18. Gottlieb M, Holladay D, Burns KM, et al. Ultrasound for airway

https://www.who.int/emergencies/diseases/novel-coronavirus-2019/

management: an evidence-based review for the emergency clinician.

situation-reports/. Accessed July 16, 2020. 3. Guan WJ, Ni ZY, Hu Y, et al. Clinical characteristics of coronavirus

Am J Emerg Med. 2019;38(5):1007-13. 19. Gottlieb M, Burns K, Holladay D, et al. Impact of endotracheal tube

disease 2019 in China. N Engl J Med. 2020;382:1708-20.

Western Journal of Emergency Medicine

twisting on the diagnostic accuracy of ultrasound for intubation

1044

Volume 21, no. 5: September 2020

Gottlieb et al.

POCUS for Intubation Confirnation of COVID-19 Patients

confirmation. Am J Emerg Med. 2020;38(7):1332-4.

28. Weaver B, Lyon M, Blaivas M. Confirmation of endotracheal tube

20. Gottlieb M, Holladay D, Peksa GD. Ultrasonography for the

placement after intubation using the ultrasound sliding lung sign.

confirmation of endotracheal tube intubation: a systematic review and meta-analysis. Ann Emerg Med. 2018;72(6):627-36.

Acad Emerg Med. 2006;13(3):239-44. 29. Sim SS, Lien WC, Chou HC, et al. Ultrasonographic lung sliding

21. Lin MJ, Gurley K, Hoffmann B. Bedside ultrasound for tracheal tube

verification in pediatric emergency department and ICU patients: a

intubation. Resuscitation. 2012;83(3):307-12.

systematic review. Pediatr Crit Care Med. 2016;17(10):e469-76.

30. Ramsingh D, Frank E, Haughton R, et al. Auscultation versus

22. Gottlieb M, Holladay D, Nakitende D, et al. Variation in the accuracy

point-of-care ultrasound to determine endotracheal versus

of ultrasound for the detection of intubation by endotracheal tube

bronchial intubation: a diagnostic accuracy study. Anesthesiology.

size. Am J Emerg Med. 2019;37(4):706-9.

2016;124(5):1012-20.

23. Gottlieb M, Holladay D, Burns K, et al. Accuracy of ultrasound for

31. Sooragonda SG, Arora S, Jain D, et al. Lung sliding sign to detect

endotracheal intubation between different transducer types. Am J

endobronchial intubation in children: An observational feasibility trial.

Emerg Med. 2019;37(12):2182-5.

Eur J Anaesthesiol. 2020;37(2):143-5.

24. Chenkin J, McCartney CJ, Jelic T, et al. Defining the learning curve of

32. Hsieh KS, Lee CL, Lin CC, et al. Secondary confirmation of

point-of-care ultrasound for confirming endotracheal tube placement

endotracheal tube position by ultrasound image. Crit Care Med.

by emergency physicians. Crit Ultrasound J. 2015;7(1):14. 25. Chou EH, Dickman E, Tsou PY, et al. Ultrasonography for

2004;32(9 Suppl):S374-7. 33. Kerrey BT, Geis GL, Quinn AM, et al. A prospective comparison

confirmation of endotracheal tube placement: a systematic review

of diaphragmatic ultrasound and chest radiography to determine

and meta-analysis. Resuscitation. 2015;90:97-103.

endotracheal tube position in a pediatric emergency department.

26. Park SC, Ryu JH, Yeom SR, et al. Confirmation of endotracheal intubation by combined ultrasonographic methods in the emergency

Pediatrics. 2009;123(6):e1039-44. 34. Lichtenstein DA, Lascols N, Prin S, et al. The “lung pulse”: an

department. Emerg Med Australas. 2009;21(4):293-7.

early ultrasound sign of complete atelectasis. Intensive Care Med.

27. Sağlam C, Unlüer EE, Karagöz A. Confirmation of endotracheal tube position during resuscitation by bedside ultrasonography. Am J

2003;29(12):2187-92. 35. Alerhand S, Tsung JT. Unmasking the lung pulse for detection of

Emerg Med. 2013;31(1):248-50.

Volume 21, no. 5: September 2020

endobronchial intubation. J Ultrasound Med. 2020. In press.

1045

Western Journal of Emergency Medicine

Letter to the Editor

COVID-19: Implications for Advanced Care Planning and End-of-life Care Mishal Reja, MD Jay Naik, MD Payal Parikh, MD

Rutgers Robert Wood Johnson Medical School, Division of Internal Medicine, Department of Medicine, New Brunswick, New Jersey

Section Editor: Dan Mayer, MD Submission history: Submitted May 5, 2020; Revision received June 26, 2020; Accepted June 24, 2020 Electronically published July 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.48049 [West J Emerg Med. 2020;21(5)1046-1047.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. Dear Editor: It was 4 am when the hospital admitting medicine service phone rang. “Ten patients with suspected COVID-19 were sent from a nursing home; it’s possible that they all may need intensive care unit [ICU] beds. How many beds are available right now?” I will never forget the series of events that followed. The urgency was palpable as evidenced by the frenzy of navigating the emergency department, careful donning and doffing of personal protective equipment, and rapid-fire triaging of each patient. It was 6 am when several more patients from that same nursing home arrived. The nasal cannulas turned into non-rebreathers, which quickly transitioned to high-flow nasal cannulas. The next obvious step was intubation. But one question persisted in our minds: “Are we doing the right thing?”

a common event during COVID-19 illness. ICUs around the globe are being filled to and/or past capacity. Studies show that patients ≥ 65 years have a 3.7x greater risk of mortality, and pre-existing cardiovascular and cerebrovascular disease also contribute to increased mortality.2 The disease is likely to be fatal for elderly and frail individuals with pre-existing conditions. For these patients, hospitalization and aggressive interventions in critical care units are unlikely to improve quality of life or survival. In a pandemic, the escalation to critical care and aggressive, life-saving measures is rapid with little time for appropriate planning. It would be beneficial to implement early advanced care planning in the outpatient setting for high-risk patients to stay home with hospice care or home health services. Prior studies have shown that patients with outpatient palliative care consultations were 2.5 times more likely to enroll in hospice, and they had lower rates of aggressive medical interventions.3

INTRODUCTION The rapid global spread of coronavirus disease of 2019 (COVID-19) has resulted in considerable emotional and physical distress in a time of limited medical resources. As healthcare systems have been pushed to the brink, advanced care planning and end-of-life life discussions are of the utmost importance. Palliative care is at a unique vantage point to help treat symptomology and provide guidance. Due to resource limitations, we aim to outline pressing, palliative care needs from a critical care and emergency medicine standpoint.

Grief Considerations The COVID-19 pandemic has disrupted the grief process for families and friends who have experienced the passing of a loved one from COVID-19. Family visits are usually limited or prohibited, and funerals and burials are held remotely. Complicated grief, secondary traumatic stress, and moral distress is to be expected.4 We must also bear in mind that families may have had multiple losses and may be in social isolation from self-quarantine. Maladaptive psychological processing will likely exacerbate post-loss bereavement, exacerbating depression, anxiety, anger, blame, and helplessness. It will be especially important to connect families to resources and self-care practices that they will need.

Advanced Care Planning and End of Life Discussions Advanced care planning involves the process of having patients and families make decisions about their last phase of life prior to losing decisional capacity.1 Unexpected death is

Emerging Technology and Artificial Intelligence Family members of critically ill COVID-19 patients with a poor prognosis face challenging scenarios. Anecdotally, those who have been resistant to withdrawing aggressive

Western Journal of Emergency Medicine

1046

Volume 21, no. 5: September 2020

Reja et al.

COVID-19: Implications for Advanced Care Planning and End-of-life Care

medical care demonstrate a lack of understanding of the disease process combined with severe psychological distress, which is exacerbated by their inability to be at the bedside. Several modalities may help engage family members in a dialogue for advanced care planning. These conversations should take place in an outpatient setting by primary care physicians prior to the need for hospitalization for high-risk patients. Evidence-based communication educational curricula can be implemented to coach providers to have difficult conversations if palliative care is unavailable. Artificial intelligence and telehealth technology can assist palliative and primary care providers to monitor and treat end-of-life symptoms at home. Furthermore, mobile health apps have been shown to be successful in goals-of-care discussions for oncology patients,5 and these can be adopted for high- risk patients at risk for COVID-19, such as the elderly, those with multiple comorbidities, or those residing in nursing homes. Video messaging with patients and families is often used, and further research needs to be done in this area. CONCLUSION End-of-life discussions are a daunting task. However, effective and empathetic goals-of-care discussions before a crisis situation are particularly important.6 Telehealth is a valuable tool to facilitate these discussions, and further research in this area is needed.7 COVID-19 has resulted in high mortality and morbidity rates in at-risk populations, and it is imperative to facilitate these discussions early on during this pandemic.

Address for Correspondence: : Mishal Reja, MD, Rutgers Robert Wood Johnson Medical School, Division of Internal Medicine, Department of Medicine, 1 Robert Wood Johnson Place, New Brunswick, NJ 08901. Email: mreja8811@gmail.com. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: ÂŠ 2020 Reja et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

3. Schneiter MK, Karlekar MB, Crispens MA, et al. The earlier the better: the role of palliative care consultation on aggressive end of life care, hospice utilization, and advance care planning documentation among gynecologic oncology patients. Support Care Cancer. 2019;27(5):1927-34. 4. Wallace CL, Wladkowski SP, Gibson A, White P. Grief during the COVID-19 pandemic: considerations for palliative care providers. J Pain Symptom Manage. 2020;60(1):e70-6. 5. Kim H, Goldsmith JV, Sengupta S, et al. Mobile health application and e-Health literacy: opportunities and concerns for cancer patients and caregivers. J Cancer Educ. 2019;34(1):3-8. 6. Bernacki RE, Block SD, American College of Physicians High Value Care Task Force. Communication about serious illness care

REFERENCES

goals: a review and synthesis of best practices. JAMA Intern Med. 2014;174(12):1994-2003.

1. Lum HD, Sudore RL, Bekelman DB. Advance care planning in the elderly. Med Clin North Am. 2015;99(2):391-403.

7. Menon PR, Stapleton RD, McVeigh U, et al. Telemedicine as a tool to provide family conferences and palliative care consultations in

2. Du RH, Liang LR, Yang CQ, et al. Predictors of mortality for patients with COVID-19 pneumonia caused by SARS-CoV-2: a prospective

critically ill patients at rural health care institutions: a pilot study. Am J

cohort study. Eur Respir J. 2020;55(5):2000524.

Hosp Palliat Care. 2015;32(4):448-53.

Volume 21, no. 5: September 2020

1047

Western Journal of Emergency Medicine

Brief Research Report

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2 * Alpert School of Medicine of Brown University, Department of Emergency Medicine, Rebecca Karb, MD, PhD*o Elizabeth Samuels, MD, MPH, MHS*o Providence, Rhode Island † Alpert School of Medicine of Brown University, Department of Internal Medicine, Rahul Vanjani, MD, MSc† † Providence, Rhode Island Catherine Trimbur, MD, MPH o Co-first authors Anthony Napoli, MD, MHL* Section Editor: Ioannis Koutroulis, MD Submission history: Submitted June 15, 2020; Revision received July 23, 2020; Accepted July 23, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48725

Introduction: The unfolding COVID-19 pandemic has predictably followed the familiar contours of well established socioeconomic health inequities, exposing and often amplifying preexisting disparities. People living in homeless shelters are at higher risk of infection with severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) compared to the general population. The purpose of this study was to identify shelter characteristics that may be associated with higher transmission of severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Methods: We conducted a cross-sectional assessment of five congregate shelters in Rhode Island. Shelter residents 18 years old and older were tested for SARS-CoV-2 from April 19–April 24, 2020. At time of testing, we collected participant characteristics, symptomatology, and vital signs. Shelter characteristics and infection control strategies were collected through a structured phone questionnaire with shelter administrators. Results: A total of 299 shelter residents (99%, 299/302) participated. Thirty-five (11.7%) tested positive for SARS-CoV-2. Shelter-level prevalence ranged from zero to 35%. Symptom prevalence did not vary by test result. Shelters with positive cases of SARS-CoV-2 were in more densely populated areas, had more transient resident populations, and instituted fewer physical distancing practices compared to shelters with no cases. Conclusion: SARS-CoV-2 prevalence varies with shelter characteristics but not individual symptoms. Policies that promote resident stability and physical distancing may help reduce SARS-CoV-2 transmission. Symptom screening alone is insufficient to prevent SARS-CoV-2 transmission. Frequent universal testing and congregate housing alternatives that promote stability may help reduce spread of infection. [West J Emerg Med. 2020;21(5)1048-1053.]

general population.1-3 Moreover, this population has a higher prevalence of baseline comorbidities that increase the risk of severe disease and mortality from SARS-CoV-2.4-7 While high rates of asymptomatic SARS-CoV-2 have been observed in homeless shelters, little is known about shelter-level risk factors and successful mitigation strategies. Many shelters have worked to comply with the US Centers for Disease Control and Prevention (CDC) recommendations to control transmission (eg, daily symptom screening and temperature checks).3 However, these mitigation strategies can be difficult

1048

Volume 21, no. 5: September 2020

Karb and Samuels et al.

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2

and costly to implement and have unclear benefits. To date, no study has examined the association of shelter characteristics with SARS-CoV-2 outbreaks. In this analysis, we describe the varying prevalence of SARS-CoV-2 infection in five congregate homeless shelters in Rhode Island as well as varying shelter characteristics and infection control practices. METHODS We conducted a cross-sectional assessment of congregate shelter residents 18 years of age and older staying in five shelters in Rhode Island, from April 19â&#x20AC;&#x201C;April 24, 2020. Testing occurred during the peak of new case identification in Rhode Island. All residents of each shelter were offered testing. At the time of testing, we measured temporal temperature and pulse oximetry and collected information on demographic characteristics, comorbidities (hypertension, diabetes, heart disease, immunosuppression), and viral symptoms. Testing was done at Shelter 5 prior to initiation of temperature and oxygen documentation. Shelter characteristics and infection control practices were assessed by structured telephone interview with shelter administrators. Of note, shelter residents testing positive for SARS-CoV-2 in Rhode Island were being isolated in a hotel with support coordinated by the Rhode Island Department of Health (RIDOH). This screening was performed in collaboration with RIDOH to identify and isolate positive shelter residents. We collected and managed data using REDCap (Vanderbilt, Nashville, TN). Nasopharyngeal swabbing was done by emergency physicians with training in appropriate nasopharyngeal swab technique. Tests were run on one of three available polymerase chain reaction (PCR) assays: Roche (specificity 99.8%, sensitivity 100%, Basel, Switzerland); Cepheid (specificity 99.2%, sensitivity 95.5%, Sunnyvale, CA); and Abbott (specificity 100%, sensitivity 93%, Chicago, IL). We used descriptive statistics to summarize participant and shelter characteristics. We compared the proportion of positive SARS-CoV-2 tests among shelters, demographic groups, medical comorbidities, and symptomatology using t-tests and Fisherâ&#x20AC;&#x2122;s exact tests using STATA (Statacorp, College Station, TX). The analysis was deemed exempt by the RIDOH Institutional Review Board. RESULTS Among 302 shelter residents across five shelters, 299 (99.0%) were tested for SARS-CoV-2; one person declined testing, and two declined to have their results included in the analysis. The overall case prevalence across all shelters was 11.7%. Approximately half of shelter residents were White (53%), about one quarter were Latinx (23%), and most were 40-64 years of age (61%, mean age 47.9 years of age) (Tables 1 and 2). More than a third reporting having asthma, chronic obstructive pulmonary disease, hypertension, diabetes, or heart disease (38%), with hypertension being the most Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? People living in congregate homeless shelters are at higher risk of infection with severe acute respiratory syndrome coronavirus 2 (SARSCoV-2). What was the research question? What are shelter-level risk factors and successful mitigation strategies that impact the spread of SARS-CoV-2? What was the major finding of the study? Resident stability and physical distancing measures may reduce SARS-CoV-2 transmission in congregate settings. How does this improve population health? Symptom screening is insufficient to prevent spread in congregate shelters. Universal testing and stable housing alternatives could reduce risk for this population.

prevalent comorbidity (23%) (Table 2). Demographic and shelter characteristics are shown in Table 1. SARS-CoV-2 prevalence in Shelters 1 and 4 was 21.6% and 35.3%, respectively, while all other shelters had no cases (Table 1). There were no differences in age, gender, or race between people testing positive and negative for SARSCoV-2 (Table 2). Only 20% of people testing positive (7/35) reported any symptoms; none had fever or hypoxia. There were no differences in the presence of symptoms between people testing positive and negative for SARS-CoV-2 (20.0% vs 14.0%, p = 0.34). People testing positive for SARS-CoV-2 had lower prevalence of comorbidities compared to people testing negative (20% SARS-CoV-2 positive vs 40% SARSCoV-2 negative, p = 0.02). Among participants with negative tests, 70.1% (185/264) had spent more than two weeks at their shelter, compared to 42.9% (15/35) of participants with positive tests (p<0.001). Regarding infection control practices, all five shelters required masks, performed daily temperature checks of clients and staff, provided onsite meals, and were open 24 hours (Table 1). Three shelters had stopped accepting new residents for at least two weeks prior to the study and had zero cases at time of testing. The shelter with the highest case positivity rate has several distinct characteristics compared to the other shelters (Table 1). The neighborhood of this shelter had higher census-tract population density compared to the

1049

Western Journal of Emergency Medicine

Karb and Samuels et al.

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2

Table 1. SARS-CoV-2 prevalence, participant and shelter characteristics, and infection control practices, by homeless shelter. Shelter All Number Tested

299

35 (11.7)

11 (21.6)

24 (35.3)

47.9 (18-85)

43.4 (18-67)

48.5 (20-72)

47.8 (25-76)

59 (20)

18 (35)

32 (67)

9 (13)

Black

59 (20)

11 (22)

17 (19)

5 (10)

17 (25)

9 (21)

White

160 (53)

18 (36)

50 (56)

35 (73)

36 (53)

21 (49)

SARS-CoV-2 +, n (%) Participants Characteristics Age, mean (range) Female, n (%)

46.7 (19-69) 53.7 (30-85)

Race, n (%)

American Indian/Alaska Native

10 (3)

1 (2)

3 (3)

3 (6)

3 (4)

Other/Unknown

70 (23)

21 (40)

19 (21)

5 (10)

12 (18)

13 (30)

Latino/a/x

68 (23)

14 (27)

15 (17)

11 (23)

12 (18)

16 (37)

Non-Latino/a/x

213 (71)

29 (57)

66 (74)

37 (77)

54 (79)

27 (63)

Other/Unknown

18 (6)

8 (15)

8 (9)

2 (3

Any comorbidities, n (%)

113 (38)

7 (13)

37 (42)

30 (63)

23 (34)

16 (37)

Any symptoms, n (%)

44 (15)

4 (8)

7 (8)

9 (19)

19 (28)

5 (12)

10,852

2,753

10,852

21,645

2,362

100

Staff and residents wear masks

Yes

Daily temperature checks

Yes

Daily symptom screenings

Daily

2x Daily

Daily

2x daily

Onsite meals offered

Yes

Sleeping spaces 6 feet apart

Yes

Open 24 hours

Yes

Daily education/updates

Yes

New residents allowed

Yes

Latino/a/x, n (%)

Shelter characteristics Census tract population density (number people per square mile) % of beds filled (previous night) % of population at shelter >14 days Infection control practices

*Data type not collected at this shelter. SARS-CoV-2, severe acute respiratory syndrome coronavirus 2.

neighborhoods of the other shelters. The resident population was also found to be more transient than that of other shelters, with only 58% (39/63) reporting staying at the shelter for more than two weeks. This low-threshold shelter has continued to keep its doors open to new residents throughout the pandemic, and given its limited capacity the shelter was unable to arrange sleeping areas at least six feet apart. DISCUSSION The range of asymptomatic prevalence of SARS-CoV-2 found in different shelters builds on growing data from other Western Journal of Emergency Medicine

cities and has important policy implications. Only one in five people with positive tests were symptomatic, which was not significantly different from those testing negative. Following initial CDC guidance, shelters have relied primarily on symptom screening to control spread of SARS-CoV-2. As this and other recent data have demonstrated, asymptomatic and pre-symptomatic transmission may be the predominant modes of SARS-CoV-2 spread in congregate settings, and thus symptom-guided identification and temperature screening are insufficient strategies to prevent SARS-CoV-2. Sheltering in place, wearing masks, and physical distancing may be at least

1050

Volume 21, no. 5: September 2020

Karb and Samuels et al.

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2

Table 2. Demographic and clinical characteristics of participants, by SARS-CoV-2 result. SARS-CoV-2 test All (N=299)

Positive (N=35)

Negative (N=264)

52 (17)

11 (31)

40 (15)

89 (30)

0 (0)

89 (34)

48 (16)

0 (0)

48 (18)

68 (23)

24 (69)

44 (17)

43 (14)

0 (0)

43 (16)

18-39

89 (30)

9 (25)

80 (31)

40-64

184 (61)

23 (66)

161 (61)

26 (9)

3 (9)

23 (8)

Female

59 (20)

9 (26)

50 (19)

Male

238 (80)

26 (74)

212 (80)

2 (1)

0 ()

2 (1)

P-value

Shelter, n (%)

< 0.001

Demographics Age, n (%)

>65

0.83

Gender, n (%)

Trans/other

0.77

Race, n (%) Black

59 (20)

7 (20)

52 (20)

White

160 (53)

17 (49)

142 (54)

Al/Alaska Native

10 (3)

1 (3)

9 (3)

Other/Unknown

70 (23)

10 (29)

60 (23)

0.88

Latino/a/x, n (%) Latino/a/x

68 (23)

6 (17)

62 (23)

Non-Latino/a/x

213 (71)

22 (63)

191 (73)

Other/Unknown

18 (6)

7 (20)

11 (4)

>14 days at current shelter

200 (67)

15 (43)

185 (70)

Slept elsewhere

48 (16)

6 (17)

42 (16)

Unknown

51 (17)

14 (40)

37 (14)

Any comorbidity

112 (38)

7 (20)

105 (40)

Asthma/COPD

52 (17)

1 (3)

51 (29)

Hypertension

68 (23)

4 (11)

64 (24)

Diabetes

32 (11)

2 (6)

30 (11)

Heart disease

23 (8)

2 (6)

21 (8)

Temperature, mean (SD)

97.1 (0.05)

96.8 (0.86)

97.2 (0.86)

0.06

Oxygen saturation, mean (SD)

96.7 (0.12)

97 (0.39)

96.7 (0.13)

0.59 0.34

0.001

Transiency, n (%) < 0.001

Clinical Comorbidities, n (%) 0.02

Symptoms, n (%) Any symptoms

44 (14.7)

7 (20)

37 (14)

Fever

5 (2)

1 (3)

4 (2)

Cough

15 (5)

2 (6)

13 (5)

Shortness of breath

11 (4)

0 (0)

11 (4)

Body aches

5 (2)

2 (6)

3 (1)

Nausea, vomiting, or diarrhea

15 (5)

2 (6)

13 (5)

Loss of smell or taste

9 (3)

2 (6)

7 (3)

SARS-CoV-2, severe acute respiratory syndrome coronavirus 2; COPD, chronic obstructive pulmonary diseas; SD, standard deviation.

Volume 21, no. 5: September 2020

1051

Western Journal of Emergency Medicine

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2 partially effective in shelters with lower occupancy and a more stable resident population. However, low-threshold shelters provide an important safety net for many people experiencing homelessness, and shelters cannot be closed without readily available supportive housing. Shelter characteristics and practices may play an important role in transmission and have not been adequately studied. This study found that shelters with more transient residents and operating at near-full capacity had higher prevalence rates. Shelters that limited the influx of new residents were able to prevent outbreaks; however, this practice comes at the cost of limiting access to individuals and families experiencing homelessness. The shelter with the highest prevalence of the virus was located in the most densely populated neighborhood. This finding underscores the potential importance of the surrounding neighborhood and indicates that future studies should examine area characteristics such as land-use mix and access to bus or train services. To maintain low-threshold shelter services, use of frequent universal testing regardless of symptoms and ability to isolate people testing positive will be necessary for reducing SARSCoV-2 transmission among people experiencing homelessness. Housing stability has been previously shown to improve health outcomes among people experiencing homelessness,8 and the importance of stable housing is readily apparent during the COVID-19 pandemic. Housing alternatives to large congregate shelters can be used to reduce transmission. This includes expansion of small, non-congregate shelter capacity as well as permanent supportive housing, which allows for more resident stability and improved physical isolation capabilities. LIMITATIONS Although this study is the first to assess shelter-level characteristics, it was limited by the cross-sectional design as well as the small number of shelters. First, at the time of our study, many shelter residents who had tested positive were already housed in a local hotel, which likely led to an underestimate of true prevalence of SARS-CoV-2 among people experiencing homelessness. Second, testing done at the shelters with more transient residents only reflects the residents present on the night of testing, not the entire group that intermittently uses shelter services. Those shelters were more likely to have residents test positive; thus, an inability to assess the full complement of those sheltersâ&#x20AC;&#x2122; residents likely dilutes the overall prevalence of positivity when all shelters are examined in aggregate. Third, shelter staff are also a potential risk to residents, particularly if they work at multiple shelters/organizations or have other personal exposures. Our analysis does not account for potential risk posed by staff. Fourth, PCR tests used may have a 20-30% false negative rate and are only adequate during viral shedding.9 Furthermore, tests were conducted in three separate labs using different PCR assays with varying sensitivities/specificities. This may have impacted Western Journal of Emergency Medicine

Karb and Samuels et al.

uniformity of test results. Lastly, since this was a crosssectional analysis we were not able to determine whether the asymptomatic positive cases were actually presymptomatic. CONCLUSION A growing body of literature has demonstrated that asymptomatic and presymptomatic spread of SARS-CoV-2 may be significant.10,11 The results of this study further underscore that symptom screening and temperature monitoring are insufficient means to mitigate transmission of SARS-CoV-2 in homeless shelters and other congregate settings. Shelter characteristics such as population density, the capacity to maintain population stability, and the ability and resources to implement preventative practices such as physical distancing, may be partially effective in mitigating disease spread. In order to prevent SARS-CoV-2 transmission while continuing to provide essential, accessible services to people experiencing homelessness, there is a need for frequent universal testing, infection control support at homeless shelters, and expanded availability of permanent housing. ACKNOWLEDGEMENTS For their collaboration and assistance, the authors would like to acknowledge and thank the Rhode Island Health State Laboratory, the Lifespan Clinical Microbiology Lab, Dr. Utpala Bandy, Dr. Philip Chan, Dr. Kim Chapin, Amy Ferguson, Deborah Garneau, MA, Sari Greene, Dr. Ewa King, Dr. Louis Marchetti, Marilyn McAllister, and Dr. James McDonald. For their assistance with test conduction and data collection, we thank Dr. Jay Baruch, Dr. Michelle Breda, Natalie Chadwick, PA, Dr. Hannah Chason, Dr. Rachel Cooper, Dr. Catherine Cummings, Dr. Giovanna Deluca, Dr. Elizabeth Goldberg, Dr. Meredith Horton, Dr. Ilse Jenouri, Dr. Josh Kaine, Dr. Anita Knopov, Dr. Larry Kogan, Dr. Victoria Leytin, Dr. Tracy Madsen, Dr. Christina Matulis, Dr. Caroline Meehan, Dr. Rory Merritt, Dr. Luke Messac, Dr. Michelle Myles, Dr. Elizabeth Nestor, Dr. David Portelli, Amy Riendeau, NP, Dr. Mahawa Sam, Dr. Jeremiah Schuur, Dr. Lynn Sweeney, and Dr. Michael Tcheyan. Address for Correspondence: Rebecca Karb, MD, PhD and Elizabeth Samuels, MD, Brown Emergency Medicine, 55 Claverick St., Providence, RI 02903. Email: rebecca_karb@brownphysicians. org and elizabeth_samuels@brown.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: ÂŠ 2020 Karb and Samuels et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http:// creativecommons.org/licenses/by/4.0/

1052

Volume 21, no. 5: September 2020

Karb and Samuels et al.

Homeless Shelter Characteristics and Prevalence of SARS-CoV-2

REFERENCES

6. Baggett TP, Chang Y, Singer DE, et al. Tobacco-, alcohol-, and drug-attributable deaths and their contribution to mortality disparities

1. Baggett TP, Keyes H, Sporn N, et al. Prevalence of SARS-CoV-2

in a cohort of homeless adults in Boston. Am J Public Health.

infection in residents of a large homeless shelter in Boston. JAMA.

2015;105(6):1189-97.

2020;323(21):2191-2 2. Mosites E, Parker EM, Clarke KEN, et al. Assessment of SARS-CoV-2

7. Schinka JA, Leventhal KC, Lapcevic WA, et al. Mortality and cause of death in younger homeless veterans. Public Health Rep.

infection prevalence in homeless shelters — four U.S. cities, March

2018;133(2):177-81.

27–April 15, 2020. MMWR Morbidity and Mortality Weekly Report. 8.

2020;69(17):521-2.

2015;105(7):1482-8.

among three affiliated homeless service sites — King County, 9.

Washington, 2020. MMWR Morbidity and Mortality Weekly Report.

www.medrxiv.org/content/10.1101/2020.04.05.20053355v2. Accessed

4. Jones CA, Perera A, Chow M,et al. Cardiovascular disease risk among

April 27, 2020.

the poor and homeless: What we know so far. Curr Cardiol Rev.

10. Arons MM, Hatfield KM, Reddy SC, et al. Presymptomatic SARSCoV-2 infections and transmission in a skilled nursing facility. N Engl J

5. Richardson S, Hirsch JS, Narasimhan M, et al. Presenting

Med. 2020;382(22):281-90.

characteristics, comorbidities, and outcomes among 5700 patients hospitalized with COVID-19 in the New York City area. JAMA.

11. Bai Y, Yao L, Wei T, et al. Presumed asymptomatic carrier transmission of COVID-19. JAMA. 2020;323(14):1406-7.

2020;323(20):2052-9.

Volume 21, no. 5: September 2020

Wikramaratna P, Paton RS, Ghafari M, et al. Estimating false-negative detection rate of SARS-CoV-2 by RT-PCR. 2020. Available at: https://

2020;69(17):523-6

2009;5(1):69-77.

Brown RT, Miao Y, Mitchell SL, et al. Health outcomes of obtaining housing among older homeless adults. Am J Public Health.

3. Tobolowsky FA, Gonzales E, Self JL, et al. COVID-19 Outbreak

1053

Western Journal of Emergency Medicine

Brief Research Report

Development and Usability Testing of a Web-based COVID-19 Self-triage Platform Justin D. Schrager, MD, MPH* Keke Schuler, PhD† Alexander P. Isakov, MD, MPH* David W. Wright, MD* Anna Q. Yaffee, MD, MPH* Kara L. Jacobson, MPH‡ Ruth M. Parker, MD* Craig Goolsby, MD, MEd§

*Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia † National Center for Disaster Medicine and Public Health, Department, Bethesda, Maryland ‡ Rollins School of Public Health, Department of Health Policy and Management, Atlanta, Georgia § Uniformed Services University of the Health Sciences, Department of Military and Emergency Medicine, Bethesda, Maryland

Section Editor: Nikhil Goyal, MD Submission history: Submitted May 15, 2020; Revision received June 10, 2020; Accepted July 24, 2020 Electronically published August 19, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48217

Introduction: The development and deployment of a web-based, self-triage tool for severe respiratory syndrome coronavirus 2 (COVID-19 disease) aimed at preventing surges in healthcare utilization could provide easily understandable health guidance with the goal of mitigating unnecessary emergency department (ED) and healthcare visits. We describe the iterative development and usability testing of such a tool. We hypothesized that adult users could understand and recall the recommendations provided by a COVID-19 web-based, self-triage tool. Methods: We convened a multidisciplinary panel of medical experts at two academic medical schools in an iterative redesign process of a previously validated web-based, epidemic screening tool for the current COVID-19 pandemic. We then conducted a cross-sectional usability study over a 24-hour period among faculty, staff, and students at the two participating universities. Participants were randomly assigned a pre-written health script to enter into the self-triage website for testing. The primary outcome was immediate recall of website recommendations. Secondary outcomes included usability measures. We stratified outcomes by demographic characteristics. Results: A final sample of 877 participants (mean age, 32 years [range, 19-84 years]; 65.3% female) was used in the analysis. We found that 79.4% of the participants accurately recalled the recommendations provided by the website. Almost all participants (96.9%) found the website easy to use and navigate. Conclusion: Adult users of a COVID-19 self-triage website, recruited from an academic setting, were able to successfully recall self-care instructions from the website and found it user-friendly. This website appears to be a feasible way to provide evidence-based health guidance to adult patients during a pandemic. Website guidance could be used to reduce unnecessary ED and healthcare visits. [West J Emerg Med. 2020;21(5)1054-1058.]

INTRODUCTION The severe respiratory syndrome coronavirus 2 (COVID-19 disease) pandemic has led to more than 287,399 deaths worldwide.1 Hospitals in the United States are experiencing surges of patients and critical shortages of personal protective 1054

Volume 21, no. 5: September 2020

Schrager et al. equipment (PPE).2 Because most hospitals routinely operate near capacity, these patient surges could lead to inadequate patient care and an insufficient number of beds, treatment spaces, and healthcare workers to evaluate patients.3 Each unnecessary patient visit creates additional potential COVID-19 exposures to healthcare workers and other vulnerable patients in the hospital, and consumes scarce PPE. A key strategy to mitigate the healthcare system burden is “forward triage.”4 For this strategy to work, a patient must be empowered to make informed decisions about the need for emergency care services, with the ultimate goal of safely reducing unnecessary emergency department (ED) visits.4 A web-based, free, educational platform, can fulfill this strategy by providing immediate instructions for next steps in care and information on potential testing sites, thereby preventing unnecessary healthcare worker exposure and exposure of other vulnerable patients to COVID-19, while conserving PPE.4,5 Web-based self-triage was first developed and deployed during the 2009 novel influenza A (H1N1) virus pandemic by researchers at Emory University. The evidence-based Strategy for Offsite Rapid Triage (SORT) was rapidly validated and integrated into web platforms hosted on the US Department of Health & Human Services flu.gov website as well as Microsoft Corporation’s H1N1 Response Center.6 The sites recorded more than 670,000 completed self-evaluations over five months during the outbreak. Success with the H1N1 self-triage website provided the groundwork for a public COVID-19 self-triage website (www.c19check.com). The original SORT algorithm has been updated to reflect the best available evidence about COVID-19 as shared by the Centers for Disease Control and Prevention and the World Health Organization, and was

A Web-based COVID-19 Self-triage Platform built iteratively with multidisciplinary input from experts in infectious disease, emergency medicine, pre-hospital medicine, epidemiology, and health literacy. On the website the user enters age, ZIP code, comorbidities, and symptoms, and the algorithm classifies risk as low, intermediate, or high (Figure). The website then provides the user CDC-based recommendations and level of risk as well as ZIP Code-specific local health department information if the user chooses to enter that piece of personal information. Widespread adoption of an effective self-triage tool has the potential to mitigate unnecessary ED visits. Objective The primary objective of this study was to determine whehter participants understood and recalled recommendations provided by a COVID-19 web-based, self-triage tool. METHOD Design Investigators at Emory University and Uniformed Services University of the Health Sciences performed usability testing of the COVID-19 self-triage website. The institutional review boards approved this study and waived the consent requirement; however, an assent script with affirmation was used. On March 19-20, 2020, staff, faculty, and students at both institutions were recruited via email for voluntary study participation. Participants received an email with instructions and a link to the self-triage website. After completing demographic information, each participant was randomly assigned a pre-written patient “script” that the participant referred to when using the COVID-19 checker website. Once participants inputted scripted information into the COVID-19 checker website and received

Figure. Visual abstract of user interface and experience as well as usability testing.

Volume 21, no. 5: September 2020

1055

Western Journal of Emergency Medicine

Schrager et al.

A Web-based COVID-19 Self-triage Platform a recommendation, they were directed to complete the userfeedback form. This form contained questions about their understanding of the recommendations, and their subjective experiences using the website. We excluded participants’ responses if they did not complete all fields in the website or the userfeedback form. The website is currently designed to triage adult patients. To test the software, some participants were randomly assigned clinical scripts with patients younger than 18 years old. These cases were not allowed to progress to the clinical questions or website guidance and were therefore removed prior to endpoint analysis. Setting The study occurred virtually with participants at Emory University and Uniformed Services University of the Health Sciences. Participants Staff, students, and faculty at Emory University and Uniformed Services University of the Health Sciences participated in the study. Main Outcome Measures We assessed participants’ ability to immediately recall the specific self-care instructions provided by the site, scored as either correct or incorrect. Self-care instructions were specific and nuanced, depending on the symptoms and comorbidities of the scripted case provided. Secondary outcomes were participant feedback about website usability. RESULTS A total of 926 participants enrolled in the study. One participant was removed for being younger than 18 years old, and participants who received pediatric scripts (n = 46) or missing scripts information (n = 2) were removed from analysis, leaving 877 participants (mean age, 32 years [range, 19-84 years]; 65.3% female) in the final analyses (Table). We found that 79.4% of the participants accurately recalled the instructions and guidance provided by the website, and 96.9% of the participants reported that the website was “easy” or “very easy” to use (responses of 4-“easy” or 5-“very easy” on a Likert scale). Responses were further broken down based on demographic characteristics. DISCUSSION This study demonstrates the feasibility of a web-based, selftriage tool to provide information to adults seeking guidance about COVID-19 symptoms and next steps. Approximately four out of five participants correctly recalled and identified the website’s recommendations, and nearly all participants described the website as easy to use. The sampling methodology allowed for rapid feedback of the website, and free-text suggestions were expeditiously incorporated into the website. Suggested modifications were Western Journal of Emergency Medicine

incorporated the day after study enrollment, and the website was launched for public use the following day. Using the prior H1N1 website experience, the current COVID-19 algorithm was created, rapidly iterated, adapted, tested, and electronically deployed within days of sustained community transmission in the US, and well before the anticipated zenith of COVID-19 in the US.1 This suggests that it is also feasible to maintain and quickly update this platform for future pandemic response. This project was conducted collaboratively between Emory University and the private company Vital Software, Inc., an innovation development partner of Emory University that provided the technical expertise and hosting at no cost. Thus far, there is high public demand for this self-triage tool. Between March 26–May 14, 2020, C19check.com has hosted 766,574 unique visitors and completed 395,895 selfassessments. In comparison, there were 320,333 unique visits to the H1N1 self-triage website during its first three months. The self-triage algorithm will be updated in real time as new guidance and data are published by national and international public health experts, and in continued consultation with health literacy experts. Future studies will validate the algorithms and test user understanding of self-assessment instructions, adherence to instructions, and intent-to-use healthcare resources before and after self-triage within the wider population of website users. Website data will also be used to map epidemiologic patterns of disease and symptoms and will continue to expand upon the list of 15 foreign-language translations available. LIMITATIONS The participants were students and faculty at two academic institutions and thus the results cannot be applied to the general population due to mismatched health literacy and recall capacity. Given the exigencies of the COVID-19 pandemic, these sites were chosen since a large number of participants could be recruited rapidly, and the free-text feedback from medicallyoriented participants was instructive for the design team. For this reason, we chose to capture a mix of free-text usability feedback and Likert-scale usability scores rather than a validated usability scale for the secondary outcome. This study tested immediate recall of instructions, rather than lasting recall of instructions, as the website was designed to be used by a person actively making the decision to seek medical care or not, and the instructions would be less relevant at a later date. Additionally, the self-care instructions were nuanced and specific by design, which could have negatively impacted recall success depending on the comorbidities and symptoms in the user’s assigned scripted case. Further, the scripted nature of the cases could also have had a negative impact on recall success. This study did not seek to identify adherence to the instructions (eg, behavior change) since the participants were not actually using their own real patient data. The study did not assess the outcomes of actual COVID-19 patients following recommendations from the site. 1056

Volume 21, no. 5: September 2020

Schrager et al.

A Web-based COVID-19 Self-triage Platform

Table. Sample demographic characteristics of 877 participants who completed COVID-19 checker testing and user-feedback responses by sample characteristics. No. (%) of participants recalled Sample demographic characteristics* No. (%) of participants correctly within characteristic Gender Female

573 (65.3)

467 (81.5)

Male

273 (31.1)

213 (78.0)

White

593 (67.6)

487 (82.1)

Asian

138 (15.7)

97 (70.3)

64 (7.3)

51 (79.7)

Race

Black or African American Native American or Alaska Native

2 (0.2)

0 (0)

Hawaiian or other Pacific Islander

1 (0.1)

1 (100)

Ethnicity Hispanic

67 (7.6)

59 (88.1)

779 (88.8)

620 (79.6)

Student, full-time

496 (56.5)

407(82.2)

Employed, full-time

304 (34.7)

238 (78.3)

Non-Hispanic Employment

Student, part-time

7 (0.8)

5 (71.4)

20 (2.3)

15 (75.0)

Homemaker

5 (0.6)

4 (80.0)

Caregiver

2 (0.2)

1 (50.0)

Full-time volunteer

2 (0.2)

2 (100)

154 (17.6)

124 (70.5)

115(13.1)

92 (80.0)

Employed, part-time

Annual Income $0 $1 to $9,999 $10,000 to $24,999

86 (9.8)

67 (77.9)

$25,000 to $49,999

135 (15.4)

115 (85.2)

$50,000 to $74,999

68 (7.8)

52 (76.5)

$75,000 to $99,999

65 (7.4)

52 (80.0)

$100,000 to $149,999

61 (7.0)

47 (77.0)

$150,000 and greater

60 (6.8)

49 (81.7)

Bachelor degree

471 (53.7)

383 (81.3)

Master degree

164 (18.7)

128 (78.0)

Doctorate degree

Highest Education

155(17.7)

128 (82.6)

Professional degree

26 (3.0)

20 (76.9)

Some college credit, no degree

11 (1.3)

9 (81.8)

High school graduate

7 (0.8)

5 (71.4)

Associate degree

4 (0.5)

1 (25.0)

1 (0.1)

0 (0)

Some high school, no diploma *Categories: Unknown, prefer not to answer, and other are not listed.

CONCLUSION This study demonstrates that adult users in an academic setting can correctly identify recommended Volume 21, no. 5: September 2020

care instructions from a self-triage website during a pandemic. Study participants found the website userfriendly. The website was adapted from a pre-existing, 1057

Western Journal of Emergency Medicine

Schrager et al.

A Web-based COVID-19 Self-triage Platform self-triage algorithm in an iterative, expeditious manner. To date, there has been high demand for the website, and it has potential to provide users valuable health information and mitigate unnecessary ED visits. Limiting unnecessary healthcare visits will benefit both patients and healthcare workers by reducing COVID-19 exposure while conserving scarce resources.

Address for Correspondence: Justin Schrager, MD, MPH, Emory University School of Medicine, Department of Emergency Medicine, 531 Asbury Circle, Annex N340, Atlanta, Georgia 30322. Email: jschrag@emory.edu.

REFERENCES

Copyright: ÂŠ 2020 Schrager et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

1. World Health Organization. Coronavirus disease 2019 (COVID-19) Situation Report â&#x20AC;&#x201C; 114. 2020. Available at: https://www.who.int/docs/ default-source/coronaviruse/situation-reports/20200513-covid-19sitrep-114.pdf?sfvrsn=17ebbbe_4. Accessed May 14, 2020. 2. Giwa AL, Desai A, Duca A. Novel 2019 coronavirus SARS-CoV-2 (COVID-19): an updated overview for emergency clinicians. Emerg Med

5. Greenhalgh T, Wherton J, Shaw S, et al. Video consultations for

Pract. 2020;21(5):1-28.

Covid-19. BMJ. 2020;368:m998.

3. Willan J, King AJ, Jeffery K, et al. Challenges for NHS hospitals during

6. Kellermann AL, Isakov AP, Parker R, et al. Web-based self-triage of

covid-19 epidemic. BMJ. 2020;368:m1117. 4. Hollander JE and Carr BG. Virtually perfect? Telemedicine for Covid-19.

Emerg Med. 2010;56(3):288-94.e6.

N Engl J Med. 2020;382:1679-81.

Western Journal of Emergency Medicine

influenza-like illness during the 2009 H1N1 influenza pandemic. Ann

1058

Volume 21, no. 5: September 2020

Review Article

The Impact of COVID-19 on Healthcare Worker Wellness: A Scoping Review Jacob Shreffler, PhD* Jessica Petrey, MSLS† Martin Huecker, MD*

*University of Louisville, Department of Emergency Medicine, Louisville, Kentucky † University of Louisville, Kornhauser Library, Louisville, Kentucky

Section Editor: Patrick J. Maher, MD Submission history: Submitted June 12, 2020; Revision received June 28, 2020; Accepted July 24, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48684

At the heart of the unparalleled crisis of COVID-19, healthcare workers (HCWs) face several challenges treating patients with COVID-19: reducing the spread of infection; developing suitable short-term strategies; and formulating long-term plans. The psychological burden and overall wellness of HCWs has received heightened awareness in news and research publications. The purpose of this study was to provide a review on current publications measuring the effects of COVID-19 on wellness of healthcare providers to inform interventional strategies. Between April 6–May 17, 2020, we conducted systematic searches using combinations of these keywords and synonyms in conjunction with the controlled vocabulary of the database: “physician,” “wellness, “wellbeing,” “stress,” “burnout,” “COVID-19,” and “SARS-CoV-2.” We excluded articles without original data, research studies regarding the wellness of non-healthcare occupations or the general public exclusively, other outbreaks, or wellness as an epidemic. A total of 37 studies were included in this review. The review of literature revealed consistent reports of stress, anxiety, and depressive symptoms in HCWs as a result of COVID-19. We describe published data on HCW distress and burnout but urge future research on strategies to enhance HCW well-being. [West J Emerg Med. 2020;21(5)1059-1066.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. INTRODUCTION The COVID-19 pandemic has resulted in significant burdens globally. Detrimental effects include high rates of infection and death, financial hardships faced by individuals, stress related to known and particularly unknown information, and fear of the uncertainty regarding continued impact. Healthcare workers (HCWs), at the heart of the unparalleled crisis of COVID-19, face challenges treating patients with COVID-19: reducing the spread of infection; developing suitable short-term strategies; and formulating long-term plans. HCWs must also continue to successfully treat non-COVID patients and maintain personal responsibilities, including taking care of their families and themselves. The psychological burden and overall wellness Volume 21, no. 5: September 2020

of HCWs has received heightened awareness, with research continuing to show high rates of burnout, psychological stress, and suicide.1 HCWs experience emotional exhaustion, which may lead to medical errors, lack of empathy in treating patients, lower productivity, and higher turnover rates.2 The ability of HCWs to adequately cope with stressors is important for their patients, their families, and themselves. Providers vary in levels of psychological resilience, the ability to positively adapt to adversity to protect themselves from stress.3 Prior to COVID-19, wide-ranging research had established the multifactorial nature of stressors in healthcare: electronic health record duties; insurance and billing issues; any patient dissatisfaction; and balancing busy work-life schedules.4 HCWs must continue to balance these existing obstacles to wellness while facing the unique challenges of a pandemic. Literature from severe acute respiratory syndrome and Middle East respiratory syndrome can provide insight on the stress, trauma, psychological morbidities, and successful interventions, but the body of evidence for the impact

1059

Western Journal of Emergency Medicine

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness COVID-19 on HCW wellness is evolving alongside the pathogen. The purpose of this study was to provide a review of current publications measuring the effects of COVID-19 on wellness of the HCWs to inform interventional strategies. METHODS Between April 6–May 17, 2020, we conducted systematic searches in PubMed, Embase, Cochrane, Clinical Key, and Web of Science using combinations of these keywords and synonyms in conjunction with the controlled vocabulary of the database: “physician,” “wellness,” “wellbeing,” “stress,” “burnout,” “COVID-19,” and “SARSCoV-2.” Results were filtered to English-language publications, retrieving a total of 107 references. We examined references in included papers and relevant excluded papers for additional studies, and a non-systematic search in Google Scholar was conducted as well. After those selections were added and duplicates were removed, 185 distinct references remained for screening. To reduce risk of bias, titles and abstracts were screened for eligibility by two independent reviewers, with a third available in the event of disagreements. Papers presenting original data regarding the evaluation or management of physicians’ well-being during the COVID-19 pandemic were included for full analysis. Some publications indexed as correspondence did contain data, so article type was not an automatic exclusion criterion, nor was study design and quality of methodology. While papers on infection control practices, personal protective equipment (PPE), or wellness in other types of HCWs were not actively sought out, any retrieved by the strategies were retained. We excluded articles without original data, research studies regarding the wellness of non-healthcare occupations or the general public exclusively, other outbreaks, or wellness as an epidemic (Figure). RESULTS We included 37 studies in this review. Multiple themes emerged from the current literature on how COVID-19 has impacted HCW wellness. The majority of studies focused on the psychological impact of COVID-19, including stress and anxiety measurements. Some evaluated burnout and sleep quality. A small portion of the studies used qualitative methodology. We have provided a summary of the articles below. Stress, Fear, Anxiety, Depression In light of the many known and unknown effects of COVID-19, exploration of stress, fear, anxiety and symptoms of depression were prevalent in the included studies, with many focusing on frontline HCWs. Frontline Workers Researchers assessed anxiety levels in 512 frontline healthcare workers in China, finding a prevalence of Western Journal of Emergency Medicine

12.5%.5 The authors found HCWs who had direct contact with COVID-19 patients were at higher risk for anxiety.5 Frontline workers were also a focus by Lu et al in 2299 HCWs (2042 medical staff and 257 administrative staff). The authors found that medical staff had greater fear, anxiety, and depression levels than administrative staff. Additionally, the investigators found that HCWs working on frontlines in departments more impacted by COVID-19 (ie, emergency department, intensive care unit, infections disease) were at greater risk for anxiety and depression and psychological disorder.6 A total of 5062 HCWs were surveyed by Zhu et al to measure psychological impact of COVID-19.7 The authors measured stress, depression, and anxiety, discovering that 29.8% of respondents reported stress, 24.1% reported anxiety, and 13.5% reported depression. Women, individuals with history of mental disorders, and HCWs with infected family members were more vulnerable to undesirable health consequences of stress, anxiety, and depression.7 Liu et al measured distress, anxiety, and symptoms of depression in 4679 Chinese HCWs.8 Results showed the prevalence of anxiety and distress was about 16% each; 34.6% of respondents experienced depressive symptoms. The investigators discovered that risk factors for developing the mental health concerns aforementioned included living alone, being a nurse, being on the frontline, and middle age.8 Li et al measured the vicarious traumatization phenomenon in three groups: frontline nurses, non-frontline nurses, and general population. Frontline nurses had lower levels of trauma than both the general public and non-frontline nurses. The authors hypothesized that frontline nurses have better training to deal with crisis.9 Similar findings were discovered in 470 HCWs in Singapore.10 Results showed nonmedical workers had greater anxiety and stress compared to medical workers. Among the 470 HCWs, 14.5% experienced anxiety with 7.7% experiencing levels of concern for posttraumatic stress disorder.10 Liang et al compared HCWs in COVID-19 associated departments to other HCWs. They found a significant portion of HCWs experienced clinically depressive symptoms, but no significant differences between frontline HCWs and non-frontline HCWs.11 A study by Cai et al measured the psychological impact of COVID-19 on 534 frontline medicalstaff members.12 The authors found that HCWs experienced anxiety about their own and their family’s safety (along with their patients) but maintained the professional obligation to effectively complete their work. The authors found that older staff had increased stress related to (lack of) PPE and longer work hours. Coping strategies used by the HCWs included adhering to strict protective measures, following isolation guidelines, and exhibiting a positive mindset.13 Guo et al studied 11,118 HCWs in China. Results showed that risk factors for anxiety and depression were being younger, employed as a nurse, and being a frontline

1060

Volume 21, no. 5: September 2020

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness

Figure. Process of systematic searches using combinations of “physician,” “wellness,” “wellbeing,” “stress,” “burnout,” “COVID-19,” and “SARS-CoV-2” to provide a scoping review on publications measuring the effects of novel coronavirus 2019 on wellness of healthcare workers.

HCW. Within the sample, about 5% experienced middle to high levels of anxiety and about 13.5% experience middle to high depression levels.13 Lai et al found that female gender predicted greater risk of psychological stress in a study that examined depression, anxiety, insomnia, and distress in 1257 HCWs.14 HCWs experienced high incidence of depression (50.4%), anxiety (44.6%), and insomnia (34%). The majority of HCWs reported distress (71.5%), with women, nurses, frontline workers, and those working in Wuhan, China, having higher negative health outcomes.14 Dai et al discovered that geographic location was a risk factor for distress in 4357 HCWs. The results showed that 39.1% of HCWs experienced distress; living in Wuhan, being isolated, worrying about family members and working on the frontline were risk factors for experiencing distress.15 HCWs were chiefly concerned about infection in colleagues or family Volume 21, no. 5: September 2020

members.15 Geographic location was also a risk factor in a study in Italy comparing stress and anxiety in healthcare workers (n = 167) to the general population (n = 186). Likelihood of exposure to disease (HCWs and individuals in highly affected Northern Italy) predicted increased stress and anxiety. Overall, HCWs reported higher levels of worry compared to the general population.16 In a letter to the editor, Du et al reported smartphone survey data on frontline HCWs in Wuhan. HCWs from two hospitals and one outreach team answered multiple questions during a five-day period. The outreach team members appeared more prepared psychologically, had more supplies, and had improved sleep, stress, and levels of depression compared to frontline workers. Fifty-nine percent of respondents had moderate to severe perceived stress, with 12.7% having at least mild depressive symptoms and 20.1%

1061

Western Journal of Emergency Medicine

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness having anxiety. Those at greatest risk were HCWs who felt less prepared, had less family support, felt less self-efficacy, perceived a higher level of stress, and those with poor sleep quality. Fear of self and colleague infection represented a top source of stress.17 Unspecified/Other Healthcare Workers Access to PPE was a key focus by Zhang et al, who surveyed 304 HCWs in Iran. The authors found that 28% of HCWs experienced anxiety, 20.1% experienced distress, and 30.6% experienced depression. Furthermore, the study revealed that access to PPE resulted in both improved physical health and job gratification and ultimately led to less distress among HCWs.18 Delgado et al measured HCW personal safety perception in 936 workers in Latin America. Overall, HCWs lacked sufficient PPE and felt limited support from human resources and public officials.19 Preparedness to fight COVID-19 was examined in 158 HCWs in England. The authors found that HCWs desired more actions (including proper education) to feel confident to fight COVID-19, particularly in the collection and management of samples.20 Suleiman et al conducted a similar study on preparedness for the COVID-19 outbreak, surveying 308 physicians in Jordan. Individuals with protocols in place and accessible PPE reported higher levels of readiness. Furthermore, 90.9% of respondents reported feelings of anxiousness regarding the transmission of the disease and fear of the increase of the volume of infected patients. The large majority (96.4%) of HCWs were worried about transmitting COVID-19 to loved ones.21 Chew et al measured stress and anxiety in HCWs in 906 HCWs in Singapore and India. The results showed that 48 (5.3%) HCWs faced moderate to very severe depression and 79 (8.7%) had moderate to extremely severe depression. Additionally, 54 (6%) HCWs experienced moderate to extremely severe stress or moderate to severe distress. After correcting for confounders, the authors noted an association between incidence of prior month physical symptoms and emotional distress during COVID-19.22 In Wuhan, China, Kang et al measured mental health and psychological wellbeing using a survey in 994 HCWs. The study revealed that 28.6% of the sample had moderate to severe mental disturbances, with young women affected the most. Within the study population, subjects who accessed mental health amenities had improved relationship between exposure risks and mental health.23 Jiang et al measured psychological impact by comparing self-efficacy and loneliness of 205 HCWs in Hubei, China. Medical staff with lower self-efficacy had higher likelihood of loneliness. The authors noted that individuals experiencing loneliness may choose undesirable coping tactics (eg, substance use).24 The protective effect of a committed relationship surfaced in 194 physicians surveyed in Oman. The researchers revealed that individuals who were married Western Journal of Emergency Medicine

and older experienced less stress compared to other HCWs. Additionally, the authors found that females may be more susceptible to stress.25 Some physician-specific studies occurred in the included literature. Chen et al surveyed pediatricians on outcomes of stress and anxiety. Of 105 respondents, 90.5% of the sample were female and 18.1% reported working in high-risk areas. The authors noted particularly high self-reported depression and anxiety during the COVID-19 outbreak.26 In a study completed by Xu et al, the researchers surveyed 60 surgical staff during a period of COVID-19 outbreak and compared them with a separate group of 60 surgical staff in a nonoutbreak period in China. The results showed that HCWs surveyed during the outbreak period had significantly higher levels of anxiety and depression.27 One researcher used the Beck Anxiety Inventory to measure anxiety in multiple sclerosis fellows in Iran. The authors had 14 respondents and only two individuals had mild levels of anxiety.28 A focused look at dentists and dental hygienists assessed the COVID-19 impact in Israel. Among the 338 surveyed, individuals with previous illness and those worried about infection from patients were inclined to higher levels of distress. HCWs in committed relationships and those with superior levels of self-efficacy reported less stress.29 Burnout Wu et al specifically measured burnout in 220 oncology medical staff working in Wuhan, China. Using the validated and widely deployed Maslach Burnout Inventory-Medical Personnel (MBI), they compared levels of burnout in frontline and other HCW groups. Frontline HCWs had significantly lower levels of burnout and were less worried about becoming ill compared to those in the “usual ward” group. The authors noted two possible explanations: frontline HCWs may perceive more control over their situation and may appreciate a closer proximity to decision-makers (with more timely provided information) compared with the other HCWs.30 Cao et al used the MBI to measure burnout and emotional distress in 37 HCWs. They found that the levels of burnout and emotional distress were not highly elevated within their sample. Connecting with family members via technology or telephone was the most prevalent coping mechanism. The study showed that 29.7% of the sample had issues obtaining proper sleep.31 Sleep Some researchers focused specifically on COVID-19’s impact on HCW sleep. Xiao et al surveyed 180 medical staff members on social support, anxiety, stress, self-efficacy, and sleep quality to determine the effects of COVID-19.32 The authors found that social support correlated significantly with both self-efficacy and quality of sleep. Anxiety and stress were significantly associated, leading to negative impacts on both self-efficacy and sleep. The authors recommended HCWs

1062

Volume 21, no. 5: September 2020

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness

to take advantage of support systems, including family and friends to stabilize emotions, share experiences, and maintain social connections, thus reducing anxiety intensities and enabling quality sleep.32 Huang and Zhao measured sleep, anxiety, and depressive symptoms in 2250 HCWs. The authors compared HCWsâ&#x20AC;&#x2122; results to individuals from the general population. Results showed that HCWs were more likely to experience poor quality sleep and develop psychological issues.33 Qi et al also measured sleep in their survey of 1306 (801 frontline) HCWs in China. The authors found that frontline HCWs had advanced anxiety, depression, and prevalence of sleep disturbances compared to non-frontline HCWs. Furthermore, the authors found that female frontline HCWs had higher prevalence of sleep disturbances compared with male frontline HCWs.34 Zhang W et al found that medical HCWs had higher levels of insomnia, anxiety, depression, somatization, and obsessivecompulsive symptoms compared to non-medical HCWs in 2182 respondents in China. Risk factors for worsened mental health included living in a rural area, being female, and having contact with infected COVID-19 patients.35 Finally, insomnia was measured in HCWs in China involved in the COVID-19 outbreak. Of the 1563 respondents, 36.1% reported insomnia symptoms. Insomnia risk factors included lower levels of education, working in a unit with isolation, being a physician, lack of support, having high levels of uncertainty, and being worried about infection. The authors called for interventions for insomnia for HCWs.36 Qualitative Approach Some researchers used qualitative methods to gain better insight into the impact of COVID-19 on HCW wellness. Liu et al interviewed nine nurses and four physicians in Hubei. Respondents described many challenges of COVID-19 including fear of infection, exhaustion, and working in a new context. Despite these challenges, the HCWs felt that they were fully responsible to care for their patients as it was part of their duty, demonstrating an immense vow to their profession. The authors noted that workplace safety including access to PPE was a top priority.37 Sun et al interviewed 20 nurses who provided care for COVID-19 patients in China. The study results indicated that anxiety and fear were prevalent in the early stages of the outbreak, leading to feelings of helplessness. The authors noted some healthy coping strategies, including team encouragement and rational thinking. Furthermore, the authors found that reflection and developing a sense of professional responsibility resulted in growth. Finally, the researchers discovered that the nurses experienced both negative and positive emotions concurrently.38 Sethi et al also used a qualitative approach to develop open-ended questions for 290 HCWs in Pakistan. They found that HCWs were anxious, overworked, and felt financially Volume 21, no. 5: September 2020

unstable. Furthermore, HCWs reported challenges in taking care of both their professional lives and their households.39 Healthcare Workers Who Became Ill This review did not intend to describe exposure, infection rate, or mortality of healthcare providers during the COVID-19 pandemic. Two publications that appeared in our literature search described providers who became infected with the virus, but we refer readers to reviews focused on this topic.40,41 Ran et al examined risk factors for HCWs who developed acute respiratory infection in Wuhan China. The authors found that longer hours, higher risk clinical setting, and suboptimal hand cleanliness were risk factors for infection.42 Researchers in Wuhan, China, studied 103 HCWs who had become infected with COVID-19. These HCWs answered questions on perceived cause of infection and psychological changes. Results showed that the large majority (84.5%) of HCWs who became infected felt it was due to their work setting, with nursesâ&#x20AC;&#x2122; top three perceived causes being suction care, swab collection, and other nursing requirements. Physicians perceived highest risk in physical examination and tracheal or manual ventilations. During the isolation period, 88.3% of these HCWs experienced stress or emotional deviations. The study showed that persons who were experiencing distress were apprehensive about their own health in addition to transmitting it to loved ones.43 DISCUSSION The review of literature revealed consistent reports of stress, anxiety, and depressive symptoms in HCWs as a result of COVID-19. Multiple studies confirmed significant anxiety regarding patient care in addition to the possibility of infecting their families.12,16,30 Access to appropriate PPE remains of paramount importance to help physicians feel physically safe. With sufficient PPE, individuals feel more protected from infection, which may lessen fear of infecting loved ones. Women and individuals in highrisk areas may have more negative psychological health outcomes. Furthermore, both individuals on the frontline and other HCWs are susceptible to distress and negative health outcomes including anxiety, poor quality sleep, and feelings of isolation. Interestingly, some frontline workers experienced better mental health outcomes. The sense of vocation / purpose in work, along with greater control of environment, could explain these findings. Given the relative novelty of this crisis, no published studies have collected data on interventions to improve psychological health and overall wellness for HCWs who face COVID-19 specific challenges. Suggestions to alleviate the burden on HCW mental health have been provided by researchers both for COVID-19 and in other crises. We found no studies that measured the same sample before and after

1063

Western Journal of Emergency Medicine

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness COVID-19 to determine how wellness or stress changed within the same individuals. Researchers could compare previous datasets on provider wellness to measure and quantify effects of COVID-19. Generally recognized for their emotional resilience, HCWs must now face additional layers of responsibilities and mental and physical hardships.1 We remain uncertain about the timeline and actions needed to effectively combat this virus but hope to reduce severity of current and future waves of infection.44 Targeted individual and organizational strategies for mental health and overall wellness for HCWs are critical for these courageous individuals. Based on the narrative review of the literature, we believe the following are necessary strategies for HCW wellness provided in the Table.

CONCLUSION We recognize the obstacles to implementing strategies to improve HCW wellness: financial barriers; worker engagement; etc.2 Burnout, stress, and the emotional burden of caring for sick patients were already affecting HCWs before COVID-19. Long-term effects of the worldwide pandemic remain unknown. We described published data on HCW distress and burnout but urge future research on strategies to enhance HCW wellbeing. To continue to provide uninterrupted, quality care, the healthcare workforce – human beings – must be empowered and encouraged to take care of themselves.4

Address for Correspondence: Jacob Shreffler, PhD, University of Louisville, Department of Emergency Medicine, 530 S Jackson St C1H17, Louisville, KY 40202. Email: Jacob.shreffler@louisville.edu.

Table. Strategies for healthcare worker wellness. 1. Immediate and individualized access to mental health resources.

2. Short-term and long-term individualized wellness and mental health interventions to address the physical and emotional tolls of COVID-19. 3. Individual AND organizational strategies to optimize wellness for healthcare providers in areas of nutrition, exercise, mindfulness, sleep quality, and reducing burnout.

Copyright: © 2020 Shreffler et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

4. Quality, accessible PPE for all HCWs to provide security and reduce likelihood of infection for themselves and their loved ones. 5. Opportunities to research and implement telehealth in a variety of settings to limit exposure to infection. 6. Reduce stigma on mental health symptoms and the psychological impact of significant stressful events within HCWs.

REFERENCES

7. Development of new HCW community groups and encouragement of participation to allow connections and reduce feelings of isolation. PPE, personal protective equipment; HCW, healthcare worker.

1. Santarone K, McKenney M, Elkbuli A. Preserving mental health and resilience in frontline healthcare workers during COVID-19. Am J Emerg Med. 2020;38(7):1530-1. 2. Penwell-Waines L, Ward W, Kirkpatrick H, et al. Perspectives on healthcare provider well-being: looking back, moving forward. J Clin

LIMITATIONS The research on effects of COVID-19, and physician wellness in general, continues to rapidly evolve. Therefore, updated reviews will be necessary in the coming months. The present review was limited by search strategies designed to retrieve publications with a focus on overall well-being, burnout, or stress; thus, studies exclusively about the physical protection, infection, and transmission rates within this population may not have been retrieved. Future research should consider assessing the psychological burden placed upon HCWs by practical and physiological aspects of disease. Finally, there was limited literature from US providers due to this scoping review being conducted in the earlier stages of the pandemic. Western Journal of Emergency Medicine

Psychol Med Settings. 2018;25(3):295-304. 3. O’Dowd E, O’Connor P, Lydon S, et al. Stress, coping, and psychological resilience among physicians. BMC Health Serv Res. 2018;18(1):730. 4. Nanda A, Wasan A, Sussman J. Provider health and wellness. J Allergy Clin Immunol Pract. 2017;5(6):1543-8. 5. Liu C-Y, Yang Y-z, Zhang X-M, et al. The prevalence and influencing factors for anxiety in medical workers fighting COVID-19 in China: a cross-sectional survey. Available at SSRN 3548781. 2020;148:e98. 6. Lu W, Wang H, Lin Y, et al. Psychological status of medical workforce during the COVID-19 pandemic: a cross-sectional study. Psychiatry Res. 2020;288:112936. 7. Zhu Z, Xu S, Wang H, et al. COVID-19 in Wuhan: immediate

1064

psychological impact on 5062 health workers. medRxiv. 2020. In press.

Volume 21, no. 5: September 2020

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness

8. Liu Z, Han B, Jiang R, et al. Mental health status of doctors and

sectional study. Brain Behav Immun . 2020;87:11-7.

nurses during COVID-19 epidemic in China. SSRN. 2020. In press.

24. Jiang N, Jia X, Qiu Z, et al. The Influence of efficacy beliefs on

9. Li Z, Ge J, Yang M, et al. Vicarious traumatization in the general

interpersonal loneliness among front-line healthcare workers during

public, members, and non-members of medical teams aiding in

the 2019 novel coronavirus outbreak in China: a cross-sectional

COVID-19 control. Brain Behav Immun. 2020;88:916-9.

study. SSRN. 2020. In press.

10. Tan BYQ, Chew NWS, Lee GKH, et al. Psychological impact of the

25. Badahdah AM, Khamis F, Mahyijari NA. The psychological well-being

COVID-19 pandemic on health care workers in Singapore. Ann Intern

of physicians during COVID-19 outbreak in Oman. Psychiatry Res.

Med. 2020.;M20-1083

2020;289:113053.

11. Liang Y, Chen M, Zheng X, et al. Screening for Chinese medical staff

26. Chen Y, Zhou H, Zhou Y, et al. Prevalence of self-reported

mental health by SDS and SAS during the outbreak of COVID-19. J

depression and anxiety among pediatric medical staff members

Psychosom Res. 2020;133:110102.

during the COVID-19 outbreak in Guiyang, China. Psychiatry Res.

12. Cai H, Tu B, Ma J, et al. Psychological impact and coping strategies

2020;288:113005.

of frontline medical staff in Hunan between January and March

27. Xu J, Xu Q-h, Wang C-m, et al. Psychological status of surgical staff

2020 during the outbreak of coronavirus disease 2019 (COVID‑19)

during the COVID-19 outbreak. Psychiatry Res. 2020:112955.

in Hubei, China. Med Sci Monit. 2020;26:e924171-1-16.

28. Naser Moghadasi A. Evaluation of the level of anxiety among Iranian

13. Guo J, Liao L, Wang B, et al. Psychological effects of COVID-19 on

multiple sclerosis fellowships during the outbreak of COVID-19. Arch

hospital staff: a national cross-sectional survey of China Mainland.

Iran Med. 2020;23(4):283.

Available at SSRN 3550050. 2020. In press.

29. Shacham M, Hamama-Raz Y, Kolerman R, et al. COVID-19 factors

14. Lai J, Ma S, Wang Y, et al. Factors associated with mental health

and psychological factors associated with elevated psychological

outcomes among health care workers exposed to coronavirus

distress among dentists and dental hygienists in Israel. Int J Environ

disease 2019. JAMA Netw Open. 2020;3(3):e203976.

Res Public Health. 2020;17(8).

15. Dai Y, Hu G, Xiong H, et al. Psychological impact of the coronavirus

30. Wu Y, Wang J, Luo C, et al. A comparison of burnout frequency

disease 2019 (COVID-19) outbreak on healthcare workers in China.

among oncology physicians and nurses working on the front lines

medRxiv. 2020. In press.

and usual wards during the COVID-19 epidemic in Wuhan, China. J

16. Simione L and Gnagnarella C. Differences between health workers

Pain Symptom Manage. 2020;60(1):e60-5.

and general population in risk perception, behaviors, and psychological

31. Cao J, Wei J, Zhu H, et al. A study of basic needs and psychological

distress related to COVID-19 spread in Italy. PsyArXiv. 2020. In press.

wellbeing of medical workers in the fever clinic of a tertiary general

17. Du J, Dong L, Wang T, et al. Psychological symptoms among

hospital in Beijing during the COVID-19 outbreak. Psychother

frontline healthcare workers during COVID-19 outbreak in Wuhan.

Psychosom. 2020:1-3.

Gen Hosp Psychiatry. 2020. In press.

32. Xiao H, Zhang Y, Kong D, et al. The effects of social support on sleep

18. Zhang SX, Liu J, Afshar Jahanshahi A, et al. At the height of the

quality of medical staff treating patients with coronavirus disease

storm: healthcare staff’s health conditions and job satisfaction and

2019 (COVID-19) in January and February 2020 in China. Med Sci

their associated predictors during the epidemic peak of COVID-19.

Monit. 2020;26:e923549.

Brain Behav Immun. 2020;87:144-6.

33. Huang Y and Zhao N. Generalized anxiety disorder, depressive

19. Delgado D, Wyss Quintana F, Perez G, et al. Personal safety

symptoms and sleep quality during COVID-19 outbreak in China: a

during the COVID-19 pandemic: realities and perspectives of

web-based cross-sectional survey. Psychiatry Res. 2020;288:112954.

healthcare workers in Latin America. Int J Environ Res Public Health.

34. Qi J, Xu J, Li B, et al. The evaluation of sleep disturbances for

2020;17(8):2798.

Chinese frontline medical workers under the outbreak of COVID-19.

20. Prescott K, Baxter E, Lynch C, et al. COVID-19: How prepared

medRxiv. 2020. In press.

are front line healthcare workers in England? J Hosp Infect.

35. Zhang W, Wang K, Yin L, et al. Mental health and psychosocial

2020;105(2):142-5.

problems of medical health workers during the COVID-19 epidemic in

21. Suleiman A, Bsisu I, Guzu H, et al. Preparedness of frontline doctors in Jordan healthcare facilities to COVID-19 outbreak. Int J Environ

China. Psychother Psychosom. 2020:1-9. 36. Zhang C, Yang L, Liu S, et al. Survey of Insomnia and related social

Res Public Health. 2020;17(9):3181.

psychological factors among medical staff involved in the 2019 novel

22. Chew NWS, Lee GKH, Tan BYQ, et al. A multinational, multicentre

coronavirus disease outbreak. Front Psychiatry. 2020;11:306.

study on the psychological outcomes and associated physical

37. Liu Q, Luo D, Haase JE, et al. The experiences of health-care

symptoms amongst healthcare workers during COVID-19 outbreak.

providers during the COVID-19 crisis in China: a qualitative study.

Brain Behav Immun. 2020;88:559-65.

Lancet Glob Health. 2020;8(6):E790-8.

23. Kang L, Ma S, Chen M, et al. Impact on mental health and

38. Sun N, Wei L, Shi S, et al. A qualitative study on the psychological

perceptions of psychological care among medical and nursing staff in

experience of caregivers of COVID-19 patients. Am J Infect Control.

Wuhan during the 2019 novel coronavirus disease outbreak: a cross-

2020;48(6):592-8.

Volume 21, no. 5: September 2020

1065

Western Journal of Emergency Medicine

Shreffler et al.

The Impact of COVID-19 on Healthcare Worker Wellness 39. Sethi BA, Sethi A, Ali S, et al. Impact of coronavirus disease (COVID-19) pandemic on health professionals. Pak J Med Sci.

hospital of Wuhan in China. Clin Infect Dis. 2020. In press. 43. Jin YH, Huang Q, Wang YY, et al. Perceived infection transmission

2020;36(COVID19-S4).

routes, infection control practices, psychosocial changes, and

40. Ing EB, Xu AQ, Salimi A, et al. Physician deaths from corona virus

management of COVID-19 infected healthcare workers in a tertiary

disease (COVID-19). medRxiv. 2020. In press.

acute care hospital in Wuhan: a cross-sectional survey. Mil Med Res.

41. Pruc M, Golik D, Szarpak L, et al. COVID-19 in healthcare workers. Am J Emerg Med. 2020. In press.

2020;7(1):24. 44. Ehrlich H, McKenney M, Elkbuli A. Protecting our healthcare

42. Ran L, Chen X, Wang Y, et al. Risk factors of healthcare workers with

workers during the COVID-19 pandemic. Am J Emerg Med.

corona virus disease 2019: a retrospective cohort study in a designated

Western Journal of Emergency Medicine

2020;38(7):1527-8.

1066

Volume 21, no. 5: September 2020

Original Research

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation Ariel Kerpel, MD* Sara Apter, MD* Noam Nissan, MD, PhD* Esther Houri-Levi, MD§ Maximiliano Klug, MD* Sharon Amit, MD, PhD§§ Eli Konen, MD* Edith M. Marom, MD*

*Department of Diagnostic Imaging, Sheba Medical Center, Tel Hashomer, affiliated with Sackler Faculty of Medicine, Tel Aviv University, Israel § Department of Medicine ‘B’, Sheba Medical Center, Tel Hashomer, affiliated with Sackler Faculty of Medicine, Tel Aviv University, Israel §§ Clinical Microbiology, Sheba Medical Center, Tel Hashomer, affiliated with Sackler Faculty of Medicine, Tel Aviv University, Israel

Section Editor: Brian Yun, MD Submission history: Submitted June 24, 2020; Revision received July 23, 2020; Accepted July 22, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48842

Introduction: Pulmonary opacities in COVID-19 increase throughout the illness and peak after ten days. The radiological literature mainly focuses on CT findings. The purpose of this study was to assess the diagnostic and prognostic value of chest radiographs (CXR) for coronavirus disease 2019 (COVID-19) at presentation. Methods: We retrospectively identified consecutive reverse transcription polymerase reactionconfirmed COVID-19 patients (n = 104, 75% men) and patients (n = 75, 51% men) with repeated negative severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) tests. Two radiologists blindly and independently reviewed the CXRs, documented findings, assigned radiographic assessment of lung edema (RALE) scores, and predicted the patients’ COVID-19 status. We calculated interobserver reliability. The score use for diagnosis and prognosis of COVID-19 was evaluated with the area under the receiver operating characteristic curve. Results: The overall RALE score failed to identify COVID-19 patients at presentation. However, the score was inversely correlated with a COVID-19 diagnosis within ≤2 days, and a positive correlation was found six days after symptom onset.Interobserver agreement with regard to separating normal from abnormal CXRs was moderate (k = 0.408) with low specificity (25% and 27%). Definite pleural effusion had almost perfect agreement (k = 0.833) and substantially reduced the odds of a COVID-19 diagnosis. Disease distribution and experts’ opinion on COVID-19 status had only fair interobserver agreement. The RALE score interobserver reliability was moderate to good (intraclass correlation coefficient = 0.745). A high RALE score predicted a poor outcome (intensive care unit hospitalization, intubation, or death) in COVID-19 patients; a score of ≥5 substantially increased the odds of having a poor outcome. Conclusion: Chest radiography was found not to be a valid diagnostic tool for COVID-19, as normal or near-normal CXRs are more likely early in the disease course. Pleural effusions at presentation suggest a diagnosis other than COVID-19. More extensive lung opacities at presentation are associated with poor outcome in COVID-19 patients. Thus, patients with more than minimal opacities should be monitored closely for clinical deterioration. This clinical application of CXR is its greatest strength in COVID-19 as it impacts patient care. [West J Emerg Med. 2020;21(5)1067-1075.]

Volume 21, no. 5: September 2020

1067

Western Journal of Emergency Medicine

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops.

Population Health Research Capsule

INTRODUCTION Coronavirus disease 2019 (COVID-19) is spreading globally.1 The World Health Organization (WHO) declared COVID-19 a pandemic on March 11, 2020.2 The most common presenting clinical symptoms are fever, cough, dyspnea, myalgia, and fatigue.3-5 Older age and medical comorbidities are linked to more severe disease.4,6-8 Men are over-represented among COVID-19 patients.3,4,6,7 Although the radiological literature mainly focuses on computed tomography (CT) findings,9,10 many patients are imaged solely with chest radiography10,11 primarily as an adjunct to reverse transcription polymerase chain reaction (RT-PCR) but in some scenarios as a triage tool,12,13 especially in resourceconstrained environments where the supply of laboratory PCR kits cannot meet the demand. Although there are nonspecific respiratory symptoms commonly observed in COVID-19 patients at presentation, some patients with COVID-19 do not present with these classic clinical manifestations, which further complicates triage and diagnosis.4 The chest radiograph (CXR) was reported as having a sensitivity of 69% for COVID-19 in one study of 64 patients.9 In that study, the common findings were bilateral peripheral opacities with a predilection to the lower lung zones. Opacities increased throughout the illness, with a peak in severity at 1012 days after symptom onset; this was shown by documenting lung opacities using a simplified radiographic assessment of lung edema (RALE) score.9,14 When the Fleischner Society consensus statement was created, which specified that chest radiography has little value early in the course of the disease, there were limited data available on the accuracy of chest radiography for the diagnosis of COVID-19.13 Data on the strengths and weaknesses of chest radiography for the diagnosis of COVID-19 are important, as CXRs are the most commonly used triage imaging tool in any patient presenting with respiratory symptoms.12 This is especially important because experts suggest that the second wave of coronavirus is likely to be even more devastating.15 Our aim was to assess the diagnostic accuracy and reliability of CXRs in patients suspected of having COVID-19 at presentation to the emergency department (ED) and to assess the prognostic value of the RALE score in patients with COVID-19. MATERIALS AND METHODS Patients and Data Source This retrospective study was approved by our institutional review board, and informed consent was waived. We identified our study population by extracting severe acute respiratory Western Journal of Emergency Medicine

What do we already know about this issue? Pulmonary opacities in coronavirus disease 2019 (COVID-19) peak after 10 days. The radiological literature focuses on computed tomography findings. What was the research question? What is the diagnostic and prognostic value of chest radiographs (CXR) for COVID-19 at presentation? What was the major finding of the study? While CXR is not a valid diagnostic tool for COVID-19, the presence of extensive opacities is associated with poor outcome. How does this improve population health? CXR’s greatest strength in COVID-19 is prognosis prediction. Patients with more than minimal opacities should be monitored closely for clinical deterioration.

syndrome coronavirus 2 (SARS-CoV-2) RT-PCR test results (positive or negative) of nasopharyngeal swabs from all consecutive patients older than 18 years analyzed at our hospital’s laboratory from the ED from March 6–31, 2020, who had a CXR at presentation (within 24 hours of the first RT-PCR). We extracted data by a database search (query) using the MDClone platform (MDClone Ltd, Be’er Sheva, Israel), a big data system for healthcare. We were granted access to the raw data in order to validate the quality and reliability of the information in the database source underlying the study. Of the RT-PCR test kits used, 90% (161/179) were Allplex 2019-nCoV assay kits (Seegene Inc. Seoul, Korea), and 10% (18/179) were kits produced in our hospital laboratory. The patients were then divided into two groups: those who had COVID-19 and those who did not. The former group comprised patients who had a positive RT-PCR test. The latter, control group comprised patients who had a negative RT-PCR result on at least two separate occasions, more than 24 hours apart (without a previous positive test result). This methodology is similar to that of previously published studies,16 as we tried to avoid the imperfect gold standard bias. We excluded patients who underwent SARS-CoV-2 testing due to an abnormal CXR and not due to clinical suspicion (n = 1 positive, n = 3 negative) based on the patients’ electronic health records (EHR) (Figure 1) to avoid partial verification bias (referral bias).17 The patients’ EHRs were reviewed to obtain demographics

1068

Volume 21, no. 5: September 2020

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation

Figure 1. Study selection process flowchart. *Overshadowing radiopaque abnormalities excluded from the subgroup analyses were pleural effusion (n = 1 COVID-19, n = 17 control), lung cancer (n = 1, control), lung metastasis (n = 1, control), and pleural plaques (n = 1, control).

and clinical data. The primary outcomes were intensive care unit (ICU) hospitalization, intubation, and mortality. COVID-19 severity was classified as severe or non-severe based on respiratory distress (≥30 breaths per minute) or oxygen saturation ≤93% on room air.18 Although lung opacities are included in some published severity criteria, we did not use CXR findings to define severity to avoid incorporation bias.17 The data cutoff date was April 21, 2020. We extracted the overall number of ED visits at our hospital during the study period using the MDClone platform database search. Overall COVID-19 new cases in Israel for the study period (26 days), and for an equal time span before and after the study period, were extracted from Israel’s Ministry of Health website.19 Imaging Protocols CXRs were acquired as computed radiographs (n = 127) or digital radiographs (n = 52) from multiple vendors. The projections were posterior-anterior (PA) (n = 108), and anterior-posterior (AP) (n=71). Image Analysis Two radiologists (EMM, a thoracic radiologist with 28 Volume 21, no. 5: September 2020

years of experience, and SA, an oncology imaging radiologist with 40 years of experience) independently reviewed all CXRs using a communication system search (PACS), Carestream, PACS Vue v12.1.5 (Carestream Health, Inc, Rochester, NY), while blinded to the RT-PCR results and clinical data. The CXRs of COVID-19 patients and the control patients were in random order. Both readers recorded pulmonary opacity characteristics, including their distribution (peripheral, perihilar or diffuse), zonal predominance (upper, lower, or equal), and laterality (bilateral or unilateral). Pleural effusion presence was recorded. Disagreements between reader 1 (R1) and reader 2 (R2) regarding the categorization of a pleural effusion as definite or questionable were resolved by an independent and blinded third reader (EK, a cardiothoracic radiologist with 21 years of experience). R1 and R2 calculated the RALE scores14 (Figure 2). The RALE score, which is used to quantitate lung opacities,14 is calculated by dividing each radiograph into quadrants and multiplying the extent (0 = no involvement, 1 = <25%, 2 = 25-50%, 3 = 50-75%, 4 = >75%) by the density (1 = hazy, 2 = moderate, 3 = dense) for each quadrant and then summing them (maximum score = 48).14 For the purposes of our study, the following density definitions were used: hazy, ranging from barely noticeable opacities to mild or veiling opacities, through

1069

Western Journal of Emergency Medicine

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation

Figure 2. An example of radiograph assessment of lung edema (RALE) scoring in a 71-year-old man with COVID-19 who presented 5 days after symptom onset, with fever, cough and fatigue. RALE scoring: reader 1: 11, reader 2: 12. Adapted from Warren et al, 2018.14

which the lung vessels can be clearly seen; moderate, in which opacities are identified, but the blood vessels are still visible; and dense, in which consolidation is apparent, and the blood vessels are not visible. For RALE scoring, we excluded CXRs with one of the following overshadowing radiopaque abnormalities: pleural effusion; pleural plaques; and pulmonary nodules or masses, whether due to lung cancer or metastatic disease. Finally, the readers gave their expert opinion regarding patient COVID-19 status based on imaging alone. All previous imaging tests were available to the readers for comparison, and any changes were recorded. Statistical Analysis To evaluate the sensitivity and specificity of categorical variables to discriminate between patients with and without COVID-19, assuming sensitivity and specificity of 80% and a 95% confidence interval (CI) of 0.2, 140 patients were needed. To evaluate the use of RALE for determining COVID-19 diagnosis using the area under the receiver operating characteristic curve (AUC), assuming an area of 0.8 with a 95% CI width of 0.2 and an equal number of participants with and without COVID-19, 78 participants were needed. We assumed that the mean RALE score for patients without poor prognosis was 2, with a mean score of 10 for patients with poor outcomes. We assumed that the standard deviation of the RALE score was 8 (range 0-48, divided by six). Using a significance level of 5% and power of 80%, and assuming a proportion of patients having poor outcomes to be 20%, a total of 53 patients were needed. We evaluated continuous variables for normal distributions using histograms. Variables that were close to being normally distributed are reported as the means and standard deviations Western Journal of Emergency Medicine

Kerpel et al.

(SD), while skewed variables are reported as the medians and interquartile ranges. Categorical variables are reported as frequencies and percentages. We used independent samples t-tests and Mann–Whitney tests to compare normally distributed variables and skewed variables between groups, respectively. Chi-square tests and Fisher’s exact tests were applied to compare categorical variables between patients with positive and negative tests. The kappa statistic was used to evaluate the agreement between readers20 and was interpreted according to Landis and Koch.21 When a kappa of 0.4 was reached, accuracy was evaluated. Diagnostic accuracy parameters were calculated by crosstabulation and included the following: sensitivity, specificity, and positive (LR+) and negative (LR-) likelihood ratios. We used the intraclass correlation coefficient (ICC) to evaluate the agreement of the two readers with regard to the RALE score.22 The AUC23 was used to evaluate the ability of the RALE score to discriminate between COVID-19 and control patients and between poor and favorable outcomes in COVID-19 patients. The discriminatory ability was also evaluated in patients who presented at early (0-2 days), intermediate (3-5 days), and late (≥6 days) time points from symptom onset. For prognostic ability, we used a RALE score cutoff threshold of 5. All statistical tests were two-sided, and p<0.05 was considered statistically significant. For statistical analyses, we used SPSS software (IBM SPSS Statistics for Windows, IBM Corp., Armonk, NY). RESULTS Patient Characteristics During the study period, 105 patients had positive RT-PCR results and had a CXR, and 78 patients had repeated negative results and had a CXR. After excluding patients who had the RT-PCR ordered due to an abnormal CXR (n = 1 COVID-19 patient, n = 3 control patients), our study group included 104 COVID-19 patients (men 78/104, 75%, mean age 57.0, SD 15.7 years) and 75 control patients (men 38/75, 51%, mean age 65.6, SD 21.4 years) (Figure 1). Table 1 shows patient characteristics and outcomes with a comparison of COVID-19 to control and non-severe to severe COVID-19 patients. The overall number of ED visits at our hospital during the study period was 8025 (all causes). The number of new cases of COVID-19 in Israel during the study period (26 days) was 5699. The number for the period immediately preceding was 17. The number for the period immediately ensuing was 9723. These numbers show that our study took place at the beginning of the first wave of COVID-19 in Israel. The mortality rate in the control group was significantly higher: 27% (20/75) vs 7% (7/104) in COVID-19 patients (P<0.001). Heart disease and active cancer were more common in the control group. Heart disease was present in 44% (n = 33/75) of the control patients compared to 16% (n = 17/104) of COVID-19 patients (p<0.001). Active cancer, defined as malignancy in the prior 12 months, was present in 24% (n =18/75) of the control patients compared to 4% (n = 4/104)

1070

Volume 21, no. 5: September 2020

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation

of COVID-19 patients (p<0.001). There was no significant difference (p>0.05) in the prevalence of other comorbidities between the control and COVID-19 patients: diabetes mellitus (34%, 29%); hypertension (32%, 38%); obesity (12%, 13%); dyslipidemia (35%, 29%); smoking (19%, 11%); respiratory disease (13%, 7%); and chronic renal failure (7%, 7%).

between patients with non-severe and severe disease. Radiographic Findings The identification of any opacity on CXRs had a moderate interobserver agreement (kappa = 0.408). When assuming that any parenchymal lung opacity could represent COVID-19 pneumonia, the diagnostic accuracy for the diagnosis of COVID-19 for both readers was sensitivity (R1-87%; R269%) and specificity (R1-25%; R2-27%), and both LR+ and LR- showed the poor diagnostic performance of CXRs for COVID-19, as most crossed or included 1 (Table 2). See the supplemental table for a summary of pulmonary opacities identified at different timeframes from symptom onset. The predominance and distribution of opacities, laterality, change from previous radiograph, and expert opinion with regard to COVID-19 status had only a fair agreement between readers (kappa = 0.399, 0.248, 0.372, 0.352, 0.249, respectively); hence, accuracy was not evaluated. The presence of a definite

Chest Radiograph Technique Most COVID-19 patients underwent a PA CXR (78/104, 75%) in a dedicated radiography room of the Corona Section emergency department (ED), while most control group patients underwent an AP CXR (45/75, 60%) (p<0.001). Among the COVID-19 patients, most patients with non-severe disease had a PA CXR (64/75, 85%), while most patients with severe disease had an AP CXR (15/29, 52%) (p<0.001). The majority of both the COVID-19 and control groups underwent computerized radiography (CR) (77/104, 74% and 50/75, 67%, respectively) (p=0.284). Similar proportions were observed

Table 1. Patient characteristics and outcomes with comparison of COVID-19 to control and non-severe to severe COVID-19. Variable

COVID-19 patients

Control

P-value

Non-severe COVID-19

Severe COVID-19

P-value

78/104 (75%)

38/75 (51%)

0.001

56/75 (75%)

22/29 (76%)

0.900

57.0 ±15.7

65.6 (21.4)

0.058

55.64 (15.1)

60.45 (16.9)

0.163

Discharge from ED

31/104 (30%)

13/75 (17%)

0.157

31/75 (41%)

0/29 (0%)

<0.001

Ward hospitalization

59/104 (57%)

51/75 (68%)

0.157

44/75 (59%)

15/29 (52%)

<0.001

ICU

14/104 (13%)

11/75 (15%)

0.157

0/75 (0%)

14/29 (48%)

<0.001

7/104 (7%)

20/75 (27%)

<0.001

0/75 (0%)

7/29 (24%)

<0.001

Intubated 14/104 (13%) 17/75 (23%) 0.108 0/75 (0%) Unless otherwise specified, data are numbers of patients, with frequency in parentheses. *mean ±SD

14/29 (48%)

<0.001

Gender (Men) Age (years)*

In-hospital mortality

Table 2. Reliability and accuracy of different radiographic characteristic and experts’ best guess to predict COVID-19 status. Radiographic variable Kappa Sensitivity (95% CI) Specificity (95% CI) LR+ (95% CI) LR- (95% CI) Any opacity (overall)

0.408

Reader 1

0.87 (0.78-0.92)

0.25 (0.16-0.37)

1.16 (1.00-1.35)

0.53 (0.30-0.94)

Reader 2

0.69 (0.59-0.78)

0.27 (0.27-0.17)

0.95 (0.94-1.14)

1.15 (0.80-1.65)

Reader 1

0.01 (<0.01-0.06)

0.81 (0.70-0.89)

0.05 (0.01-0.38)

1.22 (1.19-1.25)

Reader 2

0.01 (<0.01-0.06)

0.77 (0.66-0.86)

0.04 (<0.01-0.31)

1.28 (1.25-1.32)

Opacity predominance

0.399

Opacity distribution

0.248

Laterality*

0.372

Definite pleural effusion

0.833

Change**

0.352

Experts' best guess

0.249

When interobserver reliability did not reach a kappa of 0.4, diagnostic accuracy parameters were not calculated. . *Laterality = bilateral or unilateral. **Change = change from previous radiograph when comparison was available. LR+, positive likelihood ratio; LR-, negative likelihood ratio; CI, confidence interval.

Volume 21, no. 5: September 2020

1071

Western Journal of Emergency Medicine

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation pleural effusion had almost perfect interobserver agreement (kappa = 0.833). The accuracy parameters of the presence of a pleural effusion for the diagnosis of COVID-19 were as follows: sensitivity (R1 and R2 - 0.01%), specificity (R1-81%; R2-77%), and very low positive likelihood ratio (LR+) (R1-0.05; R-0.04); thus, the presence of definite pleural effusion at presentation makes the diagnosis of COVID-19 very unlikely (see Table 2). With regard to RALE scoring, 103 CXRs were available in the COVID-19 group after excluding one CXR due to pleural effusion, and 55 CXRs were available in the control group after excluding CXRs with the following overshadowing radiopaque abnormalities: pleural effusion (n = 17); lung cancer (n = 1); multiple metastases (n = 1); and calcified pleural plaques (n =1) (Figure 1). The RALE score interobserver reliability was moderate to good, with an ICC of 0.745 (0.665 - 0.806, p<0.001). See Table 3 for the AUC assessment summary.

Table 3. Categorization by RALE score for diagnosis and prognosis of COVID-19 by receiver operator characteristics curve analysis. Radiographic variable AUC (95% CI) P-value RALE score for diagnosis* All patients Reader 1

0.625 (0.529 – 0.721)

0.010

Reader 2

0.508 (0.412 – 0.605)

0.865

Reader 1

0.290 (0.136 – 0.443)

0.023

Reader 2

0.249 (0.095 – 0.402)

0.007

Reader 1

0.741 (0.567 – 0.916)

0.025

Reader 2

0.561 (0.351 – 0.771)

0.570

0.738 (0.571 – 0.905)

0.002

0.704 (0.551 – 0.856)

0.009

Days 0-2

Days 3-5

Days 6≥ Reader 1 Reader 2

RALE score for prognosis** Severe COVID-19 Reader 1

0.825 (0.742 – 0.907)

<0.001

Reader 2

0.755 (0.651 – 0.859)

<0.001

0.837 (0.736 – 0.937)

<0.001

Poor outcome Reader 1

Reader 2 0.772 (0.636 – 0.907) 0.001 Data are area under the receiver operating characteristic curve (AUC), with 95% confidence interval in parentheses. The RALE score intraclass correlation coefficient (ICC) was 0.745 (95% CI, 0.665-8.086), p-value <0.001. *Included only patients without radiopaque overshadowing abnormalities (N = 158). **Included COVID-19 patients without radiopaque overshadowing abnormalities (n = 103). CI, confidence interval; COVID-19, coronavirus disease 2019; RALE, radiographic assessment of lung edema.

Western Journal of Emergency Medicine

Kerpel et al.

The AUC for all patients (overall) showed no significant difference from sheer chance (R1- p = 0.010; R2- 0.865). The evaluation of the discriminatory ability of the RALE score in patients who presented early (0–2 days) showed an inverse correlation with COVID-19 diagnosis. Simply put, in patients presenting within 0-2 days of symptom onset who were clinically suspected of having COVID-19, pulmonary opacities were more likely to be due to a diagnosis other than COVID-19. For patients presenting within three to five days from symptom onset, only R1 achieved statistical significance, while for patients presenting more than six days from symptom onset, both readers reached significant discrimination ability. Thus, for patients presenting later after symptom onset, especially from day six, the higher the RALE score, the more likely a diagnosis of COVID-19. An example is seen in Figure 3, showing the sensitivity of the RALE score with a threshold of 5 for the diagnosis of COVID-19 increasing as the patients arrive later in the disease course. See Figure 4 for CXR examples of patients presenting at different timeframes from symptom onset. When the RALE score was evaluated as a prognostic indicator within the COVID-19 patient group, both readers had statistically significant discriminatory accuracy for severe disease and poor outcomes (Table 3). When a RALE score of 5 was used as a threshold for severe disease and for poor outcome, sensitivity was moderate to good, and specificity was moderate. However, LRs were encouraging, as LR+ ranged from 2.21 to 2.59 and LR- ranged from 0.10 to 0.45 (supplemental table). Hence, a RALE score <5 in COVID-19 patients at presentation substantially reduces the odds of having severe COVID-19 or poor outcome (intensive care unit hospitalization, intubation, or death), whereas a RALE score ≥5 substantially increases those odds. DISCUSSION In this study we assessed the diagnostic value of the initial CXR for diagnosing COVID-19 in patients clinically suspected of having COVID-19, as well as the prognostic value of this CXR in COVID-19 patients. The study took place in a single hospital in Israel at the beginning of the COVID-19 pandemic first wave. Our study showed that the reliability of radiographs is only moderate for any opacity and moderate to good for the RALE score. Overall, chest radiography was found not to be a valid diagnostic tool for COVID-19. However, the diagnosis of COVID-19 pneumonia by CXRs reached significant diagnostic accuracy when performed at least six days after symptom onset. For patients presenting early (0-2 days from symptom onset), a normal or near-normal CXR is more likely to be seen in a patient with COVID-19, although opacities early in the disease course do not completely rule out this condition. The presence of a definite pleural effusion indicates that the diagnosis is unlikely to be COVID-19. More extensive lung opacities are associated with poor outcome in COVID-19 patients. Previous COVID-19 studies mainly concentrated on computed tomography (CT) findings and indicated that

1072

Volume 21, no. 5: September 2020

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation

Figure 3. Sensitivity for RALE score threshold of 5 for the diagnosis of coronavirus disease 2019 in patients presenting at different timeframes from symptom onset.

Figure 4. Radiographs of three different COVID-19 patients who presented with fever and cough at different time frames from disease onset. (a) A 32-year-old man who presented one day after symptom onset. Radiograph assessment of lung edema (RALE) scoring: reader 1: 0; reader 2: 0. (b) A 64-year-old man who presented three days after symptom onset. RALE scoring: reader 1: 1; reader 2: 2. (c) A 73-year-old man who presented seven days after symptom onset. RALE scoring: reader 1: 6; reader 2: 6.

opacities are usually bilateral, with a peripheral distribution and lower zones predominance.24 We found only fair agreement with regard to the opacity predominance, distribution, and laterality, which probably relates to the lower sensitivity of CXRs compared with CT for pulmonary opacities. A previously published study reported 69% sensitivity for diagnosis on the baseline CXR,9 similar to our findings. On the other hand, we found that this high sensitivity had a trade-off with low specificity, which represents the readerâ&#x20AC;&#x2122;s avoidance of falsenegative results, offsetting with more false-positive results. This observation can only be made with a control group. This Volume 21, no. 5: September 2020

is in contrast to previous studies that did not have a control group9,11 and were only able to assess sensitivity. Moreover, LRs showed the CXR is ineffective in the ED setting as it failed to meaningfully change the estimation of disease probability from pretest to posttest. This, at the very least, raises doubts about the utility of the CXR as a triage tool. It is perhaps not surprising that the quantification of pulmonary opacities, as performed in our study with the RALE score, was not useful for assessing the entire cohort when trying to distinguish between patients with and without COVID-19, but when interpreted in the context of time from symptom onset, the accuracy improved.

1073

Western Journal of Emergency Medicine

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation Highly experienced radiologists’ expert opinions for guessing COVID-19 status were not reliable and did not reach a high enough interobserver agreement to discuss the accuracy parameters. However, poor interobserver agreement regarding specific disease status on CXRs was documented in previous studies.25,26 Despite the limited role of imaging in the diagnosis of COVID-19 as expressed by leading societies worldwide,12,13,27 the CXR is still the recommended imaging tool for any patient presenting at the ED with an acute respiratory illness.28 Future COVID-19 patients will continue to have CXRs at presentation before their disease status is known to the referring clinicians. To complicate matters, even in the ideal setting, when RT-PCR is available and results are delivered within minutes to hours, the sensitivity of the RT-PCR for SARS-CoV2 is poor,29 leaving emergency clinicians with a dilemma as to how to manage patients with non-specific presenting symptoms suggestive of COVID-19 with a negative initial RT-PCR test. This dilemma emphasizes the need to maximize available knowledge in the ED setting. Time from symptom onset is available data in this setting, and applying it to CXR interpretation may improve diagnostic accuracy. Despite not being recommended for diagnosis of COVID-19, the CXR is a tool used for the risk stratification of patients with COVID-19 and is often used as an aid to decision-making with regard to discharge vs hospitalization and the amount of close monitoring needed for specific patients.13,18,20 Our study validates this approach and shows that the amount of pulmonary opacities, as quantified by the RALE score, correlates with poor outcome. The knowledge gained from this study allows for a better understanding of the diagnostic and prognostic value of CXRs in COVID-19 patients and can aid emergency physicians in clinical decision-making. The added information can also serve educators and future researchers in understanding the strengths and weaknesses of CXRs, as this “classic” imaging modality is also the most frequently performed.

repeat the test and, hence, could underestimate the specificity.17 Even though this methodology is well accepted,16 and the motivation was to ensure having only truly non-COVID-19 patients in the control group, the trade-off eliminated many non-COVID-19 patients who might have had less remarkable radiographs. All these biases do not impact the results regarding prognosis, as these did not relate to the control group. The study’s results can be generalized to the ED setting. In a community setting, in which fewer non-COVID-19 patients have competing conditions, LRs will move further away from 1, and the test will appear more useful.31

LIMITATIONS This study has several sources of bias. Differential verification bias (double gold standard bias)17 was present in our study, as we selected patients with only one RT-PCR test for the COVID-19 group, whereas we selected only patients with two negative RT-PCR tests for the control group. Lack of clinical follow-up to confirm the absence of COVID-19 precluded incorporation of this patient population with only one negative test into our study. In our opinion, the bias reduced specificity, as the patients in the control group were sicker with almost four times the mortality rate and a higher prevalence of heart disease and active cancer. Thus, the patients in the control group probably had more lung opacities than would be expected in the general population. Similarly, spectrum bias potentially influenced our results because the control group was enriched with many “sickest of the sick,” whose clinical condition influenced the decision to

Copyright: © 2020 Kerpel et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

Western Journal of Emergency Medicine

CONCLUSION Chest radiography was found not to be a valid diagnostic tool for COVID-19. However, sensitivity increased in patients presenting later in the disease course. When presenting early, a normal or near-normal CXR is more likely in COVID-19. When a pleural effusion is present, the diagnosis is unlikely to be COVID-19. Furthermore, more extensive lung opacities at presentation are associated with poor outcome in COVID-19 patients. Thus, patients with more than minimal opacities should be monitored closely for clinical deterioration. This clinical application of chest radiography is its greatest strength in COVID-19 as it impacts patient care.

Address for Correspondence: Ariel Kerpel, MD, Department of Diagnostic Imaging, Sheba Medical Center, Ramat-Gan, Israel, 2 Sheba Rd, Ramat Gan, 52621 Israel. Email: Ariel.Kerpel@sheba. health.gov.il. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare.

REFERENCES 1. Johns Hopkins Coronavirus Resource Center. Cumulative Cases. 2020. Available at: https://coronavirus.jhu.edu/data/cumulative-cases. Accessed April 16, 2020. 2. World Health Organization‎. WHO Director-General’s opening remarks at the media briefing on COVID-19- 11 March 2020. 2020. Available at: https://www.who.int/dg/speeches/detail/who-directorgeneral-s-opening-remarks-at-the-media-briefing-on-covid-19---11march-2020. Accessed April 16, 2020. 3. Huang C, Wang Y, Li X, et al. Clinical features of patients

1074

Volume 21, no. 5: September 2020

Kerpel et al.

Diagnostic and Prognostic Value of Chest Radiographs for COVID-19 at Presentation

infected with 2019 novel coronavirus in Wuhan, China. Lancet.

of COVID-19. Radiol Cardiothorac Imaging. 2020;2(2):e200196.

2020;395(10223):497-506.

17. Kohn MA, Carpenter CR, Newman TB. Understanding the direction

4. Guan W, Ni Z, Hu Y, et al. Clinical characteristics of coronavirus

of bias in studies of diagnostic test accuracy. Acad Emerg Med.

disease 2019 in China. N Engl J Med. 2020;382:1708-20.

2013;20(11):1194-1206.

5. Bernheim A, Mei X, Huang M, et al. Chest CT findings in coronavirus

18. National Health Commission & State Administration of Traditional

disease-19 (COVID-19): relationship to duration of infection.

Chinese Medicine. Diagnosis and Treatment Protocol for Novel

Radiology. 2020;295:685-91.

Coronavirus Pneumonia (Trial Version 7). 2020. Available at:

6. Wu Z, McGoogan JM. Characteristics of and Important lessons from

https://www.chinadaily.com.cn/pdf/2020/1.Clinical.Protocols.for.

the coronavirus disease 2019 (COVID-19) outbreak in China. JAMA.

the.Diagnosis.and.Treatment.of.COVID-19.V7.pdf. Accessed June

2020;323(13):1239-42

15, 2020.

7. Wu C, Chen X, Cai Y, et al. Risk factors associated with acute

19. State of Israel Ministry of Health Coronavirus Information Dashboard.

respiratory distress syndrome and death in patients with coronavirus

2020. Available at: https://datadashboard.health.gov.il/COVID-19/.

disease 2019 pneumonia in Wuhan, China. JAMA Intern Med. March

Website in Hebrew. Accessed July 13, 2020.

2020;180(7):934-43.

20. Crewson PE. Reader agreement studies. AJR Am J Roentgenol.

8. Chow N, Fleming-Dutra K, Gierke R, et al. Preliminary estimates

2005;184(5):1391-7.

of the prevalence of selected underlying health conditions

21. Landis JR, Koch GG. A one-way components of variance model for

among patients with coronavirus disease 2019 — United States,

categorical data. Biometrics. 1977;33(4):671-9.

February 12–March 28, 2020. MMWR Morb Mortal Wkly Rep.

22. Koo TK, Li MY. A guideline of selecting and reporting intraclass

2020;69(13):382-6.

correlation coefficients for reliability research. J Chiropr Med.

9. Wong HYF, Lam HYS, Fong AHT, et al. Frequency and distribution

2016;15(2):155-63.

of chest radiographic findings in patients positive for COVID-19.

23. Obuchowski NA. Receiver operating characteristic curves and their

Radiology. 2020;296:E72-8.

use in radiology. Radiology. 2003;229(1):3-8.

10. Galperin-Aizenberg M. The chest radiograph in the COVID-19

24. Pan F, Ye T, Sun P, et al. Time course of lung changes on chest

pandemic: role, standardized reporting, and CT correlation. 2020.

CT during recovery from 2019 novel coronavirus (COVID-19)

Available at: https://www.youtube.com/watch?v=aSxOcyRd-bU.

Pneumonia. Radiology. 2020;295(3):715-21.

11. Weinstock M, Echenique A, Russell JW, et al. Chest X-ray findings in

25. Albaum MN, Hill LC, Murphy M, et al. Interobserver reliability of

636 ambulatory patients with COVID-19 presenting to an urgent care

the chest radiograph in community-acquired pneumonia. Chest.

center: a normal chest X-ray is no guarantee. J Urgent Care Med.

1996;110(2):343-350.

2020. Available at: https://www.jucm.com/documents/jucm-covid-19-

26. Sverzellati N, De Filippo M, Quintavalla M, et al. Interobserver

studyepub-april-2020.pdf/. Accessed June 15, 2020.

reliability of the chest radiograph in pulmonary embolism. Clin Appl

12. American College of Radiology. ACR Recommendations for the

Thromb Hemost. 2014;20(2):147-51.

Use of Chest Radiography and Computed Tomography (CT) for

27. Simpson S, Kay FU, Abbara S, et al. Radiological Society of North

Suspected COVID-19 Infection. 2020. Available at: https://www.

America Expert Consensus Statement on Reporting Chest CT

acr.org/Advocacy-and-Economics/ACR-Position-Statements/

Findings Related to COVID-19. Endorsed by the Society of Thoracic

Recommendations-for-Chest-Radiography-and-CT-for-Suspected-

Radiology, the American College of Radiology, and RSNA. Radiol

COVID19-Infection. Accessed June 15, 2020.

Cardiothorac Imaging. 2020;2(2):e200152.

13. Rubin GD, Haramati LB, Kanne JP, et al. The Role of Chest

28. Bancroft LW, Kransdorf MJ AR. ACR appropriateness criteria; acute

Imaging in Patient Management during the COVID-19 Pandemic:

respiratory illness in immunocompetent patients. Am Coll Radiol.

A Multinational Consensus Statement from the Fleischner Society.

2018. Available at: https://acsearch.acr.org/docs/69446/Narrative/?_

Radiology. April 2020:201365.

ga=2.17694460.497078903.1589036044-1246925013.1568210536.

14. Warren MA, Zhao Z, Koyama T, et al. Severity scoring of lung

Accessed May 1, 2020.

oedema on the chest radiograph is associated with clinical outcomes

29. Dramé M, Teguo MT, Proye E, et al. Should RT‐PCR be considered a

in ARDS. Thorax. 2018;73(9):840-6. 15. Sun LH. CDC director warns second wave of coronavirus is likely

gold standard in the diagnosis of Covid‐19? J Med Virol. 2020. In press. 30. Rodrigues JCL, Hare SS, Edey A, et al. An update on COVID-19 for

to be even more devastating. 2020. Available at: https://www.

the radiologist: a British society of Thoracic Imaging statement. Clin

washingtonpost.com/health/2020/04/21/coronavirus-secondwave-

Radiol. 2020;75(5):323-5.

cdcdirector/. Accessed April 22, 2020.

31. Jaeschke R. Users’ guides to the medical literature III. how to use an

16. Dangis A, Gieraerts C, Bruecker Y De, et al. Accuracy and

article about a diagnostic test B. What are the results and will they

reproducibility of low-dose submillisievert chest CT for the diagnosis

Volume 21, no. 5: September 2020

help me in caring for my patients? JAMA. 1994;271(9):703-7.

1075

Western Journal of Emergency Medicine

Original Research

MIST (Modified Intubating Sequence for Transmissibility) Bundle for Infectious Diseases with Aerosol Hazard Jayaraj M. Balakrishnan, MD* Sanjan Asanaru Kunju, MD, DNB* William Wilson, MD, DNB, MRCEM* Sachin Nayak Sujir, MD* Rachana Bhat, MD, DNB* K. E. Vandana, MD†

*Department of Emergency Medicine, Kasturba Medical College, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India † Department of Microbiology, Kasturba Medical College, Manipal, Manipal Academy of Higher Education, Manipal, Karnataka, India

Section Editor: Gabriel Wardi, MD Submission history: Submitted March 29, 2020; Revision received July 11, 2020; Accepted July 15, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.47473

The current global severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic has magnified the risk to healthcare providers when inititiating airway management, and safe tracheal intubation has become of paramount importance. Mitigation of risk to frontline providers requires airway management to be an orchestrated exercise based on training and purposeful simulation. Role allocation and closed-loop communication form the foundation of this exercise. We describe a methodical, 10-step approach from decision-making and meticulous drug and equipment choices to donning of personal protective equipment, and procedural concerns. This bundled approach will help reduce unplanned actions, which in turn may reduce the risk of aerosol transmission during airway management in resource-limited settings. [West J Emerg Med. 2020;21(5)1076-1079.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. BACKGROUND The emergence of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), or COVID-19, pandemic has brought infections that are transmitted via droplet and aerosol under the spotlight.1 Infections such as influenza A subtype H1N1, Nipah virus infection, Ebola virus disease, and multidrug-resistant tuberculosis are equally contagious and pose a significant risk to healthcare professionals, especially those involved in airway management.2,3 Herin, we describe a step-by-step approach to endotracheal intubation of critically ill patients with suspected or confirmed COVID-19 and other airborne diseases with the goal of limiting the risk of exposure to healthcare providers. CONCEPTS 1. Rapid sequence intubation (RSI) and invasive mechanical Western Journal of Emergency Medicine

ventilation are preferred. Non-invasive positive pressure ventilation (NIPPV) increases the risk of aerosol generation; NIPPV has been associated wth increased risk of healthcare worker infection and hence should be avoided.4-6 2. The care area is divided as follows: • Hot zone: A three-meter [9.85 feet] radius around the patient • Warm zone: The area between hot and cold zone where decontamination takes place • Cold zone: The outermost noncontaminated area. 3. The intubation team members are described in Table 1. 4. Encourage closed-loop communication. STEPS OF MIST (Modified Intubating Sequence for Transmissibility) BUNDLE 1. Pre-assessment phase and pre-briefing phase – Cold Zone Step a. Review patient clinical data to determine appropriateness of endotracheal intubation and mechanical ventilation for the patient. Step b. Team leader (TL) debriefs the intubation plan to the

1076

Volume 21, no. 5: September 2020

Balakrishnan et al.

MIST Bundle for Infectious Diseases with Aerosol Hazard

team to avoid unplanned and unarticulated actions. Step c. Infection control nurse (ICN) alerts team to any breach of protocol or infection control practice.

Population Health Research Capsule What do we already know about this issue? Several protocols for intubation during the COVID-19 pandemic aim at reducing transmissibility of infections by using sophisticated equipment.

2. Preparatory phase – Cold Zone Step a. Use continous positive airway pressure mode with non-invasive ventilation (NIV) mask for preoxygenation. The registered respiratory therapist (RRT) assembles the ventilator with its circuit including preparation of the NIV mask with a viral filter and checks for possible leaks and disconnections. Ventilator settings: pressure support 0 centimeters of water, positive end expiratory pressure as per the requirement, and fraction of inspired oxygen to 100%. Deselect the apnea setting. Step b. Review the equipment required for intubation (Figure 1A) (Table 2); the registered nurse (RN) loads pre-calculated doses of RSI medications (Table 3). Step c. The assembly (Figure 1B) of the endotracheal tube (ETT) should be preset with a catheter mount containing a viral filter, and an inflation syringe with an intubating bougie.

What was the research question? How can we reduce the risk of aerosol hazards from infectious diseases transmitted during intubation? What was the major finding of the study? Execution of intubation in suspected aerosoltransmitted infections can be performed systematically in low-resource settings. How does this improve population health? The protocol is aimed at safeguarding healthcare professionals against aerosol hazards while performing airway interventions.

3. Preoxygenation Phase: Hot Zone Step a: The RRT ensures wall-mounted suction unit is properly connected. A Yankauer suction connected to the wall-mounted suction unit should be available, but its usage should be judicious. The suction tip, if used, should be disposed of in a Ziploc bag. Step b: TL at the head of the bed places the NIV mask with the viral filter onto the patient and ensures proper sealing to avoid leak. The RRT “starts” the ventilator and preoxygenates until adequate oxygen saturation is attained.

Avoid bag-valve mask for preoxygenation. Meticulous preoxygenation should be done for 3–5 minutes. Step c: RN ensures patent intravenous access, assesses the vitals periodically, and communicates them to the TL.

Table 1. Role Allocation and Personnel Details of intubation team. S. No

Personnel

Stationed in

Responsibility

Team leader

Hot Zone (3-meter radius)

Performs tracheal intubation

Registered respiratory therapist

Hot Zone

Oversees airway and ventilator equipment

Registered nurse

Hot Zone

Ensures IV access and administers IV medications

Infection control nurse

Warm Zone

Oversees procedure and protocols

IV, intravenous; TL, Team Leader; RRT, Registered Respiratory Therapist; RN, Registered Nurse; ICN, Infection Control Nurse.

Volume 21, no. 5: September 2020

4. Peri Induction phase: Hot Zone Step a: RN administers the pre-calculated dose of the induction agent followed by the paralytic agent to the patient. Step b: Approrpiate patient positioning should be performed to maximise safe apnoea time.

Figure 1. A. List of equipment required in intubation trolley. PVC pipe sealed at one end (white arrow) filled with 1% sodium hypochlorite solution is used for discarding the soiled bougie and yellow bag (arrow head) for discarding the soiled wastes. B. Intubation unit.

1077

Western Journal of Emergency Medicine

Balakrishnan et al.

MIST Bundle for Infectious Diseases with Aerosol Hazard Table 2. List of equipment required in intubation trolley. Equipment Intubation unit (Figure 1B) Macintosh laryngoscope with size 3 and 4 blades in a sterile tray

6. Post-intubation Phase: Hot Zone Step a: Continue ventilation and monitor hemodynamics. Initiate early sedation and analgesia. Step b: Ensure all soiled equipment has been disposed of appropriately into the yellow bag for decontamination (Figure 1A).8 Step c: The order of doffing and decontamination is TL, followed by the RN, and then the RRT who should be separately overviewed and monitored by the ICN.

Numbers 1 1 each

iGel

Stethoscope

Cuffed ETT, size 7

ETT fixator

IV fluid with infusion set

Ziploc bags

CONCLUSION This sequence should guide healthcare professionals to minimize aerosol and droplet transmission during intubation and expedite better patient care. This approach does not involve significant resource intensification and can be done in resource-limited settings.

ETT, endotracheal tube; IV, intravenous.

Table 3. List of drugs used in Intubation. Drugs

Dose

Inducing agent Inj. etomidate or Inj. ketamine

0.3mg/kg IV 1-2mg/ kg IV

Paralytic agent Inj. rocuronium

1-1.2mg/kg IV

Address for Correspondence: Sanjan Asanaru Kunju, MD, DNB, Kasturba Medical College, Department of Emergency Medicine, Tiger Circle Road, Madhav Nagar, Manipal, Karnataka 576104, India. Email: sanjan.a@manipal.edu.

Inj, injection; mg, milligram; kg, kilogram; IV, intravenous.

5. Peri-intubation Phase: Hot Zone Step a: The RRT sets the ventilator on standby mode b after adequate paralysis and oxygen saturation is achieved. Step b: TL subsequently removes the NIV mask, which is disconnected from the ventilator by the RRT and placed in a Ziploc bag. Step c: TL performs laryngoscopy. During this time, the RRT is required to change the settings of the ventilator to “assist control mode ventilation.”. Once the vocal cords are visualized, the RRT hands over the intubating unit to the TL who should then pass the bougie between the cords under direct visualization. Video laryngoscope is a preferred choice for intubation of such patients, if available. Step d: The RRT assists in guiding the ET over the bougie and should subsequently inflate the cuff with the pre-filled air syringe. Step e: The RRT then removes the bougie and disposes of it in the preset disposition system (Figure 1A). Step f: The RRT then proceeds to connect the ET to the ventilator and convert the ventilator from standby to its preset settings. Simultaneously, the TL removes the laryngoscope and places it in a Ziploc bag. Step g: The RN confirms the position of the tube with fivepoint auscultation, following which the stethoscope should be disposed of in the Ziploc bag. End-tidal carbon dioxide confirmation is advised, if available. Western Journal of Emergency Medicine

REFERENCES 1.

Van Doremalen N, Bushmaker T, Morris DH, et al. Aerosol and surface stability of SARS-CoV-2 as compared with SARS-CoV-1. N Engl J Med. 2020;382:1564-7.

Li Q, Guan X, Wu P, Wang X, et al. Early transmission dynamics in Wuhan, China, of novel coronavirus-infected pneumonia. N Engl J Med. 2020;382:1199-1207.

Tang JW, Tambyah PA, Hui DS. Emergence of a novel coronavirus causing respiratory illness from Wuhan, China. J Infect. 2020;80(3):350-71.

Peng PW, Ho PL, Hota SS. Outbreak of a new coronavirus: what

Cheung JC, Ho LT, Cheng JV, Cham EY, et al. Staff safety during

anaesthetists should know. Br J Anaesth. 2020;124(5):497-501.

1078

emergency airway management for COVID-19 in Hong Kong. Lancet Respir Med. 2020;8(4):e19.

Volume 21, no. 5: September 2020

Balakrishnan et al.

MIST Bundle for Infectious Diseases with Aerosol Hazard

clinical-management-of-severe-acute-respiratory-infection-when-novel-

Detsky ME, Jivraj N, Adhikari NK, et al. Will this patient be difficult to intubate? The rational clinical examination systematic review. JAMA. 2019;321(5):493-503.

coronavirus-(ncov)-infection-is-suspected. Accessed March 29, 2020. 8.

World Health Organization. Safe management of wastes from

World Health Organization. Clinical management of severe acute

health-care activities: a summary. Available at: https://www.who.int/

respiratory infection (SARI) when COVID-19 disease is suspected -

water_sanitation_health/publications/wastemanag/en/. Accessed

Interium Guidance. Available at: https://www.who.int/publications-detail/

March 29, 2020.

Volume 21, no. 5: September 2020

1079

Western Journal of Emergency Medicine

Original Research

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario: A Crossover Study Torrey A. Laack, MD*† Franziska Pollok, MD† Benjamin J. Sandefur, MD* Aidan F. Mullan, MA‡ Christopher S. Russi, DO* Suraj M. Yalamuri, MD§

*Mayo Clinic, Department of Emergency Medicine, Rochester, Minnesota † Mayo Clinic, Mayo Clinic Multidisciplinary Simulation Center, Rochester, Minnesota ‡ Mayo Clinic, Division of Biomedical Statistics and Informatics, Rochester, Minnesota § Mayo Clinic, Department of Anesthesiology, Rochester, Minnesota

Section Editor: Ioannis Koutroulis, MD, MBA, PhD Submission history: Submitted June 5, 2020; Revision received June 30, 2020; Accepted July 29, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48574

Introduction: Barrier enclosures have been developed to reduce the risk of COVID-19 transmission to healthcare providers during intubation, but little is known about their impact on procedure performance. We sought to determine whether a barrier enclosure delays time to successful intubation by experienced airway operators. Methods: We conducted a crossover simulation study at a tertiary academic hospital. Participants watched a four-minute video, practiced one simulated intubation with a barrier enclosure, and then completed one intubation with and one without the barrier enclosure (randomized to determine order). The primary outcome measure was time from placement of the video laryngoscope at the lips to first delivered ventilation. Secondary outcomes were periprocedural complications and participant responses to a post-study survey. Results: Proceduralists (n = 50) from emergency medicine and anesthesiology had median intubation times of 23.6 seconds with practice barrier enclosure, 20.5 seconds with barrier enclosure, and 16.7 seconds with no barrier. Intubation with barrier enclosure averaged 4.5 seconds longer (95% confidence interval, 2.7-6.4, p < .001) than without, but was less than the predetermined clinical significance threshold of 10 seconds. Three complications occurred, all during the practice intubation. Barrier enclosure made intubation more challenging according to 48%, but 90% indicated they would consider using it in clinical practice. Conclusion: Experienced airway operators performed intubation using a barrier enclosure with minimal increased time to procedure completion in this uncomplicated airway model. Given potential to reduce droplet spread, use of a barrier enclosure may be an acceptable adjunct to endotracheal intubation for those familiar with its use. [West J Emerg Med. 2020;21(5)1080-1083.]

Western Journal of Emergency Medicine

INTRODUCTION Background On December 31, 2019, the World Health Organization (WHO) was first notified about a cluster of cases of pneumonia in Wuhan City, Hubei Province, China.1,2 The identified virus, named SARS-CoV-2, causes the disease now termed

1080

Volume 21, no. 5: September 2020

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario

COVID-19.2-4 On March 11, 2020, COVID-19 was officially labeled a pandemic.2,5 The infection continues to spread rapidly, and affects the majority of countries across the globe.6 Aerosol-generating procedures (AGP), such as bag-mask ventilation and endotracheal intubation (ETI), are high risk for nosocomial transmission of respiratory infections to healthcare providers.7,8 COVID-19 is transmitted by contact and droplet transmission, while aerosol spread remains uncertain.9 SARS-CoV-2 is stable in aerosol under laboratory conditions, indicating that aerosol transmission is a plausible means of transmission of COVID-19 to healthcare providers.10,11 Furthermore, SARS-CoV-2 may remain infectious in aerosols for hours.10 Transmission from critically ill patients is a significant source of anxiety for healthcare providers,12 and early reports suggested 19% of COVID-19 cases were in healthcare providers.13 Importance Guidelines have emerged to encourage safe care of patients during the COVID-19 pandemic while minimizing risk to healthcare providers.14 A central component of the guidelines is the proper use of personal protective equipment (PPE) to decrease nosocomial infection with COVID-19. However, shortages of adequate PPE are widespread.15 Novel strategies have been developed to mitigate nosocomial spread during ETI, especially given PPE shortages. Physical barriers to shield the proceduralist’s face from the patient’s airway have been developed. At the most basic level, a simple box made out of corrugated fiberboard and transparent plastic wrap has been described.16 Instructions for an “aerosol box,” which can be made inexpensively out of acrylic or polycarbonate material and is reusable after proper cleaning, are widely available on the Internet.17 This original design has since been modified to make it larger and more accommodating to different-sized patients while also better allowing other techniques, such as use of a gum elastic bougie.18,19 On May 1, 2020, the US Food and Drug Administration issued an Emergency Use Authorization for protective barrier enclosures.20 A barrier enclosure device was tested using dye and a simulated cough and was reported to potentially reduce contamination of the proceduralist.21 However, the use of barrier enclosure devices is not without criticism. Questions regarding the applicability to larger patients and the limited space in which to work for such a critical procedure as ETI remain unanswered.22,23 Prior studies have shown minimal impact of PPE use on ETI success,24,25 but use of a barrier enclosure may lead to breaches in PPE.19 Of paramount concern is that use of the barrier enclosures seems to be spreading through social media and the Internet despite little evidence supporting their safety or efficacy.22,23 Goals of This Investigation Before implementing widespread use of a novel device, Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Barrier enclosures have been developed to reduce the risk of COVID-19 transmission to healthcare providers during intubation. What was the research question? We sought to determine whether a barrier enclosure delays time to successful intubation by experienced airway operators. What was the major finding of the study? Experienced clinicians performed intubation using a barrier enclosure with minimal increased time to procedure completion. How does this improve population health? Given risk of COVID-19 transmission to healthcare providers during intubation, use of a barrier enclosure may be an acceptable adjunct for those familiar with its use.

testing is needed to establish an evidence base supporting its safety. Prior studies have demonstrated that negative patient outcomes are associated with delayed first-pass intubation success.26-28 The use of a barrier enclosure, especially by individuals who have had little or no prior experience with the device, may delay time to successful intubation or increase periprocedural complications. We sought to determine whether use of a barrier enclosure delays time to successful intubation by experienced airway operators. METHODS Study Design and Setting We conducted a nonblinded crossover simulation study involving the use of a video laryngoscope for simulated ETI under standard conditions with and without use of a barrier enclosure (Figure 1). Each participant watched an approximately four-minute video demonstrating proper use of the barrier enclosure and then had one intubation practice attempt with the barrier enclosure. Participants were assigned a number (consecutively) and randomized to either intubate with the barrier enclosure (odd numbers) or without (even numbers). They then crossed over and intubated without (odd numbers) or with the barrier enclosure (even numbers). After completion of the intubation attempts, each participant was asked to answer two brief questions. The study was reviewed and considered exempt by our institutional review board. 1081

Western Journal of Emergency Medicine

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario

Laack et al.

Enrollment

Randomization

Cross Over

Analysis

Figure 1. Design and flow of participants through the trial: All participants watched an introductory video and were then randomized into two groups. All groups performed a practice, barrier-enclosure baseline measurement, and depending on the group randomization, performed either a second trial with the barrier enclosure or no barrier enclosure. For the third trial the participants crossed over. All participants answered a two-question survey.

The barrier enclosure used was produced at our institution in collaboration with the Anatomic Modeling Lab and the Department of Engineering (Figure 2). It is modified from the box described by Hsien Yung Lai.17,18 It is constructed out of clear polycarbonate and measures 45.7 x 35.6 x 48.2 centimeters (cm) (18 x 14 x 19 inches). There are two circular cut-outs measuring 12.7 cm (5 inches) in diameter and placed 12.7 cm (5 inches) apart. The center of the holes are at a height of 30.5 cm (12 inches). The device additionally has side ports allowing oxygen inflow on one side and suction outflow on the opposite side. The idea is to create laminar flow in an attempt to capture small droplets or aerosols. The impact on aerosol and droplet capture, however, has not been confirmed. The enclosure is open on the side toward the patientâ&#x20AC;&#x2122;s feet and is covered with a disposable surgical drape to further mitigate droplet and aerosol spread and allow a second provider to pass tools to the proceduralist as needed. Western Journal of Emergency Medicine

The procedures were performed using a GlideScope video laryngoscope (Verathon Inc., Bothell, WA) with a size 3 cradle and a 7.0 millimeter endotracheal tube (ETT) with a GlideRite rigid stylet (Verathon Inc., Bothell, WA). The Airway Management Trainer (Laerdal Medical, Stavanger, Norway) mannequin was selected based on a balance of portability and realism. However, given the rigid plastic plate that secures the trainer, it was found to be higher off the bed than a sample live patient. We used a plastic storage container lid to support the enclosure (Figure 2) and better replicate the height of a sample patient, which also matched the SimMan 3G mannequin (Laerdal Medical, Stavanger, Norway) with approximately 25 cm from the highest point (chin) to the top of the barrier enclosure. Two study authors were present for each measurement. One recorded the time (FP) while the other (SMY, BJS, or TAL) performed the role of an assistant provider, assisting with tasks identically both with and without

1082

Volume 21, no. 5: September 2020

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario and the time is almost entirely dependent on the proceduralist. Secondary outcomes included recording complications, such as failed attempt at intubation or right mainstem intubation, and post-study questionnaire. After completion of the intubation attempts, each participant was asked: 1) “Did you feel that the intubation box made the procedure more challenging? YES/NO. If YES, what was most difficult about using the intubation box?” 2) “Would you consider using this device in clinical practice? YES/NO. If NO, why not?” Primary Data Analysis The primary outcome was comparison of the time to intubation (time from laryngoscope at the lips until the first ventilation) between intubation with and without the barrier enclosure. The predetermined meaningful difference in intubation outcome between arms was a time difference of greater than 10 seconds or failed intubation. With a predetermined sample size of 50 participants, we anticipated 90% power to detect a difference that was one-half the size of the standard deviation. We also considered the subjective responses from the participants regarding whether or not the barrier enclosure made the procedure more difficult and whether or not they would consider using it in clinical practice. Continuous features are summarized as medians and interquartile ranges (IQR). Categorical features are summarized as counts and percentages. Differences in intubation times between experimental conditions were assessed using pairedsample t-tests. The proportion of survey responses indicating “Yes” for each question was compared to a baseline 50% response rate using a one-proportion Z test. We computed confidence intervals (CI) for survey response rates using an asymptotic Gaussian approximation. All tests were two-sided and p-values below 0.05 were considered significant. For intubation times, a difference of greater than 10 seconds was predetermined as the threshold for clinical significance.

Figure 2. Set up with barrier enclosure placed around Airway Management Trainer (Laerdal Medical, Stavanger, Norway), GlideScope and endotracheal tube, as well as bag-valve mask within reach and visibility for participants, as well as the drape to protect the assistant. The barrier enclosure has a side port on each side, one for suction and one for oxygen insufflation to create a laminar flow and attempt to decrease droplet or aerosol spread through the circular cut outs or the draped side.

the barrier enclosure. The assistance was meant to replicate that which is generally provided during an ETI and included the following: handing the ETT to the proceduralist; assisting with removal of the stylet once requested or initiated by the proceduralist; inflating the ETT cuff once the tube was properly placed; and providing initial ventilation. The study took place in the emergency department and operating rooms at a large, tertiary academic medical center in May 2020. The simulation procedures were done in situ. Centers for Disease Control and Prevention (CDC) social distancing recommendations at the time of the study limited numbers of individuals who could meet, making large gatherings such as conferences or in-person teaching sessions impossible. The brief time requirement and in situ clinical setting allowed participants to complete the study while working clinically. Selection of Participants We recruited healthy volunteers who were employed at our hospital and are experienced clinicians who regularly perform ETI as part of their clinical practice. Participants were reached by e-mail and in person and given information about the study. Outcome Measures The primary outcome measure was time from placement of the laryngoscope blade at the lips to first successful ventilation of the lungs. This time period was chosen as it represents a period when the patient is most at risk for hypoxia Volume 21, no. 5: September 2020

RESULTS Characteristics of Study Subjects In total, 51 participants took part in this study including 22 anesthesiologists and 29 emergency physicians, nurse practitioners, or physician assistants. One participant had multiple practice attempts and was excluded. Cohort demographics are given in Table 1. Data was available for all 50 included participants for primary analysis. Main Results Table 2 provides a summary of intubation times for the practice, barrier enclosure, and no-barrier trials for each of the demographic sub-groups. Overall, time to intubation for the practice trial was the longest, with a median intubation time of 23.6 seconds (IQR: 18.8 - 28.9). Barrier enclosure trials were the second longest, with a median time of 20.5 seconds (IQR: 16.3 - 25.8), and no-barrier enclosure

1083

Western Journal of Emergency Medicine

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario Table 1. Summary of cohort demographics in trial of using a barrier enclosure box for intubation. Group 1: Box – No Box Group 2: No Box - Box Anesthesia (N = 10)

EM (N = 15)

Anesthesia (N = 11)

EM (N = 14)

Male

9 (90%)

6 (40%)

6 (55%)

9 (64%)

Female

1 (10%)

9 (60%)

5 (45%)

5 (36%)

Attending

9 (90%)

6 (40%)

10 (91%)

8 (57%)

Nurse Practitioner/ Physician Assistant

0 (0%)

2 (13%)

0 (0%)

2 (14%)

Gender

Role

Resident

1 (10%)

7 (47%)

1 (9%)

4 (29%)

Year of Residency

1 PGY 4

2 PGY 1 4 PGY 2 1 PGY 3

1 PGY 3

3 PGY 1 1 PGY 2

Prior Experience with Barrier Enclosure Yes

2 (20%)

1 (7%)

2 (19%)

0 (0%)

8 (80%)

14 (93%)

9 (81%)

14 (100%)

EM, emergency medicine; PGY, postgraduate year.

Table 2. Summary of intubation time (seconds) Practice Median [IQR]

Barrier Median [IQR]

No Barrier Median [IQR]

23.6 [18.8 – 28.9]

20.5 [16.3 – 25.8]

16.7 [10.8 – 19.1]

Male

21.0 [16.0– 24.6]

17.3 [13.2 – 23.5]

14.63 [10.1 – 18.5]

Female

27.3 [23.7 – 30.6]

22.4 [18.2 – 29.9]

17.5 [16.7 – 22.5]

27.4 [23.3 – 34.0]

24.4 [20.7 – 29.7]

17.8 [16.7 – 20.7]

Anesthesiology

17.8 [13.3 – 23.6]

15.5 [12.3 – 17.2]

10.4 [8.6 – 15.4]

Attending

23.6 [16.9 – 28.5]

17.3 [13.8 – 22.6]

15.6 [10.1 – 19.2]

Nurse Practitioner/ Physician Assistant

31.1 [28.5 – 32.8]

28.1 [25.5 – 34.8]

17.6 [16.9 – 21.4]

Resident

21.9 [20.1 – 29.8]

22.1 [19.1 – 25.6]

17.0 [16.5 – 18.8]

24.3 [19.2 – 29.8]

20.63 [16.7 – 26.7]

16.7 [12.2 – 19.4]

Yes

18.7 [17.8 – 20.9]

15.46 [12.5 – 18.7]

10.6 [7.7 – 17.0]

Overall Gender

Specialty

Role

Prior Experience with Barrier Enclosure

IQR, interquartile range; EM, emergency medicine

trials were the shortest with a median time of 16.7 (IQR: 10.8 - 19.1) seconds. There were three complications reported during the practice intubations: one right mainstem intubation and two episodes of the stylet being removed but then reinserted in order to appropriately advance the tube. No complications occurred during either the barrier enclosure trials or the no-barrier trials. Western Journal of Emergency Medicine

Figure 3 shows the difference in intubation times for the barrier enclosure and no-barrier trials for all participants. Of the 50 participants, 42 (84%) took longer in the barrier enclosure intubation compared to the no-barrier trial. The barrier intubation was found to take significantly longer than the no-barrier intubation, with an average increased intubation time of 4.5 seconds (95% CI, 2.7-6.4, p < .001). 1084

Volume 21, no. 5: September 2020

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario

Figure 3. Difference in intubation time comparing use of barrier with use of no-barrier enclosure. Positive numbers reflect a longer time without a barrier enclosure; negative numbers reflect a longer time with the barrier enclosure.

Although the increase in time did not meet the predetermined overall clinical significance threshold of 10 seconds, in nine participants the intubation time was prolonged by more than 10 seconds for the barrier enclosure compared to no barrier. In addition, when comparing the practice and no-barrier trials, the practice trials were an average of 9.8 seconds longer (95% CI, 6.3 – 13.3, p < .001). Figure 4 shows the difference in intubation times for the practice barrier enclosure and follow-up barrier enclosure trials for all participants. Overall, the practice intubation took significantly longer than the follow-up barrier shield intubation, with an average increased intubation time of 5.2 seconds (95% CI, 2.0-8.5, p = .002). Out of the 50 participants, 13 (26%) took longer in the follow-up barrier intubation compared to the practice. Table 3 provides details regarding the respondents who found the barrier enclosure made intubation more difficult and those who would consider using it in a clinical setting. Fortyeight percent (24/50) of participants indicated that the barrier enclosure made the intubation more challenging (p = 0.89). Reasons cited for increased difficulty included the following: challenges with the stylet removal; decreased dexterity and range of motion; trouble handling the ETT within the enclosure; and difficulty inserting the laryngoscope and ETT into the mouth. Ninety percent (45/50) of respondents indicated they would consider using a barrier enclosure in clinical practice (p<0.001). Twenty-one of the 24 (88%) Volume 21, no. 5: September 2020

participants who indicated that the barrier enclosure made the procedure more challenging would still consider using it in clinical practice. DISCUSSION Protection of healthcare providers from COVID-19 infection while allowing safe care of patients is paramount. PPE supply shortages have been an ongoing dilemma during the pandemic. For these reasons, innovative strategies to decrease contagion during AGP are welcome. Furthermore, even if PPE supplies are robust, breaches in PPE during AGP and in donning and doffing can occur. Therefore, strategies to decrease droplet or aerosol spread of virus during airway management can be helpful in all settings. The barrier enclosure device may offer one such benefit. However, its safety has not previously been demonstrated. The authors applaud Dr. Lai for allowing open access to the design and rationale of his novel “aerosol box.”17 We have modified our own barrier enclosure to allow additional space for tube passage, stylet removal, bag-valve mask ventilation, and even use of a bougie or other airway adjuncts. Our barrier enclosure uses tubed-in oxygen and high-flow suction to create laminar air flow within the enclosure. It seems unlikely that any barrier enclosure can eliminate aerosolization of viral particles; the term “aerosol box,” as pointed out by Chan, is somewhat of a misnomer.22 Therefore, use of a barrier enclosure does not preclude the need for full PPE. This

1085

Western Journal of Emergency Medicine

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario

Figure 4. Difference in intubation time for practice and follow-up use of barrier enclosure. Positive numbers reflect a longer time during the practice barrier enclosure intubation (prior to study); negative numbers reflect a longer time during the follow-up barrier-enclosure intubation (during study).

Table 3. Summary of â&#x20AC;&#x153;yesâ&#x20AC;? responses to survey questions. Respondents

Q1: More Challenging?

Q2: Use in Practice?

24 (48%)

45 (90%)

Male

13 (43%)

27 (90%)

Female

11 (55%)

18 (90%)

19 (66%)

26 (90%)

Anesthesiology

5 (24%)

19 (91%)

Attending

13 (39%)

29 (88%)

Nurse Practitioner/ Physician Assistant

2 (50%)

4 (100%)

Resident

9 (69%)

12 (92%)

21 (47%)

40 (89%)

Yes

3 (60%)

5 (100%)

Overall Gender

Specialty

Role

Experience with Barrier Enclosure

tempers the potential benefits of these devices and must be weighed against the potential risks of their use. In our study cohort, 48% of the participants felt using a barrier enclosure made intubation more challenging, yet 90% Western Journal of Emergency Medicine

of the participants would still consider using it with a real patient. This highlights that the participants are willing to accept a more challenging and potentially longer intubation process to further minimize droplet and aerosol spread. While 1086

Volume 21, no. 5: September 2020

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario

both video and direct laryngoscopy are regularly performed at our institution, in response to the COVID-19 pandemic, all emergent intubations are initially performed with video laryngoscopy. This is consistent with previously mentioned recommendations to maximize first-pass success.14 Video laryngoscopy has been shown to have higher first-pass success rates and fewer complications.29,30 In addition, direct laryngoscopy generally requires that the proceduralist’s face is closer to the patient’s mouth than is required for video laryngoscopy. For these reasons, we chose to test the barrier shield using the video laryngoscopy technique. Our results confirmed that first-pass ETI with the video laryngoscopy technique by experienced clinicians was delayed by an average of 4.5 seconds when using a barrier enclosure. For most situations, this level of delay is likely of no significant consequence to the patient. However, the delay was seen in an uncomplicated simulated intubation and could be much greater when dealing with a difficult airway situation. We did see an expected improvement from the initial baseline use of the barrier enclosure to the second attempt with the device, decreasing the time to intubation by an average of 5.2 seconds. We do not know whether additional practice would further narrow the delay compared to intubation without a barrier enclosure, but our findings suggest that even one practice with the device was helpful. LIMITATIONS We tested only a standard adult intubation using a video laryngoscope technique. We did not assess the impact the barrier enclosure would have on more challenging intubations or other techniques such as direct laryngoscopy or use of a gum elastic bougie. Neither the researchers nor the participants were blinded as to whether or not they were using the barrier enclosure. While the research team attempted to be consistent across groups, it is possible that the lack of blinding could have affected how assistance was given to participants. Also, the participants were aware they were being timed. While we encouraged them to try and perform the procedure as they would in an actual clinical setting, it is possible that some rushed to try and complete the procedure in a shorter period of time. The study was conducted in situ to be able to include as many participants while they were working clinically. In the interest of time, we were not able to conduct a second trial with the same participants to confirm our findings. We did record the year of each resident participant, but did not capture years of experience for attendings. While delays in first-pass intubation success have been associated with worse patient outcomes,26-28 there is not a clear cut-point as to when delays become clinically meaningful. Based on our clinical experience, we chose a delay of more than 10 seconds as potentially clinically important during this phase of the procedure when the patient is paralyzed and at highest risk for hypoxia; however, there is little data to support any specific time point. Volume 21, no. 5: September 2020

Many modifications have now been made to the originally described “aerosol box,” which may limit applicability if using a different type of barrier enclosure. Finally, this was a simulation study, which limits applicability to actual patients. While this was a simulated study and could not completely replicate actual clinical conditions, we did not feel it would be ethical to proceed with initial testing of this novel device on actual patients. However, simulation can be valuable in testing innovations in healthcare.31 Simulation has also been shown to be more effective than non-simulation techniques in teaching airway management,32 and mannequin-based models produce similar intubation times and first-pass success compared to cadaver models.33 CONCLUSION Whether or not to use a barrier enclosure is a decision that should be made carefully. Given the minimal increased time to first-pass success in an uncomplicated airway along with potential to decrease droplet spread during endotracheal intubation, use of a barrier enclosure appears to be an acceptable technique for those who are familiar with the device and the necessary adaptations to complete the procedure. Further research should focus on the impact of barrier enclosure use during difficult intubation scenarios and actual clinical encounters. Additionally, further robust investigation into how well these devices reduce droplet or aerosol spread of virus would also be of interest. ACKNOWLEDGMENTS The authors acknowledge the assistance of Dr. David A. Cook, MD, with initial planning for the study and the staff of the Mayo Clinic Multidisciplinary Simulation Center for their support of this effort. The authors also acknowledge the contributions of Dr. Jonathan M. Morris, MD, of the Anatomic Modeling Lab and the Department of Engineering in the planning, design, and production of the barrier enclosure.

Address for Correspondence: Torrey A. Laack, MD, Mayo Clinic, Department of Emergency Medicine, 200 First Street S.W. Rochester, MN 55905. Email: laack.torrey@mayo.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Laack et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

1087

Western Journal of Emergency Medicine

Laack et al.

Barrier Enclosure for Endotracheal Intubation in a Simulated COVID-19 Scenario REFERENCES

17. Lai HY. Aerosol Box. 2020. Available at: https://sites.google.com/ view/aerosolbox/home?authuser=0. Accessed May 11, 2020.

1. Lu H, Stratton CW, Tang YW. Outbreak of pneumonia of unknown

18. Malik J, Jenner C, Ward P. Maximising application of the aerosol box

etiology in Wuhan, China: the mystery and the miracle. J Med Virol.

in protecting healthcare workers during the COVID‐19 pandemic.

2020;92(4):401-2.

Anaesthesia. 2020;75(7):962-77.

2. Kakodkar P, Kaka N, Baig MN. A comprehensive literature review

19. Begley JL, Lavery KE, Nickson CP, et al. The aerosol box for

on the clinical presentation, and management of the pandemic

intubation in COVID-19 patients: an in-situ simulation crossover

coronavirus disease 2019 (COVID-19). Cureus. 2020;12(4):e7560.

study. Anaesthesia. 2020;75(8):1014-21.

3. Sohrabi C, Alsafi Z, O’Neill N, et al. World Health Organization

20. Administration USFaD. Emergency Use Authorization for protective

declares global emergency: a review of the 2019 novel coronavirus

barrier enclosures. 2020. Available at: https://www.fda.gov/

(COVID-19). Int J Surg. 2020;76:71-6.

media/137584/download. Accessed May 14, 2020.

4. WHO. Coronavirus Disease (COVID-2019) Situation Report -

21. Canelli R, Connor CW, Gonzalez M, et al. Barrier enclosure during

114. 2020. Available at: https://www.who.int/docs/default-source/

endotracheal intubation. N Engl J Med. 2020;382:1957-8.

coronaviruse/situation-reports/20200513-covid-19-sitrep-114.

22. Chan A. Should we use an “aerosol box” for intubation? 2020.

pdf?sfvrsn=17ebbbe_4. Accessed May 14, 2020.

Available at: https://litfl.com/should-we-use-an-aerosol-box-for-

5. Mahase E. Covid-19: WHO declares pandemic because of “alarming

intubation/. Accessed May 14, 2020.

levels” of spread, severity, and inaction. BMJ. 2020;368:m1036. 6. Pan A, Liu L, Wang C, et al. Association of public health interventions

23. Kearsley R. Intubation boxes for managing the airway in patients with COVID-19. Anaesthesia. 2020;75(7):962-77.

with the epidemiology of the COVID-19 outbreak in Wuhan, China.

24. Aberle SJ, Sandefur BJ, Sunga KL, et al. Intubation efficiency and

JAMA. 2020;323(19):1915-23.

perceived ease of use of video laryngoscopy vs direct laryngoscopy

7. Seto WH. Airborne transmission and precautions: facts and myths. J

while wearing HazMat PPE: a preliminary high-fidelity mannequin

Hosp Infect. 2015;89(4):225-8.

study. Prehosp Disaster Med. 2015;30(3):259-63.

8. Chan MTV, Chow BK, Lo T, et al. Exhaled air dispersion during bag-

25. Adler MD, Krug S, Eiger C, et al. [Ahead of Print]. Impact of personal

mask ventilation and sputum suctioning: implications for infection

protective equipment on the performance of emergency pediatric

control. Sci Rep. 2018;8(1):198.

tasks. Pediatr Emerg Care. February 24, 2020.

9. Cook TM. Personal protective equipment during the coronavirus

26. Sakles JC, Chiu S, Mosier J, et al. The importance of first-pass

disease (COVID) 2019 pandemic: a narrative review. Anaesthesia.

success when performing orotracheal intubation in the emergency

2020;75(7):920-7.

department. Acad Emerg Med. 2013;20(1):71-8.

10. van Doremalen N, Bushmaker T, Morris DH, et al. Aerosol and

27. Mort TC. Emergency tracheal intubation: complications

surface stability of SARS-CoV-2 as compared with SARS-CoV-1. N

associated with repeated laryngoscopic attempts. Anesth Analg.

Engl J Med. 2020;382(16):1564-7.

2004;99(2):607-13.

11. Wilson NM, Norton A, Young FP, et al. Airborne transmission of severe acute respiratory syndrome coronavirus-2 to healthcare

28. Natt BS, Malo J, Hypes CD, et al. Strategies to improve first attempt

workers: a narrative review. Anaesthesia. 2020;75(8):1086-95.

success at intubation in critically ill patients. Br J Anaesth. 2016;117 Suppl 1:i60-i68.

12. Shanafelt T, Ripp J, Trockel M. Understanding and addressing

29. Liu DX, Ye Y, Zhu YH, et al. Intubation of non-difficult airways using

sources of anxiety among health care professionals during the

video laryngoscope versus direct laryngoscope: a randomized,

COVID-19 pandemic. JAMA. 2020;323(21):2133-4.

parallel-group study. BMC Anesthesiol. 2019;19(1):75.

13. CDC COVID-19 Response Team Morbidity & Mortality Weekly Report. Prevention CfDCa. Characteristics of Health Care Personnel

30. De Jong A, Molinari N, Conseil M, et al. Video laryngoscopy versus

with COVID-19 — United States, February 12–April 9, 2020. 2020.

direct laryngoscopy for orotracheal intubation in the intensive care

Available at: https://www.cdc.gov/mmwr/volumes/69/wr/pdfs/

unit: a systematic review and meta-analysis. Intensive Care Med. 2014;40(5):629-39.

mm6915e6-H.pdf. Accessed May 14, 2020.

31. Madani A, Gallix B, Pugh CM, et al. Evaluating the role of simulation

14. Cook TM, El-Boghdadly K, McGuire B, et al. Consensus guidelines for managing the airway in patients with COVID-19: Guidelines from

in healthcare innovation: recommendations of the Simnovate Medical

the Difficult Airway Society, the Association of Anaesthetists the

Technologies Domain Group. BMJ STEL. 2017;3(Suppl 1):S8-14. 32. Kennedy CC, Cannon EK, Warner DO, et al. Advanced airway

Intensive Care Society, the Faculty of Intensive Care Medicine and

management simulation training in medical education: a systematic

the Royal College of Anaesthetists. Anaesthesia. 2020;75(6):785-99.

review and meta-analysis. Crit Care Med. 2014;42(1):169-78.

15. Artenstein AW. In pursuit of PPE. N Engl J Med. 2020;382(18):e46. 16. Lai YY and Chang CM. [Ahead of Print]. A carton-made protective

33. Pedigo R, Tolles J, Watcha D, et al. Teaching endotracheal intubation

shield for suspicious/confirmed COVID-19 intubation and extubation

using a cadaver versus a manikin-based model: a randomized

during surgery. Anesth Analg. April 10, 2020.

controlled trial. West J Emerg Med. 2019;21(1):108-14.

Western Journal of Emergency Medicine

1088

Volume 21, no. 5: September 2020

Educational Advances

Mastery Learning Ensures Correct Personal Protective Equipment Use in Simulated Clinical Encounters of COVID-19 Nicholas Pokrajac, MD Kimberly Schertzer, MD Cori M. Poffenberger, MD Alâ&#x20AC;&#x2122;ai Alvarez, MD Paloma Marin-Nevarez, MD Christopher Winstead-Derlega, MD Michael A. Gisondi, MD

Stanford University, Department of Emergency Medicine, Palo Alto, California

Section Editor: Ioannis Koutroulis, MD and Danya Khoujah, MBBS Submission history: Submitted May 11, 2020; Revision received June 25, 2020; Accepted June 24, 2020 Electronically published July 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.48132

Introduction: The correct use of personal protective equipment (PPE) limits transmission of serious communicable diseases to healthcare workers, which is critically important in the era of coronavirus disease 2019 (COVID-19). However, prior studies illustrated that healthcare workers frequently err during application and removal of PPE. The goal of this study was to determine whether a simulation-based, mastery learning intervention with deliberate practice improves correct use of PPE by physicians during a simulated clinical encounter with a COVID-19 patient. Methods: This was a pretest-posttest study performed in the emergency department at a large, academic tertiary care hospital between March 31â&#x20AC;&#x201C;April 8, 2020. A total of 117 subjects participated, including 56 faculty members and 61 resident physicians. Prior to the intervention, all participants received institution-mandated education on PPE use via an online video and supplemental materials. Participants completed a pretest skills assessment using a 21-item checklist of steps to correctly don and doff PPE. Participants were expected to meet a minimum passing score (MPS) of 100%, determined by an expert panel using the Mastery Angoff and Patient Safety standard-setting techniques. Participants that met the MPS on pretest were exempt from the educational intervention. Testing occurred before and after an in-person demonstration of proper donning and doffing techniques and 20 minutes of deliberate practice. The primary outcome was a change in assessment scores of correct PPE use following our educational intervention. Secondary outcomes included differences in performance scores between faculty members and resident physicians, and differences in performance during donning vs doffing sequences. Results: All participants had a mean pretest score of 73.1% (95% confidence interval [CI], 70.975.3%). Faculty member and resident pretest scores were similar (75.1% vs 71.3%, p = 0.082). Mean pretest doffing scores were lower than donning scores across all participants (65.8% vs 82.8%, p<0.001). Participant scores increased 26.9% (95% CI of the difference 24.7-29.1%, p<0.001) following our educational intervention resulting in all participants meeting the MPS of 100%. Conclusion: A mastery learning intervention with deliberate practice ensured the correct use of PPE by physician subjects in a simulated clinical encounter of a COVID-19 patient. Further study of translational outcomes is needed. [West J Emerg Med. 2020;21(5)1089-1094.]

Volume 21, no. 5: September 2020

1089

Western Journal of Emergency Medicine

Mastery Learning Ensures Correct PPE Use with COVID-19 Patients INTRODUCTION The pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV2) and its resultant clinical illness, coronavirus disease 2019 (COVID-19), has stressed healthcare systems across the world. Nearly 10,000 healthcare workers contracted COVID-19 in the United States (US) alone during the period from February 12â&#x20AC;&#x201C;April 9, 2020.1 SARS-CoV2 spreads by means of surface contamination, exposure to droplets containing viral particles, and through aerosolization, particularly during high-risk procedures.2 Healthcare workers are at increased risk for infection given frequent exposure to the virus during routine patient care. Proper use of personal protective equipment (PPE) by healthcare workers is well established to decrease the rate of infectious disease transmission, including by means of selfcontamination.3 However, prior studies have demonstrated that healthcare workers are inconsistent in the proper use of PPE. Contamination rates during donning and doffing of PPE range from 10-100%.3,4 Deviations from accepted protocols for donning and doffing PPE also occur in 50-87% of healthcare workers.4-6 Therefore, effective education to improve donning, wearing, and doffing of PPE is critical for healthcare worker safety. A number of educational interventions to improve correct use of PPE have been performed with varying success.3,7 A growing body of evidence suggests that simulation-based mastery learning (SBML) is more effective than other educational techniques to attain procedural skill mastery.8,9 SBML is an educational technique that must include the following: 1) baseline testing of a target skill; 2) discrete learning objectives organized by rising difficulty; 3) attentive learner engagement during the activity; 4) a defined minimum passing standard (MPS); 5) testing during the educational process to direct learning and evaluate achievement of the MPS; 6) advancement after reaching the MPS; and 7) continued practice until the MPS is achieved.10 This method often is paired with deliberate practice, which requires highly motivated students to engage in focused, repetitive practice toward a specified goal with informative feedback to correct errors. The goal of SBML is to have all participants achieve an expert level of skill with minimal to no variation, which is crucial in patient care environments. Moreover, implementation of a SBML curriculum may improve translational outcomes.11-13 The goal of this study was to determine whether mastery learning methodology can improve physician ability to correctly don and doff PPE during a simulated clinical encounter with a COVID-19 patient. METHODS Study Design and Approval Physician subjects participated in a mastery learning educational intervention with a simulated clinical encounter of a patient with COVID-19.14 Prior to the intervention, all participants had received institution-mandated N95 mask-fit Western Journal of Emergency Medicine

Pokrajac et al.

testing and training on the proper use of PPE via an online video and a supplemental online infographic demonstrating steps of donning and doffing of PPE. Participants were assessed on their ability to correctly don and doff PPE using a checklist before and after the intervention on the same day. The study was approved by our local institutional review board (IRB #55851). Participants and Study Setting Participants included clinical faculty members and emergency medicine (EM) resident physicians in the ED at a large, academic tertiary care center from March 31â&#x20AC;&#x201C;April 8, 2020. The assessment was conducted in an administrative office space designed to simulate a medical examination room with a door and no anteroom, with a patient under airborne, droplet, and contact precautions. Outcome Measures and Measuring Instrument Participants were assessed individually prior to the intervention using a 21-item checklist of steps for donning and doffing PPE using a double-glove technique (Online Supplement).15,16 The checklist was developed by one author and adapted from existing best-practices guidelines on PPE use from the US Centers for Disease Control and Prevention (Atlanta, GA) and Stanford University (Palo Alto, CA,).15 Additional authors with expertise in PPE use, medical education, and checklist design reviewed and modified the checklist. The checklist underwent final review and approval by infection control specialists at our institution to ensure completeness, compliance, and internal consistency within the hospital system. Equipment consisted of Medline isolation gowns, Medline Fitguard nitrile exam gloves (Medline Industries, Inc, Northfield, IL), and DeRoyal SPEyes Eye ShieldZ (DeRoyal Industries, Powell, TN). Due to a national shortage of N95 face masks at the time of this project, a simple substitution of quarter-inch elastic bands stapled to an 8-ounce paper bowl was used (Figure 1).

Figure 1. A simple substitution of an N95 facemask for simulated patient encounters.

1090

Volume 21, no. 5: September 2020

Pokrajac et al.

Mastery Learning Ensures Correct PPE Use with COVID-19 Patients

Adherence to checklist items during testing was assessed by four reviewers. All reviewers underwent a one-hour training on the use of the checklist as a rating instrument, which consisted of checklist review, demonstration of correct PPE use by a study author, deliberate practice, and mock assessments. Reviewers were instructed to give zero points to items not done or performed incorrectly and one point to items performed correctly. All reviewers independently scored at least 10% of participants to determine inter-rater reliability of the instrument. A MPS for correct completion of checklist items was determined by 16 experts using a combination of Mastery Angoff and Patient Safety approaches.17-19 All 16 experts were EM clinical faculty members, 10 of whom have advanced training in medical education, three in medical simulation, and one in emergency medical services.

faculty members (3.6%) and one resident physician (1.6%) successfully achieved the MPS on pre-intervention assessment. Mean pretest score among all participants was 73.1% (95% confidence interval [CI], 70.9-75.3%). There was no significant difference between the mean pretest scores of faculty members and resident physicians (75.1% vs 71.3%, p = 0.082) (Figure 2). Mean pretest doffing scores were significantly lower than donning scores (65.8% vs 82.8%, p<0.001). The items most commonly not completed or incorrectly completed included “adjusts nosepiece of mask,”, “demonstrates mask seal check,” “doffs eye shield in room,” “disposes of eye shield in room,” and “performs hand hygiene on inner gloves” (Table 2).

Educational Intervention Physician subjects individually participated in a mastery learning educational intervention if they did not achieve the MPS on the pretest assessment. The intervention consisted of an in-person expert demonstration of proper donning and doffing of PPE using the 21 steps outlined in the checklist, followed by a 20-minute opportunity for deliberate practice with feedback. If participants again did not achieve the MPS on reassessment, they were given additional opportunities for deliberate practice until the MPS was achieved. Final scores were determined by reviewers unblinded to initial participant assessments.

Table 1. Demographic information of participants in a simulationbased mastery learning course with deliberate practice to improve use of personal protective equipment. Participants (%) (N=117) Professoriate rank of faculty members Professor Associate

13 (11.1)

Assistant

28 (23.9)

Instructor

8 (6.8)

Total faculty members

56 (47.9)

Postgraduate year (PGY) of resident physicians

RESULTS A total of 117 physician subjects participated in the study, including 56 faculty members (56/88, 63.6%) and 61 EM resident physicians (61/62, 98.3%). Participant demographic information is summarized in Table 1. Standard setting using a Patient Safety approach resulted in 19/21 of the checklist items deemed critical for safety. A Mastery Angoff score calculated for the two non-critical items was 90.2%. Requiring completion of all items deemed critical from the Patient Safety approach plus 90.2% of two non-critical items resulted in the final MPS set at 100%.17-19 Agreement between assessors across checklist items ranged from moderate to strong (κ = 0.70 to 0.87). Two

PGY4

15 (12.8)

PGY3

15 (12.8)

PGY2

14 (12.0)

PGY1

16 (13.7)

Total resident physicians

100%

Percent correct

Statistical Analysis We performed statistical analysis using SPSS Statistics for Windows, Version 24, (IBM Corp., Armonk, NY). Interrater reliability was determined by calculating Cohen’s kappa statistic. We used a two-tailed paired T-test to compare pre-and posttest scores. The difference between faculty and resident physicians’ scores was calculated using a two-tailed Student’s T-test.

Volume 21, no. 5: September 2020

7 (6.0)

61 (52.1)

MPS

80% 60% 40% 20% 0%

Resident pretest score Faculty pretest score Figure 2. Comparison of resident vs faculty member pretest scores. MPS, minimum passing score.

1091

Western Journal of Emergency Medicine

Pokrajac et al.

Mastery Learning Ensures Correct PPE Use with COVID-19 Patients

Table 2. Mastery Learning Checklist* for donning and doffing personal protective equipment, with pre-intervention checklist performance by item and participant role. Items not completed

Faculty members (N = 56)

Residents (N = 61)

Total participants (N = 117)

11 (19.6)

19 (31.1)

30 (25.6)

Dons inner layer of gloves

2 (3.6)

3 (4.9)

5 (4.3)

Dons gown

9 (16.1)

3 (4.9)

12 (10.3)

Dons mask

1 (1.8)

5 (8.2)

6 (5.1)

Adjusts nosepiece of mask

14 (25)

28 (45.9)

42 (35.9)

Donning sequence Performs hand hygiene

Demonstrates mask seal check

29 (51.8)

48 (78.7)

77 (65.8)

Dons eye shield

2 (3.6)

0 (0.0)

2 (1.7)

Dons outer layer of gloves

3 (5.4)

4 (6.6)

7 (6.0)

Enters room and closes door

0 (0.0)

Doffing sequence Begins doffing 6’ from patient

7 (12.5)

3 (4.9)

10 (8.5)

Doffs gown with outer gloves

15 (26.8)

14 (23.0)

29 (24.8)

3 (5.4)

3 (4.9)

6 (5.1)

Performs hand hygiene on inner gloves

25 (44.6)

25 (41.0)

50 (42.7)

Doffs eye shield in room

30 (53.6)

42 (68.9)

72 (61.5)

Disposes of eye shield in room

30 (53.6)

43 (70.5)

73 (62.4)

Performs hand hygiene on inner gloves

43 (76.8)

52 (85.2)

95 (81.2)

2 (3.6)

3 (4.9)

5 (4.3)

41 (73.2)

44 (72.1)

85 (72.6)

Disposes of gown and outer gloves in room

Exits room and closes door Performs hand hygiene on inner gloves Removes and disposes of mask

4 (7.1)

9 (14.8)

13 (11.1)

Removes and disposes of gloves

20 (35.7)

19 (31.1)

39 (33.3)

Performs hand hygiene 2 (3.6) 1 (1.6) 3 (2.6) *Two-glove technique for personal protective equipment use with airborne, contact, and droplet precautions. See Online Supplement 1 for definitions of “complete” for each checklist item.

After our educational intervention, the mean participant score increased 26.9% (95% CI of the difference 24.7-29.1%, p<0.001) (Figure 3). No participants required more than 20 minutes to achieve mastery. DISCUSSION This study demonstrated that a (SBML) intervention with deliberate practice led to significant improvement in both faculty and resident physicians’ ability to correctly don and doff PPE. On pretest assessment, faculty and resident participants demonstrated frequent errors during donning and doffing of PPE despite completing comprehensive, institutionmandated online training. Similar to prior studies, errors were more common during doffing of protective equipment, which is when providers are at greatest risk of self-contamination.20 Therefore, these results highlight a critical role for SBML to improve correct PPE use and suggest that the sole utilization of online modules for PPE use may be inadequate for workplace safety. SBML is a highly effective method of teaching Western Journal of Emergency Medicine

procedures.21 The focus of SBML is the achievement of a fixed learning outcome; training time varies between participants to allow adequate opportunities for deliberate practice with feedback. While completion time varied, no participant in our study exceeded 20 minutes of training time and all participants met our predetermined learning outcome on post-intervention testing. This approach is in contrast to more common instructional techniques, in which teaching time is fixed and participant achievement varies. As such, rigorous adherence to SBML principles as used in this study likely represents the gold standard for procedural training.13 Previous research demonstrated SBML to be an effective method of teaching both invasive and noninvasive procedures, including lumbar puncture,9 central line insertion,11,12 paracentesis,22 and thoracentesis,8,23 among others.13 Mastery learning also achieves translational outcomes that result in better patient care (T3) as defined by lower complications rates during high-risk procedures.11,12,23 The proper donning and doffing of PPE for aerosolized infection is similarly high risk and correct completion (fixed achievement)

1092

Volume 21, no. 5: September 2020

Pokrajac et al.

Mastery Learning Ensures Correct PPE Use with COVID-19 Patients

MPS 100%

Percent correct

80%

60%

40%

20%

0% Total pretest score

Donning pretest score

Doffing pretest score

Total postest score

Figure 3. Comparison of pretest and posttest scores across all participants. MPS, minimum passing score.

of the don/doff sequence is absolutely necessary to minimize risk to patients and providers. We designed our mastery learning intervention to simulate patient care performed in locations without anterooms, which comprise the majority of hospital and outpatient clinical spaces. This allows for authenticity in the logistical challenges present when caring for a high frequency of COVID-19 patients requiring airborne, droplet, and contact precautions. In addition, double-glove technique mitigates skin contact with potential surface contaminants while exiting a patientâ&#x20AC;&#x2122;s room. Providers may also have improved hand comfort with the application of alcohol-based gels over inner gloves rather than skin, given the frequency of required hand sanitization during donning and doffing of PPE.24 The double-glove technique is presumed to increase provider safety for a variety of infectious diseases beyond COVID-19. Finally, our intervention standardized PPE use among our physicians by allowing participants the opportunity to ask nuanced questions and practice repeatedly. This supportive and psychologically safe training may be especially important to mitigate provider anxiety in anticipation of a COVID-19 surge. A shared mental model also allows observation and direct feedback by facultyresident pairs during donning and doffing of PPE. The ability to train the majority of our emergency physicians in a oneweek time period suggests that comprehensive physician training is feasible. Future study is needed to determine the potential translational impact of our intervention. Volume 21, no. 5: September 2020

LIMITATIONS This study had several limitations. First, the checklist instrument required use of a double-glove rather than single-glove technique, the latter of which was included as part of the pre-existing, institution-mandated online training for PPE use. As a result, providers may have been less familiar with the double-glove technique on pretest. Double-glove technique was chosen due to the prevalence of patient care areas without anterooms at the institution. This technique was also chosen in an effort to optimize both comfort and personal protection in light of changing guidelines during the pandemic. Second, the strict MPS of the assessment may have decreased the initial pass rate on pretest. Third, reviewers were unblinded to pretest results, which may have influenced posttest scores. Fourth, due to time constraints, participants completed repeat testing immediately following deliberate practice, which limited assessment for skill retention. Finally, while previous studies demonstrated improvements in translational outcomes following SBML, it is still unclear whether this intervention will result in an observable change of behavior during patient care. CONCLUSION Mastery learning methodology with deliberate practice is an effective and feasible educational modality for training a large number of physicians in the proper use of PPE in simulated clinical encounters of patients with COVID-19.

1093

Western Journal of Emergency Medicine

Pokrajac et al.

Mastery Learning Ensures Correct PPE Use with COVID-19 Patients Before our training intervention, very few providers passed a rigorous assessment of PPE donning and doffing despite institution-mandated, online PPE training. After undergoing the SBML intervention, all participants successfully completed assessment with 100% accuracy. In addition, the double-glove technique may offer additional provider protection when caring for a high volume of COVID-19 patients in treatment areas that lack anterooms. Further study of translational outcomes resulting from our intervention is needed.

9. Barsuk JH, Cohen ER, Caprio T, et al. Simulation-based education with mastery learning improves residents’ lumbar puncture skills. Neurology. 2012;79(2):132-7. 10. McGaghie WC, Siddall VJ, Mazmanian PE, et al. Lessons for continuing medical education from simulation research in undergraduate and graduate medical education: effectiveness of continuing medical education: American College of Chest Physicians Evidence-Based Educational Guidelines. Chest. 2009;135(3 Suppl):62S-8. 11. Barsuk JH, McGahie WC, Cohen ER, et al. Simulation-based mastery

Address for Correspondence: Nicholas Pokrajac, MD, Stanford University, Department of Emergency Medicine, 900 Welch Drive, Suite 350, Palo Alto, CA 94304. Email: pokrajac@stanford.edu.

learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med. 2009;37(10):2697-701. 12. Barsuk JH, Cohen ER, Potts S, et al. Dissemination of a simulationbased mastery learning intervention reduces central line-associated

bloodstream infections. BMJ Qual Saf. 2014;23(9):749-56. 13. McGaghie WC, Issenberg SB, Barsuk JH, et al. A critical review of simulation-based mastery learning with translational outcomes. Med Educ. 2014;48(4):375-85. 14. Matola I, Devine LA, Chung HS, et al. A critical review of simulation-

Copyright: © 2020 Pokrajac et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

based mastery learning with translational outcomes. Med Teach. 2013;35(10):e1511-30. 15. Centers for Disease Control and Prevention. Tools for protecting healthcare personnel. 2014. Available: https://www.cdc.gov/hai/prevent/ ppe.html. Accessed March 30, 2020.

REFERENCES

16. Casanova LM, Rutala WA, Weber DJ, et al. Effect of single- versus

1. Centers for Disease Control and Prevention COVID-19 Response

double-gloving on virus transfer to health care workers’ skin and clothing

Team. Characteristics of Health Care Personnel with COVID-19, United

during removal of personal protective equipment. Am J Infect Control.

States, February 12–April 9, 2020. MMWR Morb Mortal Wkly Rep.

2012;40(4):369-74.

2020;69(15):477-81.

17. Barsuk JH, Cohen ER, Wayne DB, et al. A comparison of approaches

2. Van Doremalen N, Bushmaker T, Morris DH, et al. Aerosol and surface

for mastery learning standard setting. Acad Med. 2018;93(7):1079-84.

stability of SARS-CoV-2 as compared with SARS-CoV-1. N Engl J Med.

18. Yudkowsky R, Park YS, Lineberry M, et al. Setting mastery learning

2020;382(16):1564-7.

standards. Acad Med. 2015;90(11):1495-500.

3. Verbeek JH, Rajamaki B, Ijaz S, et al. Personal protective equipment for

19. Yudkowsky R, Tumuluru S, Casey P, et al. A patient safety approach to

preventing highly infectious diseases due to exposure to contaminated

setting pass/fail standards for basic procedural skills checklists. Simul

body fluids in healthcare staff. Cochrane Database Syst Rev.

Healthc. 2014;9(5):277-82.

2020;4(4):CD011621.

20. Mumma JM, Durso FT, Casanova LM, et al. Common behaviors

4. Kwon JH, Burnham CD, Reske KA, et al. Assessment of healthcare

and faults when doffing personal protective equipment for patients

worker protocol deviations and self-contamination during personal

with serious communicable diseases. Clin Infect Dis. 2019;69(Suppl

protective equipment donning and doffing. Infect Control Hosp

3):S214–20.

Epidemiol. 2017;38(9):1077-83.

21. McGaghie WC, Issenberg SB, Cohen ER, et al. Does simulation-based

5. Mulvey D, Mayer J, Visnovsky L, et al. Frequent and unexpected

medical education with deliberate practice yield better results than

deviations from personal protective equipment guidelines increase

traditional clinical education? A meta-analytic comparative review of the

contamination risks. Am J Infect Control. 2019;47(9):1146-7.

evidence. Acad Med. 2011;86(6):706-11.

6. Zellmer C, Van Hoof S, Safdar M. Variation in health care worker

22. Barsuk JH, Cohen ER, Vozenilek JA, et al. Simulation-based education

removal of personal protective equipment. Am J Infect Control.

with mastery learning improves paracentesis skills. J Grad Med Educ.

2015;43(7):750-1.

2012;4(1):23-7.

7. Tomas ME, Kundrapu S, Thota P, et al. Contamination of health care

23. Wayne DB, Barsuk JH, O’Leary KJ, et al. Mastery learning of

personnel during removal of personal protective equipment. JAMA Intern

thoracentesis skills by internal medicine residents using simulation

Med. 2015;175(12):1904-10. 8. Barsuk JH, Cohen ER, Williams MV, et al. Simulation-based mastery

technology and deliberate practice. J Hosp Med. 2008;3(1):48-54. 24. Girard R, Bousquet E, Carré E, et al. Tolerance and acceptability of

learning for thoracentesis skills improves patient outcomes: a

14 surgical and hygienic alcohol-based hand rubs. J Hosp Infect.

randomized trial. Acad Med. 2018;93(5):729-35.

2006;63(3):281-8.

Western Journal of Emergency Medicine

1094

Volume 21, no. 5: September 2020

Educational Advances

Streamlining Care in Crisis: Rapid Creation and Implementation of a Digital Support Tool for COVID-19 Nicholas Stark, MD, MBA* Michaela Kerrissey, PhD† Madeline Grade, MD, MPH* Beth Berrean, MLIS, MBA‡ Christopher Peabody, MD, MPH*

*University of California San Francisco, Department of Emergency Medicine, San Francisco, California † Harvard School of Public Health, Department of Health Policy and Management, Boston, Massachusetts ‡ University of California San Francisco, School of Medicine Tech Division, San Francisco, California

Section Editor: Andrew W. Phillips, MD, MEd Submission history: Submitted June 8, 2020; Revision received July 4, 2020; Accepted July 10, 2020 Electronically published August 17, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48537

The unprecedented COVID-19 pandemic has resulted in rapidly evolving best practices for transmission reduction, diagnosis, and treatment. A regular influx of new information has upended traditionally static hospital protocols, adding additional stress and potential for error to an already overextended system. To help equip frontline emergency clinicians with up-to-date protocols throughout the evolving COVID-19 crisis, our team set out to create a dynamic digital tool that centralized and standardized resources from a broad range of platforms across our hospital. Using a design thinking approach, we rapidly built, tested, and deployed a solution using simple, out-of-the-box web technology that enables clinicians to access the specific information they seek within moments. This platform has been rapidly adopted throughout the emergency department, with up to 70% of clinicians using the digital tool on any given shift and 78.6% of users reporting that they “agree” or “strongly agree” that the platform has affected their management of COVID-19 patients. The tool has also proven easily adaptable, with multiple protocols being updated nearly 20 times over two months without issue. This paper describes our development process, challenges, and results to enable other institutions to replicate this process to ensure consistent, high-quality care for patients as the COVID-19 pandemic continues its unpredictable course. [West J Emerg Med. 2020;21(5)1095-1101.]

past two decades, many EDs have developed capacity to create well-defined protocols and train clinicians to use them, offering significant advantages in care quality.2-4 However, because protocols are typically intended to serve as fixed guidelines, they are rarely updated and require little ongoing access by clinicians after initial training. This often leads to an array of platforms housing these hospital- and ED-specific protocols, which was the case at our hospital, Zuckerberg San Francisco General (ZSFG). The COVID-19 pandemic upended this stability and left our hospital, like many others, scrambling to adjust. With up to 30 COVID-positive patients in our hospital on a given day – and several of those in the intensive care unit on ventilators – rapidly evolving protocols made it difficult for clinicians to stay up to date with new guidelines, adding additional stress and potential for error to an already overextended system.5-7 This instability, hallmarked by daily emails and online

1095

Western Journal of Emergency Medicine

Stark et al.

Rapid Creation & Implementation of a Digital Tool for COVID-19 folders overflowing with lengthy PDFs providing new guidance, inspired our team to create a solution that could equip frontline clinicians with accessible, up-to-date clinical protocols throughout the evolving COVID-19 crisis. Ultimately, we were able to create a digital support tool that centralized, digitized, and standardized resources from a broad range of print and digital platforms across our hospital through using accessible, offthe-shelf technology: zsfgCOVID, which is available online at https://zsfgCOVID.ucsf.edu. In what follows, we describe how we rapidly developed and deployed this digital tool, explore its utilization in our ED, and highlight lessons as well as a step-bystep process for teams endeavoring to develop a similar approach to maximize patient care during the next phase of the COVID-19 pandemic. BUILDING A COVID-19 DIGITAL SUPPORT TOOL We used an accelerated, two-pronged approach toward building a solution: 1) engaging leadership to ensure high-level support; and (2) assembling a team to iteratively build, test, and deploy a solution using the best practices of design thinking. Engaging Leadership We first approached ED leadership with our project idea, and it was received with strong support. We chose to focus on EDspecific protocols first to enable our team to quickly create, test, and implement the digital support tool in a smaller setting before expanding hospital-wide. Assembling a Multi-Disciplinary Team We assembled a team of three University of California, San Francisco (UCSF) emergency physicians who work clinically at ZSFG, as well as four members of a digital product studio at the UCSF School of Medicine. Many medical centers and health systems have similar studios; internal information technology departments can also serve as a partner group. The physicians provided the clinical perspective necessary to organize the flow of the digital support tool, while the digital product team managed the project and created the web platform. While we had a heavily resourced team, the final platform required fewer resources than we used and could easily be replicated by less resourced teams, as illustrated in Table 1. Using Design Thinking to Rapidly Develop a Solution During our initial virtual meetings, which occurred 2-3 times per week, we used a human-centered design thinking approach to further define our problem and ideate potential solutions.8 After determining that a streamlined, responsive, web-based solution would likely work best, we set out to create and test a prototype. First, our team worked to build a multilevel decision tree to organize our hospital’s COVID-related protocols in a way that could eventually be translated into a digital tool (Figure 1). Upon completing the first draft of the decision tree, our team split the project to work in parallel: the physicians worked to build specific endpoints for each protocol outcome, while the Western Journal of Emergency Medicine

digital product team began to build the digital support tool. Ten days later, we had a prototype ready for testing (Figure 2). User Testing and Adjustments Following five days of user testing with a group of eight resident and four attending emergency physicians, brief interviews were conducted for the purpose of rapidly collecting user feedback. The interviews, conducted by one of the authors over a five-day period, followed a semi-structured protocol; themes were recorded in memos by the lead interviewer immediately following the interviews. These interviews revealed two important insights: 1) the complex, multistep logic led to an unacceptable number of “clicks” to reach an endpoint; and 2) most users preferred broad overviews of protocols, rather than being directed to fine-tuned endpoints. From a platform maintenance perspective, the digital product team expressed concern that the multistep logic on the backend of the tool required extensive rebuilds each time the protocols changed. With some protocols changing as many as 10 times in two weeks, these technical challenges and user feedback led us back to design thinking to reframe the problem. A virtual brainstorming meeting the following day led us to a solution: simplify the decision-support tool by creating broad, intuitive flowcharts for the hospital protocols rather than specific informational endpoints. This approach would reduce the number of “clicks” required to reach an endpoint, allow users to see broad overviews of protocols, and minimize platform rebuilds as protocols changed. Rather than lead to specific informational endpoints – such as which PPE to wear when intubating a high-risk COVID-19 patient – these new flowchart-style endpoints provide single-page overviews, such as which PPE to wear in multiple clinical scenarios (Figure 3). After constructing each protocol flowchart in PowerPoint, our team uploaded the flowchart endpoints to the web platform (Figure 4).

Table 1. Example team roles, time commitments, and costs to develop a digital tool for patient care. Title

Role Description

Total Hours

Estimated Cost

Project Manager

Oversee project timeline, coordinate meetings, monitor progress, supervise budget, manage platform revisions.

25-35

$40-50/ hour

Platform Developer

Build web-based platform, assist with protocol format/ design, maintain platform as needed.

30-40

$75-100/ hour

Clinician

Consolidate COVID-19 protocols, organize protocol flow, standardize protocol format, update protocols as needed.

30-40

$100-200/ hour

1096

Volume 21, no. 5: September 2020

Stark et al.

Rapid Creation & Implementation of a Digital Tool for COVID-19

Figure 1. The decision tree structure to organize COVID-19 protocols. COVID, coronavirus 19; PPE, personal protective equipment; Adult EM, adult emergency medicine; AMA, against medical advice.

Following these changes to the digital support tool, we approached the same 12 resident and attending emergency physicians for a second round of user testing, which revealed a dramatic improvement in perceived usability. Our team decided to move forward with an ED-wide launch on April 7, 2020; this was 26 days after initial project brainstorming began. The tool, zsfgCOVID, is named after our hospital and publicly accessible at https://zsfgCOVID.ucsf.edu. PRODUCT ADOPTION AND UTILIZATION After usng a broad range of tactics to advertise the digital tool – including product demonstrations during department meetings, link-access within the electronic health record (EHR) system, flyers, and targeted email notifications– initial website usage data and user reviews indicate substantial uptake. Throughout the initial six weeks from launch, zsfgCOVID experienced 8-20 unique daily users for the 28 emergency clinicians working each day, or approximately 2970% of daily clinicians. Our team also conducted an institutional review boardexempt survey among emergency physicians to assess perceptions of the platform. The survey was created by adapting previously developed and validated survey measures Volume 21, no. 5: September 2020

where possible, particularly for more subjective measures such as perceived usefulness.9 After cognitive testing with two residents and one attending physician over a two-day period, which resulted in minimal updates to the survey measures for clarity in language, the online Qualtrics (Provo, UT) survey was emailed to 90 resident and attending physicians who work clinically in the ZSFG ED (Appendix A). The survey, accessible for one week with two email reminders, garnered a total of 28 responses for a response rate of 31.1%. Of the physicians surveyed, 57.4% reported lacking confidence in accessing up-to-date COVID workflows and policies prior to implementation of the digital tool; 100% responded “agree” (32.1%) or “strongly agree” (67.9%) that the digital tool has made it easier to access up-to-date COVID-related protocols, and 100% “agree” (50.0%) or “strongly agree” (50.0%) that the platform was useful in their job. In addition, 78.6% responded “agree” (35.7%) or “strongly agree” (42.9%) that the platform has affected their management of patients with COVID-19 infections. For example, one clinician commented that the platform “really helped with my ability to safely discharge a homeless patient to an isolation shelter.” Other users have noted that the platform has “helped me determine who I should be testing

1097

Western Journal of Emergency Medicine

Rapid Creation & Implementation of a Digital Tool for COVID-19

Stark et al.

PAPR

Figure 2. Version 1 of the digital tool. Obtaining information on which personal protective equipment to wear while intubating a patient at high risk for COVID-19 required a total of four “clicks” to reach an endpoint. COVID-19, coronavirus 19; EM, Emergency Medicine, PPE, personal protective equipment; PAPR, powered air purified respirator; Neb, nebulizer; NIPPV, non-invasive positive pressure ventilation; Tx, treatment.

Figure 3. Current version of the digital tool. Obtaining information on which personal protective equipment to wear requires a total of two “clicks” to reach an endpoint. COVID-19, coronavirus 19; EM, Emergency Medicine; PPE, personal protective equipment; AMA, against medical advice; PUI, patient under investigation.

Western Journal of Emergency Medicine

1098

Volume 21, no. 5: September 2020

Stark et al.

Rapid Creation & Implementation of a Digital Tool for COVID-19

Figure 4. The PPE protocol flowchart in the current version of the digital tool. All aspects of the PPE protocol are displayed in a single page. PPE, personal protective equipment; COVID, coronavirus 19; PUI, patient under investigation.

for COVID-19, and which type of test I should order,” as well as “kept me up-to-date on which PPE I should be wearing in different clinical scenarios.” Clinicians reported using the digital tool often, with 85.7% using the platform at least once per week. The platform has a net promoter score of 71%, which falls in the “Excellent” category. User feedback indicates that, in the future, clinicians would like to see additional functionality added to the digital tool, such as a way to link protocols to current scientific evidence. Our team plans to work to incorporate this feedback as the COVID-19 pandemic evolves. LESSONS LEARNED Our team learned several lessons throughout the development and deployment process, which may aid other institutions as they work to develop similar digital support tools. These lessons include the following: • Engaging appropriate stakeholders during a hectic time. Our team first reached out to ED leadership with whom we had relationships. These ED leaders became key, invested stakeholders who were able to connect us with other hospital leaders to broaden support for the digital support tool. • Developing a user-friendly, clinician-focused platform. Initial user testing demonstrated that the first version of the digital support tool required too Volume 21, no. 5: September 2020

1099

•

many “clicks” to reach an endpoint. The development of a simpler, more user-friendly final product occurred through multiple iterations based on user feedback from targeted interviews. Using a straightforward web content management platform. Our team initially built a heavily logicbased web platform foundation in the Qualtrics survey system.10 However, this approach quickly became unsustainable due to the extensive rebuilds required each time a protocol changed. Ultimately, our team found that a common web content management platform, such as Drupal (Antwerp, Belgium) was easier to use and maintain.11 Ensuring accurate, daily updates to changing protocols. To ensure all protocols are up to date, we rely heavily on the physicians on our team. Through close communication with hospital leadership, the physicians make updates to the protocol flowcharts as recommendations evolve. Both the physicians and digital product studio members have been trained in uploading the updated protocol flowcharts to the web platform; this flattened organizational structure has enabled rapid turnarounds each time recommendations change. Earning clinician trust for a new digital tool. Topdown support was key for earning clinicians’ trust and encouraging them to use the platform. Maintaining Western Journal of Emergency Medicine

Rapid Creation & Implementation of a Digital Tool for COVID-19

•

close relationships with hospital leadership and ensuring accurate information on the digital support tool is vital. Spreading the word in an information-saturated landscape. Our team quickly realized that purely email dissemination of the digital support tool would likely lead to underutilization or failure. By thinking beyond traditional information dissemination tactics, we were able to give live demonstrations of the platform at several department-wide meetings, integrate a link to the tool in our EHR system, and post flyers throughout our ED. These tactics, combined with targeted email reminders to clinicians working on a given week, have led to high utilization.

Application to Other Institutions As the COVID-19 pandemic continues its unpredictable course over the coming months to years, a centralized information source that equips clinicians with up-to-date information for the care of COVID-19 patients can help improve patient care. We believe that our approach to building a centralized, digitized, and standardized resource platform through using off-the-shelf technology is applicable across academic and community settings. We recommend the following steps for institutions interested in building a similar digital support tool: 1. Talk with frontline clinicians to determine whether similar challenges with protocol management are present at your hospital. 2. Identify and engage motivated team members, including at least one clinician and one member versed in basic website design. 3. Start small. Consider beginning with a single unit or service, rather than attempting hospital-wide implementation from the start. 4. Engage leadership at the unit or service level first, and augment to hospital-level leadership with unit leadership’s support. Leadership buy-in is key for longterm success. 5. Consolidate the existing COVID-related protocols at your hospital, and develop a decision tree to outline where each protocol should be housed on the web platform. 6. Standardize the protocols into easy-to-use flowcharts. Our team used PowerPoint for this process. 7. Build a web-based platform to house the protocols. The organization of the platform will be based on the decision tree you develop. Our team used Drupal for this step.11 8. Launch the initial version of your digital tool quickly, and test with a small group of clinicians. Briefly interview these clinicians after they use the tool to gain insight into areas for improvement. 9. Adjust the digital tool as needed based on user feedback. 10. Launch the digital tool for a broader audience, in coordination with hospital leadership to ensure support. Western Journal of Emergency Medicine

Stark et al.

LIMITATIONS While our team was able to rapidly develop a novel digital support tool to aid our hospital’s response to the COVID-19 pandemic, our user testing and surveying processes exhibit several limitations. First, our initial semi-structured interviews were targeted to updating the digital tool rather than more deeply exploring how respondents felt about and experienced the platform. Second, with a relatively low survey-response rate, our survey results may be subject to nonresponse bias as clinicians who have not used or do not like the platform may have been less likely to respond to the survey. Finally, although most survey respondents reported that the platform affected their management of patients with COVID-19 infections, our team did not assess outcome measures such as differences in PPE use or disposition times for clinicians who use the platform. Future research can explore the more complex relationships between these evolving digital tools and clinicians’ experience of – and effects on – patient care. CONCLUSION The COVID-19 pandemic will continue to affect patients and hospital systems for the foreseeable future, and it is important for clinicians to have easy access to up-to-date hospital protocols to provide exceptional patient care. Our team’s experience has shown that simple, out-of-the-box web technology can serve as a conduit to transform typically static hospital protocols into rapidly-evolving guidelines that clinicians can access within moments. We are hopeful that, through developing similar digital support tools, other institutions are able to provide similar support to frontline clinicians throughout the COVID-19 era.

Address for Correspondence: Nicholas Stark, MD, MBA, University of California San Francisco, Department of Emergency Medicine, 505 Parnassus Avenue, Room L-126, San Francisco, CA 94143. Email: Nicholas.stark@ucsf.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Stark et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

1100

Volume 21, no. 5: September 2020

Stark et al.

Rapid Creation & Implementation of a Digital Tool for COVID-19

REFERENCES

about organizing a hospital to respond to the COVID-19 outbreak: experience from a French reference centre. Clin Microbiol Infect. 2020;

1. Centers for Disease Control and Prevention. COVID-19 Situation

26(6):669-72.

Summary. 2020. Available at: https://www.cdc.gov/coronavirus/2019-

7. Liu Q, Luo D, Haase JE, et al. The experiences of health-care providers

ncov/cases-updates/summary.html. Accessed 12 May 2020.

during the COVID-19 crisis in China: a qualitative study. Lancet. 2020;

2. Reid PP, Compton WD, Grossman JH, et al. Building a better delivery

8(6):790-8.

system: a new engineering/health care partnership. NAP. 2005. 3. Hughes RG. Patient safety and quality: an evidence-based handbook for

8. Cross N. (2011). Design Thinking: Understanding How Designers Think and Work. Berg Publishers.

nurses. AHRQ. 2008. 4. Woolf SH, Grol R, Hutchinson A, et al. Potential benefits, limitations and

9. David FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly.

harms of clinical guidelines. BMJ. 1999; 318(7182):527-30.

1989;13(3):319-40.

5. Baird R. How doctors on the frontlines are confronting the uncertainties of COVID-19. 2020. Available at: https://www.newyorker.com/science/

10. Qualtrics. 2005. Available at: https://www.qualtrics.com. Accessed July 2, 2020.

medical-dispatch/how-doctors-on-the-front-lines-are-confronting-theuncertainties-of-covid-19. Accessed May 12, 2020.

11. Drupal. 2009. Available at: https://www.drupal.org. Accessed July 2, 2020.

6. Peiffer-Smadia N, Lucet JC, Bendjelloul G, et al. Challenges and issues

Volume 21, no. 5: September 2020

1101

Western Journal of Emergency Medicine

Commentary

Impact of Social Distancing on Individuals Who Use Drugs: Considerations for Emergency Department Providers Kathy T. LeSaint, MD*† Hannah R. Snyder, MD‡

*University of California San Francisco, Department of Emergency Medicine, San Francisco, California † California Poison Control System, San Francisco, California ‡ University of California San Francisco, Department of Family and Community Medicine, San Francisco, California

Section Editor: Tony Zitek, MD and Erin Dehon, PhD Submission history: Submitted April 26, 2020; Revision received July 13, 2020; Accepted July 15, 2020 Electronically published August 18, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.47896

The isolation that comes from social distancing during the COVID-19 pandemic can be particularly detrimental to the United States’ population of people who use drugs. People with substance use disorders may be at risk for return to use, exacerbation of existing mental health disorders, and risky drug practices. In this commentary, we review the risk to people who use drugs and how emergency department providers can best support these individuals during the unprecedented time of social distancing. [West J Emerg Med. 2020;21(5)1102-1104.]

Disclaimer: Due to the rapidly evolving nature of this outbreak, and in the interests of rapid dissemination of reliable, actionable information, this paper went through expedited peer review. Additionally, information should be considered current only at the time of publication and may evolve as the science develops. On January 20, 2020, the first confirmed case of coronavirus disease 2019 (COVID-19), caused by the SARSCoV-2 virus, was reported in the United States (US). As of April 16, 2020, the US death toll was at 29,998, exceeding that of any other country in the world.1 While the research community is working tirelessly to determine the most efficacious treatments for COVID-19, public health officials announced aggressive recommendations in hopes of slowing human-to-human spread. On March 19, 2020, California became the first state to mandate shelter in place, and New York was quick to follow. By April 7, 2020, 42 states, along with a number of cities and counties, had urged residents to stay in their homes except for essential trips and services. Whether under a shelter in place, safer at home, or stay home order, the concept of social distancing is strongly encouraged, defined by the US Centers for Disease Control and Prevention (CDC) as “keeping space between yourself and other people.”2 Social distancing is essential for community health but may be uniquely challenging for people who use drugs (PWUD) to comply with, and may put them at risk for drug-related harms. Western Journal of Emergency Medicine

Risks to People Who Use Drugs Approximately one third of PWUD seeking treatment for substance use disorders are unhoused or live in congregate settings including residential treatment facilities, shelters, and single-room occupancy hotels.3 In these settings, following the CDC guidance on social distancing may be difficult or impossible. This issue has led to COVID-19 clusters in some homeless shelters and has led several communities to seek alternative housing in hotels for people experiencing homelessness.4,5 Those PWUD who have the ability to practice social distancing may face an increased risk of drug-related harms. Coping with isolation and health threats may lead to increased stress and anxiety in a vulnerable population already stricken with trauma and mental health issues. Social isolation can act as a trigger and is strongly correlative with mood and substance use disorders.6 These factors can exacerbate existing substance use as patients self-treat psychiatric symptoms or lead people with a history of substance use disorder to return to use. For those who are actively using drugs, practicing harm reduction can be difficult in the setting of COVID-19. Traditional guidance is for PWUD to use with another person so that if they overdose, 911 can be called and naloxone can be used, but by following social distancing PWUD are unable to do this. As borders close and supply chains are disrupted,

1102

Volume 21, no. 5: September 2020

LeSaint et al.

Impact of Social Distancing on Individuals Who Use Drugs

PWUD may seek out drugs from places other than their usual, trusted sources and thereby be at greater risk of exposure to an adulterated or contaminated supply. In addition, many needle and syringe exchange programs changed their models in response to the COVID-19 pandemic, either closing down completely or being unable to provide their typical services and programs (eg, referral to treatment programs, harm reduction education, naloxone distribution and education, etc). Such changes place PWUD at greater risk for unsafe practices and increased risk of communicable infectious diseases, skin and soft tissue infections, and drug overdose and death. Furthermore, maintaining access to treatment and recovery services during a time of social distancing mandates is difficult. Throughout the country, support groups have been cancelled, treatment programs are limiting new patients, and inpatient treatment centers have limited visits.6-8 Opioid treatment programs, in which most patients rely on daily dispensing of medications to treat opioid use disorder, may have reduced access as well. Such changes make it harder for patients to newly access treatment and present challenges for those who are already in treatment. Without ease of access to places of recovery and medication-assisted treatment, patients are at risk of serious medical and psychological complications. Fortunately, the federal government has recently made several changes to increase access to life-saving addiction treatment. The Centers of Medicare & Medicaid Services has loosened regulations and is compensating for telemedicine services, the Drug Enforcement Agency now supports telephone and audiovisual buprenorphine prescribing, and the Office of Civil Rights at the Department of Health and Human Services approved usage of popular apps to provide telehealth without risk of penalties for noncompliance with HIPAA.9-11 In addition, opioid treatment programs are providing longer durations of take-home doses of medications for treatment of opioid use disorder.12 Emergency Department Support for People Who Use Drugs It is likely that emergency departments (ED) across the US will see an increase in the number of PWUD experiencing withdrawal, experiencing overdose, or seeking treatment for their substance use disorder. Preliminary data from the ED at San Franciscoâ&#x20AC;&#x2122;s only public hospital revealed a near twofold increase in the number of patients presenting with the chief complaint of â&#x20AC;&#x153;drug overdoseâ&#x20AC;? in March 2020 (67 patients/month; 1.2% of all ED encounters) when compared to averaged data from the prior six months (38 patients/month; 0.6% of all ED encounters). In addition, more than 35 states have reported increases in cases of opioid-related overdose and mortality.13 Therefore, in this unprecedented time of social distancing, emergency providers are placed in an additional frontline role of delivering patient-centered care for a highly at-risk population of PWUD. Emergency clinicians should provide compassionate, evidence-based care to PWUD. Establishing rapport Volume 21, no. 5: September 2020

and motivational interviewing can be difficult in a time of enhanced precautions and extra personal protective equipment. However, continuing to take the time to speak in a non-stigmatizing way is vital in the therapeutic process and is the start to effective treatment for PWUD.14 In recent years, the practice of ED initiation of buprenorphine has rapidly become the standard of care.15 We encourage emergency clinicians to offer buprenorphine to any patients presenting with opioid use disorder.16 DATA 2000 waivers are not required to administer buprenorphine in the ED. First doses of buprenorphine can be rapidly administered in the ED, and patients should be linked to ongoing treatment. In addition, while it is not the usual practice of the emergency clinician to provide long-term medication prescriptions, in this unique time we encourage providers with DATA 2000 waivers to offer longer durations of buprenorphine prescriptions (up to 28 days) to appropriate patients. During the COVID-19 pandemic, it is more essential than ever that emergency clinicians provide this service and while doing so, receive institutional support that is much needed to overcome barriers to buprenorphine administration. Individual institutions and departmental leadership can best support their clinicians by providing adequate training and resources regarding buprenorphine use, as well as assisting providers in coordination of outpatient linkage to care.17 On March 19, 2020, the Substance Abuse and Mental Health Services Administration provided additional guidance for managing the treatment of alcohol or benzodiazepine withdrawal in acute settings.18 Providing buprenorphine to treat patients with opioid use disorder and medication treatment for alcohol withdrawal is particularly essential for those patients who are diagnosed with COVID and entering quarantine. Adequately treated withdrawal and compassionate care will support them in staying for the duration of their quarantine period. As much as possible, emergency care providers must continue to offer harm reduction strategies to PWUD. Strategies of harm reduction include supporting drug use hygiene (eg, giving education on safe consumption, distributing pipes or syringes), providing overdose prevention supplies (eg, take-home or prescriptions of naloxone, fentanyl test strips), and encouraging patients to not use drugs in isolation (eg, video-chatting with a buddy, contacting support at www.neverusealone.com). Involving an ED social worker or substance use navigator who is familiar with local outpatient resources and/or changes to the outpatient landscape during this time can help facilitate linkage to care. Finally, continuing to address other social determinants of health (eg, housing insecurity, psychiatric illnesses) is paramount to providing safe discharge to the community. CONCLUSION The isolation that comes from social distancing during the COVID-19 pandemic can be particularly detrimental to the

1103

Western Journal of Emergency Medicine

LeSaint et al.

Impact of Social Distancing on Individuals Who Use Drugs population of people who use drugs. People with substance use disorders may be at risk for return to use, exacerbation of existing mental health disorders, and risky drug practices. In this time, emergency care providers have a vital role in supporting this vulnerable population of people who use drugs by establishing rapport, encouraging best practices in harm reduction, providing medication treatment, and connecting patients to outpatient resources.

the National Epidemiologic Survey on Alcohol and Related Conditions. J Clin Psychiatry. 2011;72(11):1468-76. 7. Cenziper D, Brulliard K, Jacobs J. People in addiction treatment are losing crucial support during coronavirus pandemic. Available at: https://www.washingtonpost.com/health/people-in-addictiontreatment-are-losing-crucial-support-during-coronaviruspandemic/2020/03/26/5698eae0-6ac6-11ea-abef-020f086a3fab_story. html. Accessed April 16, 2020. 8. Bebinger M. Opioid addiction is “a disease of isolation,” so pandemic

Address for Correspondence: Kathy T. LeSaint, MD, University of California San Francisco, Department of Emergency Medicine, 1001 Potrero Avenue, Building 5, Room 6A, San Francisco, CA 94110. Email: Kathy.lesaint@ucsf.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. Both authors are currently supported by a Zuckerberg San Francisco General Hospital & Trauma Center institutional grant from the Public Health Institute/California Bridge Program (agreement #: 01487). In addition, Dr. Snyder is the coprincipal investigator of the California Bridge Program. The sponsor had no role the preparation, review, or approval of this manuscript, nor the decision to submit the manuscript for publication. The contents of the article are solely the responsibility of the authors and do not necessarily represent the official views of the sponsors. Copyright: © 2020 LeSaint et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

puts recovery at risk. 2020. Available at: https://www.npr.org/sections/ health-shots/2020/03/27/820806440/opioid-addiction-is-a-disease-ofisolation-so-pandemic-puts-recovery-at-risk. Accessed April 16, 2020. 9. Centers for Medicare & Medicaid Services. Medicare Telemedicine Health Care Provider Fact Sheet. Available at: https://www.cms.gov/ newsroom/fact-sheets/medicare-telemedicine-health-care-provider-factsheet. Accessed April 24, 2020. 10. Prevoznik TW. DEA-068 letter. Available at: https://www.samhsa. gov/sites/default/files/dea-samhsa-buprenorphine-telemedicine.pdf. Accessed April 21, 2020. 11. US Department of Health & Human Services. Notification of Enforcement Discretion for Telehealth Remote Communications During the COVID-19 Nationwide Public Health Emergency. Available at: https://www.hhs.gov/hipaa/for-professionals/special-topics/emergencypreparedness/notification-enforcement-discretion-telehealth/index.html. Accessed April 21, 2020. 12. Substance Abuse and Mental Health Services Administration. Opioid Treatment Program (OTP) Guidance. Available at: https://www. samhsa.gov/sites/default/files/otp-guidance-20200316.pdf. Accessed April 21, 2020. 13. American Medical Association. Issue brief: reports of increases in

REFERENCES

opioid-related overdose and other concerns during COVID pandemic.

1. Johns Hopkins University and Medicine. Coronavirus Resource Center.

Available at: https://www.ama.assm.org/system/files/2020-07/issue-brief-

2020. Available at: https://coronavirus.jhu.edu/us-map. Accessed April 16, 2020.

increases-in-opioid-related-overdose.pdf. Accessed July 14, 2020. 14. Clement S, Schauman O, Graham T, et al. What is the impact of

2. Centers for Disease Control and Prevention. Social distancing,

mental health-related stigma on help-seeking? A systematic review of

quarantine, and isolation. Available at: https://www.cdc.gov/ coronavirus/2019-ncov/prevent-getting-sick/social-distancing.html.

quantitative and qualitative studies. Psychol Med. 2015;45(1):11-27. 15. Strayer RJ, Hawk K, Hayes BD, et al. Management of opioid use

Accessed April 16, 2020.

disorder in the emergency department: a white paper prepared for

3. Eyrich-Gerg KM, Cacciola JS, Carise D, et al. Individual characteristics

the American Academy of Emergency Medicine. J Emerg Med.

of the literally homeless, marginally housed, and impoverished in a US substance abuse treatment-seeking sample. Soc Psychiatry Psychiatr

2020;58(3):522-46. 16. Initiating buprenorphine treatment in the emergency department.

Epidemiol. 2008;43(10):831-42.

Available at: https://www.drugabuse.gov/nidamed-medical-health-

4. Mosites E, Parker EM, Clarke KE, et al. Assessment of SARS-CoV-2

professionals/discipline-specific-resources/initiating-buprenorphine-

infection prevalence in homeless shelters – four US cities, March 27-April 15, 2020. MMWR Morb Moral Wkly Rep. 2020;69(17):521-2.

treatment-in-emergency-department. Accessed April 20, 2020. 17. Im DD, Chary A, Condella AL, et al. Emergency department clinicians’

5. Frandino, N, Stapleton S, Paul K, et al. Coronavirus forces San

attitudes toward opioid use disorder and emergency department-initiated

Francisco to put homeless into hotels. 2020. Available at: https://www.

buprenorphine treatment: a mixed-methods study. West J Emerg Med.

reuters.com/article/us-health-coronavirus-homelessness/coronavirus-

2020;21(2):261-71

forces-san-francisco-to-put-homeless-into-hotels-idUSKBN21L0DY.

18. Substance Abuse and Mental Health Services Administration

Accessed April 24, 2020.

(SAMHSA). Considerations for crisis centers and clinicians in managing

6. Chou KL, Liang K, Sareen J. The association between social isolation

the treatment of alcohol or benzodiazepine withdrawal during the

and DSM-IV mood, anxiety, and substance use disorders: Wave 2 of

Western Journal of Emergency Medicine

COVID-19 epidemic: March 19, 2020. CNS Drugs. 2015;29(4):293-311.

1104

Volume 21, no. 5: September 2020

Educational Advances

COVID-19: A Driver for Disruptive Innovation of the Emergency Medicine Residency Application Process Recommendations from the Council of Residency Directors Application Process Improvement Committee

Alexis Pelletier-Bui, MD* Doug Franzen, MD† Liza Smith, MD‡ Laura Hopson, MD§ Lucienne Lutfy-Clayton, MD‡ Kendra Parekh, MD¶ Mark Olaf, DO|| Tom Morrissey, MD# David Gordon, MD** Erin McDonough, MD†† Benjamin H. Schnapp, MD, MEd‡‡ Mary Ann Edens, MD§§ Michael Kiemeney, MD¶¶

*Cooper Medical School of Rowan University, Department of Emergency Medicine, Camden, New Jersey † University of Washington School of Medicine, Department of Emergency Medicine, Seattle, Washington ‡ University of Massachusetts Medical School Baystate Health, Department of Emergency Medicine, Springfield, Massachusetts § University of Michigan Medical School, Department of Emergency Medicine, Ann Arbor, Michigan ¶ Vanderbilt University School of Medicine, Department of Emergency Medicine, Nashville, Tennessee || Geisinger Commonwealth School of Medicine, Department of Emergency Medicine, Scranton, Pennsylvania # University of Florida College of Medicine-Jacksonville, Department of Emergency Medicine, Jacksonville, Florida **Duke University, Division of Emergency Medicine, Durham, North Carolina †† University of Cincinnati College of Medicine, Department of Emergency Medicine, Cincinnati, Ohio ‡‡ University of Wisconsin School of Medicine and Public Health, BerbeeWalsh Department of Emergency Medicine, Madison, Wisconsin §§ Louisiana State University Health Shreveport, Department of Emergency Medicine, Shreveport, Louisiana ¶¶ Loma Linda University School of Medicine, Department of Emergency Medicine, Loma Linda, California

Section Editor: Danya Khoujah, MBBS Submission history: Submitted May 20, 2020; Revision received June 30, 2020; Accepted August 7, 2020 Electronically published August 19, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.8.48234

The coronavirus disease (COVID-19) pandemic has had a significant impact on undergraduate medical education with limitation of patient care activities and disruption to medical licensing examinations. In an effort to promote both safety and equity, the emergency medicine (EM) community has recommended no away rotations for EM applicants and entirely virtual interviews during this year’s residency application cycle. These changes affect the components of the EM residency application most highly regarded by program directors – Standardized Letters of Evaluation from EM rotations, board scores, and interactions during the interview. The Council of Residency Directors in Emergency Medicine Application Process Improvement Committee suggests solutions not only for the upcoming year but also to address longstanding difficulties within the process, encouraging residency programs to leverage these challenges as an opportunity for disruptive innovation. [West J Emerg Med. 2020;21(5)1105-1113.]

INTRODUCTION The coronavirus pandemic has substantially disrupted undergraduate medical education. While creative solutions have been implemented for classroom-based activities, Volume 21, no. 5: September 2020

suspension of patient contact1 causes a disproportionate impact on students in clinical rotations. Rotational experiences are critical to the development of learners into independent practitioners and comprise an important component of the 1105

Western Journal of Emergency Medicine

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process residency selection process.2 Emergency medicine (EM) is particularly vulnerable to the impact of the altered clinical learning environment as emergency departments (ED) are dealing with large volumes of coronavirus disease (COVID-19) patients and often low volumes of non-COVID patients.3 Further, EM courses are frequently only available to senior medical students, limiting students’ opportunities for exposure to the specialty.4 Given these constraints, residency programs will need to adjust their expectations for residency application materials, including numbers and types of recommendation letters, clinical experiences, and United States Medical Licensing Examination (USMLE) scores. Considering forecasts for a second surge of coronavirus in the fall or winter, paired with ongoing or reinstituted travel restrictions, the Coalition for Physician Accountability (Coalition) has recommended that all residency programs commit to virtual interviews for the entire upcoming application cycle, and the EM community has released a statement supporting this recommendation.5,6 While challenging, these changes do, however, provide an opportunity to explore alternative models for recruiting the next generation of emergency physicians while simultaneously identifying creative solutions to longstanding difficulties such as volume of applications and cost. We encourage programs to respond to the challenges presented by COVID-19 not just reactively, but with an eye toward transformative change of the application and interview process. We offer here suggestions on the application and interview process, many of which could be carried forward into future application cycles. APPLICATION REVIEW EM Rotations and Letters of Recommendation Challenges EM program directors (PD) cite the Standardized Letter of Evaluation (SLOE) as the most important component of the residency application when making interview invitation decisions.2,7-9 Many residency programs expect two SLOEs prior to making interview offers.10 Academic group SLOEs from residency programs carry the most weight,8,9 placing a greater burden on applicants from schools without a home EM residency (“orphan” students). Recognizing that evaluations from EM subspecialty rotations or EM faculty not affiliated with a residency program also have merit, the Council of Residency Directors in EM (CORD) has developed modified SLOEs (https://www.cordem.org/resources/residencymanagement/sloe/), which carry less weight.9 EM residency programs place lowest value on letters of recommendation from non-EM faculty.9 With clinical rotations suspended nationally, formerly predictable clerkship curricula are in flux, potentially delaying fourth-year rotations, including EM. Even those who are able to resume regularly scheduled fourth-year rotations are discouraged from performing away rotations to Western Journal of Emergency Medicine

Pelletier-Bui et al.

promote equity between applicants;5,6 therefore, the majority of applicants will only have one traditional SLOE in their residency application. At an even greater disadvantage are “orphan” applicants, who may have difficulty obtaining even a single EM rotation. Applicant groups that are disproportionately affected by “orphan” status are osteopathic and international medical graduate (IMG) applicants. Restricted access to fourth-year EM rotations may also impact the number of applicants to EM, as up to 36% of surveyed US medical student applicants to EM didn’t decide until their fourth-year of medical school.11 On the other hand, some students in a similar position may still attempt to pursue a career in EM only to develop specialty regret and potentially leave an unfilled position in a residency program. Suggested Solutions Decreased rotation availability and fewer EM residencyauthored SLOEs will require rethinking the current hierarchy of letters and giving more value not only to non-traditional SLOEs, but also to letters authored by non-EM faculty. To increase efficacy and make the task easier for non-EM letter writers, CORD has assembled a committee to develop a template for writers, highlighting the attributes EM PDs specifically look for in a letter and in an applicant. This template, termed the “O-SLOE” for “off-service” or “other rotation,” is now available on CORD’s SLOE webpage (https://www.cordem.org/resources/residency-management/ sloe/). This can be distributed to medical school deans and clerkship director organizations. Continued use of this template beyond this extraordinary academic year could increase the utility and rigor of non-EM evaluations in future application cycles. This same committee has updated the EMfaculty SLOE to allow writers to detail how COVID-19 has affected their student rotations (“on the reCORD,” CORD listserv communication, June 23, 2020). The Coalition and EM community recommend that away rotations be discouraged for the 2020-2021 academic year with the exception of “learners who have a specialty interest and do not have access to a clinical experience with a residency program in that specialty in their school’s system,” and “learners for whom an away rotation is required for graduation or accreditation requirements.”5,6 Given there are over 80 identified schools without an established EM residency program and there is a possibility of students being limited from performing an EM rotation at their own institution due to high volume of COVID-19 patients (Susana Tsao, DO, CORD listserv communication, May 4, 2020), we support these exceptions to the “no aways” policy. We also support the recommendation for institutions that are still hosting rotators to preferentially accept students who are unable to obtain a SLOE from their home institution.12,13 A live document of schools without a home EM residency program can be found here (https://bit.ly/37UKYEp). Proactive efforts should be made to increase EM

1106

Volume 21, no. 5: September 2020

Pelletier-Bui et al.

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process

rotation availability for EM-bound students. Potential strategies include shortening the length of the rotation (eg, from four weeks to three weeks) or reducing the number of shifts required per rotator, allowing more rotators per block. The timeframe for completion of EM rotations could be expanded beyond the traditional summer months, particularly considering the recently updated Electronic Residency Application Service (ERAS) application timeline with residency programs not being able to view residency applications until October 21, 2020.14 Even with utilization of these strategies, some EM-bound applicants may still not have a SLOE in time for file review. In such cases, we support the recommendation that an EM advisor write a letter of recommendation specifically incorporating the key elements of the SLOE.13 It should explicitly state that the applicant was unable to obtain a SLOE in time for file review secondary to COVID-19 and vouch for the applicant’s desire to pursue a career in EM, their career decision process, and potential for success. Additional insight into an applicant’s prior EM experiences beyond what can be gleaned from their curriculum vitae could also be helpful. Board Scores Challenges After SLOEs, the USMLE Step 1 and Step 2 Clinical Knowledge (CK) exams are the next biggest factors of importance in selecting applicants for interviews.2 In the 2018 National Resident Matching Program PD survey, 48% of PDs required USMLE Step 2 CK and 31% used a target score for Step 2 CK when considering applicants for interviews. Most Prometric Testing centers, which administer the USMLE Step 1 and 2 CK, were closed until June 1, 2020, and are still not running at full capacity based on governmental ordinances and advice from the Centers for Disease Control and Prevention and the World Health Organization.15 Gradual and incomplete opening, paired with the backlog of individuals competing for standardized testing, has lead to delays in testing and will likely result in later release of scores for many applicants. Likewise, USMLE Step 2 Clinical Skills (CS) testing has been suspended for 12-18 months.16 IMG applicants may be disproportionately affected by the inability to complete Step 2 CS, previously a requirement for certification by the Educational Commission for Foreign Medical Graduates (ECFMG).17 PDs may not have the full complement of USMLE scores available to them that they have traditionally relied upon for applicant screening and rank list submission. Additionally, it is unknown how these delays will affect applicants’ ability to obtain USMLE scores in time for medical school graduation. Suggested Solutions There are potential solutions to compensate for the testing bottleneck. Six US medical schools have opened regional testing centers to allow for additional testing options outside of Prometric and the USMLE is working with eligible Volume 21, no. 5: September 2020

medical schools across the US to host one-day testing events to administer Step 1 and Step 2 CK in the near future.18. The ECFMG has created five new pathways to meet the requirements for ECFMG certification for those IMG applicants who have yet to complete Step 2 CS for the 2021 match cycle.17 We encourage programs to consider these pathways as a substitute for the Step 2 CS examination. Programs that previously required Step 2 CK or CS for interview or ranking should consider temporarily revising their approach and policies. Institutional expectations or state-level requirements for licensure should be clearly articulated and communicated to allow applicants to make educated application decisions. However, we encourage residency programs to use this opportunity to take a new approach to the USMLE. Quantitative metrics such as USMLE Step 1 and 2 CK are commonly used to stratify and filter students. While scores do have some correlation with the likelihood of passing the American Board of Emergency Medicine qualifying exam,19 these scores do not correlate with clinical proficiency or success in EM residency.20 Reducing the influence of USMLE scores when screening applicants has long been discussed among EM educators, with a goal to transition toward more holistic application review. The transition of USMLE Step 1 to a pass/fail score within the next few years,21 combined with the testing disruptions of the COVID-19 pandemic, present an opportunity for more rapid change. Holistic application review involves programs performing honest self-assessment and appraisal of residency graduates to determine which character traits and attitudes are valued and associated with success in their program and then seeking out applicants with those qualities.22 Combining holistic application review with an understanding of a program’s own strengths and challenges in resident development can help programs identify and recruit applicants who are more likely to match successfully with them and succeed in training. Medical Student Performance Evaluation (MSPE) Challenges The MSPE is traditionally released at the beginning of October, marking the unofficial start of interview offers to applicants. Programs often wait for the MSPE before sending interview invitations, as it may describe professionalism or academic concerns that do not appear elsewhere in the application. With fewer SLOEs to review, and varying clinical experiences between students, the MSPE may take on additional importance this year. As a result of cancelled clinical rotations, some applicants may not have completed the core clerkships that traditionally contribute significantly to the MSPE. Suggested Solutions ERAS has amended the residency application timeline to allow for MSPE release on October 21, the same day that residency programs will be able to begin reviewing applications.14 Delaying these components of the application cycle will effectively push back the start

1107

Western Journal of Emergency Medicine

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process of interview season and might relax the time pressure on students, schools, and programs. Advising students to complete most, if not all, core clerkships before the release of the MSPE will offer applicants a better chance at receiving interview offers than if some core experiences are incomplete and updated at a later time. If not possible, a mechanism to allow regular updates to the MSPE may be useful. COVID-19 has not affected geographic regions equally. Programs will look to the MSPE to delineate the pandemic’s effect on the learning environment. We suggest adding a standardized “pandemic response” section to MSPEs. Schools should describe how they adapted, including what dates students were excluded from clinical experiences, what clerkship experiences may have been virtual, and any policies that prohibited students from seeking away rotations. Additionally, any action by students who went above and beyond to help during this time of crisis could be very useful information to EM PDs. If all schools use the MSPE to outline a school’s pandemic response, readers will be able to place a student’s record into context and identify outliers within a single school or between schools. Numbers of Applicants/Applications Challenges Cancellation of EM away rotations and delays in senior electives while students complete core clerkships may decrease the total number of EM applicants, due to decreased exposure to EM and uncertainty about the specialty. Simultaneously, the number of applications per applicant may increase due to perceived deficiencies (eg, lack of SLOEs and/ or USMLE scores, atypical MSPE) and the inability to hone residency and geographic preferences via audition rotations. Suggested Solutions While we cannot control students’ reactions to the uncertainty of this application cycle, the EM community can make efforts to mitigate other anxiety-provoking elements of the application process, foremost a lack of transparency. Clear communication will be critical, including informing students when they can expect to hear about interview offers, a program’s preferred method of contact for questions or updates, and clear expectations for wait-list status. Programs might consider allowing students to submit an optional statement of purpose or intent, in which students have the opportunity to communicate a particular interest in a given program or region. Allowing students to communicate what they perceive to be their “fit” with a program can help application reviewers identify students who are likely to be high-yield candidates and successful matches. This was instituted by the otolaryngology community as a requirement in the 2015-16 application cycle as a program-specific paragraph added to the end of an applicant’s personal statement. While this measure was Western Journal of Emergency Medicine

Pelletier-Bui et al.

found to be effective in decreasing the overall number of applications per applicant, it is also thought to have made otolaryngology appear less welcoming to medical students as there was an overall decline in the number of medical students applying to otolaryngology after this initiative. As a result, the Otolaryngology Program Directors Organization has now made the program-specific paragraph optional.23,24 Programs that wish to institute a program-specific paragraph as part of their application process are asked to publish these requests on ERAS when registering in ERAS Account Maintenance. This information will then be displayed to applicants as they research programs (Elise Lovell, MD, on behalf of Amy Mathis, Senior Director of ERAS, CORD listserv communication, May 11, 2020). One low-cost, low-effort method for students and PDs that may allow PDs to identify high-yield candidates amidst increasing application numbers is the institution of preference signaling, or a “star” system, which gives students a limited number of “stars” to allocate to their most desired programs. This method is used in other professions and has been proposed for use in orthopedics and otolaryngology.25-28 A computer-simulated model using otolaryngology match data found that applicants voluntarily adding preference data to their application enhanced the practical number of interview invitations for all applicants and could potentially allow more holistic file review of high-yield candidates to their program.28 However, implementation would require the collaboration with the Association of American Medical Colleges (AAMC) and development of technology that may be challenging to institute in time for this year’s residency application cycle. INTERVIEWS Challenges The interview and surrounding interactions are routinely cited by PDs as a major factor in ranking decisions.2 The Coalition recommends that all residency programs commit to virtual interviews for the upcoming application cycle.5 Opportunities for social events and “second looks” may also be curtailed. These interactions play key roles in assessing goodness of fit, both from the applicant’s and the program’s perspective, and are cited as particularly important for ranking decisions by under-represented minorities in medicine.29 Suggested Solutions Video Interviewing CORD supports replacing traditional in-person interviews with video interviews to try to maintain an equitable interview process for applicants and programs through this entire residency application cycle.6 Several programs have successfully demonstrated high satisfaction rates with video interviews, highlighting the advantages of reducing time and cost burdens for both the applicant and programs.30-34 Video interviews have the potential to level the playing field for applicants of low socioeconomic status who may not have the

1108

Volume 21, no. 5: September 2020

Pelletier-Bui et al.

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process

financial ability to travel for interviews.30,31. Reduced absences from clinical rotations would further enhance education in the fourth year of medical school.34 Despite some of the intrinsic appeal of video interviews,30,31 there are limitations compared to traditional onsite visits. Applicants face challenges accurately representing who they are as people, or in providing comfort in ranking a residency program,30,32 as many subtle indicators occur outside the actual interview discussion. Other perceived disadvantages are the inability to learn about a city or program and difficulty interacting with current residents and faculty.31 One study showed that neither applicants nor interviewers were comfortable making video interviewing the only means of interviewing,32 although another showed that some applicants prefer an initial video interview with the ability to later visit a program.31 This may be an option for some programs if travel restrictions and social distancing are in effect in the fall but later lifted prior to rank list submission. For recommendations on the mechanics of conducting virtual interviews, consider reviewing the tips for PDs and interviewers published by the AAMC and the Compendium of Resources published by the Coalition.35-37 Highlighting the Program Beyond the Interview We must prepare for no in-person visits to residency programs. While this may offer challenges in showcasing a residency program, this is not insurmountable. Residencies may find it beneficial to leverage existing experiences taking place at their institutions, rather than creating entirely new content for applicant consumption. For example, video of residents interacting with faculty during a small group session may provide invaluable information to applicants about didactic quality, how faculty and residents interact, and resident camaraderie. If not already available, programs should consider creation of expanded content (written, photographic, video) that highlights their program’s goals, strengths, and educational philosophy as well as what they are looking for in an applicant. New content will likely need to be created to replace the tour of the ED, resident spaces, the hospital and the surrounding geographic region. While this runs the risk of advantaging programs with the time and resources to produce professional-appearing content, these costs likely pale in comparison to that of running full interview days, and the potential advantages to program and applicants in terms of increased information to make their residency decisions may outweigh these risks. Wide distribution of content will be key. An institution residency webpage, an external website, and social media accounts, if allowed by the institution, will likely be the best options for highlighting this material. Video content could also be uploaded onto YouTube or Vimeo. Given the high utilization of the Emergency Medicine Residents’ Association (EMRA) Match website by medical students, programs should Volume 21, no. 5: September 2020

ensure that their webpages and social media sites are updated on their EMRA Match profile. Helping Applicants get a “Feel” for a Program and Assess for Fit Programs could consider hosting mini virtual-EM rotations with the ability for students to “attend” a short series of video didactic conferences at outside sites. This could be particularly beneficial for programs that traditionally depend on their EM rotation experience as a recruitment tool for outside rotators. Benefits to the student include the abilities to get a glimpse of a program’s teaching styles, facilitate interaction with faculty and residents at other EM programs beyond a single interview day, and make up for lost educational opportunities due to the reduction of the typical number of EM rotations. Virtual rotations could be particularly beneficial for IMG applicants who will likely have difficulty obtaining even one EM rotation due to travel restrictions. We encourage institutions hosting virtual EM rotations to strongly consider accepting IMG students into their rotation to help balance this inequity. To ensure a good ratio of faculty to learners, students should only be able to participate in a small number of these, equivalent to the number of away rotations typically performed. Students’ selections of which of these experiences to participate in could give programs insight into what type of program (or where) the student is ideally looking to match. For students who are invited to interview with a program, an invitation could be extended to “attend” conferences by sharing a virtual forum link. Asking selected faculty or residents to remain online with the students after the conference could be another way for applicants and faculty/ residents to get to know each other. Programs could “host” online pre-interview socials or lunch-time “hangouts” with residents, with the ability to break up into smaller rooms or even one-on-one conversations. Programs could also consider hosting a “hangout” in the spring or summer to generate interest in their program, and record/post it for interested applicants to review. A topic for consideration during these “hangouts” is the kind of resident that really shines in their program, and what kind of applicant might have struggles or find the environment less palatable. This might help attract best-fit applicants. Programs could also consider using a tool that allows interviewees to guide their interview in a way that is meaningful for them and allows a more accurate impression of themselves compared with a traditional interview, as used by one otolaryngology residency program.38 The tablet-based interactive Candidate Assessment Tool allows residency applicants to select questions via a homepage of prerecorded video clips from key leaders in the institution, covering a variety of topics, interests and Accreditation Council for Graduate Medical Education core competencies.

1109

Western Journal of Emergency Medicine

Pelletier-Bui et al.

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process EXPLORATION OF NEW METHODS/METRICS FOR APPLICANT ASSESSMENT Challenge With the potential disruption of metrics and methods that have traditionally been highly valued in the residency selection process,2 programs will have challenges identifying and stratifying applicants who may be successful in their residency program. The combination of this pandemic and USMLE Step 1 moving to pass/fail creates an opportunity for graduate medical education (GME) to develop and explore

new methods to better holistically review applications. Another goal of many EM residency programs is to increase the number of under-represented minorities in one’s program, as evidenced by the surge in under-represented minority EM clerkship scholarships.39 Suggested Solutions While the EM community was not interested in continuing the AAMC Standardized Video Interview pilot as a new metric for EM residency selection,40 there may

Table 1. Suggested solutions to address challenges in the emergency medicine residency application review process amidst COVID-19. Challenge

Suggested solutions

In-person interviewing

• Replace with video interviews. • Augment video interviews with an option to visit the program at a later date pending travel and social distancing restrictions. • For recommendations on the mechanics of conducting virtual interviews, consider reviewing the tips for program directors and interviewers published by the Association of American Medical Colleges and the Compendium of Resources published by the Coalition of Physician Accountability.

Highlighting the program beyond the interview

• Record and post videos of existing resident experiences, didactics, etc. • Create expanded content (written, photographic, video) with particular attention to replacing the traditional tour of the facility, resident spaces and geographic area. • Promote content via institution website and social media (Twitter, Instagram, Facebook, YouTube). • Ensure Emergency Medicine Residents’ Association Match profile and social media links are up to date.

Helping applicants get a “feel” for a program and assess for “fit”

• • • •

Host a mini virtual EM rotation. Invite applicants to “attend” conference virtually. Host online pre-interview and lunch socials. Host resident “hang outs” with residents fielding applicant questions about what kind of applicant shines in their program. • Use an interactive interview tool that allows applicants to guide their interview in a way that is meaningful to them.

EM, emergency medicine.

be alternative tools that we can explore to help identify successful applicants to our EM residency programs. Prehire assessments are used by eight of the top 10 US private employers and by 57% of large US employers.41 Some of these assessments have demonstrated utility in undergraduate medical education and GME as well.42-48 For example, emotional intelligence testing has been shown to have positive correlation with medical school success.43 Personality testing to assess for fit is widely used in other industries and some data indicate that it may be effective for residency selection as well.42 Situational judgment testing through methods such as the Computerbased Assessment for Sampling Personal Characteristics (CASPer), has demonstrated moderate predictive validity to national licensure outcomes in Canada44 and is required by all medical school applicants in Canada and two US medical schools.45 Its utilization in general surgery has been associated with overall performance in residency,46 allowed for more general surgery interview offers to underrepresented minorities in medicine,48 and did not detract applicants from applying to the general surgery programs that implemented its use.45,48 While it is neither feasible, nor advisable, to incorporate these assessments for all EM applicants during this application cycle, now is a better time than ever to begin exploring and validating these, or other methods, for potential future use in the EM residency application process. Volume 21, no. 5: September 2020

Altus Assessments, the developer of CASPer, and the National Board of Medical Examiners are collaborating to explore the use of CASPer during this residency application season (Elise Lovell, MD, on behalf of Altus Assessments and the National Board of Medical Examiners, CORD listserv communication, June 21, 2020). Programs interested in participating in this research project can fill out this form (http://bitly.ws/9s73). CONCLUSION While COVID-19 presents significant challenges to medical education and the residency application process, the GME community can view this pandemic as an opportunity to explore disruptive change. Just as quarantine orders eliminated barriers to virtual meetings and appointments overnight, we have the chance to make important changes to the residency application process that will benefit programs and applicants for years to come (Tables 1 and 2). We should embrace this opportunity while simultaneously working to preserve the essential components of the process. Only time will tell how the pandemic will influence the match process, and what adaptations will be most helpful. The CORD Application Process Improvement Committee is committed to making the most of the situation and plans to follow up with a report detailing the impacts and best practices for addressing similar challenges in the future.

1111

Western Journal of Emergency Medicine

Pelletier-Bui et al.

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process Address for Correspondence: Alexis Pelletier-Bui, MD; Cooper Medical School of Rowan University, Department of Emergency Medicine, 1 Cooper Plaza, 152 Kelemen, Camden, NJ 08103. Email: pelletier-bui-alexis@cooperhealth.edu.

Residency Directors’ Standardized Letter of Recommendation: The program director’s perspective. Acad Emerg Med. 2014;21(6):680-7. 9. Jarou Z. (2019). EMRA and CORD Student Advising Guide. Hillman E, Kellogg A, Lutfy-Clayton L, Pelletier-Bui A, Shandro J, eds. Irving, TX: Emergency Medicine Residents’ Association.

10. Emergency Medicine Residents’ Association. EMRA Match. Available at: https://webapps.emra.org/utils/spa/match#/search/map. Accessed April 24, 2020. 11. Ray JC, Hopson LR, Santen SA, et al. Choosing emergency medicine: influences on medical students’ choice of emergency

Copyright: © 2020 Pelletier-Bui et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http:// creativecommons.org/licenses/by/4.0/

medicine. PLoS One. 2018;13(5):e0196639. 12. Shandro J, Kessler R, Schrepel C, et al. Advising medical students during COVID-19: the case for a single emergency medicine rotation for all. AEM Education and Training. 2020. In press. 13. Katirji L, Smith L, Pelletier-Bui A, et al. Addressing challenges in obtaining emergency medicine away rotations and standardized

REFERENCES

letters of evaluation due to COVID-19. West J Emerg Med. 2020;21(3):538-41.

1. Association of American Medical Colleges. Important guidance for medical students on clinical rotations during the coronavirus (COVID-19)

14. Association of American Medical Colleges. ERAS 2021 residency timeline. Available at: https://students-residents.aamc.org/applying-

outbreak. Available at: https://www.aamc.org/news-insights/press-

residency/article/eras-timeline-md-residency/. Accessed May 14, 2020.

releases/important-guidance-medical-students-clinical-rotations-during-

15. Prometric. Special notice from Prometric regarding COVID-19

coronavirus-covid-19-outbreak. Accessed April 24, 2020.

(coronavirus). Available at: https://www.prometric.com/corona-virus-

2. National Residency Matching Program. Results of the 2018 program

update. Accessed June 24, 2020.

director survey. Available at: https://www.nrmp.org/wp-content/

16. United States Medical Licensing Examination. USMLE suspending

uploads/2018/07/NRMP-2018-Program-Director-Survey-for-WWW.

Step 2 Clinical Skills Examination. Available at: covid.usmle.org/

pdf. Accessed April 24, 2020.

announcements/usmle-suspending-step-2-clinical-skills-

3. Reed T. ERs have been quiet during COVID-19. What happens when

examination. Accessed June 24, 2020.

the patients come back? Available at: https://www.fiercehealthcare.

17. Educational Commission for Foreign Medical Graduates. Requirements

com/hospitals-health-systems/ers-have-been-quiet-during-covid-19-

for ECFMG certification for 2021 match. Available at: ecfmg.org/

what-happens-when-patients-come-back. Accessed April 24, 2020.

certification-requirements-2021-match/. Accessed June 24, 2020.

4. Mulcare MR, Suh EH, Tews M, et al. Third-year medical

18. United States Medical Licensing Examination. Preparations for event

student rotations in emergency medicine: a survey of current

testing. Available at: covid.usmle.org/announcements/preparations-

practices. Acad Emerg Med. 2011;18 Suppl 2:S41-7.

event-testing. Accessed June 24, 2020.

5. The Coalition for Physician Accountability’s Work Group on Medical

19. Harmouche E, Goyal N, Pinawin A, et al. USMLE scores predict

Students in the Class of 2021 Moving Across Institutions for Post Graduate Training. Final report and recommendations for medical

success in ABEM initial certification: A multicenter study. Western J

education institutions of LCME-accredited, U.S. osteopathic, and

Emerg Med. 2017;18(3):544–49. 20. Wagner JG, Schneberk T, Zobrist M, et al. What predicts performance?

non-U.S. medical school applicants. Available at: https://www.aamc. org/system/files/2020-05/covid19_Final_Recommendations_05112020.

A multicenter study examining the association between resident

pdf. Accessed May 14, 2020.

performance, rank list position, and United States Medical Licensing Examination Step 1 Scores. J Emerg Med. 2017;52(3):332-40.

6. American Academy of Emergency Medicine, American Academy of

21. United States Medical Licensing Examination. InCUS-Invitational

Emergency Medicine Resident and Student Association, The American College of Osteopathic Emergency Physicians, et al.

conference on USMLE scoring. Available at: https://www.usmle.org/

Consensus statement on the 2020-2021 residency application

incus/. Accessed April 21, 2020. 22. Association of American Medical Colleges. Holistic admissions.

process for US medical students planning careers in emergency medicine in the Main Residency Match. Available at: https://www.

Available at: https://www.aamc.org/services/member-capacity-

cordem.org/globalassets/files/2020-res-app-consensus-statement-v2.

building/holistic-review. Accessed April 21, 2020. 23. Stepan KO, Kaul VF, Raquib AR, et al. An evaluation of the program-

pdf. Accessed June 24, 2020.

specific paragraph in the otolaryngology residency application.

7. Negaard M, Assimacopoulos E, Harland K, Van Heukelom J.

Laryngoscope. 2018;128(11):2508-13.

Emergency medicine residency selection criteria: an update and

24. Kramer S. Is the program-specific paragraph responsible for

comparison. AEM Educ Train. 2018;2(2):146-53. 8. Love JN, Smith J, Weizberg M, et al. Council of Emergency Medicine

Western Journal of Emergency Medicine

1112

declining application numbers? A commentary. Otolaryngol Head

Volume 21, no. 5: September 2020

Pelletier-Bui et al.

COVID-19: Driver for Disruptive Innovation of EM Residency Application Process

Neck Surg. 2018;158(2):215-6.

accredited, U.S. osteopathic, and non-U.S. medical school applicants.

25. Salehi PP, Azizzadeh B, Lee YH. Preference signaling for

Available at: https://www.aamc.org/system/files/2020-05/covid19_

competitive residency programs in the NRMP. J Grad Med Educ.

Compendium_Resources_05112020.pdf. Accessed May 15, 2020.

2019;11(6):733-4.

38. Platt MP, Akhtar-Khavari V, Ortega R, et al. The interactive candidate

26. Bernstein J. Not the last word: want to match in an orthopaedic

assessment tool: a new way to interview residents. Otolaryngol Head

surgery residency? Send a rose to the program director. Clin Orthop

Neck Surg. 2017;156(6):1150-1153.

Relat Res. 2017;475(12):2845–9.

39. Society for Academic Emergency Medicine Clerkship Directors in

27. Whipple ME, Law AB, Bly RA. A computer simulation model to

Emergency Medicine. Visiting elective scholarship programs for

analyze the application process for competitive residency programs.

underrepresented minorities. Available at: https://www.saem.org/

J Grad Med Educ. 2019;11(1):30–5.

cdem/resources/medical-student-resources/underrepresented-

28. Salehi PP, Benito D, Michaelides E. A novel approach to the National Resident Matching Program—The Star System. JAMA Otolaryngol

minority-scholarships. Accessed April 24, 2020. 40. Association of American Medical Colleges. AAMC standardized video

Head Neck Surg. 2018;144(5):397–8.

interview evaluation summary. Available at: https://students-residents.

29. Agawu A, Fahl C, Alexis D, et al. The influence of gender and

aamc.org/applying-residency/article/svi-evaluation-summary/.

underrepresented minority status on medical student ranking of

Accessed April 24, 2020.

residency programs. J Natl Med Assoc. 2019;111(6):665-73.

41. Weber L. Today’s personality tests raise the bar for job seekers.

30. Shah SK, Arora S, Skipper B, et al. Randomized evaluation of a web

Available at: https://www.wsj.com/

based interview process for urology resident selection. J Urol.

articles/a-personality-test-could-stand-in-the-way-of-your-next-job-

2012;187:1380–4.

1429065001?ns=prod/accounts-wsj. Accessed April 23, 2020.

31. Daram SR, Wu R, Tang SJ. Interview from anywhere: feasibility and

42. Merlo LJ, Matveevskii AS. Personality testing may improve resident

utility of web-based videoconference interviews in the

selection in anesthesiology programs. Med Teach. 2009;31(12):e551-4.

gastroenterology fellowship selection process. Am J Gastroenterol.

43. Cook CJ, Cook CE, Hilton TN. Does emotional intelligence influence

2014;109:155-9.

success during medical school admissions and program matriculation?:

32. Edje L, Miller C, Kiefer J, et al. Using Skype as an alternative for

a systematic review. J Educ Eval Health Prof. 2016;13:40.

residency selection interviews. J Grad Med Educ. 2013;5:503–5.

44. Dore KL, Reiter HI, Kreuger S, et al. CASPer, an online pre-interview

33. Hariton E, Bortoletto P, Ayogu N. Using video-conference interviews

screen for personal/ professional characteristics: prediction of

in the residency application process. Acad Med. 2017;92(6):728-9.

national licensure scores. Adv in Health Sci Educ. 2017; 22:327–36.

34. Hariton E, Bortoletto P, Ayogu N. Residency interviews in the 21st

45. Shipper ES, Maer LM, Merrell SB, et al. Pilot evaluation of the

century. J Grad Med Educ. 2016;8:322–4.

Computer-Based Assessment for Sampling Personal Characteristics

35. Association of American Medical Colleges. Virtual interviews: Tips for program directors. Available at: https://www.aamc.org/system/

test. J Surg Res. 2017;215:211-8. 46. Gardner AK, Dunkin BJ. Evaluation of validity evidence for

files/2020-05/Virtual_Interview_Tips_for_Program_

personality, emotional intelligence, and situational judgment tests to

Directors_05142020.pdf. Accessed May 15, 2020.

identify successful residents. JAMA Surg. 2018;153(5):409-16.

36. Association of American Medical Colleges. Virtual interviews: Tips for

47. Gardner AK, Cavanaugh KJ, Willis RE, et al. Can better selection tools

interviewers. Available at: https://www.aamc.org/system/

help us achieve our diversity goals in postgraduate medical education?

files/2020-05/Virtual_Interview_Tips_for_Interviewers_05142020_0.

Comparing use of USMLE Step 1 scores and situational judgment

pdf. Accessed May 15, 2020.

tests at 7 surgical residencies. Acad Med. 2020;95(5):751-7.

37. The Coalition for Physician Accountability. Compendium of resources

48. Gardner AK, Cavanaugh KJ, Willis RE, et al. If you build it, will they

for the implementation of recommendations in the final report and

come? Candidate completion of preinterview screening assessments.

recommendations for medical education institutions of LCME-

J of Surg Educ. 2019;76(6):1534-38.

Volume 21, no. 5: September 2020

1113

Western Journal of Emergency Medicine

Brief Research Report

Leveraging Remote Research Associates During a Pandemic Alexandrea O. Cronin, MPH* Morgan A. Carlile, MD* Christian J. Dameff, MD, MS*†‡ Christopher J. Coyne, MD, MPH* Edward M. Castillo, PhD, MPH*

*University of California San Diego, Department of Emergency Medicine, San Diego, California † University of California, San Diego, Department of Biomedical Informatics, San Diego, California ‡ University of California, San Diego, Department of Computer Science and Engineering, San Diego, California

Section Editor: Dan Mayer, MD and Nikhil Goyal, MD Submission history: Submitted May 5, 2020; Revision received June 25, 2020; Accepted June 24, 2020 Electronically published July 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.48043

Introduction: The coronavirus disease 2019 (COVID-19) pandemic has seriously impacted clinical research operations in academic medical centers due to social distancing measures and stay-at-home orders. The purpose of this paper is to describe the implementation of a program to continue clinical research based out of an emergency department (ED) using remote research associates (RA). Methods: Remote RAs were trained and granted remote access to the electronic health record (EHR) by the health system’s core information technology team. Upon gaining access, remote RAs used a dual-authentication process to gain access to a host-based, firewall-protected virtual network where the EHR could be accessed to continue screening and enrollment for ongoing studies. Study training for screening and enrollment was also provided to ensure study continuity. Results: With constant support and guidance available to establish this EHR access pathway, the remote RAs were able to gain access relatively independently and without major technical troubleshooting. Each remote RA was granted access and trained on studies within one week and self-reported a high degree of program satisfaction, EHR access ease, and study protocol comfort through informal evaluation surveys. Conclusions: In response to the COVID-19 pandemic, we virtualized a clinical research program to continue important ED-based studies. [West J Emerg Med. 2020;21(5)1114-1117.]

In response to the declared coronavirus disease 2019 (COVID-19) pandemic3 and in adherence with state and local restrictions,4,5 operations within academic medical centers have been significantly impacted to ensure patient, staff, student, and volunteer safety. Clinical research tasks are necessary even during the pandemic to evaluate new protocols and treatment options, enroll new patients in ongoing studies, conduct follow-up on existing patients, and still have access to high quality and peer-reviewed data in well-designed trials. This pandemic has modified how we think of patient care and conducting research from afar beyond traditional remote chart review. Advances in technology have allowed for the remote care of some patient conditions and supportive activities such as data reporting. We hoped to expand prospective research

1114

Volume 21, no. 5: September 2020

Cronin et al.

Remote Research Associates

activities using remote access, which is not as prominent in the literature. Advances in electronic health record (EHR) capabilities have transformed how we capture, store, and summarize patient data.6 Equally important technology advances, such as security protocols and virtual networks, have allowed clinicians and researchers to access information securely and remotely. Cybersecurity specifically allows for authorized access to an established system while preventing unauthorized access. Given the abundance and importance of clinical research to improve the quality of patient care and continue educational objectives of health systems, we have successfully implemented and positioned remote RAs to continue EDbased research objectives during the COVID-19 pandemic. The purpose of this paper is to describe the implementation of a program to continue clinical research based out of an ED using remote RAs. METHODS This paper describes a remote RA program developed at an academic medical health system with two hospital EDs using a shared EHR (Epic) with a combined census of 85,000 patients annually. One hospital is an urban, academic teaching hospital (a Level 1 trauma center with the region’s only burn center) and the other is a suburban community hospital with a geriatric emergency care unit within the ED. The health system has extensive multidisciplinary research programs that rely on standard clinical research methodologies to conduct innovative research studies and clinical trials. The University of California San Diego Department of Emergency Medicine RA program was established in 2002 to provide RAs with the opportunity to gain clinical research experience in the ED while receiving academic and professional mentorship. There are usually 40-50 RAs in the program at a time who cover about 16 hours in both EDs per day. Prior to COVID-19, RAs provided active engagement with ED patients to collect data for numerous ongoing studies and support other research opportunities. Many students obtain independent study college credit for their participation in the program. All RAs are required to complete Human Subjects Protections (HSP) training and EHR training. HSP training is conducted through the Collaborative IRB Training Initiative (CITI) and set forth by the institution’s expectations. RAs must also complete Epic ASAP EHR Training. Next, RAs are scheduled for a one-hour orientation with a faculty or staff member to understand existing ED-based study needs and departmental expectations. RAs then shadow a training officer in the ED and complete a competency checklist that reinforces program expectations in order to be able to serve shifts independently. Time with the training officer varies based on individual RA level of comfort and ED-specific factors such as patient availability. Providing remote access to a group of existing RAs Volume 21, no. 5: September 2020

allowed for two specific studies to continue, while allowing RAs to gain clinical research experience and independent study credit. Student participation in these studies requires skills to complete tasks beyond remote chart review. One of the projects is an observational study that involves the evaluation of a rapidly implemented clinical care pathway to treat COVID-19 patients that began following the implementation of the stay-at-home measures; the other project is an ED-based, randomized controlled trial to refer to and assess two palliative care treatment arms for patients who had been previously diagnosed with a life-limiting illness. In support of these two studies, RAs attended a remote onehour training session for each study provided by each study’s principal investigator and RA program administrators that involved a live demonstration and discussion of study-specific inclusion and exclusion criteria. The week following the initial remote setup, a program coordinator conducted 30-minute scheduled remote consultations with individual RAs to review study criteria, troubleshoot remote access issues, and informally assess individual RA competence of study-specific expectations. To informally assess RA competence of study-specific expectations, we asked questions about navigating the EHR and other important study-specific processes. We summarized all program and study training time allotments in Table 1. Roughly seven weeks following the implementation of remote RA access, we informally assessed the perceptions of the advantages and disadvantages of conducting remote research from six RAs. Responses from the informal assessment are summarized in Table 2.

Table 1. Training types with time allocations for student research associates and faculty/staff.

Training type

Student time allocation (hours)

Faculty or staff time allocation (hours)

CITI HSP and GCP training*‡

4-6

EHR training*

1-2

2-4

Study 1*

Study 2*

‡

Orientation**‡ Shadowing in the ED**

‡

Remote access training and study-specific training

Check-ins* 0.5-1 0.5-1 *Training conducted remotely **Training conducted in person ‡ RA program onboarding (students had completed previously) RA, research associate; CITI, Collaborative Institutional Review Board Training Initiative; HSP, Human Subjects Protections; GCP, good clinical practice; EHR, electronic health record; ED, emergency department.

1115

Western Journal of Emergency Medicine

Cronin et al.

Remote Research Associates Table 2. Response themes of remote research associates’ selfreported advantages and disadvantages of conducting remote research. Advantages

Disadvantages

• Having your own (unshared) space • Having constant/direct computer access • Schedule flexibility • More exposure to more studies* • Not being exposed to COVID-19 • Being able to continue gaining research experience • Not needing to consider transportation arrangements • Being able to still work with patients

• Sometimes challenging to reach busy clinicians • Not being in person with patients to gauge indirect communication cues • Discussing sensitive topics with patients via phone • Patients not understanding why we aren’t there in person • Increased challenge of getting the information you need from a patient or clinician • Not getting to be in the ED • Reaching patients can be tough *Note: Some remote RAs previously had limited study participation. COVID-19, coronavirus disease 2019; ED, emergency department.

Expanding on existing network infrastructure protected by a host-based firewall, six existing RAs were provided guidance and training on dual authentication processes and access to a virtual private network (VPN) from which they could access the EHR. Access to the VPN is granted using a single sign-on (SSO) process in an active directory that can only be modified by core information technology (IT) team members. These RAs independently established access to a two-factor authentication system using unique credentials to access the VPN. RESULTS Leveraging skills and resources from research, IT, and clinical staff, we successfully provided access to an EHR via existing network infrastructure to enable six remote RAs in one week. Given that this research program would have otherwise been terminated due to COVID-19, RAs self-reported a high degree of program satisfaction, remote EHR access ease, and comfort with study protocol during the 30-minute check-ins with the program coordinator. For the COVID-19 pathway study, enabling remote RAs allowed them to screen for eligible study participants in the EHR and collect data via telephone from ED attending and resident providers. While there was concern regarding interrupting emergency provider workflows, emergency physicians were receptive to study participation and would communicate at a later time with our RAs if they were busy. For the second study, RAs not only screened for eligible ED patients in the EHR during the patient’s stay, but also worked with the clinical care teams via telephone to confirm subject eligibility in adherence Western Journal of Emergency Medicine

with defined study inclusion criteria. Using details from the EHR, the RAs contacted patients during their ED stay using the patient’s cell phone number or by calling the phone in the patient’s assigned room, conducted informed consent processes via telephone, as deemed appropriate by the IRB, and collected baseline study details. Overall, our remote RA program success is indicated by our ability to continue collecting data and enrolling patients in our two ED-based studies, as well as our informal assessment of the advantages and disadvantages of conducting research remotely, as summarized in Table 2. Further expansion to other observational and prospective studies and increasing research capabilities, including expansion to remote RAs not tied to college credit, with additional students is moving forward as well. Similar to our site’s ability to enroll patients for this study telephonically through remote RAs, other participating sites in the palliative care study have also been able to continue to enroll patients. DISCUSSION This remote RA approach allowed us to both initiate and continue important research studies to address the effects of a pandemic, but could also be considered as a means to improve efficiency in the future. Similar to how many healthcare settings have responded to the pandemic by reducing or eliminating visitors to reduce the spread of the virus,2 the remote RA model reduces the number of people who need to be physically present in the healthcare setting, which is beneficial for patients who may feel overwhelmed by the ED environment and further allows for the optimization and prioritization of their care. This approach opens the realm of clinical research to new opportunities. Despite the challenges that this worldwide crisis has caused, clinicians and researchers still have an opportunity to respond by rethinking the way they continue prospective clinical research. Involving RAs in research remotely fills an important gap while diversifying and expanding experiences and possibilities for RAs with underlying health conditions to gain experience beyond chart review alone. This is especially true of the COVID-19 pandemic or of similar situations where access to a clinical setting may be detrimental to an individual’s health. While this approach has positive implications for reducing the need for unnecessary exposure and personal protective equipment, it will reduce the ability of study team members seeking in-person, clinical exposure to gain necessary contextual experience. Weighing the pros and cons of this type of approach is thus important and could be cascaded into alternative research experience models. This approach also changes the way that researchers and clinicians think about subject recruitment and enrollment. While this is currently serving as a temporary alternative to existing workflows, remote EHR access and enrollment by telephone allow research teams with enrollment or other expertise to engage patients from afar, conduct research with

1116

Volume 21, no. 5: September 2020

Cronin et al.

Remote Research Associates

special populations, or conduct remote follow-up activities. Similarly, this approach transforms the need for brick and mortar structures for research teams and allows the option to work from a convenient location or designated space. However, this approach minimizes or eliminates the personal contact of in-person subject recruitment. Finally, leveraging remote RAs expands research teams to include appropriately trained students, off-site clinicians, and other research team members. Study teams are increasingly comprised of research sites or members from various locations to improve sample size and subject recruitment. Leveraging remote RAs could expand the ability of sites to more broadly share information and data with one another – while engaging with a diverse group of study team members – contributing to the improved quality and efficiency of multisite study recruitment, collaboration, and data-related efforts. Our site is one of many sites currently participating in the ED-based palliative care study and leveraging remote RAs to help with research activities for this study has helped to expand study screening and recruitment activities from afar.

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Cronin et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/ REFERENCES 1. Davis DP, Poste JC, Kelly D. The UCSD Research Associate Program: a recipe for successfully integrating undergraduates with emergency medicine research. J Emerg Med. 2005;28(1):89-93. 2. Hollander JE, Singer AJ. An innovative strategy for conducting clinical research: the academic associate program. Acad Emerg Med.

LIMITATIONS The RAs involved in our program had previously been trained in person on ED workflows and program expectations. In the event that someone without an existing program wanted to start one in a similar way, additional training considerations would need to be considered. For example, our RAs already understood ED workflows, processes and expectations, so transitioning them to remote access was not as challenging.

2002;9(2):134-7. 3. Cucinotta D, Vanelli M. WHO Declares COVID-19 a pandemic. Acta Bio Med. 2020;91(1):157-60. 4. Centers for Disease Control and Prevention. Coronavirus Disease 2019 (COVID-19): For Healthcare Professionals. Available at: https://www.cdc. gov/coronavirus/2019-ncov/hcp/index.html. Accessed April 10, 2020. 5. Executive Department State of California. Executive Order N-33-20. 2020. Available at: https://www.gov.ca.gov/wp-content/ uploads/2020/03/3.19.20-attested-EO-N-33-20-COVID-19-HEALTH-

CONCLUSION As technology improves and broadens academic medical centers’ research methods, alternative approaches may enable research continuity, even in a pandemic. While some studies may not be suitable for remote RAs, some projects may continue, and lessons from COVID-19 may be carried forward beyond the pandemic.

Volume 21, no. 5: September 2020

Address for Correspondence: Alexandrea O. Cronin, MPH, University of California San Diego, Department of Emergency Medicine, 200 West Arbor Drive, #8676, San Diego, CA 92103. Email: aocronin@health.ucsd.edu.

ORDER.pdf. Accessed April 10, 2020. 6. Evans RS. Electronic health records: then, now, and in the future. Yearb Med Inform. 2016;(Suppl 1):S48-61. 7. Jalali MS, Kaiser JP. Cybersecurity in hospitals: a systematic, organizational perspective. J Med Internet Res. 2018;20(5):e10059. 8. CITI Program. Collaborative IRB Training Initiative (CITI). Available at:

1117

https://about.citiprogram.org/en/homepage/. Accessed April 30, 2020.

Western Journal of Emergency Medicine

Commentary

Resident Research in Emergency Medicine: An Introduction and Primer James H. Paxton, MD MBA* Anne M. Messman, MD, MHPE* Nicholas E. Harrison, MD* Adrienne N. Malik, MD† Raina J. Burke, MD‡ Phillip D. Levy, MD MPH*

*Wayne State University, Department of Emergency Medicine, Detroit, Michigan † Kansas University Medical Center, Department of Emergency Medicine, Kansas City, Kansas ‡ Wheeling Hospital, Department of Emergency Medicine, Wheeling, West Virginia

Section Editor: Whitney Johnson, MS, MD Submission history: Submitted February 13, 2020; Revision received May 29, 2020; Accepted June 30, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.46520

Training in research methodology represents an important aspect of emergency medicine (EM) resident education, but best methods for design, implementation, and dissemination of resident research remain elusive. Here we describe recommendations and best practices from the existing literature on EM resident research, including helpful tips on how to best implement a resident research program. [West J Emerg Med. 2020;21(5)1118-1122.]

INTRODUCTION When René Laënnec, a French physician in 1816, failed to adequately percuss the thorax of a young woman with heart disease, he improvised. Laënnec wrote, “I rolled a quire of paper into a sort of cylinder and applied one end of it to the region of the heart and the other to my ear.”1 After numerous revisions, his invention was revealed to the medical community, and quickly caught on. Within a few years, primitive stethoscopes could be found in medical shops throughout Paris. Had Laënnec stopped with that rolledup piece of paper, his one-time improvisation would have been lost to the annals of history. Fortunately, he chose to build upon his initial discovery and, crucially, to share his breakthrough with the world. Laënnec’s journey charts an enduring and fundamental trajectory of medical innovation: from observation, through inspiration, refinement and testing, to dissemination. Development of a research project can be especially daunting to physicians already engaged in an emergency medicine (EM) residency training program. But execution of a research project during residency remains a worthwhile experience, allowing participants to meaningfully contribute to medical knowledge and develop an investigative spirit.2 Residents participating in research appear to attain greater job satisfaction,3 and can objectively frame everyday

Western Journal of Emergency Medicine

questions and methodically seek answers3 to problems including (among others) staffing issues, wait times, and communication barriers.4-5 The Accreditation Council for Graduate Medical Education Residency Review Committee for EM recognizes the importance of these efforts, mandating resident completion of a “scholarly project” prior to graduation. Their requirement cites the following as examples of qualifying activities: “… the preparation of a scholarly paper such as a collective review or case report, active participation in a research project or formulation and implementation of an original research project.”6 These activities should include problem identification, data collection, analysis, and conclusion.7 Performance and documentation of these projects are vital to the acceptance of a scholarly project, whether a case report, community project, development of medical software, or traditional research project.7 Recent reports from within the EM community have emphasized the importance of scholarly activity to EM resident education.8-9 Advancing the state of scientific knowledge is not a requirement for success in resident research, but it is a potential benefit of this exercise. It is the responsibility and privilege of those involved in residency administration to facilitate the training of EM resident researchers in the development and execution of research projects that support

1118

Volume 21, no. 5: September 2020

Paxton et al.

Resident Research in EM: An Introduction and Primer

not only the professional careers of residents but also the advancement of our specialty.8-9 What is Resident Research? It has been suggested that “resident research” is, “research where a resident has a principal role in the implementation and completion of the project.”10 We suggest that the resident research experience be defined by the engagement of the resident learner in the research process, focusing upon the educational value of the project rather than the resident’s official role or involvement in the design and execution of the project. Research studies are intended to create new generalizable knowledge that can be applied to other populations and settings.11 Consequently, we propose that “resident research” be defined as any systematic investigation designed to yield new information that actively engages the resident-learner and facilitates the acquisition of a greater understanding of the scientific method. This is in distinction to quality improvement projects, which seek to apply existing knowledge to improve healthcare outcomes within a local healthcare institution or setting.12 Setting Realistic Expectations One purpose of resident research is to expose residents to the methods by which research is conducted, creating “educated consumers” of the medical literature. However, residencies hoping to establish a resident research program de novo must recognize the additional workload that resident research projects impose upon faculty. Mentors should be primarily responsible for guiding and supervising resident research, but should be adequately vetted to ensure that the research experience yields a positive result for all involved. Research directors should provide guidance relating to funding opportunities, deadlines for abstract submission to key research conferences, important institutional and federal regulations, and departmental resources.13 Departmental leadership should create an environment in which research is actively promoted, providing appropriate funding and protected time for mentors and other research faculty.13

The Research Question A general research question must be formulated, which will generate a testable hypothesis.3,19 All possible outcomes should be considered, and at least one of them must be worthwhile.20 The FINER criteria may be used to assess the relative merits of the proposal:14-15,21 Feasible Can the project be completed within the time allotted using the given resources? Can the proposed investigation enroll enough patients to demonstrate a difference in the proposed outcome measures? Interesting Is the topic engaging enough to be worth the effort? Novel Is the proposed investigation different enough from what has been done before to add knowledge on the subject? Ethical Does the proposed investigation respect the morals of the community, the patient, and the profession? Relevant Are the results likely to be applicable to many patients? Will the results be useful and contribute to the greater good? Formulating a Hypothesis A suitably refined and meaningful research question will help in generating a hypothesis, providing a clear delineation of what the investigation will attempt to prove. Investigation of a well-designed hypothesis will be interesting even if a negative result is found.

Getting Started Clinical experiences, journal club articles, or experiences with different teaching modalities may generate an appropriate resident research topic including relevant clinical or educational questions.2, 14-15 Additional ideas may come from the resident’s personal interests or experiences. Learning Research Methodology Most programs will offer training through didactic presentations, journal clubs or evidence-based literature discussions. However, a focused educational effort specifically targeting research methodologies has been shown to correlate with improved resident skills, knowledge, and research productivity.16 Nearly one in four EM training programs offers a fixed rotation in research.5,17 A more Volume 21, no. 5: September 2020

feasible format for the busy trainee might be the Advanced Research Methodology Evaluation and Design video series available from the Society of Academic Emergency Medicine (SAEM), including “how-to” webinars and podcasts produced by senior researchers.18

The Mentor A mentor experienced in the resident’s area of research interest can be an invaluable resource by offering hints at project scope, helping with setbacks, and tailoring the learning experience to the resident’s needs.22 Most often, the mentor is an established researcher within the department but could include a specialist in another field, or even a non-physician investigator.5, 23-24 Goals and expectations should be discussed early on, to avoid frustration for both parties.5 Terregino has shown that, in general, EM residents are relatively unfamiliar with what resources are available to them, which can lead to significant amounts of time wasted.25 Most hospitals provide research support that is invisible to the outside observer, including project coordinators, departmental research directors, and biostatisticians.25 The mentor should be aware of all available institutional resources.

1119

Western Journal of Emergency Medicine

Paxton et al.

Resident Research in EM: An Introduction and Primer The Literature Search A valid research project must be informed by past work. Most literature reviews will begin with a search of PubMed. gov, the database of the National Library of Medicine, or OVID.org, which includes textbooks as well as journals.26 Search terms used must be carefully selected, and the proper Boolean operators assigned. One study has shown physicians to be especially inept at crafting effective search strings.27 Any doubts about the literature search process or its results should be referred to a librarian. Each paper identified from the literature review should be thoroughly read. Investigators should avoid citing abstracts alone, as they are often incomplete in their data presentation. This process is labor-intensive but necessary to form a strong foundation for the research project. All references cited within each article should be assessed for relevance. The selected literature should be reviewed to better understand the subject matter and to develop context for the proposed work. If adequate data from existing sources are uncovered, one may consider a retrospective evaluation of prior results including a meta-analysis.2, 28-29 Research Design The novice researcher should look to the existing medical literature for guidance in how to properly design a new study. Selection of the proper research methodology will depend upon multiple factors, including the research question, hypothesis, and predetermined outcome measures. A timeline should be implemented to ensure that all tasks are achievable within the allotted time. Resident physicians should develop a team approach, incorporating input from the faulty mentor as well as a staff epidemiologist or biostatistician. The required sample size will depend upon a variety of factors, including the acceptable level of significance, power of the study, expected effect size, underlying event rate in the population, and standard deviation in the population.30-31 Efforts should be made to collect an inclusive and truly random sampling, to avoid convenience selection bias.32 Early consultation with the biostatistician will also inform the researcher’s decisions on the most appropriate methods for the statistical analysis of data derived from the study. For further information about study design specifics, the reader is referred to several existing publications.2, 4, 13, 33-34

Conducting the Study After the research protocol has been IRB-approved or exempted, data collection can commence. Prior development of a data collection tool will greatly enhance the efficiency of this process, facilitating both IRB approval and the subsequent data analysis. Subject enrollment can also be improved with use of a trained research assistant. This problem may be circumvented through creation of an “academic associate program,” which integrates EM research with undergraduate education.35 Research Funding Resident research projects usually require little external funding. On occasion, additional costs may be incurred to help pay for statistical analysis, or the purchase of required equipment.36 Internal sources, as well as the Emergency Medicine Foundation 37 and the SAEM Foundation 38 represent potential sources for funding. Presentation and Publication Once the data have been collected and analyzed, the researcher should consider how the results will be disseminated. The annual meeting of SAEM, the Research Forum at the American College of Emergency Physicians’ annual scientific assembly, and the Annual Assembly of the Council of Residency Directors in Emergency Medicine (CORD) represent the premier locales for presentation of EM research.39 Ideally, the resident research experience should lead to a manuscript, although the lack of immediate publication must not be interpreted as failure. Only 40% of EM abstracts go on to become full article publications.15,40 Most manuscripts are published 1-2 years after initial presentation.17 Appropriate journal selection for submission enhances the likelihood of success, as does a thorough understanding of manuscript preparation techniques and review criteria.41-43

The Institutional Review Board Any research project that involves human participants or their data requires submission to the local institutional review board (IRB). Research protocols submitted to the IRB can fall into one of three categories: full submission; expedited; or exempt. Research involving greater than minimal risk to human subjects will require a thorough review by the IRB and development of an informed consent document. Prospective projects involving only minimal risk may be approved via Western Journal of Emergency Medicine

the expedited process, where a single reviewer may approve the work in lieu of the convened board. Studies that include only retrospective data from the electronic health record may be exempt from IRB review, but this determination should be made by the IRB, rather than by the investigator. Investigators should confer with their local IRB to confirm what level of IRB review is required before beginning data collection.

CONCLUSION While any research resultant from a resident’s scholarly project is unlikely to have the impact of Laënnec’s stethoscope, EM residents may still gain much from engaging in clinical research. For some, it will light an investigative fire that will burn for an entire career. At the least, resident research projects can provide an opportunity to explore issues central to the practice of EM, helping the resident to become a more well-rounded physician.

1120

Volume 21, no. 5: September 2020

Paxton et al.

Resident Research in EM: An Introduction and Primer

Address for Correspondence: James H. Paxton, MD, MBA, Wayne State University, Department of Emergency Medicine, 4201 St. Antoine Street, Detroit, MI 48201. Email: james.paxton@wayne.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Paxton et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

the 21st century. 2001. Available at: https://pubmed.ncbi.nlm.nih. gov/25057539/. June 9, 2020. 13. Biros MH, Barsan WG, Lewis RJ, et al. Supporting emergency medicine research: Developing the infrastructure. Ann Emerg Med. 1998;31(2):188-95. 14. Pollack C and Wadbrook P. Successful resident research in emergency medicine: the keys to relevance and quality. Ann Emerg Med. 1999;333(1):117-20. 15. Good AM and Driscoll P. Clinical research in emergency medicine: putting it together. Emerg Med J. 2002;19(3):242-6. 16. Rydman RJ, Zalenski RJ, Fagan JK. An evaluation of research training in a large residency program. Acad Emerg Med. 1994;1(5):448-53. 17. Chacko S, Banet G, Narsahimhalu K, et al. Predictors of residency program scholarly productivity. Acad Emerg Med. 2003;10(5):479. 18. Society for Academic Emergency Medicine (SAEM). Advanced research methodology, evaluation and design (ARMED) website.

REFERENCES

2020. Available at: https://www.saem.org/education/live-learning/

1. Simmons J. (2002). Doctors and Discoveries. Boston, MA: Houghton

advanced-research-methodology-evaluation-and-design-(armed).

Mifflin Co.

Accessed June 9, 2020.

2. Wyatt J and Guly H. Identifying the research question and planning

19. Kwiatkowski T and Silverman R. Research fundamentals: II.

the project. Emerg Med J. 2002;19(4):318-21.

Choosing and defining a research question. Acad Emerg Med.

3. Durbin CG, Jr. How to come up with a good research question:

1998;5(11):1114-7.

framing the hypothesis. Respir Care. 2004;49(10):1195-8.

20. Kahn CR. Picking a research problem. The critical decision. N Engl J

4. Goodacre S. Research methods: beyond the clinical trial. Ann Emerg Med. 2003;42(1):56-65.

Med. 1994;330(21):1530-3. 21. Hulley S, Cummings ST. (1988). Designing Clinical Research.

5. Zink B. Resident Research: The basics and beyond. Newsletter of

Baltimore: Williams and Wilkins.

the Society of Academic Emergency Medicine. 2002;14(1):20-2.

22. Panchal AR, Denninghoff KR, Munger B, et al. Scholar quest: A

6. Accreditation Council for Graduate Medical Education. ACGME

residency research program aligned with faculty goals. West J Emerg

Program Requirements for Graduate Medical Education in

Med. 2014;15(3):299-305.

Emergency Medicine. 2020. Available at: https://www.acgme.org/

23. Biros MH. Reforming a solitary passion. Acad Emerg Med.

Specialties/Program-Requirements-and-FAQs-and-Applications/

2000;7(5):421-4.

pfcatid/7/Emergency. Accessed June 9, 2020.

24. Zink B. Emergency Medicine Research - no more excuses.

7. Summers RL, Fish S, Blanda M, et al. Assessment of the “scholarly

Newsletter of the Society of Academic Emergency Medicine.

project” requirement for emergency medicine residents: report

2000;12(3):1.

of the SAEM Research Directors’ workshop. Acad Emerg Med.

25. Terregino CA, Levitt MA, Lopez BL, et al. A national profile of resident

1999;6(11):1160-5.

research experience. Acad Emerg Med. 1999;6(4):351-6.

8. Kane BG, Totten VY, Kraus CK, et al. Creating consensus: Revisiting

26. Rau JL. Searching the literature and selecting the right references.

the emergency medicine resident scholarly activity requirement. West J Emerg Med. 2019; 20(2):369-75.

Respir Care. 2004;49(10):1242-5. 27. Bronander KA, Goodman PH, Inman TF, et al. Boolean search

9. Langabeer J, Gottlieb M, Kraus CK, et al. Scholarship in emergency

experience and abilities of medical students and practicing

medicine: A primer for junior academics: Part II: Promoting your career and achieving your goals. West J Emerg Med.

physicians. Teach Learn Med. 2004;16(3):284-9. 28. Davey Smith G and Egger M. Meta-analysis. Unresolved issues and

2018;19(4):741-5. 10. Levittt MA. Can emergency medicine residents perform quality

future developments. Br Med J. 1998;316(7126):221-5. 29. Egger M, Schneider M, Davey Smith G. Spurious precision? Meta-

research?. Acad Emerg Med. 1997;4(2):154-6. 11. Office for Human Research Protections, US Department of

analysis of observational studies. Br Med J. 1998;316:140-4. 30. Kirby A, Gebski V, Keech AC. Determining the sample size in a

Health and Human Services. Code of Federal Regulations. 45 CFR 46. Available at: http://www.hhs.gov/ohrp/humansubjects/

clinical trial. Med J Aust. 2002;177:256–7. 31. Kadam P and Bhalerao S. Sample size calculation. Int J Ayurveda

guidance/45cfr46.html. Accessed June 9, 2020. 12. Institute of Medicine (US) Committee on Quality of Healthcare in

Res. 2010;1(1):55-7. 32. Simundic A-M. Bias in research. Biochem Med (Zagreb).

America. Crossing the quality chasm: A new health system for

Volume 21, no. 5: September 2020

2013;23(1):12-5.

1121

Western Journal of Emergency Medicine

Paxton et al.

Resident Research in EM: An Introduction and Primer 33. Silverman R and Kwiatkowski T. Research fundamentals: III.

foundation/grants/funding-opportunities/what-we-fund. Accessed

elements of a research protocol for clinical trials. Acad Emerg Med.

June 9, 2020.

1998;5(12):1218-23.

39. Singer AJ, Homan CS, Brody M, et al. Evolution of abstracts

34. Jones JB. Research fundamentals: statistical considerations in

presented at the annual scientific meetings of academic emergency

research design: a simple personâ&#x20AC;&#x2122;s approach. Acad Emerg Med.

medicine. Am J Emerg Med. 1999;17(7):638-41.

2000;7(2):194-9.

40. Davenport M, Grudzen C, Rao R. Publication rates in emergency

35. Hollander JE and Singer AJ. An innovative strategy for conducting

medicine: A comparison among specialties. Acad Emerg Med.

clinical research: the academic associate program. Acad Emerg Med. 2002;9(2):134-7.

2004;11(5):483. 41. Bordage G, Caelleigh AS, Steinecke A, et al. Review criteria for

36. Carden DL, Dronen SC, Gehrig G, et al. Funding strategies for

research manuscripts. Acad Med. 2001;76(9):897-978.

emergency medicine research. Ann Emerg Med. 1998;31(2):179-87.

42. DeBehnke DJ, Kline JA, Shih RD. Research fundamentals: choosing

37. Emergency Medicine Foundation. Apply for a grant. 2020. Available

an appropriate journal, manuscript preparation, and interactions with

at: https://www.emfoundation.org/grants/apply-for-a-grant/. Accessed June 9, 2020.

editors. Acad Emerg Med. 2001;8(8):844-50. 43. Gottlieb M, Lotfipour S, Murphy L, et al. Scholarship in emergency

38. Society for Academic Emergency Medicine (SAEM) Foundation.

medicine: A primer for junior academics. Part I: Writing and

What we fund. 2020. Available at: https://www.saem.org/saem-

Western Journal of Emergency Medicine

publishing. West J Emerg Med. 2018;19(6):996-1002.

1122

Volume 21, no. 5: September 2020

Original Research

Counseling on Access to Lethal Means-Emergency Department (CALM-ED): A Quality Improvement Program for Firearm Injury Prevention Kristen L. Mueller, MD* Sonya Naganathan, MD, MSBS† Richard T. Griffey, MD, MPH*

*Washington University in St. Louis School of Medicine, Department of Emergency Medicine, St. Louis, Missouri † Warren Alpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode Island

Section Editor: Erin Dehon, PhD Submission history: Submitted February 14, 2020; Revision received May 15, 2020; Accepted May 13, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46952

Introduction: Suicide is the 10th leading cause of death in the United States, with firearms reported as the cause of death in up to 50% of these cases. Our goal was to evaluate the feasibility of the Counseling on Access to Lethal Means intervention in the Emergency Department (CALM-ED) by non-physician personnel. Methods: We conducted this single-center, prospective, quality improvement study (QI) in an urban, academic ED with over 90,000 annual patient visits. The study looked at adult patients who were discharged after presenting to the ED with suicidal crisis. Assessment of access to lethal means was conducted at the bedside, followed by a counseling session regarding safe storage of lethal means and follow-up via telephone call 48-72 hours after ED discharge. We collected data on patient’s sociodemographics, psychiatric history, access to lethal means, lethal means storage methods, the patient’s specific plans for lethal means storage after discharge, and post-discharge follow-up care. Results: Of 215 eligible patients, 166 voluntarily agreed to participate in CALM-ED, of whom 84 (51%) reported access to lethal means. Following the intervention, 75% of patients described a specific storage plan for their lethal means. Patients with and without access to firearms were equally likely to participate in the follow-up telephone call. Conclusion: An ED-based CALM QI intervention is feasible for implementation by non-physician personnel and is well received by patients and families. This intervention has the potential to help saves lives at times of suicide crisis. [West J Emerg Med. 2020;21(5)1123-1130.]

INTRODUCTION Firearm-related injury and death represent a public health epidemic. Suicide is the 10th leading cause of death in the United States, with firearms reported as the cause of death in up to 50% of these cases.1 Up to 90% of suicides attempted with a firearm result in death, and the direct medical costs resulting from firearm injures are as high as $2.9 billion dollars per year.2,3 Additionally, many patients have their first point of contact with the mental healthcare system less than one month before suicide Volume 21, no. 5: September 2020

is attempted.4 While these findings suggest that many patients would benefit from outpatient mental healthcare, over 100 million people in the US live in a mental health-designated Health Professional Shortage Area, in which only 27% of the mental healthcare need is met.5 Consequently, the emergency department (ED) is a frequent point of access to care for patients with mental health crises, with nearly 1% of all US ED visits in 2013 involved in evaluation and management of suicidal ideation (SI).6 In

1123

Western Journal of Emergency Medicine

Mueller et al.

Counseling on Access to Lethal Means-ED bridging the gap to accessible mental healthcare services, there is an opportunity to improve safety from suicide in ED patients through counseling on safe storage of lethal means at times of suicidal crisis. The majority of the health professional associations that deal directly with mental healthcare have endorsed suicide risk assessment and counseling on access to lethal means, such as safe firearm storage practices.7,8 Multiple studies have shown that counseling on safe storage of lethal means can improve safe firearm storage, and thus decrease risk of firearm-related suicide.9-14 Despite this finding, routine lethal-means counseling for at-risk patients has not been widely adopted in high-volume settings such as the ED. In 2013, Betz et al found that only 22% of emergency providers regularly assess for firearm access in patients with SI.15 And a two-year survey of ED nurse managers at facilities that discharged suicidal patients identified significant gaps in asking about firearm access and counseling on safe storage when patients reported access.16 A similar needsassessment study conducted at our institution found that emergency physicians documented access to firearms in only 3% of suicidal patient encounters.17 This gap inspired a project to counsel patients on access to, and safe storage of, lethal means— especially at times of suicidal crisis. In 2014 we launched a university-wide gun violence strategic initiative to identify gaps in available research and find actionable measures to reduce gun violence.18 One result of this effort was the development of the Counseling on Access to Lethal Means (CALM) quality improvement (QI) intervention, “CALM-ED,” based upon the free, online CALM module available through the Suicide Prevention Resource Center.19 This online module was designed specifically for mental health and medical providers who counsel people at risk for suicide. A growing body of literature, including a recent survey of community-based mental healthcare workers’ knowledge and attitudes after completing an in-person CALM workshop, demonstrated a positive association between CALM training and comfort discussing CALM, and suggests that participating in CALM training improves provider self-reported comfort and increased frequency in use of CALM.20 However, there is a paucity of evidence on how to systematically provide this intervention across clinical settings.21,22 Indeed, in 2018 Betz et al reported that among 800 ED patients who screened positive for suicide risk, only 18% had a documented assessment of access to lethal means, and only 8% had documentation of a specific plan to reduce access to said lethal means.23 Leveraging the accessibility of CALM and the abundance of at-risk patients in our ED population, while cognizant of the time demands on emergency physicians, we developed the QI program CALM-ED, which was implemented by non-physician personnel. This work serves as a primer on how to improve frequency of real-time lethal means counseling among the highest risk suicidal patients in the ED setting. As a first step, in this study we evaluated the feasibility of non-physician providers successfully delivering this intervention in the ED. We also aimed Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? Suicide is the 10th leading cause of death in the US, with firearms the reported cause of death in up to 50% of these cases. What was the research question? Can the suicide prevention strategy Counseling on Access to Lethal Means be implemented in the emergency department (CALM-ED)? What was the major finding of the study? Following the CALM-ED intervention, the majority of patients described a specific storage plan for their lethal means. How does this improve population health? An ED-based CALM intervention is feasible and well received by patients and families. CALM has the potential to help save lives at times of suicide crisis.

to better stratify the at-risk population of suicidal patients presenting to the ED and evaluate whether participating in CALM-ED leads to increased compliance with outpatient mental healthcare and safe storage of lethal means. METHODS Study Design This was a single-center, prospective QI study evaluating the feasibility of delivering counseling to patients presenting to the ED with suicidal ideation. Findings are reported in accordance with SQUIRE (Standards for Quality Improvement Reporting Excellence guidelines).24,25 This study was approved by our institutional review board (IRB) and included a waiver of consent. Study Setting and Population We conducted this study between January 1, 2018–June 5, 2019, in an urban, academic ED with over 90,000 annual patient visits. Patients were eligible for inclusion if they met the following criteria: English-speaking; 18 years or older; had nursing-assigned triage chief complaints of “suicidal ideation,” “suicidal attempt,” or “depression;” had been placed on suicidal elopement precautions; had access to a telephone; and were ultimately discharged from the ED. We excluded patients who were admitted to the hospital, actively psychotic, or refused the intervention. Patients who arrived intoxicated were eligible once clinically sober. Patients received usual care of psychiatric conditions in the ED, which was provided by attending and

1124

Volume 21, no. 5: September 2020

Mueller et al.

Counseling on Access to Lethal Means-ED

resident emergency physicians. Some patients received consultation by the psychiatry service at the discretion of the ED team in the course of their usual care; psychiatric consultation was not part of the CALM-ED intervention. If psychiatry was consulted, patients only received CALM-ED if they had subsequently been cleared for discharge. Both the ED team and psychiatry consultants could provide CALM as part of their standard care. Eligible patients received CALM-ED from the study team regardless of other ED care provided. Intervention Counselors Research coordinators from our Emergency Care Research Core (seven registered nurses, one respiratory therapist, and two college-educated coordinators) completed training on administration of the CALM-ED intervention through the online CALM module available through the Suicide Prevention Resource Center, use of scripted language, and direct teaching by the study team.19 These intervention counselors were available 24 hours a day/7 days a week for enrollment and delivery of CALM-ED. Use of non-physician counselors was directed at offloading counseling from clinician staff and at making our results more generalizable to non-academic, non-tertiary care settings where nurses, advanced practice providers, medical assistants, behavioral health coordinators, and other nonphysician providers may conduct suicide risk assessment. To avoid conflicts of interest, the authors did not hire or manage the coordinators, nor did we directly provide the intervention or data collection for this study. After completing the CALM training, the majority of intervention counselors expressed comfort with their ability to suicidal ideation and CALM with ED patients.26 Identification of Participants Patients presenting to the ED with SI were identified by CASE-ED (Computer Assisted Screening and Enrollment in the ED). CASE-ED is an IRB-approved, case-finding program that notified intervention counselors via secure text messaging of patients with the aforementioned nursing-assigned, triage chief complaints. The intervention counselors approached the emergency physician team after “usual care” ED evaluation was complete to determine which patients would be discharged home. To protect patient privacy, all interventions occurred in the patient’s treatment room in the ED. Intervention Delivery The intervention counselors performed a short assessment of access to lethal means at the bedside and delivered a brief, scripted counseling session with patients and any family members or friends present regarding safe storage of firearms and other lethal means. This script, which includes stating your care role as part of the ED team, asking the patient directly about access to lethal means and how they are currently stored, and creating an individualized safe storage plan for any lethal means present, is provided in Appendix Figure 1. As CALM is supported by national organizations and practice guidelines and a valuable care Volume 21, no. 5: September 2020

service that has previously been difficult to operationalize in the ED setting, the intervention counselors identified themselves as part of the care team.27-29 Participants were given handouts developed in partnership with our Institute for Public Health (Appendix Figure 2) detailing mental health resources and local options for safe storage of firearms, and patients who possessed firearms were also given free gun locks upon discharge. A scripted follow-up via telephone call occurred 48-72 hours after ED discharge. Telephone calls were made to the phone numbers provided to the intervention counselors by patients at the time of discharge. A standardized, follow-up telephone call script was followed. Patients who responded that they were not following the safe practices discussed while in the ED were reminded of the importance of doing these things for their safety. Additionally, if a patient endorsed active suicidality at the time of the follow-up call, a warm handoff was conducted between our intervention counselors and the crisis hotline counselors at a local mental healthcare organization. Data Collection We collected data including patient age, gender, race, marital status, substance use, psychiatric history, personal history of suicide attempt, and whether this ED visit was for a suicide attempt. Additionally, we collected data on the following: access to lethal means (firearms, pills, other); how lethal means were stored; patient’s specific plans to store lethal means after discharge; and patient and/or family/friend phone contact information for follow-up phone calls. All intervention counselors were trained in data collection and were supervised in their baseline performance of the intervention by the lead intervention counselor for this study. We collected and managed study data using REDCap version 7 (Vanderbilt University, Nashville, TN) electronic data capture tools. Intervention counselors made up to three follow-up calls, inquiring whether and how lethal means were stored safely, whether the patient had established outpatient follow-up care (primary doctor, psychiatrist, therapist, or other), assessed for barriers to safe storage of lethal means, and inquired whether the patient was actively suicidal. Outcome Measures The primary outcome of this study was feasibility of implementation of the CALM-ED intervention in the ED by non-physician providers. This was informed by prior description of feasibility as an implementation outcome and included 1) the success of counselors in completing enrollment and all elements of CALM-ED implementation for the majority of eligible patients; and 2) enrolled patients’ ability to state a plan for safe storage of lethal means (when applicable) prior to discharge.30 Secondary outcomes – including better identification of patient demographics in this cohort; past psychiatric history including suicide attempt; substance use history; and types of lethal means patients had access to and their current storage practices, and current outpatient mental healthcare resources – were assessed

1125

Western Journal of Emergency Medicine

Mueller et al.

Counseling on Access to Lethal Means-ED through completion of a standardized data collection tool. Analysis We present descriptive statistics for participating sociodemographic data and self-reported outcomes. Categorical data are presented as counts and proportions. Chi-square analysis was used to evaluate the difference in follow-up participation for firearm owners vs non-firearm owners. We analyzed data using IBM SPSS Statistics for Windows, version 26 (International Business Machines Corporation, Armonk, NY). RESULTS We approached all 215 patients meeting eligibility criteria, of whom 166 (77%) voluntarily agreed to participate in CALM-ED (Figure 1). Patient demographics are described in Table 1. Of the 166 patients who received CALM-ED, 84 (51%; 95% confidence interval [CI], 0.43 - 0.58) reported access to lethal means. The other 82 patients denied access to lethal means. These included handguns, rifles, alcohol, medications, and “other” – primarily knives, jumping off a bridge or out of a car, and street drug overdose (Figure 2). Twenty-three (13.9%) patients in this cohort reported access to firearms; their pre-intervention storage methods are reported in Table 2. After receiving CALM-ED, of the 84 patients who reported having access to lethal means, 63 (75%; 95% CI 0.64 - 0.83) patients described a specific storage plan for their lethal means after discharge; one reported not having a safe storage plan, and data were missing for 20 eligible patients (Table 3). Eighty-two patients denied access to any lethal means during the CALM-ED intervention. Two patients had their disposition changed after CALM-ED: one patient received a

1372 Patients presented to ED with chief compliant SI

215 Eligible patients

psychiatric consultation, and another received continued ED observation. Psychiatry was consulted for 37 patients. Free gun locks were distributed to 45 of these patients during the intervention period.

Table 1. Characteristics of patients with suicidal ideation who received intervention regarding access to lethal means. Patient characteristics n (95% CI) Age in years (mean, IQR) Gender

Western Journal of Emergency Medicine

Female

64 (0.31-0.46)

Black

95 (0.50-0.65)

White

64 (0.31-0.46)

Hispanic/Latino

2 (<0.01-0.06)

Other

3 (<0.01-0.05)

Not documented

2 (<0.01-0.06)

Singe

115 (0.62-0.76)

Romantic partner

8 (0.02-0.09)

Married

19 (0.07-0.17)

Divorced

13 (0.05-0.13)

Widowed

3 (<0.01-0.05)

Other

3 (<0.01-0.05)

Not documented

5 (0.01-0.07)

Substance use*

Alcohol

69 (0.34-0.49)

Cocaine

29 (0.12-0.24)

Marijuana

35 (0.16-0.28)

PCP

1 (<0.01-0.06)

Heroin

19 (0.07-0.17)

Amphetamine

12 (0.04-0.12)

None

43 (0.20-0.36)

Psychiatric history**

Patients refused the intervention

Figure 1. Cohort inclusion matrix of 1,372 patients who presented to the emergency department with SI between January 1, 2018 and June 5, 2019. *some patients met more than 1 criteria for exclusion. ED, emergency departmnet; SI, suicidal ideation.

102 (0.54-0.69)

Marital status

166 Patients included in study

Male Race

1157 Patients excluded*

• Under 18 years old (3) • Discharge not anticpated (388) • Not placed on elopement for SI (190) • Acute psychosis (40) • Admitted (517) • No phone access for follow up (50) • Non-English speaking (5)

38 (27-51)

Bipolar

39 (0.18-0.30)

Schizophrenia

37 (0.17-0.29)

Personality Disorder

15 (0.05-0.14)

Depression

69 (0.34-0.49)

Anxiety

41 (0.19-0.32)

None

6 (0.02-0.08)

Not documented

9 (0.03-0.10)

Report established outpatient psychiatric care

71 (0.35-0.50)

History of suicide attempt

110 (0.59-0.73)

Current ED visit for suicide attempt 102 (0.54-0.71) *Some patients reported use of more than one substance. **Some patients reported use of more than one psychiatric diagnosis. CALM-ED, Counseling on Access to Lethal Means intervention in the emergency department; CI, confidence interval; IQR, interquartile ratio; PCP, phencyclidine.

1126

Volume 21, no. 5: September 2020

Mueller et al.

Counseling on Access to Lethal Means-ED

Figure 2. Reported lethal mean access by type.* *some patients reported access to more than one lethal mean.

Table 2. Firearm storage methods. Storage type

included, “my lethal means are household items like knives;” and “I have no one to store my medications.” Several patients expressed appreciation for our concern about their safety, as well as the reminders to store their lethal means safely and schedule outpatient follow-up appointments.

n (%)

Locked in safe

6 (26)

Unlocked and unloaded

1 (4)

Unlocked and loaded

1 (4)

Stored with friends/family

8 (35)

Refused to say

2 (9)

Not sure

1 (4)

Other unspecified storage

1 (4)

Not documented

3 (14)

Table 3 also details reported follow-up outcomes: follow-up phone calls were completed for 51 patients (31%; 95% CI, 0.24 - 0.38). Patients with and without access to firearms were equally likely to participate in the follow-up telephone call (Pearson chi-square 2.230, p 0.135). Of these 51 patients, 17 (33%; 95% CI, 0.22 - 0.47) reported safely storing their lethal means; 24 (47%; 95% CI, 0.34 - 0.60) denied safe storage practices; and 10 (20%; 95% CI, 0.10 - 0.33) denied access to lethal means before or after CALM-ED. Of the 17 patients who safely stored their lethal means, the majority did so by securing them with friends or family members. Three patients reported continued thoughts of suicide and were provided with additional resources for their mental healthcare; two of these patients were connected with the crisis line, and the third reported he was comfortable waiting until his upcoming scheduled appointment with his psychiatrist. Twentyfive patients (49%; 95% CI, 0.36 - 0.62) had arranged outpatient, follow-up care with a primary doctor, psychiatrist or therapist at time of follow-up telephone call. Common barriers to safe storage of lethal means at follow-up Volume 21, no. 5: September 2020

LIMITATIONS This was a single-center study in an academic ED, with the limitations associated with this design. The majority of patients who received this intervention were aged 27-51, single, male, Black or White, with a history of substance use and pre-existing psychiatric diagnoses. Patient inclusion was limited to English speakers only. More than half of the patients included in this study had a personal history of suicide attempt, and just over half reported access to lethal means, with 23 patients reporting access to firearms. In this pilot feasibility study, patient inclusion was limited to those presenting with a triage complaint of suicidal ideation; it is possible that relevant patients were excluded by this selection criteria. This high-acuity psychiatric population is indicative of our academic, urban ED environment. However, as this intervention is centered on interpersonal conversations and the development of multidisciplinary networks with outpatient mental healthcare systems, there is nothing that precludes generalization of these protocols to inpatient wards, clinic-based practices, and other clinical settings. The low follow-up telephone call completion rate of 33% highlights an inherent difficulty in tracking the ED population, particularly those with mental health crises, following discharge. These rates are similar to follow up-rates reported for Black (40.0%) and non-Hispanic White (50.9%) ED patients, and on par with the recently reported 30% seven-day follow-up rate in psychiatric patients after discharge from inpatient hospitalization.31,32 This potential for difficult follow-up, coupled

1127

Western Journal of Emergency Medicine

Mueller et al.

Counseling on Access to Lethal Means-ED Table 3. Storage plan after CALM-ED and at time of follow-up telephone call. After CALM-ED*

On Follow-Up

n = 166 (%)

n = 51 (%)

Locked in safe

9 (6)

2 (4)

Use gun lock

2 (1)

0 (0)

Storage type

Lock in closet

1 (1)

1 (2)

66 (40)

8 (16)

Dispose of pills safely

2 (1)

0 (0)

Other removal from home

4 (2)

1 (2)

Stored with friends/family

Not documented

20 (12)

5 (10)

Denied access to lethal means

82 (49)

10 (20)

Did not store lethal means safely n/a 24 (47) *Some reported storage plans for more than one type of lethal means. CALM-ED, Counseling on Access to Lethal Means intervention in the emergency department.

with the murky legal waters of consenting patients to enroll in a study while actively suicidal, were primary considerations in our decision to evaluate the feasibility of CALM-ED prior to attempting a randomized control trial into whether CALM leads to improved safe storage in suicidal patients. DISCUSSION Over an 18-month period non-physician providers successfully delivered CALM-ED to 166 of 215 eligible patients, demonstrating feasibility. It is our hope that the protocols, script, and data presented here will serve to assist others in implementing CALM-ED style interventions in EDs and other high-acuity clinical settings, and build on prior work in the pediatric emergency setting.13 Use of non-physician counselors avoids additional burden on physicians and increases generalizability, especially in clinical settings that use nonphysician providers to conduct suicide- risk assessment. This study takes the next step toward filling knowledge gaps previously identified to increase research and integrate patientcentered programming of firearm injury-prevention strategies such as CALM in bedside clinical practice. Encouragingly, after CALM-ED, most participants were able to state a safe storage plan for their lethal means prior to discharge from the ED, and patients who participated in the follow-up telephone phone call were receptive to continued discussion of safe storage practices. Additionally, patients who reported access to firearms were equally likely to participate in a follow-up telephone call as those with access to other types of lethal means. These results suggest that patients with access to firearms are willing to engage in conversations about safe storage of these lethal means at times of suicidal crisis. Our data do not contain a documented rationale for the lack of lethal means storage plans prior to discharge from the ED in 20 Western Journal of Emergency Medicine

patients who reported having access to said items; it is possible this was limited by intervention counselor comfort with CALMED. Survey data of our intervention counselors indicate that while the majority felt comfortable with the CALM-ED intervention, two felt “somewhat uncomfortable” talking to an individual about safe storage of lethal means, and one felt “somewhat uncomfortable” with their ability to effectively counsel patients on reducing access to medications and firearms.26 Additionally, one respondent felt “very uncomfortable” with all counseling topics related to suicidal ideation, access to lethal means, and safe storage of lethal means. Alternatively, this discussion regarding patient plans for safe storage may have taken place prior to discharge, but was not documented due to time constraints, incomplete task-switching, or failure of task completion. While most of our patients with access to lethal means were able to make plans for safe storage, many faced barriers to enacting these plans. As was demonstrated in our data, many patients rely on a social network of friends or families to help store or manage their lethal means. This may be problematic for patients who are socially isolated or live away from their families. Creation of third-party networks for temporary storage of firearms during times of suicidal crisis at police stations, gun ranges, or other repositories could help reduce access to firearms in this high-risk population. Free distribution of gun locks from clinical settings may also help mitigate risk. Despite our early successes, the suicidal crisis patient population included in this study is just the tip of the iceberg of patients at risk for death by suicide from firearms and other lethal means. As multiple prior studies of discharged ED patients with mental health presentations have documented low follow-up rates, feasibility is a reasonable first step toward a more robust assessment of efficacy. Additional next steps in this work will include expanding CALM-ED to patients with any history of depression or mental health illness (not just acute mental health crisis), substance abuse, and other at-risk populations. Given the many demands on physicians it may be more cost-effective to use non-physician counselors, as proposed in this study, to maximize CALM delivery. Given the ease of administration, especially when provided with scripting, this suggests that non-physician providers can feasibly deliver CALM. CONCLUSION While further study is needed regarding the efficacy of safe storage practices after CALM, especially in the ED population, our findings suggest that an ED-based CALM QI intervention is feasible, well received by patients and families, and has the potential to help saves lives at times of suicide crisis. ACKNOWLEDGMENTS The Institute for Public Health at Washington University in St. Louis aided in creating the CALM-ED script used in this study. Gun locks for this intervention were provided through a partnership with Women’s Voices Raised for Social Justice –

1128

Volume 21, no. 5: September 2020

Mueller et al.

Counseling on Access to Lethal Means-ED

Lock It For Love, a local community outreach organization with the purpose of educating the community about gun safety, including a focus on prevention of death or injury caused by firearms kept in homes. Provident Behavioral Health, a regional organization that provides outpatient counseling, suicide prevention resources and community outreach, provided outpatient mental healthcare and crisis line access for patients who expressed active suicidality.

injury and death in the United States: a call to action from 8 health professional organizations and the American Bar Association. Ann Intern Med. 2015;162(7):513-6. 8. Pallin R, Spitzer SA, Ranney ML, et al. Preventing firearm-related death and injury. Ann Intern Med. 2019;170(11):ITC81-96. 9. Albright TL, Burge SK. Improving firearm storage habits: impact of brief office counseling by family physicians. J Am Board Fam Pract. 2003;16(1):40-6. 10. Barkin SL, Finch SA, Ip EH, et al. Is Office-based counseling about media use, timeouts, and firearm storage effective? Results from a

Address for Correspondence: Kristen Mueller, MD, Washington University School of Medicine, Department of Emergency Medicine, 660 S. Euclid Ave, Campus Box 8072, St. Louis, MO 63110. Email: Kristen.mueller@wustl.edu.

cluster-randomized, controlled trial. Pediatrics. 2008;122(1):e15-25. 11. Carbone PS, Clemens CJ, Ball TM. Effectiveness of gun-safety counseling and a gun lock giveaway in a Hispanic community. Arch

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. This study was funded by the Foundation for Barnes Jewish Hospital and the Institute for Public Health at Washington University in St. Louis. The Institute for Public Health was involved in the design of the study. Our funding sources had no role in the conduct or reporting of the study, or the decision to submit the manuscript for publication. Dr. Griffey is funded in part by the Agency for Healthcare and Quality (AHRQ) R-18 Grant Number: 1 R18 HS025052-01. Dr. Mueller is funded in part by the Emergency Medicine Foundation and the American Foundation for Firearm Injury Reduction in Medicine (AFFIRM). There are no conflicts of interest to declare. Copyright: © 2020 Mueller et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

Pediatr Adolesc Med. 2005;159(11):1049-54. 12. Mann JJ, Apter A, Bertolote J, et al. Suicide prevention strategies: a systematic review. JAMA. 2005;294(16):2064-74. 13. Runyan CW, Becker A, Brandspigel S, et al. Lethal means counseling for parents of youth seeking emergency care for suicidality. West J Emerg Med. 2016;17(1):8-14. 14. Grossman DC, Stafford HA, Koepsell TD, et al. Improving firearm storage in Alaska native villages: a randomized trial of household gun cabinets. Am J Public Health. 2012;102 Suppl 2:S291-7. 15. Betz ME, Miller M, Barber C, et al. Lethal means restriction for suicide prevention: beliefs and behaviors of emergency department providers. Depress Anxiety. 2013;30(10):1013-20. 16. Runyan CW, Brooks-Russell A, Tung G, et al. Hospital emergency department lethal means counseling for suicidal patients. Am J Prev Med. 2018;54(2):259-65. 17. Naganathan S, Mueller KL. Physician documentation of access to firearms in suicidal patients in the emergency department. West J Emerg Med. 2019;20(5):818-21. 18. Institute for Public Health. Gun Violence Initiative. 2019. Available

REFERENCES

at: https://publichealth.wustl.edu/gun-violence-initiative/about-gunviolence-initiative/. Accessed December 15, 2019.

1. Centers for Disease Control and Prevention. Injury Prevention and

19. Frank EM, Ciocca M. CALM: Counseling on Access to Lethal Means.

Control: Statistics Query and Reporting System (WISQARS). 2017.

2009. Available at: https://www.sprc.org/resources-programs/calm-

Available at: www.cdc.gov/injury/wisqars. Accessed January 28, 2019.

counseling-access-lethal-means-0. Accessed Sept 5, 2019.

2. Gani F, Sakran JV, Canner JK. Emergency department visits for

20. Johnson RM, Frank EM, Ciocca M, et al. Training mental healthcare

firearm-related injuries In the United States, 2006-14. Health Aff

providers to reduce at-risk patients’ access to lethal means of suicide:

(Millwood). 2017;36(10):1729-38.

evaluation of the CALM Project. Arch Suicide Res. 2011;15(3):259-64.

3. Miller M, Azrael D, Hemenway D. The epidemiology of case fatality rates for suicide in the Northeast. Ann Emerg Med. 2004;43(6):723-30.

21. Sale E, Hendricks M, Weil V, et al. Counseling on Access to Lethal Means (CALM): An evaluation of a suicide prevention means

4. Ahmedani BK, Simon GE, Stewart C, et al. Health care contacts in

restriction training program for mental health providers. Community

the year before suicide death. J Gen Intern Med. 2014;29(6):870-7.

Ment Health J. 2018;54(3):293-301.

5. Bureau of Health Workforce, Health Resources and Services Administration. US Department of Health and Human Services.

22. Slovak K, Pope N, Giger J, et al. An evaluation of the Counseling

Designated Health Professional Shortage Areas Statistics. June 30,

on Access to Lethal Means (CALM) training with an area agency on

2020. Access date August 7, 2020.

aging. J Gerontol Soc Work. 2019;62(1):48-66.

6. Owens PL, Fingar KR, Heslin KC, et al. (2017). Emergency

23. Betz ME, Kautzman M, Segal DL, et al. Frequency of lethal means

Department Visits Related to Suicidal Ideation, 2006–2013. In:

assessment among emergency department patients with a positive

Healthcare Cost and Utilization Project (HCUP) Statistical Brief.

suicide risk screen. Psychiatry Res. 2018;260:30-5.

Rockville, MD: Agency for Healthcare Research and Quality (US).

24. Ogrinc G, Davies L, Goodman D, et al. SQUIRE 2.0 (Standards for QUality Improvement Reporting Excellence): revised publication

7. Weinberger SE, Hoyt DB, Lawrence HC 3rd, et al. Firearm-related

Volume 21, no. 5: September 2020

1129

Western Journal of Emergency Medicine

Mueller et al.

Counseling on Access to Lethal Means-ED guidelines from a detailed consensus process. BMJ Qual Saf.

Prevention. 2012. Available at: https://pubmed.ncbi.nlm.nih.

2016;25(12):986-92. 25. Ogrinc G, Mooney SE, Estrada C, et al. The SQUIRE (Standards for

gov/23136686/. Accessed January 28, 2019. 29. Daigle MS. Suicide prevention through means restriction: assessing

QUality Improvement Reporting Excellence) Guidelines for Quality

the risk of substitution. A critical review and synthesis. Accid Anal

Improvement Reporting: explanation and elaboration. Qual Saf Health Care. 2008;17 Suppl 1:i13-32.

Prev. 2005;37(4):625-32. 30. Proctor E, Silmere H, Raghavan R, et al. Outcomes for

26. Mueller KL, Chirumbole D, Naganathan S. Counseling on Access to

implementation research: conceptual distinctions, measurement

Lethal Means in the Emergency Department: a script for improved

challenges, and research agenda. Adm Policy Ment Health.

comfort. Community Ment Health J. 2020. In press. 27. Suicide Resource Prevention Center. Caring for adult patients with

2011;38(2):65-76. 31. Menchine M, Oberfoell S, Schriger D, et al. Improving telephone

suicide risk: a consensus-based guide for emergency departments.

follow-up for patients discharged from the emergency department:

2015. Available at: https://www.sprc.org/edguide. Accessed January

results of a randomized controlled trial. Acad Emerg Med.

17, 2019. 28. Office of the Surgeon General (US); National Action Alliance

2013;20(5):456-62. 32. Benjenk I, Chen J. Variation of follow-up rate after psychiatric

for Suicide Prevention (US). 2012 National Strategy for Suicide

hospitalization of medicare beneficiaries by hospital characteristics

Prevention: Goals and Objectives for Action: A Report of the U.S.

and social determinants of health. Am J Geriatr Psychiatry.

Surgeon General and of the National Action Alliance for Suicide

2019;27(2):138-48.

Western Journal of Emergency Medicine

1130

Volume 21, no. 5: September 2020

Original Research

Intention to Leave Emergency Medicine: Mid-career Women Are at Increased Risk Michelle D. Lall, MD, MHS* Sarah M. Perman, MD† Nidhi Garg, MD‡ Nina Kohn, MBA, MA§ Kristy Whyte, MD¶ Alexa Gips, MD|| Tracy Madsen, MD# Jill M. Baren, MD** Judith Linden, MD††

*Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia † University of Colorado School of Medicine, Department of Emergency Medicine, Denver, Colorado ‡ Hofstra/Northwell Health, Department of Emergency Medicine, Long Island, New York § Northwell Health, Feinstein Institutes for Medical Research, Long Island, New York ¶ Vituity/DeKalb Emergency Physicians, Emory Decatur Hospital, Emory Hillandale Hospital, Atlanta, Georgia || University of Colorado School of Medicine, Department of Hospice & Palliative Medicine, Denver, Colorado # Alpert Medical School, Brown University, Department of Emergency Medicine, Providence, Rhode Island **Perelman School of Medicine, University of Pennsylvania, Department of Emergency Medicine, Philadelphia, Pennsylvania †† Boston University Medical Center, Department of Emergency Medicine, Boston, Massachusetts

Section Editor: Donna Mendez, MD Submission history: Submitted March 16, 2020; Revision received May 26, 2020; Accepted May 23, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.47313

Introduction: Burnout is prevalent among emergency physicians and may cause physicians to consider leaving the practice of emergency medicine (EM). This study sought to determine whether there is a gender difference in reporting burnout and seriously considering leaving the specialty of EM, and secondarily to explore the factors reported as contributing to burnout. Methods: This was a secondary analysis of the 2014 American Board of Emergency Medicine Longitudinal Survey of Emergency Physicians. We used multiple logistic regression to determine which factors were associated with reporting serious consideration of leaving EM, when stratified by years in practice and adjusting for individual, departmental, and institutional factors. Results: The response rate was 82%, (n = 868); 22.6% (194) were female and 77.4% (664) were males; and 83.9% (733) White. The mean age of men responding was significantly higher than women (52.7±11.9 vs. 44.9±10.4, p<0.001). Overall, there were no significant gender differences in reporting having had serious thoughts of leaving EM in either unmatched or age-matched analyses. More women reported that burnout was a significant problem, while men more often were equivocal as to whether it was a problem. When stratified by years in practice, mid-career women had a sevenfold increase in the odds ratio (OR) of seriously considered leaving EM, compared to men of similar years in practice (OR 7.07, 95% confidence interval, 2.45-20.39). Autonomy at work, control over working conditions, fair compensation, personal reward, and a sense of ownership were factors associated with a lower rate of reporting considering leaving EM. Conclusion: Our findings suggest that the intention to leave EM is not more prevalent in women. However, mid-career women more often reported seriously considering leaving the specialty than mid-career men. Further research on the factors behind this finding in mid-career women in EM is needed. [West J Emerg Med. 2020;21(5)1131-1139.]

Volume 21, no. 5: September 2020

1131

Western Journal of Emergency Medicine

Lall et al.

Intention to Leave Emergency Medicine among Women INTRODUCTION Burnout, defined as “a state of emotional exhaustion, depersonalization, and a lack of sense of personal accomplishment,” is prevalent among physicians.1 In the medical literature, burnout is often measured using the Maslach Burnout Inventory (MBI),1 Oldenburg Inventory,2 or single-item measures of emotional exhaustion and depersonalization.3 However, prior work on emergency physicians (EP) demonstrated that self-reported burnout (as assessed by “have you thought you are experiencing burnout”) accurately predicted burnout as defined by MBI scores 72% of the time.4 Burnout in physicians is associated with many negative effects including decreased job satisfaction,5 an increase in intention to leave a job,6,7 decreased job productivity,8 increased medical errors and decreased patient safety,9,10 and substance use disorders.11 The prevalence of burnout in attending physicians across medical specialties is more than twice that observed in the general adult working population, with EPs reporting one of the highest burnout rates – between 48-70%.12,13,14 Prior studies reveal gender differences in reported burnout, with women reporting burnout at higher rates than men, yet little is known about gender differences in burnout among EPs specifically.15-21 In one study of internal medicine residency program directors, women had higher rates of emotional exhaustion and depersonalization.18 A study of American surgeons revealed that women suffered from higher rates of burnout than men and also had higher rates of emotional exhaustion, a factor that has been shown to be associated with a desire to leave clinical practice.20 Data on gender differences in burnout in emergency medicine (EM) is lacking. Previous studies have shown that older age may be protective against burnout.22,23,24 Older age has also been shown to be a positive factor in EP job satisfaction.25 Since women in EM are often of a younger age and may also have more non-clinical and family responsibilities,26,27 these factors may contribute to higher burnout rates among mid-career women. Mid-career seems to be a particularly vulnerable time for female physicians.17,28,29,30 Additionally, the notable decline in women who rise to higher ranks of leadership and seniority in EM28,31 could be either a contributor to, or a result of, potentially higher burnout rates for women. Physicians suffering from burnout are significantly more likely to leave healthcare6,7,32,33,34, and those who report an intent to leave is a strong predictor of actual departure.33 Prior to leaving healthcare, physicians often reduce their work hours and change their clinical work environment in an attempt to ameliorate burnout.13 Burnout and the attrition of physicians from healthcare is quite costly.34,35,36,37 Our primary objective was to determine whether there is a gender difference in reported serious consideration of leaving the field of EM. As a secondary objective, we sought to determine whether previously identified domains contributing to burnout are associated with burnout among EPs. Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? There is attrition of women in academic emergency medicine. Women are less likely to achieve senior leadership positions or promotions than their male counterparts. What was the research question? Is there a gender difference in reporting burnout and seriously considering leaving EM? What was the major finding of the study? Mid-career women had a seven-fold increased odds ratio of seriously considering leaving EM, compared to mid-career men. How does this improve population health? Diversity and longevity in the EM workforce are critical; the factors contributing to mid-career women seriously considering leaving EM need exploration.

METHODS Study Design and Setting This is a secondary data analysis of the 2014 American Board of Emergency Medicine (ABEM) Longitudinal Study of Emergency Physicians (LSEP). The ABEM LSEP is a 36-page questionnaire that was sent every five years to an ongoing cohort of EPs, from 1994-2014. A full text of the survey can be found on the ABEM website.38 This study was approved by the Emory University Institutional Review Board as an exempt protocol. Selection of Study Participants The first LSEP cohort identified in 1994 was selected via a stratified, random sampling of representative EPs within four different stages in the development of the specialty ensuring a representative sample of those who had completed EM residency and those who had not. Since that time, new cohorts were identified for inclusion every five years, until the final survey in 2014. Since 1999, all new cohorts are participants of Accreditation Council for Graduate Medical Education-approved EM residency programs. For the purpose of this study, all participants who responded to the 2014 questionnaire were considered for inclusion into this analysis. Subjects were excluded if they were not actively engaged in the practice of EM (ie, considered themselves fully retired), if they did not report their gender, or if they did not respond to the question “Have you ever seriously considered leaving the specialty of EM?”, as these variables were necessary for the primary outcome for this analysis (Figure 1).

1132

Volume 21, no. 5: September 2020

Lall et al.

Intention to Leave Emergency Medicine among Women

Study Outcomes and Variables The primary outcome was to determine whether there was a gender difference in board-certified EPs having considered leaving the specialty of EM. To explore this question, we analyzed the question “Have you ever seriously considered leaving the specialty of Emergency Medicine” with a dichotomous answer (Yes/No). Serious consideration of leaving the specialty was used as a surrogate for burnout. Additionally, we hypothesized that a greater proportion of women would report burnout to be a problem in their everyday work experience. To answer this question, we used the query “How much of a problem is burnout in your day-to-day work for pay?” with 1 not being a problem, and 5 being a serious problem (five-point Likert scale). The secondary outcome was to explore the prevalence of previously identified domains associated with burnout19,21,39,40,41 in this cohort of EPs. Specifically, what proportion of survey respondents who were seriously considering leaving the specialty of EM self-identified the following burnout domains: 1) workload: demands of the job exceed capability; 2) lack of control; 3) lack of perceived reward: more about recognition, less about salary; 4) lack of community: socially toxic environment, lack of fairness/equity, lack of respect; and 5) value conflict: a disconnect between values that give meaning to life and day-today work.42,43 Statistical Analysis De-identified data were provided to the research team. All

analyses were carried out using STATA, v14.1 (StataCorp, College Station, TX). We used standard descriptive statistics to characterize the men and women who completed the survey. Categorical variables were compared using chi-squared tests, while we compared continuous variables using analysis of variance (ANOVA). Analysis for Decision to Leave Emergency Medicine To further explore the associations between gender and the intention to leave EM, we performed unadjusted logistic regression to determine whether there was an association between gender and the intention to leave EM. Subsequently, we also conducted multivariable logistic regression that accounted for years in practice. In review of the demographics within the cohort, male respondents were significantly older than the female respondents. To account for this, we created a 1:1 age-matched cohort of men and women in EM. Within this cohort, we explored the association between gender and the intention to leave EM via unadjusted analysis. Analysis of Burnout in Day-to-Day Work To explore burnout, we asked the question “how much of a problem is burnout in your day-to-day work for pay?” This question was reported as a five-point Likert scale, and we determined the proportion of respondents who considered leaving EM. The five-point Likert scale was then reconfigured into three categories with 1) burnout is a problem (score of 4 or 5); 2)

34,816 2014 ABEM Delegates

1,102 2014 LSEP Participants 220 (20.0%) Surveys not returned

Excluded: 24 1 retired 13 missing gender 10 missing answer to “Ever consider leaving EM”

882 (80.0%) Completed Surveys

858 Participants (77.9%) 664 Men (77.4%) 194 Women (22.6%)

Figure 1. Flow diagram of participant exclusion in survey of emergency physicians. ABEM, American Board of Emergency Medicine; LSEP, Longitudinal Study of Emergency Physicians; EM, emergency medicine.

Volume 21, no. 5: September 2020

1133

Western Journal of Emergency Medicine

Lall et al.

Intention to Leave Emergency Medicine among Women burnout is not a problem (score of 1 or 2); and 3) ambivalent if burnout is or is not a problem (score of 3). Using these three categories, we used ANOVA to test for differences in gender and burnout. Next, we explored the association between gender and the decision to leave EM accounting for burnout and duration of time in practice using multivariate logistic regression. The secondary objectives of this study were to explore the prevalence of variables identified as contributors to burnout in both male and female survey respondents who were considering leaving EM. These variables included the following: 1) autonomy at work; 2) compatible colleagues; 3) control over working conditions; 4) fair compensation; 5) personal reward; and 6) sense of ownership. The proportion of men and women who acknowledged the presence of these conditions in their current positions was reported using descriptive statistics. We used logistic regression to explore the association between each domain, gender and “seriously considering leaving EM” by including interaction terms between gender and each domain variable.

In all analyses, a p-value of 0.05 was considered statistically significant. We conducted analyses using STATA v14.1. RESULTS The 2014 LSEP surveys were sent to 1102 EPs. A total of 882 physicians responded (80% response rate for all respondents, 82% for practicing physicians). We excluded 10 participants because they did not answer the primary study question, “have you seriously considered leaving the specialty of EM?” An additional 13 participants were excluded for not reporting their gender. Finally, we excluded one individual who did not answer any questions pertaining to practice, which we surmised meant he or she was retired. In total, the final cohort analyzed was 858 study participants (Figure 1). Of those included, 22.6% (n = 194) were women and 77.4% (n = 664) were men. Table 1 describes the differences between the men and women who responded to this survey. Most notably, women who responded to the survey were younger than the men who

Table 1. Participant characteristics in the cohort of emergency physicians completing the American Board of Emergency Medicine survey stratified by gender. Variable

Women

Men

P-value

44.9 ± 10.4

52.7 ± 11.9

<0.001

White

82.4% (159)

87.2% (574)

0.086

Non-White

17.6% (34)

12.8% (84)

0-1 years

12.3% (20)

3.4% (18)

2-3 years

1.2% (2)

1.1% (6)

4-6 years

22.1% (36)

12.2% (65)

7-10 years

7.4% (12)

6.2% (33)

11-20 years

31.9% (52)

22.1% (118)

21-30 years

16.6% (27)

24.9% (133)

>30 years

8.6% (14)

30.2% (161)

Married

74.7% (145)

88.5% (587)

Divorced/Separated

7.8% (15)

3.9% (26)

Single, cohabitating

5.7% (11)

2.9% (19)

Single, living as single

11.3% (22)

3.9% (26)

Widowed

0.5% (1)

0.8% (5)

1.5 (0, 2)

2.0 (1.5, 3)

Exceptionally healthy

21.7% (42)

25.6% (169)

No health concerns

46.9% (91)

36.7% (242)

Some minor health concerns

28.4% (55)

32.6% (215)

Some serious health concern

3.1% (6)

5.2% (34)

97% (84, 100)

90% (55, 100)

Age (n=853) Race (n=851)

# years in Emergency Medicine (n=697) <0.001

Marital status (n=857)

Children

<0.001

Current state of health (n=854) (Likert scale 1,2 = health concerns, 3,4 = no concerns)

Clinical practice (n=846)

Western Journal of Emergency Medicine

1134

0.066

<0.001

Volume 21, no. 5: September 2020

Lall et al.

Intention to Leave Emergency Medicine among Women

were surveyed (44.9 ± 10.4 vs 52.7 ± 11.9, p<0.001), and a greater proportion of women had been practicing for 10 years or less in comparison to men (43.0% vs 22.9%). The younger age and shorter length of time spent in the field of EM was significant between men and women. The overall cohort was predominantly White, which did not allow for sufficient analysis of the effect of race/ethnicity in our findings. In unadjusted analysis, there was no significant gender difference in the desire to leave EM (odds ratio [OR] 1.04, 95% confidence interval [CI], 0.74-1.43). Given the notable difference in age of men and women in our cohort, we used multivariable logistic regression to explore the association between gender and serious thoughts of leaving EM when accounting for years in practice (Table 2). When categorized into career state (early, mid, late), mid-career women (7-10

Table 2. Multivariable analysis exploring the association between gender and “seriously considering leaving EM,” stratified by years in clinical practice. Variable Women (vs men)

95% CI

P-value

1.23

0.82-1.84

0.309

Years in practice 0-1 years

REF

2-3 years

1.89

0.30-11.73

0.496

4-6 years

1.12

0.40-3.10

0.829

7-10 years

7.07

2.45-20.39

<0.01

11-20 years

2.61

1.02-6.64

0.045

21-30 years

4.96

1.94-12.70

0.001

>30 years

4.01

1.56-10.30

0.004

EM, emergency medicine; OR, odds ratio; CI, confidence interval; REF, reference.

years of practice) had a seven-fold increase in the OR of “seriously considered leaving EM” compared to those in early stage careers (OR 7.07, 95% CI, 2.45-20.39) (Table 2). This trend persisted into late-stage career, where women had a higher likelihood of having serious thoughts about wanting to leave EM (Table 2). Due to the smaller proportion of women in the dataset, the CIs were large; nevertheless, we still observed a trend indicating women had a higher likelihood of considering leaving the specialty later into their careers in EM. To account for the marked difference in age of men and women who responded to the survey, we also created an age-matched cohort of men and women. The matched cohort included 374 individuals (187 age-matched female-male pairs). In this age-matched cohort, the mean age of men and women was 45.0 ± 10.5 years. In this cohort, there was no association between gender and seriously considering leaving EM (OR 1.33, 95% CI, 0.87-2.04). Among all survey respondents, we hoped to establish an association between considering leaving the field of EM and whether the participants reported burnout in their day-to-day work for pay. We found that an increasing desire to leave the field of EM was correlated with greater reporting that burnout heavily affected one’s day-to-day work for pay (Figure 2), and that the proportion of those reporting they had seriously considered leaving EM increased as self-reported burnout increased, with 76% of those in the highest burnout group reporting that they had considered leaving EM. Table 3 reports gender-based responses to the question of whether burnout is a problem, with a higher proportion of women stating it was a problem in comparison to men (37.3% vs 32.0%, p = 0.013). Unadjusted, burnout was significantly associated with seriously considering leaving EM (OR 5.2, 95% CI, 3.657.27). Adjusting for gender did not alter this association significantly (OR 5.2, 95% CI, 3.68-7.36).

No, I have not seriously considered leaving EM Yes, I have seriously considered leaving EM

Degree to which burnout is a problem in your day to day work Figure 2. Association between burnout and considering leaving emergency medicine for all respondents (n = 852). Responses were pooled in a Likert scale of 0,1,2 (burnout is not a problem in everyday work for pay), 3 (ambivalent if it is or is not), 4,5 (burnout is a problem in everyday work for pay).

Volume 21, no. 5: September 2020

1135

Western Journal of Emergency Medicine

Lall et al.

Intention to Leave Emergency Medicine among Women Finally, we explored gender differences in the variables most associated with burnout. Table 4 outlines how frequently respondents reported having these conditions in their current work dichotomized by gender. There was no statistically significant gender difference except for women reporting less frequently “having control over working conditions” than men (50.3% vs 62.2%, p = 0.003). In addition, lack of autonomy at work, fair compensation, personal reward, and sense of ownership were associated with seriously considering leaving EM. When respondents reported having these conditions within their workplace, they were less likely to report having seriously considered leaving EM. DISCUSSION In the population of physicians in this ABEM survey, there was no significant gender difference in the proportion of participants who had seriously considered leaving EM in the overall or age-matched cohort analyses. As of December 2014, ABEM had 32,238 diplomats: 56% male, 25% female, and 19% unknown gender (personal communication with ABEM in

Table 3. Self-reported assessment of burnout as a problem stratified by gender. Men (n = 656)

Women (n = 193)

Burnout IS NOT a problem (n=363)

45.4% (298)

33.7% (65)

Burnout IS a problem (n=282)

32.0% (210)

37.3% (72)

I have no preference on if burnout is a problem or not (n=204)

22.6% (148)

29.0% (56)

P-value

0.013

August 2018). In addition, we found that overall, 33.2% of all respondents in this cohort reported that “burnout was a significant problem in day-to-day work for pay,” which was lower than that reported in the literature.4,13,44 We found an association between burnout in day-to-day work for pay and consideration of leaving EM in both genders with mid-career women being significantly more likely to indicate that they had seriously considered leaving EM. Women were more likely to report that burnout was a significant problem, or that they were ambivalent as to whether it was or not, while men were more likely to answer that burnout was not a significant problem. Finally, we found that women were more likely than men in this cohort to endorse “lack of control over working conditions.” The methodology of this study does not allow us to explain the reason for this observation. It is notable, however, that women are more likely than men to have a greater number of home and family responsibilities, and this finding may actually represent work-home conflict, and the lack of control over work-home responsibilities related to this conflict. Several studies have shown that physician and research scientist women have greater work-home conflict than male peers as they are often still responsible for a majority of household tasks, child caring and child rearing, and other unpaid work.26,27,43 Workhome conflict may be part of the reason why women are less likely to stay in academia and continue to lag behind their male counterparts in leadership positions and at the associate and professor ranks.45 In 2015, women made up 37.3% of EM residents and this number was relatively unchanged as compared to 2005.46 Women make up 43% of instructors, 37% of assistant professors, 24% of associate professors, and 17% of professors in EM.47 Additionally, there is a dearth of female chairs in EM. A 2015 report using American Academy of Medical Colleges (AAMC) faculty roster data revealed only 10 female chairs (11%) in EM as compared to 87 male chairs.48

Responses were pooled in a Likert scale of 0,1,2 (burnout is not a problem in everyday work for pay), 3 (ambivalent if it is or is not), 4,5 (burnout is a problem in everyday work for pay).

Table 4. Availability of various work conditions stratified by gender and the association with “ever seriously considered” leaving emergency medicine.

Is each of the following work conditions available in your current position? (Y)

Men

Univariate association with leaving EM OR (all gender)

95% CI

P-value

Autonomy at work (n = 851)

93.8% (180) 93.2% (612)

0.770

0.34

0.20-0.60

<0.001

Compatible colleagues (n = 849)

98.4% (188) 97.7% (643)

0.549

0.59

0.23-1.49

0.262

50.3% (97) 62.2% (407)

0.003

0.49

0.37-0.66

<0.001

Fair compensation (n = 845)

83.9% (162) 87.7% (572)

0.171

0.45

0.30-0.67

<0.001

Personal reward (n = 839)

85.4% (164) 90.3% (584)

0.058

0.33

0.21-0.52

<0.001

Sense of ownership (n = 845)

61.7% (119) 59.8% (390)

0.646

0.48

0.36-0.64

<0.001

Control over working conditions (n = 847)

Women

P-value (difference between men and women)

CI, confidence interval; OR, odds ratio; EM, emergency medicine.

Western Journal of Emergency Medicine

1136

Volume 21, no. 5: September 2020

Lall et al.

Intention to Leave Emergency Medicine among Women

Similar to prior literature, we found that women in mid-career practice may be especially vulnerable to considering leaving EM, compared to men at a similar point in career.17,30 It is important to consider potential contributors, and implications of our finding that women at mid-career were more likely to report having considered leaving EM. Since mid-career is the time when women often have more childcare and family responsibilities, such responsibilities may contribute to burnout. A study by Dyrbye et al found that mid-career physicians had the lowest satisfaction with their specialty choice and work-life balance, and had the highest rates of emotional exhaustion and burnout.30 Additionally, mid-career physicians were the most likely to plan to leave the practice of medicine for reasons other than retirement.30 Work-life concerns also prevent women from seeking promotion and leadership positions.28 The finding that midcareer women are more often considering leaving EM speaks to the importance of determining and incorporating best practices for flexibility to support women during this time in their career. Best practices for supporting women with family-friendly policies in academic EM have been identified, including flexible scheduling, emergency back-up childcare options, and policies that support healthy pregnancies and lactation areas.49 Ensuring that there is parity in advancement and compensation between women and men is also essential for retention of mid-career women.49 Many of these practices apply to both academic and community EM. If we can retain more mid-career women, not only will this lead to a more diverse community of practicing EPs but it will help to increase the number of women in senior EM ranks and positions, including at the associate and full professor ranks in academic EM, and as department leaders. Historically, burnout was viewed as an individual-level problem and a sign of personal weakness, and as a result suggested interventions or modifying factors were focused on the individual level without consideration of organizational influences.7 As the body of literature on the prevalence of physician burnout continues to grow, there has been an increasing recognition of organizational and environmental causes for burnout. These causes include bureaucratic tasks (charting, paperwork), long work hours, electronic health records, lack of respect,50 insufficient compensation, lack of control/autonomy, feeling like a cog in a wheel, and profits over patients.13 Not surprisingly, we found that factors such as lack of autonomy at work, lack of control over working conditions, and lack of fair compensation all contributed to burnout, in both genders. The association between these previously established domains and our outcome of consideration of leaving EM supports the validity of our survey question on ever seriously considering leaving EM. Efforts to minimize these factors that contribute to burnout can be protective in maintaining the EM clinical workforce irrespective of gender and should be addressed in both community and academic settings.

Volume 21, no. 5: September 2020

LIMITATIONS There are several limitations to our study. First, the proportion of women in the sample was small (22%), which may have limited our ability to detect differences by gender. This is somewhat lower than the national percentage of women in EM as reported by ACEP (25%), ABEM (25%). (ABEM, Carmen Swinger, personal communication 11/22/2017), (ACEP personal communications 5/22/2018). The AAMC Group on Women in Medicine and Science, reported 37% of academic faculty in EM in 2015 were women.31 Second, the data only includes physicians who are currently practicing EM. We were limited in the conclusions we could draw about physicians who are no longer practicing or consider themselves fully retired, and what role burnout may have contributed to their reasons for leaving clinical practice. In addition, the interpretation of the results of this survey is limited by the questions themselves as well as the format of the questions. This survey did not use a validated measure of burnout (MBI). Rather, we used the response to the question â&#x20AC;&#x153;How much of a problem is burnout if your everyday work for pay,â&#x20AC;?, and responses were dichotomized to a five-point Likert scale. This question has not been validated for identifying burnout but self-reported burnout in EPs has been shown to correlate with the MBI in a previous study.4 We also acknowledge that burnout is not a static condition, but rather often varies over time, as does consideration of leaving EM. Depending on the current situation at the time of the survey, the respondentsâ&#x20AC;&#x2122; recall of ever having considered leaving EM may vary. Additionally, we acknowledge that there are non-burnout related reasons that may result in someone responding in the affirmative to the question of seriously considering leaving EM. As mentioned in the discussion, the age of the female cohort was younger than that of the male cohort. We accounted for this in our age-matched cohort, leading to a smaller sample size in the age-matched cohort, and found no difference in results. Finally, we were not able to perform any analysis related to minorities, given the very small number of respondents who identified as under-represented minorities (11% total). CONCLUSION Mid-career women were more likely to have ever seriously considered leaving EM than mid-career men. Women were more likely to report that burnout was a significant problem in their day-to-day work for pay, while men were more likely to be ambivalent, or report that it was not a significant problem. Identifying and addressing the various factors that contribute to burnout and intention to leave the field of EM is critical, and emphasizing gender differences in these factors is necessary for retention and advancement. Mid-career women represent a particularly vulnerable group for increased rates of burnout and intention to leave the practice of EM, and intentional programming to support and promote this cohort is warranted.

1137

Western Journal of Emergency Medicine

Lall et al.

Intention to Leave Emergency Medicine among Women Address for Correspondence: Michelle D. Lall, MD, MHS, Emory University School of Medicine, Department of Emergency Medicine, 49 Jesse Hill Jr Drive, SE, Atlanta, GA 30303 Email: michelle.d.lall@emory.edu

working population between 2011 and 2014. Mayo Clin Proc. 2015;90(12):1600–13. 13. Medscape National Physician Burnout, Depression & Suicide Report 2019. Medscape. Available at: https://www.medscape.com/

slideshow/2019-lifestyle-burnout-depression-6011056. Accessed on March 4, 2020. 14. Shanafelt TD, West CP, Sinsky C, et al. Changes in burnout and satisfaction with work-life integration in physicians and the general US working population between 2011 and 2017. Mayo Clin Proc. 2019;94(9):1681-1694. 15. Naugle KE, Behar-Horenstein LS, Dodd VJ, Tillman MD, Borsa PA. Perceptions of wellness and burnout among certified athletic trainers: sex differences. J Athl Train. 2013;48(3):424–30. 16. Blom V. Contingent self-esteem, stressors and burnout in working women and men. Work Read Mass. 2012;43(2):123–31. 17. Norlund S, Reuterwall C, Höög J, Lindahl B, Janlert U, Birgander LS. Burnout, working conditions and gender: results from the northern

REFERENCES

Sweden MONICA Study. BMC Public Health. 2010;10:326.

1. Maslach C JS, Leiter MP. Maslach Burnout Inventory. 3rd ed. Eval

18. West CP, Halvorsen AJ, Swenson SL, McDonald FS. Burnout and

Stress B Resour. 1997;191–218.

distress among internal medicine program directors: results of a

2. Halbesleben JR, Demerouti E. The construct validity of an alternative measure of burnout: investigating the English translation of the

national survey. J Gen Intern Med. 2013;28(8):1056–63. 19. Linzer M, McMurray JE, Visser MRM, Oort FJ, Smets E, de Haes

Oldenburg Burnout Inventory. Work & Stress. 2005;19(3):208-20.

HCJM. Sex differences in physician burnout in the United States and

3. West CP, Dyrbye LN, Sloan, JA et al. Single item measures of

The Netherlands. J Am Med Womens Assoc 1972. 2002;57(4):191–3.

emotional exhaustion and depersonalization are useful for assessing

20. Dyrbye LN, Shanafelt TD, Balch CM, Satele D, Sloan J, Freischlag J.

burnout in medical professionals. J Gen Intern Med.

Relationship between work-home conflicts and burnout among

2009;24(12):1318–21.

American surgeons: a comparison by sex. Arch Surg.

4. Goldberg R, Boss RW, Chan L, et al. Burnout and its correlates in

2011;146(2):211–7.

emergency physicians: four years’ experience with a wellness booth.

21. Chambers CNL, Frampton CMA, Barclay M, McKee M. Burnout

Acad Emerg Med. 1996;3(12):1156–64.

prevalence in New Zealand’s public hospital senior medical

5. Shanafelt TD, Bradley KA, Wipf JE, Back AL. Burnout and self-

workforce: a cross-sectional mixed methods study. BMJ Open.

reported patient care in an internal medicine residency program. Ann Intern Med. 2002;136(5):358–67.

2016;6(11):e013947. 22. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with

6. Estryn-Behar M, Doppia M-A, Guetarni K, et al. Emergency

work-life balance among US physicians relative to the general US

physicians accumulate more stress factors than other physicians:

population. Arch Intern Med. 2012;172(18):1377–85.

results from the French SESMAT study. Emerg Med J.

23. Peisah C, Latif E, Wilhelm K, Williams B. Secrets to psychological

2011;28(5):397–410.

success: why older doctors might have lower psychological distress

7. Stehman CR, Testo Z, Gershaw RS, Kellogg AR. Burnout, drop out,

and burnout than younger doctors. Aging Ment Health.

suicide: physician loss in emergency medicine, Part 1. West J Emerg Med. 2019;20(3):485-94.

2009;13(2):300–7. 24. LaFaver K, Miyasaki JM, Keran CM, et al. Age and sex differences in

8. Kuhn G, Goldberg R, Compton S. Tolerance for uncertainty, burnout,

burnout, career satisfaction, and well-being in US neurologists.

and satisfaction with the career of emergency medicine. Ann Emerg

Neurology. 2018;91(20):e1928-e1941.

Med. 2009;54(1):106–113.e6.

25. Lloyd S, Streiner D, Shannon S. Burnout, depression, life and job

9. Shanafelt TD, Balch CM, Bechamps G, et al. Burnout and medical

satisfaction among Canadian emergency physicians. J Emerg Med.

errors among American surgeons. Ann Surg. 2010;251(6):995–1000. 10. Hall LH, Johnson J, Watt I, Tsipa A, O’Connor DB. Healthcare staff

1994;12(4):559–65. 26. Jolly S, Griffith KA, DeCastro R et al. Gender differences in time

wellbeing, burnout, and patient safety: a systematic review. PloS

spent on parenting and domestic responsibilities by high-achieving

One. 2016;11(7):e0159015. 11. Wallace JE, Lemaire JB, Ghali WA. Physician wellness: a missing

young physician-researchers. Ann Int Med. 2014;160(5):344-53. 27. Baptiste D, Fecher AM, Dolejs SC et al. Gender differences in

quality indicator. Lancet. 2009;374(9702):1714–21.

academic surgery, work-life balance and satisfaction. J Surg Res.

12. Shanafelt TD, Hasan O, Dyrbye LN, et al. Changes in burnout and satisfaction with work-life balance in physicians and the general US

Western Journal of Emergency Medicine

2017;218:99-107. 28. Ellinas EH, Fouad N, Byars-Winston A. Women and the decision to

1138

Volume 21, no. 5: September 2020

Lall et al.

Intention to Leave Emergency Medicine among Women

leave, linger, or lean in: predictors of intent to leave and aspirations to

satisfaction, job stress, mental and physical health. Health Care

leadership and advancement in academic medicine. J Womens

Manage Rev. 2010;35(2):105–15.

Health (Larchmt). 2018;27(3):324-332.

40. Shanafelt TD, Dyrbye LN, Sinsky C, et al. Relationship between

29. Lewiss RE, Silver JK, Bernstein CA, Mills AM, Overholser B, Spector

clerical burden and characteristics of the electronic environment with

ND. Is academic medicine making mid-career women physicians

physician burnout and professional satisfaction. Mayo Clin Proc.

invisible? J Womens Health (Larchmt). 2020 .;29(2):187-192.

2016;91(7):836–48.

30. Dyrbye LN, Varkey P, Boone SL, Satele DV, Sloan JA, Shanafelt TD.

41. Shanafelt TD, Gorringe G, Menaker R, et al. Impact of organizational

Physician satisfaction and burnout at different career stages. Mayo

leadership on physician burnout and satisfaction. Mayo Clin Proc.

Clin Proc. 2013;88(12):1358–67.

2015;90(4):432–40.

31. Lautenberger D, Dander VM, Raezer CL, Sloane RA. (2014). The

42. Linzer M, Visser MRM, Oort FJ, Smets EMA, McMurray JE, de Haes

state of women in academic medicine: the pipeline and pathways

HCJM. Predicting and preventing physician burnout: results from the

to leadership 2013-2014. Washington, DC: Association of American

United States and the Netherlands. Am J Med. 2001;111(2):170–5.

Medical Colleges.

43. Langballe EM, Innstrand ST, Aasland OG, Falkum E. The predictive

32. Landon BE, Reschovsky JD, Pham HH, Blumenthal D. Leaving

value of individual factors, work‐related factors, and work–home

medicine: the consequences of physician dissatisfaction. Med Care.

interaction on burnout in female and male physicians: a longitudinal

2006;44(3):234-42.

study. Stress Health. 2011;27(1):73–87.

33. Sinsky CA, Dyrbye LN, West CP, Satele D, Tutty M, Shanafelt TD.

44. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with

Professional satisfaction and the career plans of US physicians.

work-life balance among US physicians relative to the general US population. Arch Intern Med. 2012;172(18):1377–85.

Mayo Clinic Proc. 2017;92(11):1625-1635. 34. Han S, Shanafelt TD, Sinsky CA et al. Estimating the attributable

45. Carr PL, Raj A, Kaplan SE, Terrin N, Breeze JL, Freund KM.

cost of physician burnout in the United States. Ann Intern Med.

Gender differences in academic medicine: retention, rank, and

2019;170:784-90.

leadership comparisons from the National Faculty Survey. Acad Med.

35. Hamidi MS, Bohman B, Sandbor C et al. Estimating institutional

2018;93(11):1203-9.

physician turnover attributable to self-reported burnout and

46. AAMC. GME Track Table 2: Distribution of Residents by Specialty,

associated financial burden: a case study. BMC Health Serv Res.

2005 compared to 2015. 2016. Available at: https://www.aamc.org/

2018;18(1):851.

system/files/reports/1/2015table2.pdf. Accessed May 4, 2020.

36. Dewa CS, Loong D, Bonato S et al. How does burnout affect

47. AAMC. Faculty Roster Table 4a: Distribution of Full-time Women

physician productivity? A systematic literature review. BMC Health

M.D. Faculty by Department and Rank. 2015. Available at: https://

Serv Res. 2014;14:325.

www.aamc.org/system/files/reports/1/2015table4a.pdf. Accessed May

37. Shah S. The Studer Group. The real cost of emergency department physician turnover. 2016. Available at: https://www.studergroup.

4, 2020. 48. AAMC. Faculty Roster Table 11: Distribution of charis by

com/resources/articles-and-industry-updates/insights/january-2016/

department, gender, and race/ethnicity. December 31,

the-real-cost-of-emergency-department-physician-tu. Accessed

2015. Available at: https://www.aamc.org/system/files/

October 18, 2019.

reports/1/2015table11.pdf. Accessed May 4, 2020.

38. ABEM. Longitudinal Study of Emergency Physicians. 2014. Available

49. Choo EK, Kass D, Westergaard M, et al. The Development of Best

at: https://www.abem.org/public/docs/default-source/default-

Practice Recommendations to Support the Hiring, Recruitment, and

document-library/2014-five-year-survey.pdf?sfvrsn=0. Accessed on

Advancement of Women Physicians in Emergency Medicine. Acad

March 4, 2020. 39. Williams ES, Konrad TR, Scheckler WE, et al. Understanding

Emerg Med. 2016;23(11):1203–9. 50. Mareiniss DP. Emergency Medicine burnout and abuse - One

physicians’ intentions to withdraw from practice: the role of job

Volume 21, no. 5: September 2020

doctor’s perspective. Am J Emerg Med. 2018;36(4):720-1.

1139

Western Journal of Emergency Medicine

Original Research

Using an Online Vaccination Registry to Confirm Tetanus Status in Children with Tetanus-prone Wounds Cristina Zeretzke-Bien, MD* Janelle McCall, MD† Todd Wylie, MD‡ Muhammad A.B. Chowdhury, MPH, MPS, MSc* Meenakshi Balakrishnan, PhD, MPH* Phyllis Hendry, MD‡ Colleen J. Kalynych, MSH, EdD§ Hac-Tu J. Chung, BS*

*University of Florida in Gainesville, Department of Emergency Medicine, Gainesville, Florida † University of Texas Southwestern, Department of Emergency Medicine, Dallas, Texas ‡ University of Florida in Jacksonville, Department of Emergency Medicine, Jacksonville, Florida § University of Florida in Jacksonville, Office of Educational Affairs, Jacksonville, Florida

Section Editor: Donna Mendez, MD, EdD Submission history: Submitted January 16, 2020; Revision received April 1, 2020; Accepted June 2, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.46582

Introduction: Tetanus vaccination status is an important consideration for emergency physicians managing patients with tetanus-prone wounds. Physicians must identify at-risk patients, but vaccination histories are often unknown and commonly lack documentation. The study objective was to determine the potential impact of an online immunization registry (Florida SHOTS – State Health Online Tracking System) on the appropriate administration of tetanus prophylaxis for pediatric patients managed in the emergency department (ED). Methods: We conducted a retrospective review of all patients less than 18 years old who received ED tetanus prophylaxis at two separate sites between January 2011–May 2015. The Florida SHOTS database was accessed to determine vaccination status for each patient in the study group at the time of the encounter. We compared vaccination status for each patient, as documented in the electronic health record (EHR), with Florida SHOTS data to determine whether tetanus prophylaxis was indicated. The proportion of patients receiving tetanus prophylaxis in the ED, who were subsequently identified as up to date with tetanus vaccination per Florida SHOTS, was determined. Results: We identified 743 patients who received ED tetanus prophylaxis. Forty-three (6%) were listed as “up to date” on the EHR and 656 (93%) were listed as “not up to date.” In comparison, 209 (30%) of the study group were identified as “up to date” via Florida SHOTS, and 477 (70%) were not. We accessed the Florida SHOTS record retrospectively to determine whether the vaccine was required. It was determined that 174 (25%) of the patients received tetanus prophylaxis unnecessarily as they were already up to date per Florida SHOTS documentation. Conclusion: Twenty-five percent of patients vaccinated for tetanus in the ED could have been spared if Florida SHOTS data had been used by providers at the time of the encounter. Access to Florida SHOTS provides valuable information regarding vaccination status that impacts patient care and resource utilization in the ED. [West J Emerg Med. 2020;21(5)1140-1146.]

INTRODUCTION Tetanus is a life-threatening disease caused by the bacterium Clostridium tetani. Mortality is high in those who are not immunized and do not receive treatment. Thankfully, it is now rare in the developed world due to tetanus vaccination Western Journal of Emergency Medicine

programs. Widespread use of tetanus-toxoid containing vaccines and tetanus immune globulin (TIG) for wound management has led to a 95% decline in the number of tetanus cases and a 99% decrease in the number of tetanus-related deaths since the 1940s.1 However, to achieve and maintain

1140

Volume 21, no. 5: September 2020

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children

appropriate immunization status, children must receive the complete primary series of tetanus vaccinations and subsequent booster vaccinations as indicated. Currently, the US Centers for Disease Control and Prevention (CDC) recommend that the DTaP (diphtheria, tetanus, and acellular pertussis vaccine) be given as a 5-dose series at ages 2, 4, and 6 months, as well as at ages 15-18 months and 4-6 years. Tdap (tetanus, diphtheria, and pertussis) is administered at 11-12 years of age. Following that, a Td booster should be given every 10 years. In the emergency department (ED), it is recommended to give the tetanus vaccine under the following conditions: if tetanus vaccine status is unknown; if the patient has had less than three doses (would also receive TIG for dirty wound), or if the patient has had at least three doses but it has been over five years since the last dose (10 years for “clean” wound; but of note, most wounds are considered “dirty wounds” in the ED).1 Tetanus vaccination status is an important consideration for emergency physicians (EP) managing pediatric patients with tetanus-prone wounds. EPs must decide which patients are at risk for tetanus based on their vaccination status; however, vaccination histories are often unknown by parents and/or caretakers and commonly lack documentation.2 A recent study suggests that multiple formulations of tetanus vaccinations and fragmented documentation of immunizations increase the prevalence of medication errors related to tetanus vaccinations.3 Most of the literature surrounding ED tetanus vaccination demonstrates inaccuracies of judgment of patients’ tetanus vaccination status. A recent pediatric study in Utah showed that providers incorrectly assessed tetanus vaccination status 8.8% of the time.4 Of these, 85% (7.4% of the entire group) were incorrectly identified as being up to date. Therefore, if they had a clinical indication for the tetanus vaccine, providers would have missed giving it. A longitudinal study in Taiwan following 770,000 adult patients over eight years discovered that more than 160,000 unnecessary tetanus boosters were given.5 In an effort to provide all practitioners with access to up-to-date vaccination records, the CDC has supported initiatives to develop local and state immunization registries. The CDC reports that every state in the union is either developing or operating a state or regional immunization registry. Florida SHOTS (State Health Online Tracking System) is a free, statewide, centralized online vaccination registry, which was created in 2003. The registry is an online Health Insurance Portability and Accountability Act (HIPAA)exempt immunization registry available to all healthcare providers (Florida Statute, Section 456.057). Once an account is set up, the database is easily accessible to medical personnel in the ED as well as in outpatient clinics. This is a centralized database tracking vaccine administration, which can be accessed easily by healthcare providers, as opposed to individual hospital or clinic EHRs, which may or may not communicate with each other. Presumably, utilization of a state immunization registry would allow ED providers to correctly identify a patient’s Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Emergency physicians must assess a patient’s need for tetanus prophylaxis, but often the patient’s vaccine status is unknown. What was the research question? Can an online immunization registry improve the accuracy of determining a patient’s tetanus vaccine status in the ED? What was the major finding of the study? 25% of patients could have been spared the tetanus vaccine if the immunization registry had been used. How does this improve population health? If an immunization registry were implemented in the ED, the costs of redundant tetanus vaccines to both the patient and the system could be saved.

vaccination status. Reliably determining a patient’s vaccination status could potentially reduce unnecessary tetanus vaccine administration in patients with tetanus-prone wounds. Our objective was to determine whether use of an online immunization registry would impact the provision of tetanus prophylaxis for pediatric patients managed in the ED. METHODS We designed this study to retrospectively review the EHR of pediatric patients who received tetanus vaccination in the ED and compare the data from Florida SHOTS on vaccination status to determine whether vaccination was indicated at the time of presentation to the ED. The institutional review boards at both facilities approved this retrospective review to be conducted. Trained research assistants performed chart review and data entry. We used the REDCap (Research Electronic Data Capture) application to store data in order to retrospectively review the EHR of all pediatric patients who received a tetanus vaccine in the ED from January 1, 2011–May 31, 2015. This also included the patients for whom the tetanus vaccine was ordered but who received it after leaving the ED (on the floor or in the intensive care unit). This included all forms of tetanus vaccines given to pediatric patients under age 18, including DTaP, DT, Dtap/Hepatitis B/Polio (Pediarix), Td, Tdap, and tetanus toxoid. Children who did not receive a tetanus vaccine (including those for whom it may have been indicated) were excluded due to the method of data extraction (children who received the vaccine).

1141

Western Journal of Emergency Medicine

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children We conducted the study at two EDs in Florida, located about 70 miles from each other. Both EDs are located within academic institutions, and there are approximately 20,00025,000 pediatric visits to each site annually. Variables collected for patients included age, gender, chief complaint, insurance status, primary care provider, and registration status in Florida SHOTS. The Florida SHOTS database was accessed to determine vaccination status at the time of the ED visit. This was compared to documentation about vaccination status in the hospitals’ EHRs. Of note, the EHR vaccination status was obtained from the immunization tab, which can be updated by physicians and/or nurses at any time, at any visit to the hospital or a clinic within the same system. Both nursing documentation and physician documentation about tetanus vaccination status were also examined by reviewing the notes in the EHR. We performed descriptive statistics to summarize demographic variables. Documentation of the pediatric patients’ vaccination status in both the EHR and Florida SHOTS at the time of ED encounter were reported in frequencies and percentages. We performed all data analysis using SAS version 9.4 software (SAS Institute, Cary, NC).

RESULTS We identified 703 patients who received some form of tetanus prophylaxis in the ED. Of those patients, 438 (62.3%) were seen at the first site, and 265 (37.7%) were seen at the second site. Seventy percent of all patients were male, and the median age was 12.4 years old. Fifty-three percent were White, and 38% were Black; 58% were Medicaid patients, and 10% were self-pay. Sixty percent of the patients reportedly had a “delayed” vaccination schedule, according to the EHR. Most of the chief complaints fell into the category of laceration/wound/ puncture (73%), and the remaining complaints were burns, trauma, and other. The EHR documented that 487 patients (70%) had a primary care provider (PCP), 175 (25%) did not, and 37 (5%) were unknown. This relates to our data because if a PCP is reported, immunization data is more likely to be documented in the online vaccination registry. The primary care physician’s office is typically responsible for updating the vaccination registry. In this group of 703 patients, only 2.5% (18 children) were not registered in Florida SHOTS. The demographics are summarized below in Table 1. When discussing the results, “up to date” indicates that the child’s tetanus vaccination status was current, and the tetanus

Table 1. Demographic characteristics of pediatric patients receiving the tetanus vaccine in the emergency department. Variables Age (years), median (IQR)

Site 1 (n = 438)

Site 2 (n = 265)

Total (n = 703)

11.70 (8.60)

14.40 (6.35)

12.4 (6.9)

Male

299 (68.26)

191 (72.08)

490 (69.7)

Female

139 (31.74)

74 (27.92)

213 (30.3)

White

288 (66.06)

83 (31.32)

371 (52.92)

Black

104 (23.85)

161 (60.75)

265 (37.8)

44 (10.09)

21 (7.93)

65 (9.28)

226 (51.6)

185 (69.81)

411 (58.46)

161 (36.76)

44 (16.6)

205 (29.16)

Self-Pay/Charity

41 (9.36)

26 (9.81)

67 (9.53)

Other

10 (2.28)

10 (3.77)

20 (2.84)

Gender, Number (%)

Race, Number (%)

Other (Hispanic, Asian, Native American, Multiracial) Payer Status, Number (%) Medicaid Commercial

Chief Complaint, Number (%) Burn

25 (5.72)

6 (2.26)

31 (4.42)

305 (69.79)

205 (77.36)

510 (72.65)

55 (12.59)

22 (8.3)

77 (10.97)

52 (11.9)

32 (12.08)

84 (11.97)

Yes

327 (75.35)

160 (60.38)

487 (69.67)

105 (24.19)

70 (26.42)

175 (25.04

2 (0.46)

35 (13.21)

37 (5.29)

Laceration, Wound, Puncture Trauma Alert (Activated Level 1 or Level 2) Other Does patient have a primary care provider? Number (%)

Unknown IQR, interquartile range.

Western Journal of Emergency Medicine

1142

Volume 21, no. 5: September 2020

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children

vaccine was not indicated during the ED visit; thus, the vaccine was unnecessarily administered. “Not up to date” means that the tetanus vaccine was indicated and thus, appropriately administered. As stated previously, all the children in the study received the tetanus vaccine in the ED (except for 15 of the patients, or about 2%, who were given the vaccine subsequently during the hospitalization after it was ordered in the ED). Interestingly, we collected data from both the nursing notes as well as the physician notes in the EHR. Nursing documentation reported 481 (69%) patients were “up to date,” 90 (12%) were “not up to date,” and 129 (18%) were “unknown.” Physician documentation reflected 85 (12%) as “up to date,” with 383 (54%) as “not up to date,” and 234 (33%) as “unknown.” The breakdown by site was similar to the overall results. The reasons for these differences are unclear, but it highlights the issue of discrepancies in obtaining the vaccination status of patients in the ED and the need for a vaccination registry with more accurate information. We examined whether the Florida SHOTS data (patient’s entire vaccine record) appeared in the EHR at the time of the ED visit. It was not present in 386 (56%) of the records, but 303 (44%) did contain the complete Florida SHOTS data in the EHR. There was a large discrepancy between sites: Site 1’s EHR contained the Florida SHOTS data only 25% of the time, while Site 2’s EHR contained the Florida SHOTS data 75% of the time. We also reviewed whether the tetanus vaccine given in the ED was documented in the Florida SHOTS record: 281 (41%) of the Florida SHOTS records did not contain documentation of the tetanus vaccine given in the ED, and 410 (59%) did contain the vaccine administered in the ED. Again, there was wide variability here with Florida SHOTS containing documentation of the tetanus vaccine given at Site 1 only 50% of the time, whereas it documented those given at Site 2 73% of the time. The EHR review reflected that 43 (6%) of patient records were listed as “up to date” and 656 (93%) patient records were listed as “not up to date,” thus requiring a tetanus vaccine. When comparing Florida SHOTS data, 209 (30%) patients were listed as “up to date” (not requiring vaccine), and 477 (70%) were “not up to date” (did require vaccination). Of the 209 patients who were listed as “up to date” in Florida SHOTS, only 35 of them were documented as being “up to date” in the EHR as well. This means that 174 (25% of the entire patient population) patients were documented as “up to date” in Florida SHOTS but as “not up to date” in the EHR. These patients likely received the tetanus vaccine unnecessarily. This data is shown by site in Table 2, and the summary data is outlined in Table 3 below. It is important to note that patients for whom the tetanus vaccination status was missing from the EHR and/or Florida SHOTS were marked as “not up to date.” DISCUSSION As mentioned above, in this group of 703 patients, only 2.5% of the patients (18 children) were not registered in Volume 21, no. 5: September 2020

Florida SHOTS. These patients may not have been Florida residents. The other 97.5% of the children registered in Florida SHOTS had the potential to benefit from the vaccination registry. About 70% of the children were noted to have a PCP. This is significant because the PCP’s office is the primary site where data is documented into Florida SHOTS. There was a slight discrepancy between the sites: 75% of the patients at Site 1 had a PCP, while only 60% at Site 2 had a PCP. As mentioned above, EHR documentation showed that 43 (6%) patient records were listed as “up to date” and 656 (93%) patient records were listed as “not up to date,” thus requiring a tetanus vaccine. It is unclear why the patients in the group of 43 (6%) were administered a tetanus vaccine when the EHR indicated that they were already up to date. One reason this may have occurred is that the risk of the injury may have been so great that an additional vaccine was administered due to the high concern for developing tetanus. Additionally, it is possible that the tetanus vaccine was given during the initial trauma resuscitation, prior to family members arriving to provide the vaccine history. According to Florida SHOTS records, the tetanus vaccine was indicated and administered appropriately to the majority of the pediatric patients who received the tetanus vaccine in our ED settings (70%). However, almost a third of the patients studied may have received the vaccine unnecessarily. Additionally, there were several discrepancies between the EHR and Florida SHOTS records. There were even larger discrepancies between nursing and physician documentation within the EHR. For patients who received the vaccine unnecessarily, there were likely multiple factors that led to the vaccine unnecessarily being administered. This may include the non-utilization of Florida SHOTS at the time of administration. Florida SHOTS does require a login, and while the nurses in the primary care clinics routinely access this resource, the ED nurses may not have access or be appropriately trained to access Florida SHOTS. Additionally, the ED is inherently busy, so time was likely a factor for both nurses and physicians deciding to access Florida SHOTS. In the academic hospital settings for this study, there are also multiple residents from different backgrounds (pediatrics, emergency medicine [EM], family medicine), and not all of them have access to Florida SHOTS, which would also have contributed to their inability to verify immunization status. Pediatric residents who also work in the pediatric continuity clinics have access to Florida SHOTS. However, the EM and family medicine residents did not have access. It is also important to note that 40% of these patients who received the tetanus vaccine in the ED never had their Florida SHOTS records updated to reflect this. The ED providers do not routinely update Florida SHOTS with immunizations provided in the ED. It is therefore up to the PCP’s office to complete this task. However, many patients

1143

Western Journal of Emergency Medicine

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children

Table 2. Comparison of tetanus vaccination status in the electronic health record and Florida SHOTS for the pediatric patients receiving the tetanus vaccine by site. Site 1 (n = 438)

Site 2 (n = 265)

Total (n = 703)

According to EHR tetanus dates, did the patient need a vaccination? Frequency (Percentage) No (Up to date)

39 (8.99)

4 (1.51)

43 (6.15)

395 (91.01)

261 (98.49)

656 (93.85)

No (Up to date)

159 (36.81)

50 (19.69)

209 (30.47)

Yes (Not up to date)

273 (63.19)

204 (80.31)

477 (69.53)

289 (66.44)

192 (72.45)

481 (68.71)

Yes (Not up to date) According to Florida SHOTS tetanus dates, did the patient need a vaccination? Frequency (Percentage)

Immunization status - per nursing documentation Frequency (Percentage) Up to date Not up to date

75 (17.24)

15 (5.66)

90 (12.86)

Unknown

71 (16.32)

58 (21.89)

129 (18.43)

Immunization status (per physician documentation) Up to date

72 (16.48)

13 (4.91)

85(12.11)

Not up to date

171 (39.13)

212 (80)

383(54.56)

Unknown

194 (44.39)

40 (15.09)

234(33.33)

Was tetanus vaccination given in the ED? Frequency (Percentage) No

15 (3.43)

Yes

15 (2.14)

422 (96.57)

265 (100)

687 (97.86)

85 (19.77)

18 (6.79)

103 (14.82)

Tetanus vaccine type given in ED, Frequency (Percentage) DTaP DT

4 (0.93)

6 (2.26)

10 (1.44)

DTap, Hepatitis B, Polio (Pediarix)

2 (0.47)

10 (3.77)

12 (1.73)

16 (3.72)

40 (15.09)

56 (8.06)

286 (66.51)

191 (72.08)

477 (68.63)

Td Tdap Tetanus Toxoid (Booster)

37 (8.6)

386 (56.02)

Does the Florida SHOTS vaccination data appear on the EHR Immunizations record? Frequency (Percentage) No

319 (75.06)

67 (25.38)

386 (56.02)

Yes

106 (24.94)

197 (74.62)

303 (43.98)

211 (49.53)

70 (26.42)

281 (40.67)

Does the vaccination from the date of ED encounter appear on Florida SHOTS? Frequency (Percentage) No

Yes 215 (50.47) 195 (73.58) 410 (59.33) EHR, electronic health record; Florida SHOTS, Florida State Health Online Tracking System; ED, emergency department; D, diphtheria; T, tetanus; aP, acellular pertussis. The case indicates amount of each ingredient in the vaccine.

do not have a PCP or may not follow up with them after an ED visit. If they do follow up, they may forget to report that they received the vaccine, especially if it is not noted in their discharge paperwork. This is important because these patients may then receive the vaccine in the PCPâ&#x20AC;&#x2122;s office when the booster was previously scheduled to be due, or if they sustain another injury, they may again receive the vaccine in the ED unnecessarily. Western Journal of Emergency Medicine

Over 30% of patients receiving the tetanus vaccine in error translates to major costs for both the patients and the hospitals. The pediatric population is especially sensitive to painful injections and often requires extra measures, such as involving child life specialists to make the experience less traumatic. They may require an extra person to help hold them while the injection is being administered. Besides requiring the extra attention from busy ED personnel, each medication

1144

Volume 21, no. 5: September 2020

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children

Table 3. Overall comparison of tetanus vaccination status in the electronic health record and Florida SHOTS for the pediatric patients receiving the tetanus vaccine. According to FL SHOTS tetanus dates, did the patient need a vaccination? Frequency (Percentage) According to EMR Tetanus Dates, did the patient need a vaccination? Frequency (Percentage)

No/Up to date

Yes/Not up to date

Total

35 (5.10)

8 (1.17)

43 (6.27)

Yes/Not up to date

174 (25.36)

469 (68.37)

643 (93.73)

Total

209 (30.46)

477 (69.54)

686 (100)

No/Up to date

P-value <0.001

Note: p-value was calculated using chi-square test. EHR, electronic health record; SHOTS, State Health Online Tracking System.

administered comes with a monetary cost. Compared to outpatient costs for medications and vaccines, costs in the ED are substantially higher. These costs may not be covered by insurance, and they can add to the family’s financial burden. Since the completion of this study, the EHR integrated Florida SHOTS directly into its immunizations section so that the Florida SHOTS records are automatically updated in the EHR when accessed. This will likely decrease the discrepancies between the EHR and Florida SHOTS and possibly decrease rates of inappropriate administration of the tetanus vaccine. This change was made in 2017; so it would be interesting to examine the data after another 1-2 years. Interestingly, a prospective adult study in Rome comparing patients’ memory to a rapid immunochromatographic test (Tetanus Quick Stick [Nephrotek Lab, Rungis, France) found that the TQS was able to save unnecessary tetanus vaccines 57% of the time.6 A similar study of 200 adults showed that almost 40% of them had incorrect recall of their tetanus vaccination status.7 However, one contrasting adult study in France did find that patients self-reported that their tetanus vaccines were up to date correctly about 96% of the time.8 It may be interesting to pursue a prospective study in the pediatric ED comparing patients’/parents’ memories, EHR, and state vaccination registry to a tetanus rapid immunochromatographic test. LIMITATIONS A limitation of this study was that only the medical records of patients who received the tetanus vaccine during their ED visit/hospitalization were reviewed. Therefore, we did not examine cases of children in which the tetanus vaccine may have been indicated but was not provided. This was a result of the selection of cases from the EHR by those for whom the tetanus vaccine had been ordered in the ED. Another limitation of the study was its retrospective design. A few patients were missing some of the data points because they were not recorded in the EHR (three patients were missing nursing documentation of tetanus status, and one was Volume 21, no. 5: September 2020

missing physician documentation). However, this is unlikely to have significantly affected the results. Also, it was not possible to determine with certainty why the discrepancies existed between the EHR and Florida SHOTS or even between the various medical personnel (nurses, physicians) taking care of the patient. CONCLUSION This retrospective review of the electronic health records and the state vaccination registry of 703 pediatric patients seen at two EDs between 2011–2015 showed that 25-30% of them received tetanus prophylaxis when it was not indicated. Access to Florida SHOTS provides valuable information regarding vaccination status that impacts patient care and resource utilization in the ED. If the physicians and/or nurses were readily able to access the vaccination registry from the ED, the costs of the tetanus vaccine to the patient and system could be saved. In 2017 (after the conclusion of this study), Florida SHOTS was incorporated directly into the hospital’s EHR. This will likely decrease the number of patients receiving the tetanus vaccine unnecessarily. It would be interesting to review the data again after this change was implemented.

Address for Correspondence: Janelle McCall, MD, University of Texas Southwestern, 1935 Medical District Drive, Dallas, Texas 75235. Email: Janelle.McCall@UTSouthwestern.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Zeretzke-Bien et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http:// creativecommons.org/licenses/by/4.0/

1145

Western Journal of Emergency Medicine

Zeretzke-Bien et al.

Online Vaccination Registry to Confirm Tetanus Status in Children REFERENCES

double-blind study. Ann Ig. 2015;27(2):467-74. 7. McVicar J. â&#x20AC;&#x153;Should we test for tetanus immunity in all emergency

1. Blain A, Tiwari T (2017). Tetanus. In: Manual for the Surveillance of

department patients with wounds?â&#x20AC;? Emerg Med J. 2013;30(3)177-9.

Vaccine-Preventable Diseases. Atlanta, GA: US Department of

8. Auffret Y, Rousseaux JY, Gatineau F, et al. Should we believe

Health and Human Services, CDC; 2017.

emergency department patients self-reported tetanus vaccine status?

2. Eppert, H. Disease prevention update: tetanus toxoid, reduced

Am J Emerg Med. 2019;37(5):983-90.

diphtheria toxoid, and acellular pertussis: who, what, when, why and

9. Kretsinger K, Broder KR, Cortese MM, et al. Preventing tetanus,

how? J Emerg Nurs. 2010;36(2):122-4.

diphtheria, and pertussis among adults: use of tetanus toxoid,

3. Fletcher M, Rankin S, Sarangarm P. The effect of pharmacy-driven education on the amount of appropriately administered tetanus vaccines

reduced diphtheria toxoid and acellular pertussis vaccine

in the emergency department. Hosp Pharm. 2019;54(1):45-50.

recommendations of the Advisory Committee on Immunization Practices (ACIP) and recommendation of ACIP, supported by the

4. Drapkin Z, Madsen TE, Weaver N, et al. Accuracy of parentalreported tetanus vaccination status for children with lacerations.

Healthcare Infection Control Practices Advisory Committee

Pediatr Emerg Care. 2019. In press.

(HICPAC), for use of Tdap among health-care personnel. MMWR Recomm Rep. 2006;55(RR-17):1-37.

5. Liu SW, Huang LC, Chung WF, et al. Tetanus vaccination and

10. McConnochie KM, Roghmann KJ. Immunization opportunities missed

extra-immunization among adult populations: eight-year follow up

among urban poor children. Pediatrics. 1992;89(6.1):1019-26.

cohort study of 771,443 Adults in Taiwan, 2006-2013. Int J Environ

11. Meissner C. How much do you know about tetanus? 2015. Available

Res Public Health. 2018;15(8):1622.

at: https://www.aappublications.org/content/36/4/8.1. Accessed

6. Orsi GB, Modini C, Principe MA, et al. Assessment of tetanus immunity status by tetanus quick stick and anamnesis: a prospective

Western Journal of Emergency Medicine

1146

September 27, 2016.

Volume 21, no. 5: September 2020

Original Research

A Method for Grouping Emergency Department Visits by Severity and Complexity B. Jason Theiling, MD* Kendrick V. Kennedy, MD* Alexander T. Limkakeng Jr., MD* Pratik Manandhar, BA† Alaatin Erkanli, PhD† Stephen R. Pitts, MD‡

*Duke University School of Medicine, Department of Emergency Medicine, Durham, North Carolina † Duke University School of Medicine, Department of Biostatistics and Bioinformatics, Durham, North Carolina ‡ Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia

Section Editor: Stormy Monks, PhD, MPH Submission history: Submitted August 9, 2019; Revision received April 30, 2020; Accepted June 19, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.44086

Introduction: Triage functions to quickly prioritize care and sort patients by anticipated resource needs. Despite widespread use of the Emergency Severity Index (ESI), there is still no universal standard for emergency department (ED) triage. Thus, it can be difficult to objectively assess national trends in ED acuity and resource requirements. We sought to derive an ESI from National Hospital Ambulatory Medical Care Survey (NHAMCS) survey items (NHAMCS-ESI) and to assess the performance of this index with respect to stratifying outcomes, including hospital admission, waiting times, and ED length of stay (LOS). Methods: We used data from the 2010-2015 NHAMCS, to create a measure of ED visit complexity based on variables within NHAMCS. We used NHAMCS data on chief complaint, vitals, resources used, interventions, and pain level to group ED visits into five levels of acuity using a stepwise algorithm that mirrored ESI. In addition, we examined associations of NHAMCS-ESI with typical indicators of acuity such as waiting time, LOS, and disposition. The NHAMCS-ESI categorization was also compared against the “immediacy” variable across all of these outcomes. Visit counts used weighted scores to estimate national levels of ED visits. Results: The NHAMCS ED visits represent an estimated 805,726,000 ED visits over this time period. NHAMCS-ESI categorized visits somewhat evenly, with most visits (42.5%) categorized as a level 3. The categorization pattern is distinct from that of the “immediacy” variable within NHAMCS. Of admitted patients, 89% were categorized as NHAMCS-ESI level 2-3. Median ED waiting times increased as NHAMCS-ESI levels decreased in acuity (from approximately 14 minutes to 25 minutes). Median LOS decreased as NHAMCS-ESI decreased from almost 200 minutes for level 1 patients to nearly 80 minutes for level 5 patients. Conclusion: We derived an objective tool to measure an ED visit’s complexity and resource use. This tool can be validated and used to compare complexity of ED visits across hospitals and regions, and over time. [West J Emerg Med. 2020;21(5)1147-1155.]

INTRODUCTION In the current practice of emergency medicine, triage functions to quickly prioritize care and sort patients by anticipated resource needs. While the goal of accurate Volume 21, no. 5: September 2020

prioritization is direct improvement in the quality of care of individuals, the intention of predicting resource utilization is to streamline emergency department (ED) operations without causing harm. The second goal has become increasingly 1147

Western Journal of Emergency Medicine

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity important as the number of ED visits continues to rise, hospitals function under reduced available capacity, and reliance on the ED as the safety net of hospital systems increases.1 This dual function of ED triage was proposed by Wuerz, who pioneered a five-level Emergency Severity Index (ESI).2 Several studies have evaluated ESI’s reliability and validity.2-10 Other five-level triage schemes have been developed outside the United States: the Canadian Triage and Acuity Scale (CTAS),11-12 and the Australasian Triage Scale (ATS).13 Both CTAS and ATS assign each triage level a target “time to be seen,” which in turn allows comparisons between sites on the basis of compliance with these targets. The ATS is unique in explicitly accepting a third role as a data source for describing case-mix to generate the adjusted estimates of ED visit characteristics that inform national policy.12 Comparing ED performance and ED visit characteristics is more problematic in the US. Despite geographic variation in important ED characteristics (eg, the proportion of safetynet visits,14 population-based ED visit rates,15 and hospital admission rates16) there is still no national mandated standard for categorizing the acuity and resource complexity of ED visits. Central to the ESI system is the idea of “immediacy,” a marker of how acutely ill a patient is believed to be and thus how “immediately” they may need to be seen. Unfortunately, because of the continued widely disparate triage procedures and non-response in surveys, adjustment of ED data on the basis of the “immediacy” item alone is potentially biased and may ignore the dimension of care complexity. Additionally, the acuity or “immediacy” of a patient, as denoted by ESI, often does not linearly correlate with resource utilization during that ED visit. The purpose of this study was to evaluate the “immediacy” variable in existing National Hospital Ambulatory Medical Care Survey (NHAMCS) survey data, and to create a practical alternative method for grouping ED visits by both acuity and resource complexity in a manner analogous to the ESI. To minimize data loss we sought to derive an ESI from NHAMCS survey items with low frequencies of item non-response. We assessed the performance of this index with respect to several outcomes, including hospital admission, waiting time, and overall ED length of stay (LOS). This study uses the combined 2010–2015 ED components of the NHAMCS.16 The NHAMCS is a probability sample of US hospital EDs and outpatient departments conducted annually since 1992. It is one of a family of healthcare surveys performed by the Centers for Disease Control and Prevention’s (CDC) National Center for Health Statistics (NCHS). The US Census Bureau is responsible for field operations and data collection. Although one of its data items is currently a five-level item called “immediacy with which patient should be seen,” with additional checkboxes for “no triage” and “unknown,” other measures of urgency have been abstracted from ED charts Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? Triaging prioritizes care and sorts patients by anticipated resource needs. Despite widespread use of the Emergency Severity Index (ESI), no universal standard exists. What was the research question? Derive an ESI tool from a national survey item and assess the performance of this index with respect to stratifying outcomes. What was the major finding of the study? This tool can be used to compare complexity of ED visits across hospitals and regions, and over time. How does this improve population health? ESI may not reflect resource needs in linear fashion. Our tool helps to compare data across regions and time periods.

in the past. From 1992–1996 the survey captured a highly subjective two-level “Urgent/emergent vs. Non-urgent” item, which led to the widely cited and heavily criticized conclusion that “55% of ED visits are non-urgent.”17 In 1997 this item was replaced by a four-level variable to capture more degrees of immediacy, each succeeding level associated with a progressively longer target “time to be seen.” In 2005 “immediacy” was promoted to the current five-level item, each level again associated with target times. METHODS The NHAMCS is a four-stage probability sample, sampled in the following sequence: 1) 112 geographic primary sampling units of approximately county size; 2) probability sample of nonfederal, short-stay, general hospitals with EDs or outpatient departments or both, within the sampled primary sampling units, selected from a publicly available database of all US hospitals; 3) emergency service areas within 24-hour EDs and clinics within outpatient departments; and 4) a sample of about 100 visits within the selected EDs or outpatient departments during a randomly assigned four-week reporting period throughout the year. We limited our analysis to the ED component of NHAMCS and downloaded data from the NHAMCS website (ftp://ftp.cdc.gov/ pub/Health_Statistics/NCHS/Datasets/NHAMCS). Hospital staff were asked to complete a patient record form (PRF) for a sample of visits during a four-week reporting period, from which the data were abstracted and coded. The NHAMCS was approved by

1148

Volume 21, no. 5: September 2020

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity

Duke University Insitutational Review Board. A report published elsewhere describes the plan and operation of the NHAMCS in greater detail.17 Unless otherwise noted, all estimates in this report are weighted to give national estimates. We considered estimates based on an unweighted count of less than 30 to be unreliable.

upon or shortly after presentation. Thus, ESI-N uses the presenting complaint rather than the final diagnosis as the main component. For NHAMCS-ED, this complaint is abstracted directly from the actual ED chart into up to three free-text entry fields on the PRF. The PRFs are then batched, and the handwritten text is converted to standard codes by the Constella Group, Inc. (Durham, NC). According to the reason for visit classification for ambulatory care (RVC), there is a very low rate (<1%) of nonresponse. Additionally, vital signs have been recorded since 2001 and can be used to modify triage class just as the ESI does. Vital signs are not obtained on every visit,

Creating The NHAMCS-ESI Index We based the NHAMCS-ESI (ESI-N) on the published ESI,2 but used only variables available in NHAMCS (Table 1). Since ESI and other tools are used in the initial triage process, they are dependent on data available immediately

Table 1. Illustrates in detail the procedure we used to derive Emergency Severity Index levels, specifying variable names and values taken from naming conventions in NHAMCS-ED* input programs for public use files.18 Patient conditions

Variable name

Occurrences

Dead on arrival (RFV code)

RFV1-RFV3

Respiratory arrest

RFV1-RFV3

Cardiac arrest

RFV1-RFV3

140

Cardiopulmonary arrest

RFV1-RFV3

Unconscious on arrival

RFV1-RFV3

860

DOA

Pulse â&#x2030;¤50 and age >25

PULSE; AGE

793

Endotracheal intubation

ENDOINT

373

CPR

225

Level 1

Dead on arrival (checkbox)

Cardiopulmonary resuscitation Systolic blood pressure â&#x2030;¤80 and Age >25

BPSYS; AGE

324

Any of level 1 criteria

2,365

% of total weighted (95% CI)

1.4 (1.3, 1.5)

Level 2 (if not in level 1) Fainting (Syncope)

RFV1-RFV3

1,689

Hostile behavior

RFV1-RFV3

536

Neurological weakness or speech difficulty

RFV1-RFV3

488

Shortness of breath/breathing problem

RFV1-RFV3

10,339

Gastrointestinal bleeding

RFV1-RFV3

109

Retention of urine

RFV1-RFV3

472

Sepsis, septicemia

RFV1-RFV3

Ischemic heart disease

RFV1-RFV3

100

Violence/self-harm

RFV1-RFV3

1,277

Rape

RFV1-RFV3

162

RFV1-RFV3; AGE

144

Altered level of consciousness Abdominal pain (elderly) Abdominal pain, vomiting and diarrhea

RFV1-RFV3

Abdominal pain (youth)

RFV1-RFV3; AGE

391

Head Trauma (infants)

RFV1-RFV3; AGE

113

Level 3 exceeding vital sign thresholds

AGEDAYS; PULSE; TEMPF

AGEDAYS; AGE; PULSE *NHAMCS-ED, National Hospital Ambulatory Medical Care Survey-emergency department visits. RFV, reason for visit; CI, confidence interval.

Volume 21, no. 5: September 2020

1149

564

Western Journal of Emergency Medicine

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity Table 1. Continued. Patient conditions

Variable name

Occurences

Level 3 exceeding vital sign thresholds (continued) AGE; PULSE

879

AGE; PULSE

26,814

Any of level 2 criteria

40,178

% of total weighted (95% CI)

24.5 (24.1, 25.0)

Level 3 (if not in level 1-2) More than 1 resource used

(see Resources below)

53,704

Severe pain

PAINSCALE

29,291

Pediatric fever

AGE; TEMPF

785

Motor vehicle accident

2,044

Any of level 3 criteria

70,230

% of total weighted (95% CI)

42.5 (41.7, 43.3)

Level 4 (if not in level 1-3) One resource used

(see Resources below)

% of total weighted (95% CI)

24,974

15.1 (14.6, 15.7)

Level 5 (if not in level 1-4) No resource used

(see Resources below)

% of total weighted (95% CI) RFV, reason for visit; CI, confidence interval.

16.4 (15.7, 17.1)

especially among the pediatric population; thus, when vital signs were missing, we considered the ESI-N unchanged rather than missing. Unlike the ESI, the retrospective ESI-N tallies actual resources used, rather than predicted resources, and cannot account for any resources not listed in the PRF. Describing The Acuity Of The Patients And Validating The Index We described the acuity of the study population by generating basic descriptive statistics including the mean ESI-N level and corresponding confidence intervals (CI) for selected patient and hospital characteristics. To validate the ESI-N, we examined associations with typical indicators of acuity such as waiting time, LOS, disposition, and mode of arrival. Waiting time was defined as the number of minutes between the time of arrival and the time seen by a physician. We defined LOS as the number of minutes between the time of arrival and time of ED discharge. All of these outcomes measures of acuity (waiting time, LOS, disposition, and mode of arrival) were measured in their respective units or proportions for each level of the derived ESI-N and compared using 95% CIs. We conducted all analyses using Stata v10 (StataCorp, College Station, TX).19 RESULTS The “immediacy” item in NHAMCS-ED (IMMED) was unknown, or triage was not performed, in 15.0% of visits. Among the remaining visits, the maximum value of Western Journal of Emergency Medicine

27,408

IMMED was less than five in 145 (26.2%) of 553 emergency service areas in the 457 hospitals surveyed. This suggests the continued use of triage schemes with fewer than five levels. Unfortunately, having four or fewer triage levels creates a bias toward lower (more acute) immediacy levels when incorporated in aggregate estimates. These results are given both in raw counts of 20102015 patient record forms, and in a nationally representative estimate of percent of all visits. Figure 1 compares this frequency distribution both to the distribution of the variable IMMED in the same data, and to the distribution of actual ESI levels assigned by triage nurses in a prospective validation study of two ED populations.20 We assessed mean ESI levels for a variety of patient characteristics, as demonstrated in Table 2. For each of the four patient-level characteristics that we assessed (age, gender, race/ethnicity, and payer type), there were statistically significant differences (p<0.0001) between mean ESI-N levels. Of note, there was a monotonic increase in acuity with increasing age. Within their respective categories, visits by women and by non-Hispanic Whites had on average more acute ESI-N scores. Among payer types, Medicare visits had on average the most acute ESI-N, with the least acute categories being worker’s compensation, “no charge,” and Medicaid/State Children’s Hospital Insurance Program. We assessed validity of the ESI-N against several

1150

Volume 21, no. 5: September 2020

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity

Figure 1. Distribution of Emergency Severity Index levels based on the presenting complaint. IMMEDR, immediacy.

outcomes, including hospital admission, intensive care unit (ICU) admission, ambulance arrival, and leaving without being seen and in each case found the expected relationships (Table 3). As can be seen, the ESI-N is able to differentiate all visits into five levels of care, with only a minority being classified in the most severe, “Immediate” category. Relative to their proportion of total visits, more severe acuity ESI-N levels account for a higher proportion of the patients admitted to the hospital and those admitted to a critical care unit. Furthermore, ESI-N is able to differentiate visits (Table 4) by likelihood of being admitted to the hospital and ED LOS. The 95% CIs of percent admitted or LOS in minutes between categories largely do not overlap as the ESI-N increases in severity from Level 5 “Nonurgent” to Level 1 “Immediate.” ESI-N does not appear to differentiate well among categories for time waiting to be seen by a physician, however. DISCUSSION The NHAMCS-ED is the only nationally representative survey of ED visits; thus, it is a valuable resource for policymaking and has provided data for many published studies. However, most causal inferences of interest, such as the effect of demographic variables like race/ethnicity on hospital admission, length of ED visit, and other aspects of quality and cost, are potentially confounded by the main determinant of these outcomes: the seriousness or acuity of the patient’s initial problem. A related reason to measure acuity is the need for fair comparisons across regions or over time, i.e., for Volume 21, no. 5: September 2020

case-mix adjustment. We have derived a five-level index of acuity called the ESI-N, because a similar existing item in the survey, called “immediacy,” frequently offered difficulties in chart abstraction: either a triage score was not obtained, or, in some cases when the score was obtained, it was absent from the chart. In addition, even when a triage score was present, there still is no universally accepted standard of triage in the US. In many cases, EDs may use fewer than five triage levels; in others the identical levels in different systems may have different definitions or applications. We attempted to overcome these limitations by deriving ESI-N, a severity index based primarily on the patient’s RFV codes, modified by values in a small number of other data fields, including age, vital sign extremes, and resuscitative procedures. Finally, we separated the less acute visits by number of resources used, emulating the previously derived and validated ESI.2 The American College of Emergency Physicians (ACEP) and the Emergency Nurses Association (ENA) formed a task force in 2002 dedicated to the goal of creating and promoting a standard measure of presenting patient acuity.21 However, to this day, a number of EDs continue to use locally designed triage guidelines of varying complexity and evidence quality or do not perform formal triage.22 In fact, this continued lack of a standard resulted in the ACEP and ENA revising its position statement and advocating for “implementing a standardized emergency department (ED) triage scale and acuity categorization process” and endorsing ESI as that process.23 While ESI remains a prominent triage tool, it does not always adequately reflect resource need in linear fashion. That is, ESI Level 1s do not always necessitate the greatest

1151

Western Journal of Emergency Medicine

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity Table 2. Mean ESI-N for patient characteristics. Patient characteristics

Weighted patient # (in 1000s)

Mean ESI-N level (95% CI)

805,726

3.21 (3.19, 3.23)

Female

445,253

3.15 (3.13, 3.17)

Male

360,473

3.27 (3.25, 3.29)

Under 15 years

152,469

3.76 (3.72, 3.79)

15-24 years

124,430

3.20 (3.18, 3.23)

25-44 years

227,839

3.12 (3.10, 3.15)

45-64 years

176,474

3.05 (3.03, 3.07)

65-74 years

54,185

2.94 (2.91, 2.97)

75 years and over

70,328

2.88 (2.86, 2.91)

Non-Hispanic White

476,805

3.15 (3.13, 3.18)

Non-Hispanic Black

180,130

3.26 (3.23, 3.29)

Hispanic

124,909

3.33 (3.30, 3.36)

Non-Hispanic other

23,882

3.23 (3.18, 3.28)

10,470

3.37 (3.26, 3.48)

Unknown

48,878

3.25 (3.20, 3.30)

Private insurance

230,145

3.22 (3.19, 3.24)

Medicare

146,598

2.93 (2.90, 2.95)

Medicaid or CHIP

227,873

3.35 (3.32, 3.37)

6,857

3.57 (3.49, 3.65)

105,473

3.22 (3.19, 3.25)

All Visits Gender

Age

Race

Expected source of payment All sources of payment are blank

Worker's compensation Self-pay No charge/charity

7,113

3.15 (3.08, 3.23)

22,320

3.19 (3.14, 3.24)

Northeast

140,858

3.25 (3.21, 3.29)

Midwest

187,086

3.19 (3.15, 3.24)

South

310,329

3.19 (3.16, 3.22)

West

167,453

3.21 (3.17, 3.25)

2010

129,843

3.19 (3.16, 3.23)

2011

136,296

3.17 (3.14, 3.21)

2012

130,870

3.21 (3.17, 3.24)

2013

130,353

3.22 (3.18, 3.25)

2014

141,420

3.25 (3.20, 3.29)

2015

136,943

3.21 (3.15, 3.26)

Other Region

Visit Year

ESI-N, Emergency Severity Index levels based primarily on the patient’s “reason for visit” code as presented in the National Hospital Ambulatory Care Survey; CI, confidence interval; CHIP, Children’s Health Insurance Program.

resources in the ED, with ESI 2s requiring less, and so on. In fact, one paper found that ESI Level 2 and 3 patients are actually very similar in their resource needs, but hospitalization varied dramatically.24 In contrast, we found Western Journal of Emergency Medicine

positive associations between ESI-N severity and hospital admission, ICU admission, and ambulance transport.2 Unlike one prominent early study,22 we found that patients who left without being seen were much more likely to have a low 1152

Volume 21, no. 5: September 2020

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity

Table 3. Number of emergency department visits by Emergency Severity Index-N levels. ESI-N level Weighted patient # (in 1000s)

Immediate (level 1) (%)

Emergent (level 2) (%)

Urgent (level 3) (%)

Semi-urgent (level 4) (%)

Non-urgent (level 5) (%)

All visits

805,726

1.4 (0.1)

24.5 (0.2)

42.5 (0.4)

15.1 (0.3)

16.4 (0.4)

Admitted to hospital

83,607

4.7 (0.2)

40.6 (0.5)

49.2 (0.7)

3.4 (0.2)

2.2 (0.6)

Admitted to critical care unit

10,875

15.0 (1.1)

46.3 (1.4)

35.9 (1.3)

1.8 (0.3)

Arrived by ambulance

122,246

5.0 (0.2)

33.1 (0.5)

48.1 (0.6)

8.2 (0.3)

5.5 (0.4)

Criterion

Left without 8,012 * 28.4 (1.4) 41.4 (1.7) 9.5 (1.2) 19.4 (1.3) being seen *Figure does not meet standard or reliability of precision. ESI-N, Emergency Severity Index levels based primarily on the patient’s “reason for visit” code as presented in the National Hospital Ambulatory Care Survey.

Table 4. Rates of hospital admission by ESI-N levels. ESI-N level (Weighted patient # [in 1000s], %) Criterion

Immediate (level 1)

28,981

Urgent (level 3)

18.0% (16.8, 19.2)

34,117

12.3% (11.4, 13.2)

Semi-urgent (level 4) 2,274

2.5% (2.1, 2.9)

Non-urgent (level 5)

Admitted to hospital

3,425

Length of stay

201.4 (191.3, 211.5)

180.2 (174.2, 186.3)

182.7 (177.2, 188.3)

119.5 (115.4, 123.5)

82.9 (80.0, 85.8)

13.5 (11.8, 15.2)

22.5 (21.2, 23.8)

24.4 (23.0, 25.8)

24.5 (22.6, 26.3)

24.9 (23.3, 26.5)

Waiting time

37.9% (34.5, 41.2)

Emergent (level 2)

1,504

1.5% (0.6, 2.4)

ESI-N, Emergency Severity Index levels based primarily on the patient’s “reason for visit” code as presented in the National Hospital Ambulatory Care Survey.

severity by ESI-N, a change that could reflect increased use of the ED as a “safety net” rather than for emergencies. This is congruent to findings of some more recent studies.25-26 Waiting time is often cited as an indicator of ED quality that depends both on hospital capacity as well as the average ED volume and acuity of visits. Recent studies suggest that ED crowding has increased waiting times even for serious problems.27 But excluding triage category 1, the ESI-N index discriminates poorly, possibly in part because the nurse assigning a triage category also controls the patient’s waiting time. LIMITATIONS While the ESI-N derived here appears to be a consistent, valid measure of acuity and complexity, there are limits on its use. It is complex: Creating the index requires an algorithm executed by a computer program. It is derived from the 20102015 NHAMCS-ED survey files, which were formatted in a specific way; however, variable names and values, such as the RVF classification scheme, have changed and will continue to change over time. Use of the algorithm in survey years other than 2010-2015 may require its modification. Like the ESI itself, devising the ESI-N algorithm required subjective judgment. Volume 21, no. 5: September 2020

The index reflects two separate dimensions of ED visits: acuity and complexity. Distinguishing between these dimensions is impossible when evaluating aggregate data. In this paper we describe and validate the ESI-N. It has not been validated and tested for reliability. CONCLUSION Two technical tasks commonly required in health services research include accounting for confounding of a causal relationship, and adjusting for case-mix to minimize selection bias when comparing groups.28 Our derivation of a five-level severity index for data abstracted from ED charts addresses these research needs. The ESI-N can be used to stratify results, or as an ordinal exposure or outcome variable in regression or propensity score models, increasing statistical power by reducing the need to include multiple covariates in a model. Caution should be exercised in its application. While it includes observations lost with the use of IMMED, it is less predictive of ED waiting time. It might be used together with IMMED to reduce residual confounding in some analyses. It is important to understand the origin of the index in complaint codes modified by age and a few other variables, and that it segregates less acute

1153

Western Journal of Emergency Medicine

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity visits based on a list of specified services performed. Since this list is short, and since some of the data abstracted are relatively nonspecific (eg, five-level instead of 10-level pain scores), the ESI-N will misclassify some patients compared to a prospectively determined ESI. Future directions include validating ESI-N on another independent source of ED visits and testing its use in a prospective cohort of ED visits.

10. Bullard MJ, Unger B, Spence J, et al. Revisions to the Canadian Emergency Department Triage and Acuity Scale (CTAS) adult guidelines. Can J Emerg Med. 2008;10(2):136-42. 11. Warren DW, Jarvis A, LeBlanc L, et al. Revisions to the Canadian Triage and Acuity Scale paediatric guidelines (PaedCTAS). Can J Emerg Med. 2008;10(3):224-43. 12. Australasian College of Emergency Medicine. Policy on the Australasian Triage Scale. 2000. Available at: http://www.acem. org.au/media/policies_and_guidelines/P06_Aust_Triage_Scale_-_ Nov_2000.pdf. Accessed October 23, 2018.

Address for Correspondence: B. Jason Theiling, MD, Duke University Medical Center, Division of Emergency Medicine, 3401 Erwin Road, Durham, NC, 27710. Email: Jason.theiling@duke.edu.

13. Burt CW and Arispe IE. (2004). Characteristics of emergency departments serving high volumes of safety-net patients: United States, 2000. In National Center for Health Statistics (Eds.), Vital and Health Statistics (pages 1-16). Hyattsville: U.S. Dept. of Health

and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics. 14. Cunningham PJ. What accounts for differences in the use of hospital emergency departments across U.S. communities? Health Affair. 2006;25(Suppl1):w324-36.

Copyright: © 2020 Theiling et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

15. Pitts SR, Niska RW, Xu J, et al. National Hospital Ambulatory Medical Care Survey: 2006 emergency department summary. Natl Health Stat Report. 2008;(7):1-38. 16. Fernandes CM, Tanabe P, Gilboy N, et al. Five-level triage: a report from the ACEP/ENA Five-level Triage Task Force. J Emerg Nurs. 2005;31(1):39-118. 17. McCaig LF and McLemore T. (1994). Series 1: Programs and Collection Procedures. Plan and Operation of the National Hospital

REFERENCES

Ambulatory Medical Survey (pages 1-78). Hyattsville: U.S. Dept. of Health and Human Services, Public Health Service, Centers for

1. Ieraci S, Digiusto E, Sonntag P, et al. Streaming by case complexity:

Disease Control and Prevention, National Center for Health Statistics.

evaluation of a model for emergency department Fast Track. Emerg

18. Wuerz R. Emergency Severity Index triage category is associated

Med Australas. 2008;20(3):241-9.

with six-month survival. Acad Emerg Med. 2008;8(1):61-4.

2. Wuerz RC, Milne LW, Eitel DR, et al. Reliability and validity of a new

19. Centers for Disease Control and Prevention (CDC). NHAMCS-ED

five-level triage instrument. Acad Emerg Med. 2000;7(3):236-42.

SAS input statements. Available at: ftp://ftp.cdc.gov/pub/Health_

3. Wuerz RC, Travers D, Gilboy N, et al. Implementation and refinement

Statistics/NCHS/Dataset_Documentation/NHAMCS/sas . Accessed

of the Emergency Severity Index. Acad Emerg Med. 2001;8(2):170-6.

October 25, 2008.

4. Eitel DR. The Emergency Severity Index triage algorithm version 2 is reliable and valid. Acad Emerg Med. 2003;10(10):1070–80.

20. McHugh M, Tanabe P, McClelland M, et al. More patients are triaged

5. Tanabe P, Gimbel R, Yarnold PR, et al. The Emergency Severity

using the Emergency Severity Index than any other triage acuity system in the United States. Acad Emerg Med. 2012;19(1):106-109.

Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs. 2004;30(1):22-9.

21. Burt CW, McCaig LF, Valverde RH. Analysis of ambulance transports and diversions among US emergency departments. Ann Emerg Med.

6. Tanabe P, Gimbel R, Yarnold PR, et al. Reliability and validity of

2006;47(4):317-26.

scores on the Emergency Severity Index version 3. Acad Emerg Med. 2004;11(1):59-65.

22. American College of Emergency Physicians. ACEP Policy Statements: Triage Scale Standardization. 2010. Available at: from

7. Baumann MR. Evaluation of the Emergency Severity Index

http://www.acep.org/Content.aspx?id=29828&terms=triage%scale.

(version 3) triage algorithm in pediatric patients. Acad Emerg

Accessed June 1, 2011.

Med. 2008;12(3):219-224. 8. Worster A, Gilboy N, Fernandes CM, et al. Assessment of inter-

23. Hocker MB, Gerardo CJ, Theiling BJ, et al. NHAMCS Validation of Emergency Severity Index as an indicator of emergency department

observer reliability of two five-level triage and acuity scales: a

resource utilization. West J Emerg Med. 2018;19(5):855-62.

randomized controlled trial. Can J Emerg Med. 2004;6(4):240-5. 9.

Worster A, Fernandes CM, Eva K, et al. Predictive validity

24. Baker DW. Patients who leave a public hospital emergency

comparison of two five-level triage acuity scales. Eur J Emerg Med.

department without being seen by a physician. Causes and

2007;14(4):188-92.

consequences. JAMA. 1991;266(8):1085-90.

Western Journal of Emergency Medicine

1154

Volume 21, no. 5: September 2020

Theiling et al.

A Method for Grouping ED Visits by Severity and Complexity

25. Hall J and Jelinek GA. Characteristics and outcomes of patients

27. Wilper AP, Woolhandler S, Lasser KE, et al. Waits to see an

who “did not wait” after attending Perth public hospital emergency

emergency department physician: U.S. trends and predictors, 1997–

departments, 2000–2003. Med J Australia. 2007;187(11-12):626–9.

2004. Health Affair. 2008;27(Suppl 1):w84-95.

26. Bragulat E, Ortega M, Miro O. Increasing number of patients who

28. Deeks JJ, Dinnes J, D’Amico R, et al. Evaluating non-randomised

leave the ED without being seen. Emerg Med J. 2008;25(1):60.

Volume 21, no. 5: September 2020

intervention studies. Health Technol Assess Rep. 2003;7(27):iii-173.

1155

Western Journal of Emergency Medicine

Brief Research Report

Triage in the Time of Diphtheria Hadiki Habib, MD Hesty Kurniawaty, RN

Emergency Unit, Dr. Cipto Mangunkusumo Hospital, Jakarta, Indonesia

Section Editor: Elissa Perkins, MD, MPH Submission history: Submitted December 6, 2019; Revision received April 19, 2020; Accepted June 22, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.46094

Introduction: A diphtheria outbreak occurred in 2017 in Jakarta, Indonesia, during which our hospital was appointed as a referral hospital where patients with upper respiratory tract symptoms were sent for confirmation of the diagnosis and medical intervention. In this study we review the implementation of the emergency department (ED) triage process and patient flow management during the diphtheria outbreak. No previous study in Indonesia has provided a detailed report on the triage process during infectious disease outbreaks. Method: We modified our pre-existing hospital triage method according to the “identify, isolate, and inform” principle. We developed novel criteria for triage to identify triage-suspected cases and also a diphtheria package to simplify the diagnostic process. Four separate rooms were modified to isolation spaces to enable medical staff to observe these patients. We obtained data from the ED outbreak registry and electronic health records. Results: Of 60 cases of triage-suspected diphtheria, six were classified as suspected diphtheria. The mean time from “identify” to “isolate” was 3.5 minutes, and from “isolate” to “inform” was 10 minutes. Mean ED length of stay for probable diphtheria was 24.46 hours. No medical personnel in the ED showed any signs of diphtheria 30 days after the outbreak had abated. Conclusion: The modified criteria can help triage officers detect suspected diphtheria cases and measure the triage response time. Use of the diphtheria package and four separate rooms in the ED could act as an infection control procedure and facilitate the improvement of the diagnostic process. [West J Emerg Med. 2020;21(5)1156-1159.]

INTRODUCTION Diphtheria is a contagious infection caused by Corynebacterium diphtheria. The infection spreads through droplets and contact of mucous membranes with bodily fluid containing the causative bacteria. Diphtheria was last reported in 1987 and between 1990–1995 in the United Kingdom and the Soviet Union, respectively.1,2 Even though diphtheria can be prevented by immunization, it remains a major health problem in developing countries such as India, Thailand, and Bangladesh.3-5 During an outbreak, the emergency department (ED) has an important role in identifying suspected cases and providing acute medical management, while controlling infection transmission. It needs to act fast by implementing the World Health Organization (WHO) guidelines for suspected infectious disease.6,7 Western Journal of Emergency Medicine

In 2017, a diphtheria outbreak occurred in Jakarta, Indonesia.8,9 Our hospital was appointed as a referral hospital where patients with upper respiratory tract symptoms were examined for case findings of suspected diphtheria. Our goal in this study was to review the implementation of ED triage and patient flow management during the outbreak, as well as to present data on the new cases of diphtheria. No previous studies in Indonesia have reported on the triage process during infectious disease outbreaks. METHODS Participants, Study Design and Setting We included all patients who presented to the ED with triage-suspected diphtheria. Informed consent was obtained, which included permission to take complete notes from medical

1156

Volume 21, no. 5: September 2020

Habib et al.

Triage in the Time of Diphtheria

records. This study was approved by the ethics committee of the hospital. We reviewed patients’ medical records from presentation to the ED until discharge. Data were collected during the outbreak in Jakarta from December 2017 to March 2018. We evaluated in-hospital transmission by interviewing medical personnel in the ED 30 days after the outbreak abated, to determine whether they showed signs of infection. During the start of the outbreak, we vaccinated all medical personnel in the ED who were at risk of being exposed to diphtheria patients.

Population Health Research Capsule What do we already know about this issue? Diphtheria is a re-emerging infectious disease. Proper triage and early recognition could improve emergency department flow and prevent local transmission. What was the research question? We sought to summarize our modification on of ED triage and patient flow during the diphtheria outbreak.

Triage Criteria We implemented the pre-existing, three-tier triage categories. Critical patients who required immediate treatment due to life-threatening conditions were included in the resuscitation category. In the urgent category we included non-critical patients whose symptoms were uncontrolled or could not be tolerated by the patient (eg, moderate to severe pain, vomiting, diarrhea, and severe dizziness) and who needed early medical intervention to prevent further clinical deterioration. Patients who did not belong to either of the two groups were classified into the non-urgent category.

What was the major finding of the study? Modified criteria help triage officers detect suspected cases and increase response time.

Diphtheria Diagnosis Suspected diphtheria patients are defined as those with pharyngitis, laryngitis, or tonsillitis with greyish, thick pseudomembrane adhering to the pharynx, larynx, or mucosa of the nose that bleeds easily after applying light pressure using a wooden tongue depressor. Laboratory-confirmed diphtheria patients were defined as those with Corynebacterium diphtheria detected in their specimen cultures.10 ED Outcome We defined the outcome as the patient’s condition immediately before he or she left the ED. These were as follows: deceased; discharged as outpatient; transferred to inpatient ward; or referred to another hospital. The length of stay was defined as the time from admission to outcome determination in the ED. Identify, Isolate, and Inform Principle.11-13 To increase the awareness of the triage officer (nurse) on triage-suspected diphtheria, novel identifying criteria were included as follows: 1) those presenting with any one of the complaints such as sore throat, cough, or shortness of breath within less than three days of onset during the diphtheria outbreak; 2) those referred from other healthcare facilities with a diagnosis of suspected diphtheria; and 3) those who presented with a concern of having acquired the infection. The triage officer’s duty was to identify these conditions immediately after the patient arrived at the triage counter. Any one of the three criteria was sufficient to categorize the patient as having triage-suspected diphtheria and the diagnosis had to be clinically confirmed by an ear, nose and throat (ENT) specialist. Volume 21, no. 5: September 2020

How does this improve population health? This report highlights the importance of triage and early recognition to improve patient flow and prevent local transmission in the busy emergency department.

A complete vital sign examination was performed after placing a surgical mask on the patient.11,13 Triage-suspected patients were then transferred to a separate isolation room in the ED, to ensure that droplet precautions were done. There was no pressure gradient between the isolation room and the surrounding zone. Our unit provided four separate rooms for triage-suspected diphtheria, and each room could contain one patient. An emergency physician using a surgical mask received the patient in the isolation room and performed initial assessment and early intervention. The “inform” process involved the emergency physician contacting the ENT specialist for throat examination by using the diphtheria package available in ED pharmacy. The diphtheria package consisted of one surgical mask with a face shield, a pair of non-sterile gloves, three cotton buds, one disposable apron, two disposable wooden tongue depressors, and a biohazard plastic bag.13 ENT specialist examination was then validated by an infectious disease specialist to designate the case as suspected diphtheria or not. A suspected case was reported by the hospital call center to the public health center and the infection center’s hospital for referral purposes. Data Management We analyzed data descriptively, with no subgroup analysis. Incomplete filing in the medical records was considered as missing data.

1157

Western Journal of Emergency Medicine

Habib et al.

Triage in the Time of Diphtheria RESULTS During the study period, 12,778 patients visited the ED of the Cipto Mangunkusumo Hospital. There were 60 cases of triage-suspected diphtheria, among which six were cases of suspected diphtheria. There were no initially non-suspected cases of diphtheria at our ED who later returned as suspected or diagnosed cases during the study period. Only suspected cases underwent microbiology culture. The demographic characteristics of the patients with suspected diphtheria are shown in Table 1. Almost half of the triage-suspected cases (45%) were referred from outpatient primary care, paediatricians, and ENT specialist clinics. All referred patients presented to the ED on their own and brought the referral letter from the previous physicians. Upon arrival to the ED, the referred patient underwent a reassessment at the triage counter. Among those with suspected diphtheria, five patients were brought by an ambulance to the infection center’s hospital to be admitted, and one patient was self-referred to our hospital. The mean time from “identify” to “isolate” was 3.5 minutes, and from “isolate” to “inform” was 10 minutes. Patients with suspected diphtheria showed several observable signs and symptoms, as detailed in Table 2.

Table 1. Demographic characteristics of patients with triagesuspected diphtheria. Characteristics

N (%)

Gender Male Female

29 (48.3) 31 (51.7)

Age (years) <18 ≥18

25 (41.7) 35 (58.3)

ED Admission Self-referred Referred from other medical facilities

33 (55) 27 (45)

Chief Complaint Sore throat Fever Shortness of breath

51 (85) 45 (75) 1 (1.7)

Clinical Manifestation Neck mass Membrane in respiratory tract mucosa

11 (18.3) 38 (63.3)

Exposure History Positive Negative

4 (6.7) 56 (93.3)

Diphtheria Diagnosis Suspected Diphtheria Not Diphtheria

6 (10) 54 (90)

Length of Stay (LoS) Identify to Isolate (minutes) Isolate to inform (minutes) Emergency unit LoS probable diphtheria (hours) ED, emergency department.

Western Journal of Emergency Medicine

Mean 3.5 10 24.46

DISCUSSION Identifying suspected patients is an important protocol during an outbreak. Although not all patients with positive results during triage screening have the disease, deciding which patient group is at a high risk of contracting the disease and in need of further medical examination is an important part of infection control.14 Syndromic surveillance is the process of identifying symptoms and signs indicative of the disease during screening.15 During the recent outbreak of diphtheria, we modified our ED to create four separate rooms to contain suspected cases. This measure was adapted from that followed in Taiwan and Toronto during an epidemic of severe acute respiratory syndrome (SARS) where a temporary, high-efficient filtration system unit was built outside the ED for screening and isolating SARS patients.7 The “identify, isolate, and inform” method had been used previously during the Middle East respiratory syndrome coronavirus, Ebola virus disease, and measles epidemics.7,11,12,16 By using this method for triage, we were able to measure the response time (“identify” to “isolate” and “isolate” to “inform” time) for all suspected diphtheria cases that presented to the ED (Table 1). ED length of stay was prolonged (24.46 hours) because of the diagnostic process and referral communication at the infection center’s hospital, which took time. Half of the suspected diphtheria patients were adults. This is supported by the findings of a systematic review of diphtheria in children and teenagers.17 Adult patients with a history of diphtheria immunization can get infected later in life because of a decrease in immunity from vaccination with age. Therefore, booster vaccines are needed, especially during an outbreak.18 Diphtheria is often difficult to distinguish from other upper respiratory tract infections as the symptoms are not very specific (Table 2). Nandi reported that the most common form of diphtheria is pharyngeal diphtheria, and 70% of patients have no immunization history.19 The most common symptoms are tonsillar exudate, sore throat, dyspnea, and fever.19 Clinical signs, such as a greyish pseudomembrane, are still very common and enabled easy identification of diphtheria at our center. However, a report from Brazil found that only 52% of diphtheria patients manifest that sign.20 LIMITATIONS This was a single-center study in a referral teaching hospital, and thus did not represent the incidence in the community. Additionally, incomplete data records could have resulted in potential bias. Another limitation was that a follow-up of microbial culture results from probable cases was not done as suspected diphtheria was used as a working diagnosis and is dependent on clinical assessment. CONCLUSION Modified criteria help triage officers detect suspected diphtheria cases and increase the triage response time. The diphtheria package and the four separate rooms in the ED could act as an infection control procedure and facilitate the

1158

Volume 21, no. 5: September 2020

Habib et al.

Triage in the Time of Diphtheria

Table 2. Clinical manifestations among patients with probable diphtheria. Name

SBP

Temp

Sore throat

Fever

Dyspnea

Pseudomembrane

Comorbidity

Emergency unit outcome

130

Referred

100

110

Referred

100

110

36.5

Referred

110

36.7

In-patient

100

110

Referred

Gender

Age

GCS

4 5

6 M 2 15 130 24 110 36.8 + + + Referred GCS, Glasgow Coma Scale score; HR, heart rate (beats per minute); RR, respiratory rate (breaths per minute); SBP, systolic blood pressure; temp, body temperature; referred, sent the patient to infection control hospital for further in-patient management.

improvement of the diagnostic process. Further multicenter studies should be conducted for outbreaks other than diphtheria.

November 1, 2018. 8.

Tosepu R, Gunawan J, Effendy DS, et al. The outbreak of diphtheria in Indonesia. Pan Afr Med J. 2018;31:249.

Address for Correspondence: Author, Hadiki Habib, MD, Emergency Unit, Dr. Cipto Mangunkusumo Hospital, 10430, Jakarta Pusat, Indonesia. Email: hadikihabib@gmail.com

Karyanti MR, Nelwan EJ, Assyidiqie IZ, et al. Diphtheria epidemiology in Indonesia during 2010-2017. Acta Med Indones. 2019;51(3):205-13.

10. World Health Organization. Diphtheria. WHO Vaccine-Preventable Surveillance Standard. 2018:4-6. Available at: https://www.who.int/

immunization/monitoring_surveillance/burden/vpd/standards/en/. Accessed November 1, 2018. 11. Koenig KL. Identify-Isolate-Inform: A modified tool for initial detection and management of Middle East respiratory syndrome patients in the emergency department. West J Emerg Med. 2015;16(5):619-24.

Copyright: ÂŠ 2020 Habib et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

12. Koenig KL, Alassaf W, Burns MJ. Identify-isolate-inform: a tool for initial detection and management of measles patients in the emergency department. West J Emerg Med. 2015;16(2):212-9. 13. The College of Emergency Medicine. Ebola Guidance for Emergency Departments. 2014. Available at: https://www.rcem.ac.uk/docs/ College%20Guidelines/5a.%20Ebola%20guidance%20for%20 emergency%20department.pdf. Accessed November 1, 2018. 14. Evans MI, Galen RS, Britt DW. Principles of screening. Semin Perinatol.

REFERENCES 1.

2005;29(6):364-6. 15. Hiller KM, Stoneking L, Min A, et al. Syndromic surveillance for

Walters RF. Diphtheria presenting in the accident and emergency

Influenza in the emergency department: a systematic review. Plos One.

department. Arch Emerg Med. 1987;4(1):47-51. 2.

2013;8(9):e73832.

Dittman S, Wharton M, Vitek C, et al. Successful control of epidemic diphtheria in the states of the former Union of Soviet Socialist Republics:

3. 4. 5.

16. Centers for Disease Control and Prevention. Identify, isolate, inform:

lessons learned. J Infect Dis. 2000;181:S10-22.

Emergency department evaluation and management for patients under

Saikia L, Nath R, Saikia NJ, et al. A diphtheria outbreak in Assam, India.

investigation (PUIs) for Ebola virus disease (EVD). 2016. Available at:

Se Asian J Trop Med 2010;41(3):647-52.

www.cdcgov/vhf/ebola/clinicians. Accessed November 1, 2018.

Pantukosit P, Arpornsuwan M, Sookananta K. A diphtheria outbreak in

17. Murhekar M. Epidemiology of diphtheria in India, 1996-2016:

Buri Ram, Thailand. Se Asian J Trop Med. 2008;39(4):690-6.

implications for prevention and control. Am J Trop Med Hyg.

Finger F, Funk S, White K, et al. Real-time analysis of the diphtheria

2017;97(2):313-8.

outbreak in forcibly displaced Myanmar nationals in Bangladesh. BMC

18. Tansel O, Ekuklu G, Eker A, et al. Community-based seroepidemiology of diphtheria and tetanus in Edirne, Turkey. Jpn J Infect Dis.

Med. 2019;17(1):58. 6.

2009;62(4):275-8.

Barajas G, Zembower T, Silkaitis Ch, et al. Triage form optimization: travel and infectious disease screening. Am J Infec Control.

19. Nandi R, De M, Browning S, et al. Diphtheria: the patch remains. J Laryngol Otol. 2003;117(10):807-10.

2017;45:S16-93. 7.

Department of Homeland Security. Infectious disease outbreaks.

20. Santos LS, Santâ&#x20AC;&#x2122;anna LO, Ramos JN, et al. Diphtheria outbreak in

Establishing separate triage and assessment facilities. 2003.

Maranhao, Brazil: microbiological, clinical and epidemiological aspects.

Available at: https://www.hsdl.org/?view&did=776154. Accessed

Epidemiol Infect. 2015;143(4):791-8.

Volume 21, no. 5: September 2020

1159

Western Journal of Emergency Medicine

Original Research

Academic Emergency Medicine Faculty Experiences with Racial and Sexual Orientation Discrimination Dave W. Lu, MD, MS* Ava Pierce, MD† Joshua Jauregui, MD, MEd* Sheryl Heron, MD, MPH‡ Michelle D. Lall, MD, MHS‡ Jennifer Mitzman, MD§ Danielle M. McCarthy, MD, MS¶ Nicholas D. Hartman, MD, MPH|| Tania D. Strout, PhD, MS#

*University of Washington School of Medicine, Department of Emergency Medicine, Seattle, Washington † University of Texas Southwestern Medical School, Department of Emergency Medicine, Dallas, Texas ‡ Emory University School of Medicine, Department of Emergency Medicine, Atlanta, Georgia § The Ohio State University College of Medicine, Department of Emergency Medicine, Columbus, Ohio ¶ Northwestern University Feinberg School of Medicine, Department of Emergency Medicine, Chicago, Illinois || Wake Forest School of Medicine, Department of Emergency Medicine, WinstonSalem, North Carolina # Tufts University School of Medicine – Maine Medical Center, Department of Emergency Medicine, Portland, Maine

Section Editor: David Thompson, MD Submission history: Submitted February 28, 2020; Revision received May 21, 2020; Accepted June 29, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.47123

Introduction: Despite the increasing diversity of individuals entering medicine, physicians from racial and sexual minority groups continue to experience bias and discrimination in the workplace. The objective of this study was to determine the current experiences and perceptions of discrimination on the basis of race and sexual orientation among academic emergency medicine (EM) faculty. Methods: We conducted a cross-sectional survey of a convenience sample of EM faculty across six programs. Survey items included the Overt Gender Discrimination at Work (OGDW) Scale adapted for race and sexual orientation, and the frequency and source of experienced and observed discrimination. Group comparisons were made using t-tests or chi-square analyses, and relationships between race or sexual orientation, and we evaluated physicians’ experiences using correlation analyses. Results: A total of 141 out of 352 (40.1%) subjects completed at least a portion of the survey. Non-White physicians reported higher mean racial OGDW scores than their White counterparts (13.4 vs 8.6; 95% confidence interval (CI) for difference, -7.7 – -2.9). Non-White EM faculty were also more likely to report having experienced discriminatory treatment based on race than were White EM faculty (48.0% vs 12.6%; CI for difference, 16.6% – 54.2%), although both groups were equally likely to report having observed race-based discrimination of another physician. EM faculty who identified as sexual minorities reported higher mean sexual minority OGDW scores than their heterosexual counterparts (11.1 vs 7.1; 95% CI for difference, -7.3 – -0.6). There were no significant differences between sexual minority and heterosexual faculty in their reports of experiencing or observing discrimination based on sexual orientation. Conclusion: EM faculty from racial and sexual minority groups perceived more discrimination based on race or sexual orientation in their workplace than their majority counterparts. EM faculty regardless of race or sexual orientation were similar in their observations of discriminatory treatment of another physician based on race or sexual orientation. [West J Emerg Med. 2020;21(5)1160-1169.]

Western Journal of Emergency Medicine

1160

Volume 21, no. 5: September 2020

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

INTRODUCTION Approximately half of all students enrolled in United States medical schools in 2019 self-reported as non-White.1 Despite the increasing diversity of individuals entering medicine, physicians from racial minority groups continue to experience racial bias and discrimination in the workplace, including disparities in career satisfaction, job turnover, federal research grants, and academic promotion.2-7 Many studies have detailed racial discrimination of minority medical students and physicians.2,6,8,9 There is currently little data describing racial discrimination in academic emergency medicine (EM).10-12 A better understanding of the current workplace environment with regard to racial discrimination will aid efforts to promote equity, inclusion, and diversity within the emergency physician workforce. Many physicians who identify as lesbian, gay, bisexual, or transgender (LGBT) also report workplace harassment, social ostracization, and discriminatory treatment.13,14 A significant proportion of LGBT physicians, trainees, and medical students cited concerns of discrimination and harassment for their need to conceal their sexual or gender identities.15-18 LGBT providers’ discomfort with this disclosure is one contributor to their higher levels of distress, burnout, and depression compared to their heterosexual colleagues.14-16,19 Few studies have examined the experiences with workplace discrimination among physicians who identify as sexual minorities.13,14,20 Current data on this understudied provider population will fill an important knowledge gap and inform the aforementioned diversity efforts of both EM and healthcare in general. The objective of this study was to determine the current experiences and perceptions of discrimination by race and sexual orientation among academic EM faculty. We hypothesized that racial and sexual minority emergency physicians would have greater perceptions of and more experiences with discrimination compared to their nonminority colleagues. METHODS Study Design This study was a cross-sectional survey of a convenience sample of EM faculty on their perceptions of and experiences with racial and sexual identity discrimination in the workplace. Data from the same study examining the experiences of EM faculty with workplace gender discrimination have been presented previously.21 Details of the same methodology are summarized and briefly presented here. Study Setting and Population All EM faculty, except the study authors, at six urban, academic training programs were eligible for this study. Study sites were departments of EM located in the following regions: New England (one); Southeast (two); South (one); Midwest (one); West (one). The survey was administered over February– March 2019. Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Studies have shown that physicians from racial and sexual minority groups experience bias and discrimination in the workplace. What was the research question? What are the experiences of academic EM faculty with racial and sexual orientation discrimination in the workplace? What was the major finding of the study? Racial and sexual minority faculty perceived greater discrimination based on race and sexual orientation than their peers. How does this improve population health? There is cultural momentum to confront discrimination based on race and sexual orientation. Efforts to promote equity and diversity within the emergency physician workforce are needed.

Study Protocol An anonymous electronic survey was emailed to all eligible subjects. Subjects consented to the voluntary study by completing the survey on an online, secure platform. Three reminder emails were sent to non-responders. The study was either approved or deemed exempt from review by each site’s institutional review board. Measurements No single, well-validated instrument could be found that satisfactorily measured the multiple aspects of workplace racial and sexual identity discrimination that were of interest. Based on a review of the current literature, we created a 31-item survey composed of questions adapted from surveys used in similar work among populations of physicians from multiple specialties. The survey was pre-tested by EM faculty at five institutions to ensure respondent comprehension. We measured subjects’ perceptions of discrimination using five questions adapted from the Overt Gender Discrimination at Work (OGDW) Scale, an instrument that assesses the perception of gender biases in the workplace, by substituting references to gender with race or sexual identity.22,23 The scale asks: “How strongly do you agree with the following statements about your current place of work?: 1) I have been treated unfairly at work because of my [race or sexual orientation]; 2) The people I work with sometimes make [racist or anti-LGBTQ] statements and/or decisions; 3) I feel that some of the policies and practices of this

1161

Western Journal of Emergency Medicine

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination organization are [racist or anti-LGBTQ]; 4) At work, I sometimes feel that my [race or sexual orientation] is a limitation; and 5) At work, I do not get enough recognition because of my [race or sexual orientation]. Responses are based on a 1-5 Likert scale, with 1 = strongly disagree; 3 = neutral; and 5 = strongly agree. Scores range from 5-25, with higher scores indicating higher perceptions of discrimination. Using questions adapted from prior work,24 we also asked subjects to report the frequency with which they have experienced discriminatory treatment based on their race or sexual orientation, as well as the frequency with which they have observed such discriminatory treatment of another physician. Responses included weekly, monthly, annually, rarely, and never. Those respondents who reported weekly, monthly, or annually to either experiencing discriminatory treatment or having observed discriminatory treatment of another physician based on race or sexual orientation were subsequently asked to identify the source of the discriminatory treatment. Sources included the following: university / medical school / hospital administrator; consulting or admitting physician; EM attending physician; resident physician; medical student; nursing staff; clerical staff; emergency medical services personnel; patient; and other. Subjects were asked to report the frequency with which they had experienced or had observed discriminatory treatment from each source (weekly, monthly, annually, rarely, and never). Developed by Bruce and colleagues,24 these items were designed to categorize the scope, type, and source of gender-based discrimination in medicine. We substituted gender with race or sexual identity for purposes of this study. We collected limited demographic information (Table 1) to prevent easy identification of otherwise anonymous responses and to encourage honest reporting. We did not obtain information linking subjects by study site. Data Analysis Data were collected electronically using Qualtrics (Qualtrics, Provo, UT) software and exported into SPSS for Windows v25.0 (SPSS, Inc., Chicago, IL) for analysis. Continuous variables (eg, age, modified OGDW scores) were examined for normality using visual inspection of histograms, P-P plots, and Pearson’s skewness statistic. We used the t-test for independent samples to compare group means for continuous variables. In addition, Pearson’s chi-square analysis or Fisher’s exact test was employed to compare proportions across categorical variables. In some cases, for example, in categorizing respondents as having experienced or observed racial or sexual orientation discrimination, response categories were collapsed into dichotomous categories a priori to aid in result interpretation (“never” and “rarely” vs “weekly,” “monthly,” and “annually”). Similarly, the anticipated small numbers of racial and sexual minority participants (Table 1) necessitated a priori collapse of these individual response categories into dichotomous variables (eg, non-White vs White, sexual minority vs non-minority) to aid analysis. To assess the strength and direction of relationships Western Journal of Emergency Medicine

Table 1. Participant characteristics in emergency medicine faculty racial and sexual orientation discrimination survey. Characteristics

Participants (N = 141)

Age (years) ≤39

(47.3)

40-49

(37.3)

50-59

(14.5)

≥60

(0.9)

1-5

(25.2)

6-10

(30.5)

11-15

(19.8)

16-20

(11.5)

≥21

(13.0)

Male

(61.1)

Female

(38.9)

104

(79.4)

Black/African American

(4.6)

Hispanic/Latino

(3.8)

Asian/Pacific Islander

(9.2)

American Indian/Alaska Native

(1.5)

Other

(1.5)

Years out of training

Gender

Race/Ethnicity White

Sexual Orientation Straight / Heterosexual

120

(90.9)

Gay / Lesbian / Homosexual

(6.1)

Bisexual

(1.5)

Decline to answer Data are reported as n (%).

between variables, we used Pearson’s correlation coefficient or Spearman’s rho, as appropriate for the data. Partial correlations were also used to evaluate relationships between variables, while controlling for the effect of a covariate (race or gender orientation). Data are presented as frequencies, proportions, means and 95% confidence intervals (CI) around differences between means. All p-values are two-tailed and we accepted an alpha of less than 0.05 as statistically significant. RESULTS A total of 141 out of 352 (40.1%) subjects completed at least a portion of the survey. Respondents were mostly male (n = 80, 61.1%) and White (n = 104, 79.4%) (Table 1). The mean age reported by participants was 41.3 years (range 30-64 years) with the majority of respondents (n = 73, 55.7%) having completed residency training within 10 years. In contrast, according to 2017 data from the Association of American Medical Colleges (AAMC), 72.4% of active emergency

1162

Volume 21, no. 5: September 2020

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

physicians in the US were male,25 with 65.2% under 55 years of age.25 In addition, 2018 AAMC data of full-time US medical school faculty revealed that 63.9% were White, 3.6% were Black or African American, 3.2% were Hispanic or Latino, 19.3% were Asian or Pacific Islander, and 0.2% were American Indian or Alaska Native.26 Although our sample appears to be younger, less male, and more White than national samples, direct comparisons were not possible due to differences in queried age and racial / ethnic categories. In our sample, Cronbach’s alpha for the five items of the OGDW scale was 0.84, suggesting an acceptable level of internal consistency. The mean racial OGDW score for all respondents was 9.4 (standard deviation 4.7; 95%, CI, 8.6–10.2), with nonWhite physicians reporting significantly higher mean racial OGDW scores than their White counterparts (13.4 vs 8.6, respectively; t = -4.502, degrees of freedom [df] = 28.543, p < 0.001, equal variances not assumed; mean difference -5.3, 95% CI for difference, -7.7 – -2.9). Non-White EM faculty were also significantly more likely to report having experienced discriminatory treatment based on race than were White EM faculty (48.0% vs 12.6%, respectively; p < 0.001) (Figure 1). Having experienced discriminatory treatment based on race was significantly associated with higher racial OGDW scores (mean racial OGDW 14.5 vs. 8.5, t = -5.905, df = 31.210, p < 0.001, equal variances not assumed; mean difference -6.0, 95% CI for difference, -8.1 – -3.9). Although non-White physicians were more likely than White physicians to report having experienced race-based discriminatory treatment, the proportion of non-White (50%) and White (29.1%) EM faculty who reported observing race-based discriminatory treatment of another physician was statistically similar (χ2 = 3.832, df = 1, p = 0.050) (Figure 1). Having observed race-based discriminatory treatment of another physician was significantly associated with higher racial OGDW scores (12.4 vs 8.2, t = -5.744, df = 131, p < 0.001; mean difference -4.2; 95% CI for difference, -5.6 – -2.7). Respondent age was not significantly correlated with racial OGDW scores nor observations of discriminatory treatment (r = 0.104, p = 0.454; r = -0.009, p = 0.927, respectively). However, there was an association between age and having experienced race-based discrimination (r = 0.282, p = 0.003), with older respondents reporting more discriminatory experiences. Similarly, respondents’ years in practice were not significantly correlated with racial OGDW scores (r = 0.115, p = 0.189) nor observations of discrimination (r = -0.009, p = 0.922). Yet those respondents with more years in practice reported more racebased discriminatory experiences (r = 0.309, p < 0.001). For those respondents who had experienced discriminatory treatment based on race at least annually, the three most frequent sources of the treatment were patients; university, medical school, or hospital administrators; and consulting or admitting physicians (Figure 2). For those respondents who had observed discriminatory treatment based on race at least annually, the three most frequent sources were patients; nursing staff; and consulting or admitting physicians (Figure 2). Volume 21, no. 5: September 2020

Figure 1. Percentage of participants who (A) experienced or (B) observed race-based discriminatory treatment by racial minority status and frequency.

Cronbach’s alpha for the five items of the OGDW sexual orientation scale was 0.79 in this sample, supporting acceptable internal consistency reliability. The mean sexual minority OGDW score for all participants was 7.1 (SD 3.3, 95% CI, 6.5–7.6), with respondents who identified as sexual minorities reporting significantly higher mean sexual minority OGDW scores than their heterosexual counterparts (11.1 vs 7.1, respectively; t = -2.643, df = 9.461, p = 0.026, equal variances not assumed; mean difference -4.0, 95% CI for difference, -7.3 – -0.6). There were no significant differences between sexual minority and heterosexual respondents in their reports of experiencing discrimination based on sexual orientation, with 10% of minority and 2.5% of heterosexual EM faculty reporting these experiences (p = 0.279) (Figure 3). Having experienced discriminatory treatment based on sexual orientation was

1163

Western Journal of Emergency Medicine

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

Figure 2. Sources of (A) experienced or (B) observed race-based discriminatory treatment by average frequency.

significantly associated with higher OGDW scores (mean sexual minority OGDW 12.5 vs 7.34, t = -3.684, df = 128, p < 0.001; mean difference -5.2, 95% CI for difference, -7.9 â&#x20AC;&#x201C; -2.4). Sexual minority and heterosexual EM faculty were equally likely to report having observed discriminatory treatment of another physician based on sexual orientation (20% vs 10.3%, Ď&#x2021;2 = 0.892, df = 1, p = 0.345) (Figure 3). Having observed discriminatory treatment of another physician based on sexual orientation was also associated with higher sexual minority OGDW scores (mean sexual minority OGDW 10.7 vs 7.1, t = -4.917, df = 127, p < 0.001). There were no consistent relationships between respondent age or years in practice and sexual minority OGDW scores or personal experiences of discriminatory treatment. However, there was an association between both age and years in practice with having observed discrimination of another physician based on sexual orientation (r = 0.227, p = 0.018; r = 0.233, p Western Journal of Emergency Medicine

Lu et al.

Figure 3. Percentage of participants who (A) experienced or (B) observed sexual orientation-based discriminatory treatment by sexual minority status and frequency.

= 0.008, respectively), with older respondents reporting more discriminatory observations. For those respondents who had experienced discriminatory treatment based on sexual orientation at least annually, the three most frequent sources of the discriminatory treatment were university, medical school, or hospital administrators; other EM attending physicians; and nursing staff (Figure 4). For those respondents who had observed discriminatory treatment based on sexual orientation at least annually, the most frequent sources were patients; nursing staff; and other EM attending physicians and residents (Figure 4). DISCUSSION In our study academic emergency physicians who identify as racial or sexual minorities differed significantly when compared to their non-minority colleagues in their perceptions of and experiences with workplace discrimination. Non-

1164

Volume 21, no. 5: September 2020

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

Figure 4. Sources of (A) experienced or (B) observed sexual orientation-based discriminatory treatment by average frequency.

White EM faculty were significantly more likely to report experiencing discriminatory treatment based on their race than their White colleagues. This is consistent with studies among physicians across multiple specialties that showed racial minority physicians were significantly more likely to report having experienced racial discrimination both during their medical careers and in their current workplace, including discrimination related to career advancement, punitive behaviors, practice barriers, and hiring barriers.2,27,28 Although we did not ask respondents to detail these reported instances of discrimination, prior research revealed that physicians from racial minorities frequently described encountering microaggressions in the workplace.29-31 Microaggressions are defined as brief, commonplace, daily, verbal, nonverbal, environmental slights, insults, invalidations, and indignities – intentional or unintentional – directed toward a marginalized group.32 There is literature that details the detrimental mental health effects of microaggressions.31,33 Volume 21, no. 5: September 2020

Microaggressions and other forms of workplace discrimination may also have deleterious effects on physicians’ careers. Previous work demonstrated that experiences with racial discrimination, and not physician race, was significantly associated with higher rates of job turnover, with approximately 25% of racial minority physicians reporting that they have left at least one job due to personally experienced workplace discrimination.6 Within EM, a national survey of faculty found disparities in rank and leadership positions for physicians of under-represented minority groups.12 These data suggest that racial discrimination in the workplace may not only be harmful to the health of minority physicians but it may also significantly impact career trajectories and the retention of a diverse physician workforce.6 Although EM faculty who identified as sexual minorities reported more experiences of discriminatory treatment based on their sexual identity compared to their non-sexual minority peers, this was not statistically significant likely due to the limited numbers of respondents who identified as a sexual minority in our sample. Nonetheless, both racial and sexual minority OGDW scores were significantly higher for racial and sexual minority EM faculty than their nonminority counterparts. As expected, having more experiences with and observations of discriminatory treatment based on race and sexual orientation correlated with higher OGDW scores. Interestingly EM faculty regardless of race or sexual orientation were equally likely to report observing discriminatory treatment of another physician based on race or sexual orientation. So although someone may not have direct experience with racial or sexual orientation discrimination, he or she can identify and recognize it when it occurs with another physician. We did not query respondents about whether they said or did something when they saw these instances of discrimination of another physician, nor did we ask respondents who reported having experienced discrimination whether others intervened on their behalf when there were witnesses. Prior work showed that racial minority physicians were uncomfortable voicing race-related concerns at work,29 and among those who did, minority physicians were more likely to find no change in their situation following submission of a complaint compared to their White colleagues.28 Similarly, in a national survey of surgery residents, none of the LGBT residents who experienced homophobic remarks reported the event due to fears of reprisal, not wanting to create more “trouble,” or a belief that nothing would be done about the event.17 Institutional policies and guidance on how individuals can and should respond to instances of racial or sexual minority discrimination may be helpful. For example, the British Medical Association launched a national campaign in 2001 to inform both patients and providers that racial harassment would not be tolerated in the National Health Service (NHS). This campaign was supplemented by training for all NHS employees focusing on available institutional resources and skills individuals can

1165

Western Journal of Emergency Medicine

EM Faculty Experiences with Racial and Sexual Orientation Discrimination use to respond to instances of racial discrimination in the workplace.34 Other suggested actions to mitigate discriminatory behavior and promote diversity include the identification of best practice efforts to recruit and retain faculty from minority groups, addressing obstacles to advancement, and implementing strategies to promote members of minority groups to positions of leadership.28 In our study EM faculty who have been in practice longer were more likely to report having encountered racist behaviors as well as discrimination based on sexual orientation. Prior studies revealed similar findings with regard to racial discrimination2 and sexual orientation.15 It is unclear whether longer-practicing respondents have had more time in the medical profession to encounter these behaviors, those behaviors were more common in the past, or whether they felt more empowered to report these instances since they may be more established in the field and have less fear of reporting. Future work documenting these trends will be helpful to clarify this question. Sources of experienced or observed discriminatory treatment based on race were most commonly from patients. This is consistent with recent work that demonstrated that a majority of healthcare providers, including physicians, reported offensive comments from patients about their age, gender, race or ethnicity, weight, or other personal traits.35 Physicians from minority backgrounds were more likely to describe discriminatory treatment from patients, with 70% of Black and Asian physicians reporting biased comments from patients.35,36 Patients were also the most common source of observed discriminatory treatment based on sexual orientation. This may stem from underlying racist or homophobic beliefs that exist within our culture and society. For example, in a 2008 survey of patients, about a third of respondents indicated they would change providers if they found out their provider was gay or lesbian,Â and a similar number would change practices if they found out gay or lesbian providers were employed there.37Â Prejudiced comments and behaviors in the healthcare setting are particularly challenging to deal with because physicians have a responsibility to provide appropriate medical care to these patients. Physicians who were subject to discriminatory treatment from patients often experienced an emotional toll that included exhaustion, self-doubt, and cynicism.38 Many of these targeted physicians also expressed a need for training on how to deal with biased patients and for clear institutional policies to guide responses.38,39 The next most common source of experienced or observed discriminatory treatment based on race or sexual orientation was other medical staff. Racism and homophobia within the medical profession have been previously documented.12-14 Prior work found that racial minority faculty were substantially more likely than majority faculty to perceive racial bias in the workplace, with nearly half reporting experiencing racial discrimination by a work superior or colleague.2 Racial minority faculty also described feelings Western Journal of Emergency Medicine

Lu et al.

of isolation and invisibility, disrespect with overt and covert bias/discrimination, different performance expectations, devaluing of research on health disparities, the unfair burden of being identified with affirmative action, and responsibility for diversity efforts.40 Similarly, among medical students who have experienced anti-LGBT discrimination, the most frequent source originated from fellow medical students.41 In a study of surgical residents, the majority of respondents reported having witnessed homophobic remarks by nurses and residents, and about 30% heard similar remarks made by surgical attending physicians.17 Among EM residents specifically, 2.5% of trainees reported feeling uncomfortable with other LGBT physicians, and discriminatory LGBT comments were reported from both fellow residents (17%) and faculty (10%).42 Unfortunately, discriminatory treatment of sexual minority providers is not uncommon after medical school and residency training. Among practicing physicians who identify as LGBT, approximately 10% reported that they were denied referrals from heterosexual colleagues, 15% had been harassed by a colleague, 22% had been socially ostracized, 65% had heard derogatory comments about LGBT individuals, and 27% had witnessed discriminatory treatment of an LGBT coworker.13 Achieving diversity within the physician workforce has been a national priority over the last three decades.43,44 Most recent data demonstrated that 35.7% of full-time faculty in US medical schools identified as non-White, with 9.7% from under-represented minority (URM) groups.26 In EM, approximately 27.0% of full-time faculty identified as non-White, with 10.3% from URM groups. While 7.7% of the 14,254 matriculated US medical students voluntarily identified as lesbian, gay, or bisexual in a 2018 survey, the percentage of practicing physicians who identify as a sexual minority is unknown because neither sexual or gender identity is a required demographic field currently collected by the Accreditation Council for Graduate Medical Education (ACGME) or the AAMC.45 Diversity in healthcare is important because it enhances the quality of training for all students and trainees.46,47 Diversity within medical faculty is particularly significant for the role modeling and mentorship it provides to students and trainees of similar backgrounds.46,47 A diverse physician workforce has also been shown to reduce healthcare disparities in terms of access and quality.43,48 In an effort to promote workforce diversity, both the ACGME and the Liaison Committee on Medical Education have detailed elements that residency and medical education programs should have when they are assessed for accreditation.49,50 Other actions that healthcare organizations can take include bias training, cultural competency and sensitivity training, patient-physician communication training, compensation equity, and workforce diversity initiatives.48 As the US population becomes increasingly diverse,51 issues regarding physician workforce diversity will remain salient in the future.

1166

Volume 21, no. 5: September 2020

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

LIMITATIONS Similar to what we reported previously,21 our study population was a convenience sample of EM faculty at six urban academic sites and our results may not be generalizable to practicing emergency physicians in non-urban and nonacademic settings. Approximately 40% of eligible subjects responded to the survey and response bias may have played a role in our results. We were unable to compare characteristics of respondents with those of non-respondents due to the anonymous nature of our survey methodology. Therefore, we do not know whether more respondents who identify as racial/ethnic or sexual minorities chose to participate in the study, nor do we know whether their experiences with discrimination or harassment played a role in their study participation. The low numbers of respondents who identify as racial/ethnic and sexual minorities also limited our analyses such that dichotomization of data to White and nonWhite as well as sexual-minority and non-sexual minority groups were necessary. The OGDW scale was originally intended to measure perceptions of gender discrimination, and its validity in measuring racial and sexual minority discrimination has not been examined. Although our questions measuring experiences and observations of racial and sexual orientation discrimination were modeled after prior work and have face validity, their reliability as well as criterion and construct validity have not been established. In addition, our results were based on physicians’ self-reports of perceived or experienced discrimination, and thus we were unable to corroborate respondents’ self-reported experiences and observations with racial or sexual orientation discrimination. Nonetheless, researchers have found that self-reports of discrimination are accurate and reliable when validated against other data sources.52 Finally, our study did not use qualitative methods to explore in-depth our respondents’ varied and multi-faceted experiences with workplace discrimination that may provide additional context to our survey findings. CONCLUSION Racial and sexual minority EM faculty perceived more discrimination based on race and sexual orientation, respectively, in their workplace than their non-minority counterparts. Perceptions of discrimination were associated with direct experience with and observations of discriminatory treatment. Although non-White EM faculty were more likely to experience racial discrimination than their White colleagues, both groups were similar in their observations of discriminatory treatment of another physician based on race. Similarly, EM faculty regardless of sexual orientation were similar in their observations of discriminatory treatment of another physician based on sexual orientation. Future work examining the prevalence and characteristics of racial and sexual orientation discrimination in a larger and more diverse sample of emergency physicians is necessary. Volume 21, no. 5: September 2020

Address for Correspondence: Dave W. Lu, MD, MS, University of Washington School of Medicine, Department of Emergency Medicine, 1959 NE Pacific St, Seattle, WA 98195. Email: davelu@uw.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Lu et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1. Association of American Medical Colleges. Total U.S. Medical School Enrollment by Race/Ethnicity and Sex. 2019. Available at: https:// www.aamc.org/system/files/2019-11/2019_FACTS_Table_B-3.pdf. Accessed December 4, 2019. 2. Peterson NB, Friedman RH, Ash AS, et al. Faculty self-reported experience with racial and ethnic discrimination in academic medicine. J Gen Intern Med. 2004;19(3):259-65. 3. Palepu A, Carr PL, Friedman RH, et al. Minority faculty and academic rank in medicine. JAMA. 1998;280(9):767-71. 4. Fang D, Moy E, Colburn L, et al. Racial and ethnic disparities in faculty promotion in academic medicine. JAMA. 2000;284(9):1085-92. 5. Nunez-Smith M, Ciarleglio MM, Sandoval-Schaefer T, et al. Institutional variation in the promotion of racial/ethnic minority faculty at US medical schools. Am J Public Health. 2012;102(5):852-8. 6. Nunez-Smith M, Pilgrim N, Wynia M, et al. Health care workplace discrimination and physician turnover. J Natl Med Assoc. 2009;101(12):1274-82. 7. Ginther DK, Schaffer WT, Schnell J, et al. Race, ethnicity, and NIH research awards. Science. 2011;333(6045):1015-9. 8. Hung R, McClendon J, Henderson A, et al. Student perspectives on diversity and the cultural climate at a U.S. medical school. Acad Med. 2007;82(2):184-92. 9. Dhaliwal JS, Crane LA, Valley MA, et al. Student perspectives on the diversity climate at a U.S. medical school: the need for a broader definition of diversity. BMC Res Notes. 2013;6:154. 10. McNamara RM, Whitley TW, Sanders AB, et al. The extent and effects of abuse and harassment of emergency medicine residents. The SAEM In-service Survey Task Force. Acad Emerg Med. 1995;2(4):293-301. 11. Li SF, Grant K, Bhoj T, et al. Resident experience of abuse and harassment in emergency medicine: ten years later. J Emerg Med. 2010;38(2):248-52. 12. Madsen TE, Linden JA, Rounds K, et al. Current status of gender

1167

Western Journal of Emergency Medicine

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination and racial/ethnic disparities among academic emergency medicine

29. Nunez-Smith M, Curry LA, Berg D, et al. Healthcare workplace

physicians. Acad Emerg Med. 2017;24(10):1182-92.

conversations on race and the perspectives of physicians of African

13. Eliason MJ, Dibble SL, Robertson PA. Lesbian, gay, bisexual, and

descent. J Gen Intern Med. 2008;23(9):1471-6.

transgender (LGBT) physicians’ experiences in the workplace. J

30. Almond AL. Measuring racial microaggression in medical practice.

Homosex. 2011;58(10):1355-71.

Ethn Health. 2019;24(6):589-606.

14. Ko M and Dorri A. Primary care clinician and clinic director

31. Ogunyemi D, Clare C, Astudillo YM, et al. Microaggressions in the

experiences of professional bias, harassment, and discrimination in

learning environment: a systematic review. J Divers High Educ.

an underserved agricultural region of California. JAMA Netw Open.

2020;13(2):97–119.

2019;2(10):e1913535.

32. Sue DW, Capodilupo CM, Torino GC, et al. Racial microaggressions

15. Eliason MJ, Streed C, Jr., Henne M. Coping with stress as an

in everyday life: implications for clinical practice. Am Psychol.

LGBTQ+ health care professional. J Homosex. 2018;65(5):561-78.

2007;62(4):271-86.

16. Lapinski J and Sexton P. Still in the closet: the invisible minority in

33. Sue DW. (2010). The manifestation of racial, gender, and sexual-

medical education. BMC Med Educ. 2014;14:171.

orientation microaggressions. Microaggressions in Everyday Life:

17. Lee KP, Kelz RR, Dube B, et al. Attitude and perceptions of the other

Race, Gender, and Sexual Orientation (p. 3-20). Hoboken: John

underrepresented minority in surgery. J Surg Educ. 2014;71(6):e47-52. 18. Risdon C, Cook D, Willms D. Gay and lesbian physicians in training:

Wiley & Sons, Inc. 34. Collins, M. (2001). Racial harassment in the NHS. In Coker N (Eds.),

a qualitative study. CMAJ. 2000;162(3):331-4.

Racism in Medicine: An Agenda for Change (p. 169-190). London:

19. Przedworski JM, Dovidio JF, Hardeman RR, et al. A comparison of

King’s Fund.

the mental health and well-being of sexual minority and heterosexual

35. WebMD. Credentials don’t shield doctors, nurses from bias.

first-year medical students: a report from the medical student

2017. Available at: https://www.webmd.com/a-to-z-guides/

CHANGE study. Acad Med. 2015;90(5):652-9.

news/20171018/survey-patient-bias-toward-doctors-

20. Chester SD, Ehrenfeld JM, Eckstrand KL. Results of an institutional

nurses?ecd=medscape&print=true. Accessed December 4, 2019.

LGBT climate survey at an academic medical center. LGBT Health.

36. Tedeschi B. 6 in 10 doctors report abusive remarks from patients,

2014;1(4):327-30.

and many get little help coping with the wounds. 2017. Available at:

21. Lu DW, Lall MD, Mitzman J, et al. #MeToo in EM: A multi-center

https://www.statnews.com/2017/10/18/patient-prejudice-wounds-

survey of academic emergency medicine faculty on their experiences with gender discrimination and sexual harassment. West J Emerg

doctors/?s_campaign=medscape. Accessed December 4, 2019. 37. Lee RS, Melhado TV, Chacko KM, et al. The dilemma of disclosure:

Med. 2020;21(2):252-60.

patient perspectives on gay and lesbian providers. J Gen Intern Med.

22. Miller J, Katz D. Gender differences in perception of workplace

2008;23(2):142-7.

experience among anesthesiology residents. J Educ Perioper Med.

38. Wheeler M, de Bourmont S, Paul-Emile K, et al. Physician

2018;20(1):E618.

and trainee experiences with patient bias. JAMA Intern Med.

23. Jagusztyn NE. Perceived Workplace Discrimination as a Mediator

2019;179(12):1678-85.

of the Relationship Between Work Environment and Employee

39. Paul-Emile K, Smith AK, Lo B, et al. Dealing with racist patients. N

Outcomes: Does Minority Status Matter? Graduate Thesis and

Engl J Med. 2016;374(8):708-11.

Dissertations. 2010. Available at: http://scholarcommons.usf.edu/

40. Pololi L, Cooper LA, Carr P. Race, disadvantage and faculty

etd/3657. Accessed January 5, 2019.

experiences in academic medicine. J Gen Intern Med.

24. Bruce AN, Battista A, Plankey MW, et al. Perceptions of gender-

2010;25(12):1363-9.

based discrimination during surgical training and practice. Med Educ

41. Nama N, MacPherson P, Sampson M, et al. Medical students’

Online. 2015;20:25923.

perception of lesbian, gay, bisexual, and transgender (LGBT)

25. Association of American Medical Colleges. Active Physicians by

discrimination in their learning environment and their self-reported

Sex and Specialty, 2017. 2018. Available at: https://www.aamc.org/

comfort level for caring for LGBT patients: a survey study. Med Educ

data-reports/workforce/interactive-data/active-physicians-sex-and-

Online. 2017;22(1):1368850.

specialty-2017. Accessed May 21, 2020.

42. Moll J, Krieger P, Heron SL, et al. Attitudes, behavior, and comfort of

26. Association of American Medical Colleges. U.S. Medical School

emergency medicine residents in caring for LGBT patients: What do

Faculty by Sex, Race/Ethnicity, and Department, 2018. 2019.

we know? AEM Educ Train. 2019;3(2):129-35.

Available at: https://www.aamc.org/download/495054/data/18table16.

43. Institute of Medicine. (2004). Transforming the Institutional Climate to

pdf. Accessed May 21, 2020.

Enhance Diversity In Health Professions. In Smedley BD, Butler AS,

27. Nunez-Smith M, Pilgrim N, Wynia M, et al. Race/ethnicity and

and Bristow LR (Eds.), In the Nation’s Compelling Interest: Ensuring

workplace discrimination: results of a national survey of physicians. J

Diversity in the Healthcare Workforce (p. 143-177). Washington, DC:

Gen Intern Med. 2009;24(11):1198-1204.

National Academies Press.

28. Coombs AA and King RK. Workplace discrimination: experiences of

44. Nickens HW, Ready TP, Petersdorf RG. Project 3000 by 2000.

practicing physicians. J Natl Med Assoc. 2005;97(4):467-77.

Western Journal of Emergency Medicine

Racial and ethnic diversity in U.S. medical schools. N Engl J Med.

1168

Volume 21, no. 5: September 2020

Lu et al.

EM Faculty Experiences with Racial and Sexual Orientation Discrimination

1994;331(7):472-6.

CPRResidency2020.pdf. Accessed May 21, 2020.

45. Association of American Medical Colleges. Matriculating student

50. Liaison Committee on Medical Education. LCME Consensus

questionnaire. 2019. Available at: https://www.aamc.org/system/files/

Statement Related to Satisfaction with Element 3.3, Diversity/Pipeline

reports/1/msq2018report.pdf. Accessed December 4, 2019.

Programs and Partnerships. 2015. Available at: https://ttuhscep.

46. Hurtado S. Linking diversity with the educational and civic missions of

edu/som/curriculum/accreditation/_documents/7_lcme-consensus-

higher education. Rev Higher Ed. 2007;30(2):185-96.

statement-with-intro-element3-3.pdf. Accessed May 21, 2020.

47. Odom KL, Roberts LM, Johnson RL, et al. Exploring obstacles to and

51. United States Census Bureau. The Nationâ&#x20AC;&#x2122;s Population Is Becoming

opportunities for professional success among ethnic minority medical

More Diverse [press release]. 2017. Available at: https://www.census.

students. Acad Med. 2007;82(2):146-53.

gov/newsroom/press-releases/2017/cb17-100.html. Accessed

48. Silver JK, Bean AC, Slocum C, et al. Physician workforce disparities

December 4, 2019.

and patient care: a narrative review. Health Equity. 2019;3(1):360-77.

52. Bodo LD, Oliver ML, Johnson JH, et al. (2002). Surveying racial

49. Accreditation Council for Graduate Medical Education. Common

discrimination: analyses from a multiethnic labor market. Prismatic

Program Requirements (Residency). 2020. Available at: https://

Metropolis: Inequality in Los Angeles (p. 523-560). New York: Russell

www.acgme.org/Portals/0/PFAssets/ProgramRequirements/

Sage Foundation.

Volume 21, no. 5: September 2020

1169

Western Journal of Emergency Medicine

Brief Research Report

Improving Understanding of Screening Questions for Social Risk and Social Need Among Emergency Department Patients Gia Ciccolo, MPH* Alexa Curt, BA* Carlos A. Camargo Jr, MD, DrPH† Margaret Samuels-Kalow, MD, MPhil, MSHP†

*Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts † Harvard Medical School, Massachusetts General Hospital, Department of Emergency Medicine, Boston, Massachusetts

Section Editor: Erik Anderson, MD Submission history: Submitted January 15, 2020; Revision received May 12, 2020; Accepted May 12, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46536

Introduction: With recent healthcare policy changes, including the creation of accountable care organizations, screening for social risks such as food and housing insecurity has become increasingly common in the healthcare system. However, the wide variety of different tools used for screening makes it challenging to compare across systems. In addition, the majority of tools used to measure social risks have only been tested in primary care settings and may not be optimal for emergency department (ED) use. Therefore, the goal of this study was to create a brief social screening tool for use in EDs. Methods: We developed an initial tool using publicly available questions corresponding to the five core categories of the Centers for Medicare & Medicaid Services’ Accountable Health Communities Screening Tool. Iterative cycles of cognitive interviews with purposively sampled participants were performed using a hybrid model of think-aloud and verbal probing to understand/experience answering questions and potential comprehension challenges. After thematic saturation was reached in one cycle, the tool was changed per participant input; cycles were completed until thematic saturation was reached overall. Results: A total of 16 participants (six patient guardians and 10 patients) completed cognitive interviews throughout three cycles. Participant feedback included suggestions for further clarification and simplification of survey questions for improved comprehension. The survey was thus reduced and simplified from 16 questions concerning five domains to 10 questions concerning four domains. Conclusion: We used an iterative cognitive interviewing process to develop a social screening tool for use in EDs. This process demonstrates the importance of patient input to refine questionnaires, and provides a brief screening tool for ED use. [West J Emerg Med. 2020;21(5):1170-1174.]

INTRODUCTION With recent policy changes, including the movement toward accountable care organizations as health delivery systems, there has been an increasing priority placed on both screening for social risk factors,1-3 (defined as the “adverse social conditions that are associated with poor health”)4 and assessing social needs, or the patient’s prioritization of social interventions.4 Although emergency department (ED) patient populations have a high prevalence of social risk,5 optimal

Western Journal of Emergency Medicine

strategies for identifying these factors within the busy and time-limited setting of the ED have yet to be described. Currently, a major barrier to identifying and addressing social risk and social need in EDs is the wide variety of different tools used across studies5 and the lack of a “criterion standard” assessment. As stated in a systematic review of social needs in the ED, “a concise yet comprehensive material needs [social risk and social need] screening tool has not yet been created and validated for ED patients.”5 Efforts to

1170

Volume 21, no. 5: September 2020

Ciccolo et al.

Improving Screening Questions for Social Risk and Need Among ED Patients

standardize screening questions3 have been limited by the copyright restrictions on recommended questions and the total length of the survey, which limits the applicability of most tools in the ED. To date, studies using shorter tools have focused on screening for an individual social risk (eg, food insecurity) rather than assessing the multiple social risks that may affect patients’ health.5-7 Few tools assess both social risks and social needs in one assessment. The lack of consistency in ascertainment techniques and screening tools presents an obstacle to researchers, policy makers, and health systems to design interventions to address social risk and social need in the ED patient population. In developing a social screening tool, it is of critical importance to ensure the screening questions are easy to understand and interpreted in a consistent manner.8 Cognitive interviewing has been proposed as a method for improving the validity of response processes, by allowing the researcher to understand how participants interpret questions.7 The hybrid cognitive interview methodology involves two parts. The first is “think aloud” in which the intention of the interviewer is to guide the participant in providing verbal insight into his/her thought process and understanding while walking through the survey. The second is “verbal probing” in which the participant responds to specific probes concerning understanding of certain areas of the survey. Survey changes informed by this process ensure that the respondent is interpreting and responding to questions as intended in the survey.8 Survey changes based on information from cognitive interviewing data, such as those in this study, are used to clarify the intention of the question to the reader, improve survey comprehension,4 and have been used to optimize other self-report assessment tools.5,9 Modifying a screening tool using this technique can thus increase the ability of the tool to assess risks and needs consistently. The goal of this study was to develop and optimize a social risk and social need screening tool for ED patients that would be both brief and understandable to patients in order to connect them to potential interventions. METHODS We conducted a cognitive interview study with patients and parents of patients in the ED. The initial 16-question survey was developed in both English and Spanish, through a systematic review of existing social risk and social need screening tools using web-based searches and PubMed. Questions were included if they addressed one of the five core domains of the Accountable Health Communities screening tool: 1) food insecurity; 2) housing instability; 3) transportation needs; 4) utility needs; and 5) interpersonal safety.3 This tool was available in the public domain, without copyright restrictions. A cognitive interview guide of open-ended questions (Online Supplement Table 1) was developed by the study team, piloted and refined. Edits were made to best capture Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Emergency department (ED) patient populations have a high prevalence of social risk yet optimal strategies for identifying these factors remain to be described. What was the research question? To develop and optimize a brief and understandable social risk and social need screening tool for ED patients. What was the major finding of the study? Patient informed changes supported a more concise, understandable, and dependable screening tool. How does this improve population health? This screening tool serves to assess social risk and social need in ED patients, enable linkage to relevant resources and improve overall health outcomes.

patient understanding and feedback concerning the proposed questions. The guide was developed in English and Spanish, with interviews in the patient’s choice of language. Iterative cycles of cognitive interviews were performed and recorded. Transcripts were reviewed by investigators, the questionnaire was modified in response, and re-tested in a subsequent cycle of interviews until thematic saturation was achieved and no novel feedback was obtained. Patients were purposively sampled by language spoken (English or Spanish) as well as health literacy level (adequate or limited) in order to reduce bias in representation in the patient population and recruited from a large, urban ED. A bilingual research assistant screened patients for eligibility. Eligibility criteria included adult patients or parent/guardians of pediatric patients, fluency in either English or Spanish, provider approval for approach, and plans for discharge home. We excluded patients on an involuntary mental health hold or with active intoxication. Patient participants completed a brief demographic survey and a health literacy assessment (Newest Vital Sign)10 in either English or Spanish, as well as the cognitive interview. Cognitive interviewing used the “think aloud” and “verbal probing” methods and was employed to understand the participant’s thought process, while going through the survey and comprehension of each survey question. The interviewer received cognitive interview training, and direct feedback following each interview, from a researcher trained

1171

Western Journal of Emergency Medicine

Improving Screening Questions for Social Risk and Need Among ED Patients in the technique. All interviews were recorded and professionally transcribed. All “think aloud” and “verbal probing” responses were reviewed and a cycle was complete when no new responses were given. All changes to the survey were made by consensus of the study team. The study was approved by the Institutional Review Board (IRB) of Partners HealthCare. RESULTS In total, 16 patients completed cognitive interviews over the course of three cycles. Of the 16 participants, four (25%) were primarily Spanish-speaking and five (31%) were categorized as having limited health literacy (Online Supplement Table 2). Based on participant feedback, the survey was reduced and simplified from 16 questions, concerning the five domains of social risk and social need to 10 questions concerning four of the five original domains; neighborhood safety was excluded (Table). Questions concerning neighborhood safety were removed as participants did not have consensus on the meaning of “safety.” For example, some participants felt these questions were referencing crime in the surrounding area [“You can safely walk around your neighborhood without feeling endangered” (adequate health literacy)]. Others felt they were referring to domestic violence [“Que se refiere como que si alguien que vive con un hijo, me lo golpeara, me va a hacer un maltrato agresivo como ¿violencia doméstica?”] [That it refers to like if someone that lives with a son/daughter, were to hit him/her, [or] is going to aggressively mistreat me like domestic violence? (adequate health literacy)] and [“I would assume, there, that you are referring to something that would be more like domestic abuse” (adequate health literacy)]. The lack of consistency in definition caused difficulty in interpreting a positive answer, and determining the appropriate community resource for response. The alternative option of adding further questions to clarify the type of safety need would have made the survey excessively long for ED use and overlapped with existing ED screening protocols for intimate partner violence. For these reasons, in addition to the limited community resources available to address safety, we removed the domain of safety from the question set. For other domains, participants mentioned confusion in the wording and subsequent description of response options [“I think the wording is a little confusing after you’ve just gone through questions that are more direct yes or no…And so I had to switch gears and be like, ‘Oh wait. Okay. So now it’s often true, sometimes true, never true thing…’ (Adequate health literacy)]. Responses were thus simplified to binary options for improved participant understanding and ease in taking the survey [“Again, I just don’t like those sometimes, nevers, often, always. I think people get thrown off with that.” (Adequate health literacy)]. Questions were also removed for similarity to one another Western Journal of Emergency Medicine

Ciccolo et al.

[“3a, I guess it’s fine. 3B is fine as well. They’re both pretty similar” (Adequate health literacy)]. LIMITATIONS Limitations of the study include recruitment limited to those who spoke English or Spanish. In this study, English- and Spanish-speaking patients had similar survey feedback; thus, all changes were made to both versions of the survey. The tool will need to be translated and tested in other languages. Interviews with participants with limited health literacy tended to be shorter with less feedback provided, suggesting that additional techniques to improve cognitive interviewing may be needed in this population. Social risk and social need screening results were not collected from the participants, so we cannot compare perceptions of the question to measured risk or need. We were only able to interview to thematic saturation across the categories of language and health literacy, which were chosen because they were believed to have the greatest impact on patient comprehension of the questions. Additionally, we do not have data on those who declined participation. Therefore, we were unable to compare those who did and did not participate in the study. Because there is no “criterion standard” for social risk and social need assessment,11 a larger study to understand the performance of the questions compared to other measurements of socioeconomic status, social risk, and social need will be the next step to better understand performance of this screening tool. CONCLUSION The cognitive interviews provided important information concerning how to improve an assessment tool for measuring social risk and social need in the ED. After addressing a variety of the study participant concerns (including word choice, response categories, terminology, and question clarity), the final assessment tool (Online Supplement Final Survey) as compared to the original version, is more concise, understandable, and more likely to measure these factors as intended. Importantly, this tool includes both social risk and social need and was developed in both English and Spanish and among patients with a range of health literacy. This short screening tool was developed to be of use to ED clinicians attempting to link patients to community resources, health system administrators developing programs to address adverse social determinants of health, and researchers working to improve care and outcomes for patients with social risk and social need. Given the importance and goal of integration of social determinant measures in clinical practice,12,13 we encourage future work to focus on testing the tool across multiple EDs, comparison with population level data, as well as implementationscience work regarding best practices for screening patients, and connecting them to appropriate community resources to improve health outcomes.

1172

Volume 21, no. 5: September 2020

Ciccolo et al.

Improving Screening Questions for Social Risk and Need Among ED Patients

Table. Social risk and social need survey tool changes through three rounds of cognitive interview. (Abbreviated version of online supplement Table 3). Original survey questions

Final survey questions

Domain 1 1a. In the last month, have you slept outside, in a shelter or in a place not meant for sleeping?

[Removed]*

1b. In the last month, have you had concerns about the condition or quality of your housing?

1a. In the last month, have you had concerns about the condition or quality of your housing?

1c. In the last 12 months, how many times have you or your family moved from one home to another?

[Removed]**

1d. Are you worried that in the next 2 months, you may not have stable housing?

1b. Are you worried that in the next month, you may not have stable housing? *** 1c. Would you like resources to help with housing?∆

Domain 2 2a. Within the past 12 months, you worried whether your food would run out before you got money to buy more.

2a. In the past 12 months, have you worried that your food would run out before you got money to buy more?∆∆

Response Options: Often true, sometimes true, never true, don’t know/refuse

Response Options: “Yes, Often/Sometimes” “No, Never” ∆∆∆

2b. Within the past 12 months, the food you bought just didn’t’ last and you didn’t have money to get more.

2b. In the past 12 months, has your food run out and you didn’t have money to get more?∆∆

Response Options: Often true, sometimes true, never true, don’t know/refuse

Response Options: “Yes, Often/Sometimes” “No, Never” ∆∆∆ 2c. Would you like resources to help with obtaining food?∆

Domain 3 3a. How often is it difficult to get transportation to or from your medical or follow-up appointments?

3a. How often is it difficult to get transportation to or from your medical or follow-up appointments?

Response Options: Does not apply, never, sometimes, often, always

Response Options: “Always/often” “Sometimes/Never” ∆∆∆

3b. How often is it difficult to get transportation to or from your other non-medical activities (work, school etc)?

[Removed]

Response Options: Does not apply, never, sometimes, often, always 3b. Would you like resources to help with transportation?∆ Domain 4 4. In the past 12 months, have you had any utility (electric, gas, water or oil) shut off for not paying your bills?

4a. In the past 12 months, have you worried that any utility (electric, gas, water or oil) would be shut off for not paying your bills?†

4b. Would you like resources to help with paying for your utility bills?∆ -Note that changes to questions from the original to final survey are italicized in the final version. *Respondents reported wanting a more definitive reference for a place “not meant for sleeping.” **Participants reported people may be uncomfortable answering the question. Also other domain questions capture homelessness sufficiently. ***Number of months was changed from 2 to 1 to be consistent with previous questions. ∆ Questions reworded to clarify that interviewer is not providing said “help.” Also, reordered to directly follow questions about specific domain, for improved flow. ∆∆ Reworded because of respondent confusion by question presentation. ∆∆∆ Responses simplified to a binary option as respondents expressed difficulty with multiple options. † Reworded as participants expressed experience “being close” to having a utility shut off. †† The domain was removed, as there was a lack of consensus among participants about the meaning of safety.

Volume 21, no. 5: September 2020

1173

Western Journal of Emergency Medicine

Ciccolo et al.

Improving Screening Questions for Social Risk and Need Among ED Patients Table. Continued.

Original survey questions

Final survey questions

Domain 5 5a. Do you have any concerns about safety in your neighborhood?

[Removed] ††

5b. Are you afraid you might be hurt in your apartment building or house?

[Removed] ††

Need H1. Would you like help with shelter or housing?

[Moved] ∆

H.2 Would you like help with obtaining food?

[Moved] ∆

H.3 Would you like help with transportation?

[Moved] ∆

H.4 Would you like help paying for your utility bills?

[Moved] ∆

H.5 Would you like help regarding your personal or [Moved] ∆ then [Removed] †† neighborhood safety? †† The domain was removed, as there was a lack of consensus among participants about the meaning of safety. ∆ Questions reworded to clarify that interviewer is not providing said “help.” Also, reordered to directly follow questions about specific domain, for improved flow.

health-related social needs in clinical settings: the accountable health

Address for Correspondence: Margaret Samuels-Kalow, MD, MPhil, MSHP, Harvard Medical School, Massachusetts General hospital, Department of Emergency Medicine, 125 Nashua Street, Suite 920, Boston, MA 02114. Email: msamuels-kalow@partners.org.

communities screening tool. 2017. Available at: https://nam.edu/wpcontent/uploads/2017/05/Standardized-Screening-for-Health-RelatedSocial-Needs-in-Clinical-Settings.pdf. Accessed August 14, 2019.

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. This work was supported by the Emergency Medicine Foundation. Dr. SamuelsKalow is supported by the Harvard Catalyst | The Harvard Clinical and Translational Science Center (National Center for Advancing Translational Sciences, National Institutes of Health Award UL 1TR002541) and financial contributions from Harvard University and its affiliated academic healthcare centers. The content is solely the responsibility of the authors and does not necessarily represent the official views of Harvard Catalyst, Harvard University and its affiliated academic healthcare centers, or the National Institutes of Health. There are no conflicts of interest to declare. Copyright: © 2020 Ciccolo et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

4. Alderwick H, Gottlieb LM. Meanings and misunderstandings: a social determinants of health lexicon for health care systems. Milbank Q. 2019;97(2):407-19. 5. Malecha PW, Williams JH, Kunzler NM, et al. Material needs of emergency department patients: a systematic review. Acad Emerg Med. 2018;25(3):330-59. 6. Bottino CJ, Rhodes ET, Kreatsoulas C, et al. Food insecurity screening in pediatric primary care: Can offering referrals help identify families in need? Acad Pediatr. 2017;17(5):497-503. 7. Martel ML, Klein LR, Hager KA, et al. Emergency department experience with novel electronic medical record order for referral to food resources. West J Emerg Med. 2018;19(2):232-7. 8. Ryan K, Gannon-Slater N, Culbertson M. Improving survey methods with cognitive interviews in small- and medium-scale evaluations. Am J Eval. 2012;33(3):414-30. 9. Samuels-Kalow ME, Rhodes KV, Henien M, et al. Development of a patient-centered outcome measure for emergency department asthma patients. Acad Emerg Med. 2017;24(5):511-22. 10. Osborn CY, Weiss BD, Davis TC, et al. Measuring adult literacy in health

REFERENCES

care: performance of the newest vital sign. Am J Health Behav. 2007;31

1. Garg A, Sheldrick RC, Dworkin PH. The inherent fallibility of validated

Suppl 1:S36-46.

screening tools for social determinants of health. Acad Pediatr.

11. Beck AF, Cohen AJ, Colvin JD, et al. Perspectives from the society

2018;18(2):123-4.

for pediatric research: interventions targeting social needs in pediatric

2. Center for Health Care Strategies, Inc. Prioritizing social determinants of health in Medicaid ACO programs: a conversation with two pioneering

clinical care. Pediatr Res. 2018;84(1):10-21. 12. Seymour CK, Griffin C, Holmes SM, et al. Structural differential

states. 2018. Available at: https://www.chcs.org/prioritizing-social-

- a 32 year old man with persistent wrist pain. N Engl J Med.

determinants-health-medicaid-aco-programs-conversation-two-

2018;379(25):2385-8.

pioneering-states/. Accessed June 13, 2018.

13. Metzi JM, Hansen H. Structural competency: theorizing a new medical

3. Billioux A, Verlander K, Anthony S, et al. Standardized screening for

Western Journal of Emergency Medicine

engagement with stigma and inequality. Soc Sci Med. 2014;103:126-33.

1174

Volume 21, no. 5: September 2020

Original Research

Buprenorphine for Opioid Use Disorder in the Emergency Department: A Retrospective Chart Review Kathy T. LeSaint, MD* Brent Klapthor, MD, MS† Ralph C. Wang, MD* Curtis Geier, PharmD‡

*University of California, San Francisco, Department of Emergency Medicine, San Francisco, California † Kaiser Permanente San Diego Medical Center, Department of Emergency Medicine, San Diego, California ‡ University of California, San Francisco, Department of Clinical Pharmacy, San Francisco, California

Section Editor: Erik Anderson, MD Submission history: Submitted January 6, 2020; Revision received April 12, 2020; Accepted June 11, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.46452

Introduction: Emergency care providers routinely treat patients with acute presentations and sequelae of opioid use disorder. An emergency physician and pharmacist implemented a protocol using buprenorphine for the treatment of patients with opioid withdrawal at an academic, Level I trauma center. We describe our experience regarding buprenorphine implementation in the emergency department (ED), characteristics of patients who received buprenorphine, and rates of outpatient follow-up. Methods: We conducted a retrospective chart review of all patients in the ED for whom buprenorphine was administered to treat opioid withdrawal during an 18-month period from January 30, 2017–July 31, 2018. Data extraction of a priori-defined variables was recorded. We used descriptive statistics to characterize the cohort of patients. Results: A total of 77 patients were included for analysis. Thirty-three patients (43%) who received buprenorphine did not present with the chief complaint of opioid withdrawal. Most patients (74%) who received buprenorphine last used heroin, and presented in moderate opioid withdrawal. One case of precipitated withdrawal occurred after buprenorphine administration. Twenty-three (30%) patients received outpatient follow-up. Conclusions: This study underscores the safety of ED-initiated buprenorphine and that buprenorphine administration in the ED is feasible and effective. [West J Emerg Med. 2020;21(5)1175-1181.]

INTRODUCTION As of 2017, 2.1 million Americans were suffering from opioid use disorder (OUD), a condition associated with a 20-fold increase in rates of early death.1-2 While medications with proven benefit exist for the treatment of OUD, their use has not yet become widespread.1,3,4 Emergency departments (ED) are a natural setting for the improvement of this care, as providers routinely treat patients with acute presentations and sequelae of OUD. Calls from the Office of the Surgeon General, the US

Volume 21, no. 5: September 2020

Centers for Disease Control and Prevention, and numerous state governments have specifically suggested this be done through expanded ED use of buprenorphine.5-7 Prior investigators have shown the potential of the ED as a critical point of access for patients suffering from OUD, finding that ED-initiated medications for opioid use disorder (MOUD) is feasible, efficacious, and associated with significantly increased rates of engagement in addiction treatment.8 In the state of California, the California Bridge Program seeks to expand and increase access to MOUD

1175

Western Journal of Emergency Medicine

LeSaint et al.

Buprenorphine for OUD in the ED: A Retrospective Chart Review whereby participating EDs implement protocols to treat patients with OUD and connect those patients with outpatient treatment centers for sustained MOUD.9 However, there remains a relative lack of formal research regarding EDinitiated MOUD protocols and especially so given the scale of the current opioid overdose epidemic.6,10 Differing approaches to the use of buprenorphine for opioid withdrawal in the ED have been proposed (eg, meeting a certain Clinical Opiate Withdrawal Scale [COWS] threshold before medication administration), but description of their use in the clinical setting has thus far been limited.11-13 The goal of this study was to describe our experience regarding implementation of a protocol using buprenorphine for patients presenting with opioid withdrawal in the ED of an academic, Level I trauma center. Specifically, we sought to describe the main adverse event associated with buprenorphine administration (precipitated withdrawal) and rates of linkage to care. MATERIALS AND METHODS No Patient and Public Involvement This research was done without patient involvement. Patients were not invited to comment on the study design nor were any consulted to develop patient-relevant outcomes or interpret the results. Patients were not invited to contribute to the writing or editing of this document for readability or accuracy. This study was approved by the University of California, San Francisco’s institutional review board and informed consent was waived given the minimal risk to subjects involved in a retrospective review of health records. Setting This retrospective cohort study was conducted at Zuckerberg San Francisco General Hospital & Trauma Center (ZSFG). ZSFG is the only public hospital for San Francisco, and is the highest volume ED in the San Francisco Bay Area. Over 73,000 patients are treated annually at ZSFG, and it is the only Level I trauma center for the city and county of San Francisco. From June 2016–May 2018, the ZSFG ED saw 633 unique patients with opioid withdrawal or OUD, although this number is likely an underestimate as only the primary diagnosis is coded by the hospital’s billing services. Implementation and Treatment Protocol Addiction care for ZSFG ED patients with OUD has been growing since early 2017. In January 2017, with minimal funding and technical assistance from the California Health Care Foundation (CHCF), an emergency physician (EP) champion and a clinical pharmacist worked together to implement a protocol using buprenorphine for the treatment of patients with opioid withdrawal. The implementation was part of CHCF’s creation of a project aimed at piloting a treatment model that had previously proven successful in other hospital settings. The EP champion and pharmacist also received Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? Buprenorphine is an effective treatment for opioid use disorder. Description of its use in the emergency department (ED) is limited. What was the research question? What were the characteristics and outcomes of patients who were administered buprenorphine for opioid withdrawal in the ED? What was the major finding of the study? Buprenorphine was administered to 77 patients; 1% had precipitated withdrawal, and 30% received outpatient follow-up. How does this improve population health? Buprenorphine administration in the ED is feasible and can help optimize treatment for patients with opioid use disorder.

coaching and technical assistance from CHCF’s pilot lead. This study pre-dates the now widely known California Bridge Program, which offers formalized guidance regarding ED initiation of MOUD.9 Prior to the initiation of ZSFG’s treatment protocol, the EP champion and clinical pharmacist performed literature reviews and had several meetings with the CHCF pilot lead to develop a thorough understanding of buprenorphine. The EP champion also met with directors of several outpatient clinics and opioid treatment programs to come to an agreement on a single outpatient site where discharged patients could follow up for continued access to buprenorphine. Approval for the protocol implementation was obtained from ED leadership and the hospital-wide Pharmacy & Therapeutics Committee. The two site leads performed teaching of the protocol to ED providers at several on-site faculty meetings, pre-shift nursing team huddles, and at residency conferences. This “start-up” period totaled approximately three months. In addition, for the first six months of the protocol implementation, the EP champion carried a 24/7 pager to provide as-needed technical assistance to all ED providers. For the first few months of the study, patients with suspected opioid withdrawal were assessed using the Short Opiate Withdrawal Scale (SOWS).14 After seven months, however, the protocol was revised to use COWS in an effort to standardize opioid withdrawal assessment in the ED and inpatient units.15 The final ZSFG protocol (Figure 1) was based on the suggested algorithm by Herring et al and as described by the current California Bridge Program.6, 16, 17

1176

Volume 21, no. 5: September 2020

LeSaint et al.

Buprenorphine for OUD in the ED: A Retrospective Chart Review

Figure 1. Buprenorphine treatment protocol in the emergency department at Zuckerberg San Francisco General Hospital. Mg, milligram; ED, emergency department; ZSFG, Zuckerberg San Francisco General Hospital.

Patients who met the threshold for moderate withdrawal (SOWS ≥10; COWS ≥8) were administered buprenorphine (8 milligrams [mg], per protocol). Withdrawal reassessment was then performed 30-60 minutes later. Subsequent dosing of 4-8 mg of buprenorphine was given at the provider’s discretion. All patients who received buprenorphine in the ZSFG ED were given a referral for next business day follow-up at a single outpatient clinic in San Francisco. Patients who were unable to attend outpatient follow-up within 24 hours were given a prescription for buprenorphine until follow-up could be established. Selection of Participants In this study, an ED pharmacist identified all patients for whom buprenorphine was ordered by a clinician via the medication administration record during an 18-month period from January 30, 2017–July 31, 2018. Patients who were not administered the ordered buprenorphine were excluded from the study. Subsequently, we surveyed electronic health records (EHR) to determine the reason buprenorphine was given. Additional patients were excluded if they were not in opioid withdrawal (eg, the patient was on chronic buprenorphine therapy and wanted a dose/refill of their medication). All patients who received buprenorphine for opioid withdrawal in the ED were included in the data analysis. Methods of Measurement and Data Collection and Processing We defined all variables for data collection a priori. A researcher-made data extraction form was developed in accordance with the study objective, which included the patient’s demographic characteristics; date of service; ED length of stay; SOWS or COWS score assessments; dosages of administered buprenorphine; occurrence of precipitated withdrawal; and whether the patient followed up at the designated outpatient site within one week of ED discharge. Follow-up was tracked by reviewing patients’ EHRs for a clinic progress note, as the designated outpatient site uses the Volume 21, no. 5: September 2020

same EHR as the hospital. The study data were collected from the same medical charts by two abstractors (BK and CG). Both abstractors were hospital employees and so were well versed in the EHR. Training included reviewing 10% of all charts together with a third investigator (KL). The investigators met periodically to resolve discrepancies. The third investigator (KL) would settle any unresolved disputes by review of the specific chart. The inter-rater reliability of two variables of interest – prevalence of precipitated withdrawal, and proportion of patients who followed up at one week – were compared for inter-rater agreement. Cohen’s kappa statistic, κ, between our abstractors for the presence of precipitated withdrawal (1.0) and outpatient follow-up (0.85) was excellent (100% and 93.4%, respectively). We collected the data in a secure onsite location and database to avoid the loss of charts and confidential information. Primary Data Analysis We used descriptive statistics to characterize the cohort of patients in our study. We calculated medians and interquartile ranges (IQR) to describe the distribution of skewed numerical values such as age, while categorical variables such as race, chief complaint, and last opioid used were tabulated and reported as percentages. Data and all calculations were evaluated with Microsoft Excel 2011 (Microsoft Corporation, Redmond, WA) and STATA statistical software release 13 (StataCorp LP, College Station, TX). RESULTS During the study period (January 30, 2017–July 31, 2018), buprenorphine was ordered for 102 ED patients. Of those, 77 patients were included for analysis (Figure 2). Table 1 summarizes the baseline characteristics of the cohort. The median age of patients was 37 years (IQR 31-50), and 20 (26%) were female. The largest proportion of patients were White (48%), followed by Black (30%), Latino/Hispanic (20%), and Asian (1%), while race was unknown in 1% of the

Figure 2. Flowchart for patients included in primary analysis of patients for whom buprenorphine was ordered in the emergency department for opioid withdrawal.

1177

Western Journal of Emergency Medicine

Buprenorphine for OUD in the ED: A Retrospective Chart Review Table 1. Baseline characteristics of emergency department patients who received buprenorphine for opioid withdrawal (N = 77). Age in years (median, IQR)

37 (31-50)

Female gender

20 (26%)

Race Asian

1 (1%)

Black

23 (30%)

Latino/Hispanic

15 (20%)

White

37 (48%)

Unknown

1 (1%)

Chief complaint Opioid withdrawal, requesting detoxification

24 (31%)

Gastrointestinal upset

14 (18%)

Requesting buprenorphine

4 (5%)

Generalized pain

2 (3%)

Other

33 (43%)

Last opioid used prior to presentation Heroin

57 (74%)

Buprenorphine

6 (8%)

Methadone

4 (5%)

Oxycodone

4 (5%)

Other

3 (4%)

Unknown

3 (4%)

Time since last opioid use in hours (median, IQR) Methadone

84 (60-276)

Non-methadone opioids

24 (13-48)

ED length of stay in hours (median, IQR)

6.1 (4.7-9.0)

Withdrawal assessment SOWS performed

19 (25%)

COWS performed

43 (56%)

No SOWS or COWS performed

15 (19%)

Disposition Home or self-care

68 (88%)

Jail

9 (12%)

Follow-up at OUD clinic within 1 week Yes

23 (30%)

IQR, interquartile range; SOWS, short opiate withdrawal scale; COWS, Clinical Opiate Withdrawal Scale; OUD, opioid use disorder.

cohort. Thirty-three (43%) patients who received buprenorphine presented to the ED without a chief complaint of opioid withdrawal. Of these patients, 12 presented with complaints of localized pain (eg, arm, back, chest, flank, foot, knee, pelvic, tooth). Others presented with a primary psychiatric complaint such as suicidal ideation or anxiety (N = Western Journal of Emergency Medicine

LeSaint et al.

4); after an assault (N = 3); with abscesses (N = 2;, or generalized weakness (N = 2). Other, less common, chief complaints included altered mentation, foreign body ingestion, rectal bleeding, seizure, and urinary retention. Most patients (N = 57, 74%) who received buprenorphine in the ED last used heroin prior to being diagnosed with opioid withdrawal. Other commonly used opioids prior to presentation included buprenorphine (N = 6, 8%); methadone (N = 4, 5%); oxycodone (N = 4, 5%); and fentanyl (N = 1, 1%). For non-methadone opioids, the median time since last opioid use was 24 hours. Table 2 details buprenorphine administration for the 77 patients in our cohort. Eighteen patients were initially assessed with the SOWS, while 41 patients had an initial COWS. Sixteen patients deemed to be in opioid withdrawal did not receive either assessment scores. There was considerable variation in practice, such as patients continuing to receive buprenorphine despite not receiving additional scoring or not meeting the set thresholds for precipitated withdrawal. However, in a majority of the cases, providers followed the protocol set in place. One case of documented precipitated withdrawal occurred in our cohort. A 54-year-old man with a history of daily heroin insufflation presented to the ED with the chief complaint of nausea and anxiety requesting detoxification from opioids after last having used heroin four hours prior to arrival. He had never received medications for OUD in the past. His triage vital signs were as follows: blood pressure 132/91 millimeters of mercury (mm Hg), heart rate 98 beats per minute (bpm), respiratory rate 18 breaths per minute, and oxygen saturation 99% on room air. The patient was initially seen by an advanced practice provider in our EDâ&#x20AC;&#x2122;s provider in triage area. His physical exam was unremarkable: he had normal vital signs; a soft and non-tender abdomen; and a normal respiratory and cardiovascular examination. His initial COWS score, performed by the treating provider, was 11, and buprenorphine 8 mg was subsequently administered. Within an hour after receiving buprenorphine, he developed restlessness, body aches, runny nose, gastrointestinal upset, anxiety, and gooseflesh skin. He did not have diaphoresis, dilated pupils, tremors, or yawning. He was moved to the main ED and was subsequently treated by an attending physician. His repeat vital signs were: blood pressure 164/88 mm Hg, heart rate 110 bpm, respiratory rate 18 breaths per minute, an oxygen saturation 98% on room air. Over the course of five hours, he was treated with ondansetron, ketorolac, lorazepam, and intravenous fluids. During his hospital course, neither blood tests nor toxicology-specific testing were performed. The patientâ&#x20AC;&#x2122;s repeat COWS score prior to discharge, as performed by his bedside nurse, was 6. Buprenorphine was not continued. The patient was discharged to self-care 9.2 hours after his triage time and did not follow up at the designated outpatient clinic. All patients who received buprenorphine in the ED were

1178

Volume 21, no. 5: September 2020

LeSaint et al.

Buprenorphine for OUD in the ED: A Retrospective Chart Review

Table 2. Assessment scores and buprenorphine dose administered. Buprenorphine 4 mg

Buprenorphine 8 mg

No buprenorphine

1/18 (5.6%)

17/18 (94.4%)

SOWS Initial SOWS (N = 18) <10 (N = 0) 10 or above (N = 18) 2nd SOWS (N = 16) <10 (N = 8)

8/8 (100%)

10 or above (N = 5)

5/5 (100%)

No repeat score (N = 3)

3/3 (100%)

COWS Initial COWS (N = 43) <8 (N = 2) 8 or above (N = 39)

1/2 (50%)

1/39 (2.6%)

38/39 (97.4%)

2/2 (100%)

No initial score (N = 2) 2nd COWS (N = 31) <8 (N = 12) 8 or above (N = 15) No repeat score (N = 4)

1/12 (8.3%)

10/12 (83.3%)

11/15 (73.3%)

2/15 (13.3%)

2/4 (50%)

3/16 (18.75%)

13/16 (81.25%)

Clinical judgment (or some other) 1st dose given (N = 16)

2 dose given (N = 4) 2/4 (50%) 2/4 (50%) mg, milligram; SOWS, short opiate withdrawal scale; COWS, Clinical Opiate Withdrawal Scale. nd

discharged to home or jail. No patients were admitted to inpatient units. The median length of stay was 6.1 hours (IQR 4.7-9.0). Twenty-three (30%) patients followed up at the designated outpatient OUD clinic within one week. LIMITATIONS Over the course of 18 months, the number of patients who were administered buprenorphine was relatively low compared to the number of patients who present to our ED with billing codes reflecting OUD or opioid withdrawal. We did not formally assess the barriers to buprenorphine initiation during this study period. We suspect this relatively low volume was due to the slow uptake of a novel protocol amidst the changing landscape of substance use disorder treatment in emergency medicine. Prior to the implementation of our protocol, many of our clinical staff had not heard of buprenorphine. In the first few months of the study, many of the pager calls and questions received by the EP champion were related to general buprenorphine use and to allay clinician discomfort with using the treatment protocol. Other limitations of this study include those inherent to retrospective studies. For example, the EHR is limited to the completeness of the data recorded (eg, the time since last opioid use was not known in all cases). In addition, we reported that 16 patients received buprenorphine but did not have either a SOWS or COWS assessment performed. Volume 21, no. 5: September 2020

However, because documentation of the assessments was not compulsory in our EHR, some of these patients may have had formal assessments that were not recorded. A final limitation was a transition from SOWS to COWS during the study period, which made it difficult to adequately compare the two. As previously mentioned, the protocol was revised to use COWS in an effort to standardize opioid withdrawal assessment in the ED and inpatient units in our hospital. DISCUSSION Our study adds to the growing body of evidence regarding the feasibility of implementing a protocol to provide buprenorphine to ED patients in opioid withdrawal. As others have shown, buprenorphine remains a safe treatment option with minimal risk for precipitated withdrawal and offers an opportunity to connect these patients to ongoing addiction treatment.11, 12 In addition, we uniquely demonstrate that initiation of buprenorphine administration in the ED setting can be achieved with relatively few start-up resources: a single medical provider and pharmacist championed our protocolâ&#x20AC;&#x2122;s execution. As previously mentioned, Dâ&#x20AC;&#x2122;Onofrio et al first showed the feasibility of ED-initiated buprenorphine, although they did so with the use of research associate-led interviews and referrals.8 Dunkley et al also conducted a retrospective review of 95 patients who received buprenorphine induction

1179

Western Journal of Emergency Medicine

LeSaint et al.

Buprenorphine for OUD in the ED: A Retrospective Chart Review over a five-month period. While this study enrolled a large number of patients, the protocol required consultation from a specialty service, formal assessment of OUD, and admission to a clinical decision/observation unit.12 Therefore, although a robust protocol including consultants and specialists may lead to higher rates of buprenorphine induction in the ED setting, we demonstrate that patients with OUD may still receive adequate treatment without such resources. Lowenstein et al studied barriers and facilitators for ED initiation of buprenorphine and showed that the largest barriers were related to patient social challenges, patient engagement to treatment, and availability of treatment referrals.13 Based on our study, additional barriers to the initiation of buprenorphine for OUD may be unclear chief complaint (eg, not presenting with “opioid withdrawal” or symptoms suggestive of withdrawal), inadequate screening for OUD, long ED lengths of stay, and lack of familiarity with buprenorphine or the protocol in place. This study also underscores the safety of ED-initiated buprenorphine. Despite variations in dosing administration, most patients did not experience significant adverse events. One patient experienced precipitated withdrawal. The patient, while with an initial COWS score of 11, had last used heroin only four hours prior to ED presentation. While a formal assessment using a withdrawal scale was performed, this patient case illustrates the limitations of such scales as a screening tool. The precipitated withdrawal was most likely related to the patient’s very recent use of heroin. In our population, 30% of patients followed up at our protocol’s designated clinic within one week of ED discharge. Our proportion of patients who attended followup was lower than has been seen in other studies.8,11,18 This can be partially attributed to other studies using an opt-in form of OUD clinic referral, selecting for patients who were more ready for pursuing treatment, rather than our referral of all-comers approach of simply providing the clinic location and instructing patients to present for follow-up on the next business day after their ED discharge.18 Other programs evaluated intake and retention over longer time horizons, such as 30 days, although increased lag time between time of referral and date of initial rehab intake is associated with lower rates of follow-up.11,19,20 So, despite less than an ideal follow-up rate in our study, our intervention still very likely led to an overall reduction in days of opioid use, which in itself has been shown to improve health outcomes.19,21 However, this potential benefit must be balanced by the fact that the time period around MOUD discontinuation is associated with increased risk of overdose death, meaning treatment initiation without retention may actually undermine benefits.19 CONCLUSION Given the magnitude of the opioid use epidemic in the United States, more formal studies of this kind are needed to Western Journal of Emergency Medicine

demonstrate appropriate protocols for buprenorphine administration in the ED. In addition, directions for future research include the impact of the current California Bridge Program and qualitative studies to improve the rates of outpatient follow-up. It is in this way that we will be able to most adequately treat the current large proportion of vulnerable patients with opioid use disorder.

Address for Correspondence: Kathy T. LeSaint, MD, Zuckerberg San Francisco General Hospital, Department of Emergency Medicine, 1001 Potrero Avenue, San Francisco, CA 94110. Email: kathy.lesaint@ucsf.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. At the time of this study, Dr. LeSaint was supported by a grant from the California Health Care Foundation. Dr. LeSaint is currently supported by a Zuckerberg San Francisco General Hospital & Trauma Center institutional grant from the Public Health Institute/California Bridge Program. Dr. LeSaint also receives grant funding from the California Department of Health Care Services, Substance Abuse and Mental Health Services Administration, and Sierra Health Foundation related to the treatment of patients with opioid use disorder. All funders had no role in the preparation, review, or approval of this manuscript, nor the decision to submit the manuscript for publication. The contents of the article are solely the responsibility of the authors and do not necessarily represent official views of the funders. . Copyright: © 2020 LeSaint et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1. National Academies of Sciences, Engineering and Medicine. (2019). Introduction. In Leshner AI & Mancher M (Eds.), Medications for Opioid Use Disorder Save Lives (15-32).Washington, D.C.: National Academies Press. 2. Bose J, Hedden SL, Lipari RN, et al. (2016). Summary. In Key Substance Use and Mental Health Indicators in the United States: Results from the 2015 National Survey on Drug Use and Health (13). United States: U.S. Deparment of Health and Human Services. 3. Schuckit MA. Treatment of opioid-use disorders. N Engl J Med. 2016;375(4):357-68. 4. Dugosh K, Abraham A, Seymour B, et al. A systematic review on the use of psychosocial interventions in conjunction with medications for the treatment of opioid addiction. J Addict Med. 2016;10(2):93-103.

1180

Volume 21, no. 5: September 2020

LeSaint et al.

Buprenorphine for OUD in the ED: A Retrospective Chart Review

5. Houry DE, Haegerich TM, Vivolo-Kantor A. Opportunities for

for emergency department initiation of buprenorphine: A physician

prevention and intervention of opioid overdose in the emergency

survey. Am J Emerg Med. 2019;37(9):1787-90.

department. Ann Emerg Med. 2018;71(6):688-90.

14. Gossop M. The development of the short opiate withdrawal scale.

6. Herring AA, Perrone J, Nelson LS. Managing opioid withdrawal in

Addict Behav. 1990;15(5):487-90.

the emergency department with buprenorphine. Ann Emerg Med.

15. Wesson DR and Ling W. The Clinical Opiate Withdrawal Scale

2019;73(5):481-7.

(COWS). J Psychoactive Drugs. 2003;35(2):253-9.

7. U.S. Department of Health & Human Services. Facing addiction in

16. Emergency buprenorphine treatment. Buprenorphine (BUP)

America: the surgeon general’s spotlight on opioids. 2018. Available

Algorithm. 2018. Available at: https://static1.squarespace.

at: https://addiction.surgeongeneral.gov/sites/default/files/Spotlight-

com/static/5adf7f1fa2772cb4f86018b2/t/5afce60070a6ad7647

on-Opioids_09192018.pdf. Accessed May 7, 2019.

ef8893/1526523398244/QUICK+BUP+ALGORITHM+MAY+2018.pdf.

8. D’Onofrio G, O’Connor PG, Pantalon MV, et al. Emergency

Accessed May 7, 2019.

department-initiated buprenorphine/naloxone treatment

17. D’Onofrio G, Chawarski MC, O’Connor PG, et al. Emergency

for opioid dependence: a randomized clinical trial. JAMA.

department-initiated buprenorphine for opioid dependence with

2015;313(16):1636-44.

continuation in primary care: outcomes during and after intervention.

9. California Bridge Program. Available at: http://www.bridgetotreatment. org. Accessed April 13, 2020.

J Gen Intern Med. 2017;32(6):660-6. 18. Liebschutz JM, Crooks D, Herman D, et al. Buprenorphine treatment

10. Love JS, Perrone J, Nelson LS. Should buprenorphine be

for hospitalized opioid-dependent patients. JAMA Intern Med.

administered to patients with opioid withdrawal in the emergency

2014;174(8):1369-76.

department?. Ann Emerg Med. 2018;72(1):26-8.

19. Larochelle MR, Bernson D, Land T, et al. Medication for opioid use

11. Kaucher KA, Caruso EH, Sungar G, et al. [Ahead of Print].

disorder after nonfatal opioid overdose and association with mortality:

Evaluation of an emergency department buprenorphine induction and medication-assisted treatment referral program. Am J Emerg Med.

A cohort study. Ann Intern Med. 2018;169(3):137-45. 20. Pollini RA, McCall L, Mehta SH, et al. Non-fatal overdose and

July 30, 2019.

subsequent drug treatment among injection drug users. Drug Alcohol

12. Dunkley CA, Carpenter JE, Murray BP, et al. Retrospective review of a novel approach to buprenorphine induction in the emergency

Depend. 2006;83(2):104-10. 21. Stein M, Anderson B. Injection frequency mediates health service use

department. J Emerg Med. 2019;57(2):181-6.

among persons with a history of drug injection. Drug Alcohol Depend.

13. Lowenstein M, Kilaru A, Perrone J, et al. Barriers and facilitators

Volume 21, no. 5: September 2020

2003;70(2):159-68.

1181

Western Journal of Emergency Medicine

Original Research

Post-traumatic Stress Disorder in Family-witnessed Resuscitation of Emergency Department Patients Mert Erogul, MD Antonios Likourezos, MA, MPH Jodee Meddy, DO Victoria Terentiev, MD Dâ&#x20AC;&#x2122;anna Davydkina, MPA Ralph Monfort, BS Illya Pushkar, MPH Thomas Vu, MD Madhu Achalla, MD Christian Fromm, MD John Marshall, MD

Maimonides Medical Center, Department of Emergency Medicine, Brooklyn, New York

Section Editor: Erin Dehon, PhD Submission history: Submitted December 25, 2019; Revision received April 30, 2020; Accepted June 22, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.46300.

Introduction: Family presence during emergency resuscitations is increasingly common, but the question remains whether the practice results in psychological harm to the witness. We examine whether family members who witness resuscitations have increased post-traumatic stress disorder (PTSD) symptoms at one month following the event. Methods: We identified family members of critically ill patients via our emergency department (ED) electronic health record. Patients were selected based on their geographic triage to an ED critical care room. Family members were called a median of one month post-event and administered the Impact of Event Scale-Revised (IES-R), a 22-item validated scale that measures post-traumatic distress symptoms and correlates closely with Diagnostic and Statistical Manual of Mental DisordersIV criteria for post-traumatic stress disorder (PTSD). Family members were placed into two groups based on whether they stated they had witnessed the resuscitation (FWR group) or not witnessed the resuscitation (FNWR group). Data analyses included chi-square test, independent sample t-test, and linear regression controlling for gender and age. Results: A convenience sample of 423 family members responded to the phone interview: 250 FWR and 173 FNWR. The FWR group had significantly higher mean total IES-R scores: 30.4 vs 25.6 (95% confidence interval [CI], -8.73 to -0.75; P<.05). Additionally, the FWR group had significantly higher mean score for the subscales of avoidance (10.6 vs 8.1; 95% CI, -4.25 to -0.94; P<.005) and a trend toward higher score for the subscale of intrusion (13.0 vs 11.4; 95% CI, -3.38 to .028; P = .054). No statistical significant difference was noted between the groups in the subscale of hyperarousal (6.95 vs 6.02; 95% CI, -2.08 to 0.22; P=.121). All findings were consistent after controlling for age, gender, and immediate family member (spouse, parent, children, and grandchildren). Conclusion: Our results suggest that family members who witness ED resuscitations may be at increased risk of PTSD symptoms at one month. This is the first study that examines the effects of family visitation for an unsorted population of very sick patients who would typically be seen in the critical care section of a busy ED. [West J Emerg Med. 2020;21(5)1182-1187.]

Western Journal of Emergency Medicine

1182

Volume 21, no. 5: September 2020

Erogul et al.

PTSD in Family-witnessed Resuscitation of ED Patients

INTRODUCTION The issue of family-witnessed resuscitation (FWR) has been debated since the late eighties when doctors at Foote Hospital in Michigan published their observations suggesting a benefit to family members who stayed to watch their critically ill loved ones getting cardiopulmonary resuscitation (CPR).1 Since then, authors have considered the question from medicolegal and ethical perspectives, from practical standpoints related to physician stress and potential interference with resuscitation and from the angle of potential harm or benefit to the family members themselves.2-7 Even as this debate has continued, the practice has spread to many emergency departments (ED) in North America and worldwide. Numerous professional bodies have endorsed FWR.7-12 Despite the growing consensus, there remains the question of the remote psychological cost of this experience to the witness. In particular, concern has been raised about the development of post-traumatic stress disorder (PTSD) in witnesses of resuscitation.13-16 PTSD is a syndrome of hyperarousal, vivid re-living of events, and inability to achieve a state of safety that results from, among other things, exposure to the actual or threatened death of a family member or friend in a way that is “accidental or violent.”17,18 Symptoms must be present for a month and the pathophysiology is thought to relate to memory being encoded in alternative, more persistent pathways that resist being extinguished. The literature supports the development of PTSD immediately or up to months after witnessing traumatic events, but whether or not this happens is related to multiple factors including the victim’s interpretation of the event, their pre-existing beliefs, and prior experiences.19 ED resuscitations can be brutal and memorable events in which the human form is treated violently or disfigured by invasive procedures; so it stands to reason that they might constitute the kind of traumatic event that can trigger PTSD in the brain of the witness. The experimental literature about FWR and the development of PTSD is conflicting and not of high quality.13-16 Moreover, all of the studies thus far have limited themselves to the subset of patients getting CPR. There is no literature about the much broader category of critically patients who are resuscitated but do not receive CPR, yet family members frequently ask to be present for the presorted range of ED resuscitations, even if they do not involve CPR. In view of the noted gaps, the divergent outcomes, and uneven quality of the current literature, there is a clear justification for further study. Our objective was to assess whether family and close contacts who witness ED resuscitations exhibit an increase in PTSD symptoms in the months following an event. METHODS Study Design and Setting We conducted a prospective cross-sectional study, from July 2011–June 2016, comparing family members who Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Family-witnessed resuscitation has become accepted in the emergency department (ED) despite the question of remote psychological cost of this experience to the witness. What was the research question? Do family who witness ED resuscitations exhibit an increase in PTSD symptoms in the months following an event? What was the major finding of the study? There was an association between family presence during resuscitation and increased PTSD symptoms at one month. How does this improve population health? Adding to the growing body of literature about family-witnessed resuscitation moves us closer to sound recommendations for family of critically ill ED patients.

witnessed resuscitation (FWR group) and family members who did not witness resuscitation (FNWR group) to assess post-traumatic distress symptoms. We conducted this study at a 711-bed urban, community teaching hospital with an annual ED census of greater than 120,000 visits. The goal was to enroll 150 participants in each arm. Patient screening, enrollment, and data collection were performed by study investigators. The hospital’s institutional review board approved the study. Selection of Participants At our institution, critically ill patients who need immediate, lifesaving attention are treated in a specific resuscitation room. We generated a list of patients who were treated in the resuscitation room and identified family members of those patients via our ED electronic health record. We included family members of critically ill patients aged 18 and older who underwent resuscitation in the ED. We excluded non-English speaking patients and those whose primary resuscitations occurred out of hospital. We did not capture whether resuscitations were medical or related to trauma, or whether or not they were successful. Patients did not require CPR to be entered into the study, although many of them did get CPR, were intubated, or received other procedures in a timesensitive manner. Family members were given the choice to be present in the room during all resuscitations and were called via telephone one month after the event. They were placed into two groups based on whether they self-reported that they had

1183

Western Journal of Emergency Medicine

Erogul et al.

PTSD in Family-witnessed Resuscitation of ED Patients witnessed the resuscitation (FWR group) or had not witnessed the resuscitation (FNWR group). Data Collection The phone interviewers, consisting of college-educated and trained research assistants, administered the Impact of Event Scale-Revised (IES-R), a reliable 22-item validated scale that measures post-traumatic distress symptoms and correlates closely with the Diagnostic and Statistical Manual of Mental Disorders-IV criteria for PTSD.20 It is best suited for recent, not remote, traumatic events. Interviewers made up to three attempts to contact study subjects between the hours of 8 am and 6 pm Monday through Friday. The total IES-R score ranges from 0 to 88, with scores >24 associated with clinical concern for PTSD or partial PTSD, >33 with probable PTSD, and >39 with PTSD severe enough to suppress immune system function, even 10 years after the impact. The IES-R has three subscales that correspond to the classic features of PTSD, specifically the alternation between avoidance (deliberate efforts not to think about the event) and intrusion (nightmares, involuntary thoughts of the event, and interfering feelings). A third subscale assesses hyperarousal, which relates to persistence of sympathomimetic excitability such as feeling on guard, and experiencing sweats and palpitations. Subjects were asked how distressing each of 22 components of the IES-R was in the last week with respect to their family member’s illness event, getting from zero to four points depending on the severity of the distress. Composite scores above 24 suggest a clinical concern for PTSD, whereas scores above 37 are associated with symptoms profound enough to cause immune dysfunction. Outcomes Measures The primary outcome included a difference in total IES-R score between the groups. Secondary outcomes included a comparative difference between the three subscale scores of intrusion, avoidance, and hyperarousal. Data Analyses The investigators recorded all data on data sheets (separate from clinical data), entered them into Microsoft Excel (Microsoft Corporation, Redmond, WA), and then imported the data into SPSS 24.0 (IBM Corp, Armonk, NY) for statistical analyses. Data were described in terms of mean (standard deviation [SD]) or 95% confidence limits for continuous variables, and frequency (percentage) for categorical variables. Data analyses included chi-square test, independent sample t-test, and linear regression controlling for gender, age, and immediate family member (spouse, parent, children, and grandchildren). A P-value <.05 denoted statistical significance between the groups. RESULTS An estimated 3000 family members qualified for the study, of which approximately 1200 were reached by the phone Western Journal of Emergency Medicine

interviewers. A convenience sample of 423 family members completed the IES-R, a response rate of 35%. The median duration between traumatic event and interview was 33 days (range 18-67), and there was no difference between groups in this regard. Of the 423 family members who completed the survey, 250 self-reported in the FWR and 173 the FNWR group. Family members consisted of immediate family members (children, parents, spouses, or grandchildren) as well as close friends, cousins, nieces, nephews, aunts, daily caretakers, and in-laws. The FWR group had more immediate family members (82.7% vs. 73.4%; P<.05). The mean age for the FWR group was 53.5 (±14.7) and for the FNWR group was 53.4 (±15.1; P=.905). The FWR group consisted of 71.7% females and FNWR of 63.2% females (P=.069) (Table 1). The FWR group had significantly higher mean total IES-R scores: 30.4 vs 25.6 (95% confidence interval [CI], -8.73 to -0.75; P<.02). Additionally, the FWR group had a significantly higher mean score for the subscales of avoidance (10.6 v. 8.1; 95% CI: -4.25 to -0.94; P<.005) and a trend toward a higher score for the subscale of intrusion (13.0 vs 11.4; 95% CI: -3.38 to .028; P=.054). No statistically significant difference was found between the groups in the subscale of hyperarousal (6.95 v. 6.02; 95% CI, -2.08 to 0.22; P =.121). All findings were consistent after controlling for age, gender, and immediate family member (spouse, parent, children, and grandchildren) in a linear regression equation (Table 2). DISCUSSION In our prospective cross-sectional study, we found an association between family presence during resuscitation and increased PTSD symptoms at one month as measured by the IES-R. The IES-R is reported as a composite score with three subscales. In our study, we found significant differences in the composite scores as well as in the avoidance subscale. There was also a trend toward significance for the intrusion subscale. Prior experimental studies as detailed below employed the IES-R as a measure of PTSD, although the IES-R does not capture hyperarousal symptoms, thus limiting our ability to compare and generalize with those studies. Robinson et al13 conducted a small, prospective, semirandomized survey of 13 family members who had witnessed ED resuscitation of cardiac arrest after being given the choice to do so, and 12 who were not given the choice to

Table 1. Participant characteristics. FWR

FNWR

% Male

28.3

46.8

% Female

71.7

63.2

% Immediate family

82.7

73.4

33 (18-67)

33 (20-67)

Days from resuscitation to interview

FWR, family-witnessed resuscitation; FNWR, family not witnessed resuscitation.

1184

Volume 21, no. 5: September 2020

Erogul et al.

PTSD in Family-witnessed Resuscitation of ED Patients

Table 2. IRS-R Total mean scores and subscales intrusion, avoidance, and hyperarousal No FWR

FWR

Mean difference

95% CI

P-value

Mean total score

25.6

30.4

-4.74

-8.73 to -0.75

.020*

Mean intrusion score

11.4

13.0

-1.68

-3.38 to 0.028

.054

Mean avoidance

8.24

10.6

-2.33

-3.89 to -0.76

.003**

Mean hyperarousal

6.02

6.95

-0.93

-2.08 to 0.22

.121

*Statistical significant difference between the groups at P<.05; **Statistical significant difference between the groups at P<.005. FWR, family-witnessed resuscitation; FNWR, family not witnessed resuscitation; CI, confidence interval.

witness CPR (and who additionally did not ask to witness). Successful resuscitations were excluded. The family members were queried using the IES-R at one month and six months by mail. The study was stopped early because the participating staff became convinced of the value of providing family access to resuscitations. It was limited by the use of a multiplicity of outcome measures and found no significant difference between the two groups in the development of PTSD symptoms. Compton et al14 conducted a small, prospective, nonrandomized cohort study of 54 family members of patients who had undergone failed resuscitation of out-of-hospital arrest. The family members were surveyed using the PTSD Symptom Scale Interview (PSSI) by telephone a month after the ED visit. The 34 who had witnessed the CPR had considerably higher (almost double) PSSI measures than the 20 who had not. The study’s limitations included lack of randomization, blinding, participant decay, and differing characteristics between the study groups. Compton et al 15 prospectively compared two hospitals, one at which families were permitted to witness non-traumatic resuscitations (#24) and another at which families were not (#41). The subjects were interviewed by telephone at one and two months and evaluated using the PTSD-self report and the Center for Epidemiologic Studies Depression scale tools. The only significant difference in bereavement-related PTSD symptoms between the groups was an increase in arousal in the FWR group at two months. There were pre-intervention differences between the two groups, possibly related to cachement populations of the respective hospitals. The study was small and not randomized.

The one randomized and controlled study to date was conducted by Jabre et al16 in Paris. In their health system, ambulances known as “mobile ICUs” [intensive care units] are staffed by emergency physicians. Half of the local ambulance teams gave family members the choice to be present and the other half did not. Ninety days after the resuscitation, a blinded psychologist conducted a telephone questionnaire that included the IES-R. They followed 570 family members for one year to compare psychological outcomes in those who had been given the option to be present during resuscitation (n = 239) and those who had not, as well as between those who had witnessed resuscitation and those who had not. The authors found that family members who had not witnessed resuscitation displayed significantly more PTSD symptoms than those who had (IES 26 to 21, p 0.007) (Table 3). To summarize the findings of the above studies, one showed more PTSD symptoms with FWR, one showed fewer PTSD symptoms with FWR, and two showed no meaningful difference. Furthermore, there is a diversity of studied scenarios. Robinson excluded successful resuscitations from analysis. In another case, only cardiac arrest or violent trauma were included. Notably, all of these studies were limited to patients receiving CPR. Yet family members frequently ask to be present for all manner of ED resuscitations, even if they do not involve CPR. To our knowledge there is no data for family members who witness the type of presorted range of resuscitations that occur in the ED. Our findings support the hypothesis that witnessing resuscitation, which is often sudden, unexpected, violent, or frightening to the observer, may lead to PTSD symptoms. This

Table 3. Prior experimental studies of post-traumatic stress disorder in family-witnessed resuscitation. Study

Inclusion criteria

PTSD outcome measures

ED cardiac arrest / multi-trauma

IES1

Compton, 200914

Out of hospital cardiac arrest

PSSI2

Compton, 201115

ED cardiac arrest

PSS-SR3

Robinson, 1998

Jabre, 2014 Out of hospital cardiac arrest IES1 PSSI, Post-Traumatic Stress Disorder Symptom Scale Interview; ED, emergency department; IES, Impact of Events Scale; PSS-SR, post-traumatic stress disorder Symptom Scale—Self Report. 16

Volume 21, no. 5: September 2020

1185

Western Journal of Emergency Medicine

Erogul et al.

PTSD in Family-witnessed Resuscitation of ED Patients is the first study that examines the effect of family visitation in an unsorted population of very sick patients that would typically be seen in the critical care section of a busy ED. In contrast to the reviewed studies that excluded patients who survived, we did not track whether or not the resuscitations were successful. It can be argued that considering all-comers (successful resuscitation and not) is more relevant from a policy perspective as it is difficult to tell in advance whether or not a resuscitation will be effective and the important question is related to the strategy of how to manage family members of a presorted population of all critically ill patients. In studies limited to patients getting CPR – and particularly in those limited to patient deaths – family members may display a different pattern of symptoms intertwined in complicated ways with their grief reaction. The complex interrelationship between PTSD and bereavement in this context deserves further consideration. In our study, the composite score and the avoidance subscore showed significant changes in the FWR group. The two other subscales did not show any changes. In the four prior cited studies, Compton (2008) suggested increased hyperarousal their FWR group and Compton (2011) showed an increase in hyperarousal at 60 days but not at 30 days. The other studies used the IES tool, which did not measure symptoms of hyperarousal. Our study adds to the limited body of literature on the topic of PTSD symptoms in FWR in the ED. It is too soon to make evidence-grounded recommendations about whether or not families should be encouraged or permitted to witness resuscitations. For starters, the evidence is conflicting and incomplete – there is no consistent message in the literature. Moreover, a potential increase in PTSD symptoms is only one of many outcome measures that would be relevant to such a recommendation. In deciding whether to permit family members to witness resuscitations, one must weigh a slight increase in the development of PTSD symptoms as evidenced in our study against potential psychological benefits that are as yet unstudied. For instance, there is an emerging body of literature suggesting that PTSD symptoms such as rumination and intrusive thoughts may be a bridge to post-traumatic growth.21 Other measures such as development of complicated grief, acute stress disorder, and depression would also be relevant. Studies show that when asked, families overwhelmingly prefer to be given the option to be present.22-25 To limit autonomy based on paternalistic impulse is to defy the prevailing momentum of increased transparency in medicine set in motion during the era of patient rights fifty years ago, and should require a very high bar of evidence. At this point, there is still no reason to disagree with authors who advocate giving family members an informed choice to be present during resuscitations.26 LIMITATIONS The primary limitation of our design is that the sorting of subjects into FWR and FNWR groups was not randomized but rather left to the discretion of the participants. Subjects who Western Journal of Emergency Medicine

chose to witness resuscitations may have done so because of closer bonds with their family member, and were therefore more likely to develop PTSD, potentially skewing the results. Additionally, the FWR group was somewhat more likely to include immediate family, although on regression analysis the presence of immediate family did not correlate with PTSD symptoms. There was a self-selection bias of the convenience sample of family members who agreed to participate. Finally, our analysis could have benefited from a reassessment of outcome measures at a later time point, as patients can develop PTSD symptoms even after many months. CONCLUSION In our prospective cross-sectional study of an unsorted population of ED resuscitations, there was an association between family presence during resuscitation and increased PTSD symptoms as measured by the IES-R scale. There are numerous relevant questions that remain unanswered, and our suggestions for future research would include a longer follow-up period to assess later development of symptoms as well as inclusion of measures of post-traumatic psychological growth. There is also the question of whether the presence of social workers or chaplains or other informed guides might affect the outcome of witnessed resuscitations. Finally, it would be important to replicate this study in a pediatric patient population.

Address for Correspondence: Mert Erogul, MD, Maimonides Medical Center, Department of Emergency Medicine, 4802 10th Avenue, Brooklyn, NY 11209. Email: MErogul@maimonidesmed.org. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Erogul et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1.

Doyle CJ, Post H, Burney RE, et al. Family participation during resuscitation: an option. Ann Emerg Med. 1987;16(6):673-5.

Mian P, Warchal S, Whitney S, et al. Impact of a multifaceted intervention on nurses’ and physicians’ attitudes and behaviors toward family presence during resuscitation. Crit Care Nurse. 2007;27(1):52-61.

1186

Tsai E. Should family members be present during cardiopulmonary resuscitation? N Engl J Med. 2002;346(13):1019-21.

Volume 21, no. 5: September 2020

Erogul et al. 4. 5.

PTSD in Family-witnessed Resuscitation of ED Patients

DeWitt, S. Should family-witnessed resuscitation become our

14. Compton S, Grace H, Madgy A, et al. Post-traumatic stress

standard? J Emerg Med. 2015;49(4):500-2.

disorder symptomology associated with witnessing unsuccessful

Brasel KJ, Entwistle JW, Sade RM. Should family presence be

out-of-hospital cardiopulmonary resuscitation. Acad Emerg Med.

allowed during cardiopulmonary resuscitation? Ann Thorac Surg.

2009;16(3):226-9.

2016;102(5):1438–43. 6. 7.

15. Compton S, Levy P, Griffin M, et al. Family-witnessed resuscitation:

Boyd R. Witnessed resuscitation by relatives. Resuscitation.

bereavement outcomes in an urban environment. J Palliat Med. 2011

2000;43(3):171-6.

Jun;14(6):715-21.

Lippert FK, Raffay V, Georgiou M, et al. European Resuscitation

16. Jabre P, Belpomme V, Azoulay E, et al. Family presence during

Council Guidelines for Resuscitation 2010 Section 10. The

cardiopulmonary resuscitation. N Engl J Med. 2013;368(11):1008-18.

ethics of resuscitation and end-of-life decisions. Resuscitation. 8.

17. Shalev AY, Marmar CR. (2017). Post-traumatic Stress Disoder

2010;81(10):1445-51.

In Sadock BJ, Sadock VA, Ruiz P (Eds.), Kaplan and Sadock’s

Morrison LJ, Kierzek G, Diekema DS, et al. Part 3: ethics: 2010

Comprehensive Textbook of Psychiatry, 10th ed. (1812-1826).

American Heart Association Guidelines for Cardiopulmonary

Philadelphia, PA: Wolters Kluwer/Lippincott Williams & Wilkins.

Resuscitation and Emergency Cardiovascular Care. Circulation.

18. American Psychiatric Association. (2013). Post-traumatic Stress

2010;122(18 Suppl 3):S665-75. 9.

Disorder. Diagnostic and Statistical Manual Of Mental Disorders, 5th

Oczkowski SJ, Mazzetti I, Cupido C, et al. Family presence during

ed. (271-280). Arlington: American Psychiatric Association.

resuscitation: a Canadian critical care society position paper. Can

19. Yehuda R. Post-traumatic stress disorder. N Engl J Med.

Respir J. 2015;22(4):201-5.

2002;346(2):108-14.

10. Australian Resuscitation Council. Guideline 10.6 Family Presence

20. Weiss, DS. The impact of event scale: revised. Cross-cultural

during Resuscitation. 2016. Available at: https://secureservercdn.

assessment of psychological trauma and PTSD. Boston: Springer; 2007.

net/184.168.47.225/777.066.myftpupload.com/download/section_10/

21. Tedeschi RG, Calhoun LG. Posttraumatic growth: conceptual

anzcor-guideline-10-6-family-presence-during-resuscitation-

foundations and empirical evidence. Psychological Inq.

aug-2016.pdf; Accessed April 23, 2020.

2004;15(1):1-18.

11. Emergency Nurses Association. Clinical Practice Guideline:

22. Barratt F, Wallis DN. Relatives in the resuscitation room: their point of

Family Presence During Invasive Procedures and

view. J Accid Emerg Med. 1998;15(2):109–11.

Resuscitation. 2017. Available at: https://www.ena.org/docs/

23. Meyers TA, Eichhorn DJ, Guzzetta CE. Do families want to be present

default-source/resource-library/practice-resources/cpg/

during CPR? A retrospective survey. J Emerg Nurs. 1998;24(5):400-5.

familypresencecpg3eaabb7cf0414584ac2291feba3be481.

24. Booth MG, Woolrich L, Kinsella J. Family witnessed resuscitation in

pdf?sfvrsn=9c167fc6_18; Accessed April 23, 2020.

UK emergency departments: a survey of practice. Eur J Anaesthesiol.

12. Bossaert, LL, Perkins GD, Askitopoulou H, et al. European

2004;21(9):725-8.

Resuscitation Council Guidelines for Resuscitation 2015 Section 11.

25. Boie ET, Moore GP, Brummett C, et al. Do parents want to be

The ethics of resuscitation and end-of-life decisions. Resuscitation.

present during invasive procedures performed on their children in the

2015;95:302-11.

emergency department? A survey of 400 parents. Ann Emerg Med.

13. Robinson SM, Mackenzie-Ross S, Campbell Hewson GL, et al.

1999;34(1):70-4.

Psychological effect of witnessed resuscitation on bereaved relatives.

26. Kramer DB, Mitchell SL Weighing the benefits and burdens of

Lancet. 1998;352(9128):614-7.

Volume 21, no. 5: September 2020

witnessed resuscitation. N Engl J Med. 2013;368(11):1058-9.

1187

Western Journal of Emergency Medicine

Original Research

Point-of-sale Naloxone: Novel Community-based Research to Identify Naloxone Availability Travis Olives, MD, MPH, MEd*§ *Hennepin Healthcare, Department of Emergency Medicine, Minneapolis, Minnesota † Laurie A. Willhite, PharmD†§ Hennepin Healthcare, Minneapolis, Minnesota § Samantha C. Lee, PharmD, DABAT ‡University of Calgary, Department of Emergency Medicine, Calgary, Alberta, Canada § Danika K. Evans, DO* Minnesota Poison Control System, Minneapolis, Minnesota § Ashley Jensen, PharmD Hsiao-Ting Regelman, PharmD§ Eric S. McGillis, MD‡ Section Editor: Tony Zitek, MD Submission history: Submitted April 9, 2020; Accepted July 9, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.7.2020.47252

Introduction: Expanding naloxone availability is important to reduce opioid-related deaths. Recent data suggest low, variable urban naloxone availability. No reports describe naloxone availability at the point of sale (POSN). We characterize POSN without prescription across a Midwestern metropolitan area, via a unique poison center-based study. Methods: Pharmacies were randomly sampled within a seven-county metropolitan area, geospatially mapped, and distributed among seven investigators, who visited pharmacies and asked, “May I purchase naloxone here without a prescription from my doctor?” Following “No,” investigators asked, “Are you aware of the state statute that allows you to dispense naloxone to the public under a standing order?” Materials describing statutory support for POSN were provided. Responses were uploaded to REDCap in real time. We excluded specialty (veterinary, mail order, or infusion) pharmacies a priori. POSN availability is presented as descriptive statistics; characteristics of individual sites associated with POSN availability are reported. Results: In total, 150 pharmacies were prospectively randomized, with 52 subsequently excluded or unavailable for survey. Thus, 98 were included in the final analysis. POSN was available at 71 (72.5%) of 98 pharmacies. POSN availability was more likely at chain than independent pharmacies (84.7% vs 38.5%, p<0.001); rural areas were more commonly served by independent than chain pharmacies (47.4% vs 21.5%, p = 0.022). Five chain and five independent pharmacies (18.5% each) were unaware of state statutory support for collaborative POSN agreements. Statutory awareness was similar between independent and chain pharmacies (68.8% vs 54.6%, p = 0.453). Rationale for no POSN varied. Conclusion: POSN is widely available in this metropolitan area. Variability exists between chain and independent pharmacies, and among pharmacies of the same chain; awareness of statutory guidance does not. Poison centers can act to define local POSN availability via direct inquiry in their communities. [West J Emerg Med. 2020;21(5)1188-1194.]

INTRODUCTION Opioids continue to account for a large proportion of drug-related deaths in the United States, and were involved

Western Journal of Emergency Medicine

in over half of all deaths related to drug overdose from 2013–2017.1 Nearly 10% of all substances reported in fatal drug exposures reported to US poison centers were attributed

1188

Volume 21, no. 5: September 2020

Olives et al.

Naloxone Availability

to opioids in 2017, making them the second most commonly cited exposure category involved in overdose deaths.2 The importance of efforts to decrease or eliminate morbidity and mortality attributed to opioid use, misuse, and overdose remains a public health priority, within which primary and secondary prevention efforts have been increasingly accompanied by efforts to broaden the distribution of the opioid reversal agent, naloxone.2,3 Naloxone has long been recognized as a competitive opioid receptor antagonist when administered parenterally or intranasally,4-6 and remains, in conjunction with the restoration of respiration, the reversal agent of choice for the treatment of acute opioid toxicity. Naloxone is increasingly considered an important component of tertiary prevention and harm reduction in the fight against opioids,7 in addition to its utility in the care of individual patients at risk for opioid overdose. Naloxone distribution has been shown to be a cost-effective way to decrease overdose mortality,8 and evidence modeling naloxone distribution at a time of increasing fentanyl adulteration suggests a survival benefit to naloxone distribution.9 Although penetrance of naloxone prescribing varies,10 the practical availability of point-of-sale naloxone (POSN) without a medical prescription remains ill-defined. In a manner analogous to previously controversial but now widely accepted needle-exchange programs to prevent the spread of HIV and other communicable diseases,11,12 expanded availability of POSN without a medical prescription offers the hope of increased access to a life-saving antidote with fewer acquisition barriers. Although previous studies have characterized the prevalence of POSN availability without a medical prescription within pre-specified geographic areas,13,14 to our knowledge none have characterized naloxone availability through organized, in-person assessments at the level of individual pharmacies. The purpose of this study was to define the availability of POSN within a Midwestern metropolitan area, and to describe site characteristics associated with POSN availability. METHODS Study Design and Setting This was a cross-sectional study of POSN availability conducted at community pharmacies within a large, sevencounty metropolitan area in Minnesota with a total population of approximately 3,000,000. The study was identified as exempt from review by the governing institutional review board. Although the greater metropolitan area is entirely within the seven counties, some rural areas within these counties are also represented. Pharmacy locations were defined as rural if located in a community of fewer than 50,000 people, per the US Census Bureau definition,15,16 and entirely outside of the US interstate 494/694 ring clearly defining the central metropolitan area. The remainder Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Changes to legislation have facilitated the availability of point-of-sale naloxone (POSN) in many states. What was the research question? What is the prevalence of POSN availability at pharmacies in a large Midwestern metropolitan area? What was the major finding of the study? Of 98 pharmacies approached on foot by seven Poison Center professionals, 72.5% offered POSN. How does this improve population health? When pharmacies are approached directly, POSN availability varies. This variability persists across chain and independent pharmacies despite statutory awareness

were defined as urban. This definition served to avoid the inclusion of communities of fewer than 50,000 people, but geographically located immediately contiguous with urban areas. Study Protocol The Minnesota State Board of Pharmacy lists 569 operational pharmacies within the metro area. From this list we randomly sampled 150 pharmacies to approach for this study by using a random number generator in Excel 2013 (Microsoft Corporation, Redmond WA) and sorting on the randomly generated numbers to select the first 150 sites. We intended to analyze approximately 100 sites, anticipating limitations to time and resources preventing additional sampling. A priori exclusion criteria included known sub-specialization, including mail order, veterinary, and infusion center pharmacies. We then geospatially mapped the remaining pharmacies using arcGIS online 2019 (Environmental Systems Research Institute, Redlands, CA) and divided them by geographic location. Seven investigators, all based at a single, accredited poison center were trained equivalently on approaching pharmacies to inquire about the availability of POSN. These investigators were assigned to evaluate sampled pharmacies within a specified geographic region. To minimize the impact of evolving pharmacy protocols over time, all pharmacies were visited within a 24-hour period, the majority of which occurred over a single morning. Once assigned, investigators

1189

Western Journal of Emergency Medicine

Olives et al.

Naloxone Availability approached each pharmacy in person and asked a series of scripted questions: 1) “May I purchase naloxone here without a prescription from my doctor?” “No” responses to question 1 were followed by a request to ask the question of the onsite pharmacist for verification purposes, if initially answered by a nonpharmacist. Following an answer of “no,” the response to a follow-up question was recorded: 2)“Are you aware of the state statute that allows you to dispense naloxone to the public under a standing order?” To simulate anticipated clinical circumstances, the protocol did not specify that a pharmacist had to be approached and queried. Rather, investigators were instructed to question the employee greeting them at the pharmacy. Pharmacies were provided with information from the state Board of Pharmacy describing statutory support for collaborative agreements for standing orders for naloxone. Data including pharmacy name, survey responses, and geographic address including county and retail status (chain or independent) were entered into the REDCap mobile app and uploaded in real time to a central REDCap v8.11.3 (Vanderbilt University, Nashville, TN) database. REDCap is a web-based clinical research tool for creating and storing databases. Chain community pharmacies were substantially overrepresented in the initial sample. Because of the imbalance that under-representation of independent pharmacies introduced to the dataset, one third of sites from each chain pharmacy with greater than 10 sites sampled were replaced with randomly selected independent pharmacies using the method described above. We chose one third to maintain prominent representation of community chain pharmacies, which are common throughout the study area, while still affording opportunity for a meaningful comparison with independent pharmacies. Stata/ IC 15.1 (StataCorp, College Station, TX) was used to assess the association between pharmacies and POSN availability. We employed Pearson’s χ2 to assess the association between the outcome of interest, POSN availability, and independent variables. Where fewer than five observations per cell were encountered, we employed Fisher’s exact test. RESULTS After a priori exclusions of 15 pharmacies for clear evidence of a business model (mail order, veterinary medicine, or infusion center) intended for non-retail or non-human customers, 135 pharmacies (Figure 1) were approached by seven investigators comprised of seven Poison Center staff (five female and two male, of whom four were pharmacists/ specialists in poison information, one an emergency medicine resident, one a medical toxicology fellow, and one a medical toxicologist). Median pharmacies approached by a single investigator was 20 (range 16 – 24). Of the 135 Western Journal of Emergency Medicine

Figure 1. Study flowchart of pharmacies chosen for in-person query regarding over-the-counter naloxone availability.

pharmacies approached, 37 (27.4%) were excluded due to closure (22, 16.3%); other than community pharmacy (eight, 5.9%); membership requirements (two, 1.5%); not at the described location (two, 1.5%); or other (three, 2.2%). Thus, we included a total of 98 (73.1%) in the study. A single investigator approached 12 (9%) pharmacies the evening prior to the four-hour study period due to time constraints. Of included pharmacies, 75 (76.5%) were urban; the remaining 23 (23.5%) were rural (Table 1). Pharmacies were widely distributed across the seven-county metropolitan area. Naloxone was available at the point of sale at 71 of 98 pharmacies surveyed (72.5%, Figure 2). Pharmacy location was not associated with POSN availability (rural 65.2% vs urban 74.7%, p = 0.375, Table 2). Chain pharmacies were more likely to report POSN availability than independent pharmacies (chain 84.7% vs independent 38.5%, p<0.001), and rural areas were more commonly served by independent pharmacies than chain pharmacies (47.4% vs 21.5%, p = 0.022). Independent pharmacies in rural settings were less likely to offer POSN than chain pharmacies (30.0% vs 92.3%, p = 0.006). Among chain pharmacies with more than one location sampled, POSN availability varied from 66.7-100% (Table 3) across geographic locations. Among those without POSN availability, 17 (63%) were aware of the state statute allowing for the provision

1190

Volume 21, no. 5: September 2020

Olives et al.

Naloxone Availability

Table 1. Characteristics of pharmacies. Pharmacies Number sampled Excluded a priori or at time of assessment Included for analysis

N (%) 150 (100) 52 (34.7) 98 (65.3)

Urban

75 (76.5)

Rural

23 (23.5)

Chain

72 (73.5)

of POSN via collaborative agreement with a prescribing practitioner. No differences in statutory familiarity were apparent when stratified by pharmacy location or retail type (Table 4). Reasons given for not providing POSN included a lack of consumer demand, incomplete stocking plans, no physician collaborator, and refusal to provide naloxone despite availability (“Yes, I am aware of the statute, but I can use my discretion and I won’t be giving it out this time”). Still others indicated that POSN could be provided “if the patient had risk factors for overdose,” or “if the patient was actively overdosing.” A single pharmacy denied POSN availability, was prompted to revisit internal pharmacy guidelines, and then identified POSN as available. In a post hoc sensitivity analysis to determine the effect of oversampling independent pharmacies, we excluded 25 randomly selected independent pharmacies to account for oversampling. Compared to the overall POSN availability in our primary analysis, POSN availability in our sensitivity analysis suggested that our oversampling of independent pharmacies modestly underestimated availability in this sample (77.8% vs 72.5%). Differences in availability across chain and independent pharmacies remained. DISCUSSION Pharmacy-based POSN is an evolving harm reduction approach to limit morbidity and mortality from opioid overdose. Collaborative naloxone prescribing has developed in parallel with other models of increasing naloxone availability, including point-of-contact,17 emergency department-based,18 and pharmacist-driven distribution.19 The present study suggests that POSN availability is more widespread in this metropolitan area than it was in 10 New Jersey cities that were assessed by telephone survey.10 This finding may represent a meaningful difference in the availability of POSN across the two regions, but it also may be attributable to the time lapse of 24 months between the two studies. Support for POSN via collaborative agreements with medical providers is stipulated in Minnesota statutes (Minn. Stat. §151.37, subd. 13 [2019]), but is predicated on the availability of a collaborating healthcare professional willing to provide a standing order to dispense naloxone. Currently, all states but Wyoming and Kansas have active naloxone-access laws.20 Despite this, discrepancies continue to be reported at Volume 21, no. 5: September 2020

Figure 2. Geospatial distribution of point-of-sale naloxone availability in the Minneapolis/St. Paul, Minnesota metropolitan area.

the point of sale.21 Our data suggest that chain community pharmacies were more likely than independent pharmacies to provide POSN, and episodes of within-chain variability were common, with availability ranging from 66.7-100%. This finding is consistent with previously reported variability in POSN availability between chain and independent pharmacies,14,22 as well as limited practical availability of other medications whose delivery has previously been limited, such as emergency contraception.23 Areas served predominantly by independent pharmacies are less likely to have access to POSN than those served by chain pharmacies despite similar awareness of Minnesota statutory support. Previous research has revealed that rural areas of Minnesota are predominantly served by independent pharmacies,24 potentially placing rural populations at risk of poorer access to POSN than their urban counterparts. Important differences exist between this study and previous investigations of naloxone availability. Early studies of comparative survey methodologies demonstrated differences in response rates and content between telephonic and in-person surveys of household drug use,25 but few if any studies have described differences in healthcare professional responses to telephonic vs in-person queries of available services. It is likely that our results may have differed if we had contacted pharmacies by telephone, rather than presenting in person. Importantly, at least one point of contact with a community pharmacy led to the correction of a pharmacist’s erroneous belief that POSN was not available in her pharmacy. This community-based research, in which investigators sought to contribute to broader medical knowledge while at the

1191

Western Journal of Emergency Medicine

Olives et al.

Naloxone Availability Table 2. Comparison of point-of-sale naloxone availability by pharmacy characteristic. Pharmacy characteristic

POSN N (%)

Table 3. Point-of-sale naloxone availability across chain pharmacies with more than one site sampled. Pharmacy:

P-value

61/72 (84.7)

18/19

15/16

7/9

7/8

7/7

2/3

2/2

Independent

10/26 (38.5)

94.7

93.8

77.8

87.5

100

66.7

100

Urban

56/75 (74.7)

Rural

15/23 (65.2)

Chain

<0.001 0.375

POSN, point-of-sale naloxone.

same time effecting change to improve community health, is previously described,26 and represents a unique form of community advocacy undertaken at the level of the individual poison center. The finding that up to 45% of pharmacy staff reporting no POSN availability were unaware of statutory support for collaborative prescribing protocols highlights two important findings from this study. First, an opportunity exists to better educate community pharmacists such that POSN is acknowledged as an option. When coupled with the community-based approach to survey data collection, this finding also provided an immediate opportunity to educate pharmacists at the point of contact regarding statutory support for POSN. Poison centers are well-known agents of change with respect to legislative lobbying;27 however, affecting change at the level of individual pharmacies through face-to-face interaction is unique even among poison centers. Although a national trend in increased naloxone prescriptions is evident, 2018 saw more than half a million prescriptions for naloxone written, compared to more than 38 million prescriptions for highdose opioids.28 An opportunity exists to augment naloxone dispensation relative to opioid prescriptions; poison centers may hold multiple roles in this endeavor. In addition to the isolated finding of a pharmacist prompted to revisit retail protocols to verify POSN availability despite her previous understanding to the contrary, still other reported reasons for failure to provide POSN were uncovered. These ranged from a perceived need to demonstrate overdose risk factors prior to providing POSN, to using personal discretion in deciding not to provide POSN despite an acknowledged capacity and institutional policy to provide it. These responses highlight opportunities for additional education within pharmacies to optimize naloxone distribution. Although assessing the impact of this survey on naloxone availability was not a formal study aim, future investigations might consider iterative assessments of naloxone availability following similar surveys. LIMITATIONS A number of limitations apply to this study. Of particular note, we sampled a fixed, random sample of pharmacies within the metropolitan area. Although nearly one out of every Western Journal of Emergency Medicine

five community pharmacies were successfully approached, a broader sample would have added strength to our findings. However, the in-person approach to surveying resulted in successful assessments of all pharmacies included in the study, and thus our sample likely represents as good or better an appraisal as a telephonic survey would have, accounting for likely non-responders. Nonetheless, our study did not account for medication stocking maintenance or other barriers to dispensation previously reported to affect individual pharmaciesâ&#x20AC;&#x2122; capacities to provide POSN.29 Similarly, our resources limited us from investigating naloxone availability within a broader geographic region including more rural communities. Our findings are thus more limited in their generalizability. Nonetheless, the finding that independent pharmacies are less likely to provide POSN, contextualized in previous data suggesting that rural areas in the region are heavily served by independent pharmacies, suggests that rural regions are less likely to have access to POSN via collaborative prescription protocols. An additional geographic limitation of this study was the specificity of our findings to a single state. Legislative efforts to promote POSN availability are variable across states, impacting state-to-state availability of POSN. Differences in availability of POSN secondary to legislative differences would likely be found by the current study protocol, given the â&#x20AC;&#x153;boots on the groundâ&#x20AC;? approach to data collection; however, the evaluation of stateto-state differences in POSN availability was beyond the scope of the current study. We chose to oversample independent pharmacies intentionally, at the expense of chain pharmacies. While this may have introduced a degree of bias to results, oversampling of independent pharmacies provided for a more balanced population and assessment of the impact of pharmacy type

Table 4. Awareness of state statutory support for point-of-sale naloxone among pharmacists reporting no access to point-ofsale naloxone. Pharmacy characteristic POSN N(%) P-value Chain

17/27 (63.0)

Independent

11/16 (68.8)

Urban

12/19 (63.2)

Rural

5/8 (62.5)

0.687 1.000

POSN, point-of-sale naloxone.

1192

Volume 21, no. 5: September 2020

Olives et al.

Naloxone Availability

on POSN availability, an association that we predicted based on previously published research. Ultimately this decision likely led to an underestimate of POSN availability in our community, although our sensitivity analysis suggested a magnitude of underestimation to be approximately 5%. Finally, we did not collect data on the role of the employee approached at individual pharmacies. It is plausible that answers would differ meaningfully between pharmacy technicians and pharmacists. We attempted to address this possible confounder by following up negative responses delivered by non-pharmacists with requests to speak directly with a pharmacist. Delivering the study question to the pharmacy professional who greeted researchers, best reflecting actual conditions at the point of sale, was determined to be a better reflection of reality than directing the study question solely to pharmacists. CONCLUSION Point-of-sale naloxone is more widely available in this Midwestern metropolitan area than in recently described metropolitan areas in other regions of the United States. Although collaborative prescribing protocols are one of many naloxone distribution strategies contributing to harm reduction efforts, the survey method used in this study represents a unique “boots on the ground” for poison center professionals to champion change.

3. National Institute on Drug Abuse. Naloxone for opioid overdose: life-saving science. 2017. Available at: https://www.drugabuse.gov/ publications/naloxone-opioid-overdose-life-saving-science/naloxoneopioid-overdose-life-saving-science. Accessed November 6, 2019. 4. Jasinski DR, Martin WR, Haertzen CA. The human pharmacology and abuse potential of N-allylnoroxymorphone (naloxone). J Pharmacol Exp Ther. 1967;157(2):420-6. 5. Wanger K, Brough L, Macmillan I, et al. Intravenous vs subcutaneous naloxone for out-of-hospital management of presumed opioid overdose. Acad Emerg Med. 1998;5(4):293-9. 6. Sporer KA, Firestone J, Marshal Isaacs S. Out-of-hospital treatment of opioid overdoses in an urban setting. Acad Emerg Med. 1996;3(7):660-7. 7. Stopka TJ, Donahue A, Hutcheson M, et al. Nonprescription naloxone and syringe sales in the midst of opioid overdose and hepatitis C virus epidemics: Massachusetts, 2015. J Am Pharm Assoc . 2017;57(2S):S34-44. 8. Townsend T, Blostein F, Doan T, et al. Cost-effectiveness analysis of alternative naloxone distribution strategies: first responder and lay distribution in the United States. Int J Drug Policy. 2020;75:102536. 9. Irvine MA, Buxton JA, Otterstatter M, et al. Distribution of takehome opioid antagonist kits during a synthetic opioid epidemic in British Columbia, Canada: a modelling study. Lancet Public Health. 2018;3(5):e218–25. 10. Dunne RB. Prescribing naloxone for opioid overdose intervention. Pain Manag. 2018;8(3):197-208. 11. Des Jarlais DC, Arasteh K, Friedman SR. HIV among drug users at

Address for Correspondence: Travis Olives, MD, MPH, MEd, Minnesota Poison Control System, 701 Park Avenue, Minneapolis, MN 55415. Email: travis.olives@gmail.com. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Olives et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

Beth Israel Medical Center, New York City, the first 25 years. Subst Use Misuse. 2011;46(2-3):131-9. 12. Sawangjit R, Khan TM, Chaiyakunapruk N. Effectiveness of pharmacy based needle/syringe exchange programme for people who inject drugs: a systematic review and meta-analysis. Addiction. 2017;112(2):236-47. 13. Lozo KW, Nelson LS, Ramdin C, et al. Naloxone deserts in NJ cities: sociodemographic factors which may impact retail pharmacy naloxone availability. J Med Toxicol. 2019;15(2):108-11. 14. Stone RH, Hur S, Young HN. Assessment of naloxone availability in Georgia community pharmacies. J Am Pharm Assoc (2003). 2020;60(2):357-61. 15. United States Census Bureau. 2010 Census Urban and Rural Classification and Urban Area Criteria. 2012. Available at: https:// www.census.gov/programs-surveys/geography/guidance/geo-areas/ urban-rural/2010-urban-rural.html. Accessed October 31, 2019. 16. Minnesota State Demographic Center. Our Estimates. 2019.

REFERENCES

Available at: https://mn.gov/admin/demography/data-by-topic/

1. Scholl L, Seth P, Kariisa M, et al. Drug and opioid-involved

population-data/our-estimates/. Accessed October 31, 2019.

overdose deaths — United States, 2013–2017. MMWR.

17. Madah-Amiri D, Clausen T, Lobmaier P. Rapid widespread

2018;67(5152):1419027.

distribution of intranasal naloxone for overdose prevention. Drug

2. Gummin DD, Mowry JB, Spyker DA, et al. 2017 Annual Report of the American Association of Poison Control Centers’ National Poison

Alcohol Depen. 2017;173:17-23. 18. Gunn A, Smothers Z, Schramm-Sapyta N, et al. The emergency

Data System (NPDS): 35th Annual Report. Clin Toxicol (Phila.).

department as an opportunity for naloxone distribution. West J Emerg

2018;56(12):1213-1415.

Med. 2018;19(6):1036-42.

Volume 21, no. 5: September 2020

1193

Western Journal of Emergency Medicine

Olives et al.

Naloxone Availability 19. Morton KJ, Harrand B, Floyd CC, et al. Pharmacy-based statewide

areas: Is the problem geographic access or financial access? J Rural

naloxone distribution: a novel “top-down, bottom-up” approach. J

Health. 2002;18(3):467-77.

Amer Pharm Assoc. 2017;57(2S):S99-106.

25. Aquilino WS. Telephone versus face-to-face interviewing for

20. MonQcle. Prescription drug abuse policy system. Available at: http://

household drug use surveys. Int J Addict. 1992;27(1):71-91.

pdaps.org/about. Accessed June 22, 2020.

26. Israel BA, Schulz AJ, Parker EA, et al. Review of community-based

21. Luthern, A. Under Wisconsin law, it should be easy to buy the

research: assessing partnership approaches to improve public

drug to reverse an opioid overdose. It’s not. 2018. Available at:

health. Annu Rev Publ Health. 1998;19:173-202.

https://www.jsonline.com/story/news/local/wisconsin/2018/08/23/

27. Institute of Medicine. (2004). Chapter 9. In A. Editor & B. Editor

wisconsin-pharmacies-unclear-narcan-laws-reduce-opioid-heroin-

(Eds.) Forging a Poison Prevention and Control System. Washington

overdoses/1013650002/. Accessed June 22, 2020.

(DC):National Academies Press.

22. Egan KL, Foster SE, Knudsen AN, et al. Naloxone availability in retail

28. Guy GP, Haegerich TM, Evans ME, et al. Vital signs: Pharmacy-

pharmacies and neighborhood inequities in access. Am J Prev Med.

based naloxone dispensing – United States, 2012 – 2018. MMWR.

2020;58(5):699-702. 23. Chau VM, Stamm CA, Borgelt L, et al. Barriers to single-dose

2019;68(31):679-86. 29. Sisson ML, McMahan KB, Chichester KR, et al. Attitudes and

levonorgestrel-only emergency contraception access in retail

availability: a comparison of naloxone dispensing across chain and

pharmacies. Womens Health Iss. 2017;27(5):518-22.

independent pharmacies in rural and urban areas in Alabama. Int J

24. Casey MM, Klingner J, Moscovice I. Pharmacy services in rural

Western Journal of Emergency Medicine

Drug Policy. 2019;74:229-35.

1194

Volume 21, no. 5: September 2020

Original Research

The Utility of Serum Creatinine Kinase in Emergency Department Patients with Possible Substance-use Related Conditions Mohammad S. Alzahri, MD

King Saud University, Department of Emergency Medicine, Riyadh, Saudi Arabia

Section Editor: Jeffrey Suchard, MD Submission history: Submitted January 23, 2020; Revision received May 12, 2020; Accepted May 12, 2020 Electronically published September 4, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46678

Introduction: Our goal was to assess the diagnostic utility and temporal kinetics of serum creatine kinase (CK) measurement as a predictor of acute kidney injury (AKI) in emergency department (ED) patients who present with possible substance-use related conditions. Methods: This was a retrospective chart review of ED patients with a urine drug screen (UDS) ordered and resulted between 2009â&#x20AC;&#x201C;2013. Data was extracted electronically from EPIC Systems electronic health records, populated into a Microsoft Excel file, and includes demographics, chief complaint, vital signs, neuro-psychiatric physical examination findings, laboratory findings, psychiatric consult order time, ED medications given, orders, disposition and its time, and diagnosis. Results: Of 74,970 patients with an ED UDS, 22,101 (29%) had at least one CK measured. After inclusion and exclusion criteria, 2858 (13%) remained. Mean (standard deviation [SD]) age was 43.3 (12.5) years, 73% were male, 61% Black, 22% White, and 17% Hispanic. Mean (SD) ED length of stay was 10.4 (5.8) hours, and 56.7% were hospitalized. On average, CK was higher at 6-12 hours (p<0.001) and 12-18 hours (p=0.016) compared to 6 hours. CK was lower at 42-56 hours (p = 0.011), 72 hours (p<0.001), and over 72 hours (p<0.001), compared to 6 hours. Maximum CK was determined in those with >2 CK measures. We defined AKI risk as a creatinine of >1.4 milligrams per deciliter based on RIFLE criteria. AKI risk was calculated among those with at least two creatinine values in 522 patients. We identified five (1%) patients as having AKI risk. The odds of AKI risk were not associated with increase in CK over time. Conclusion: In 74,970 ED patients undergoing UDS testing for potential substance abuse, there was no identifiable CK level associated with AKI risk. In patients with possible substance-use conditions, CK continued to trend up even after six hours from door time and began to decrease after 42 hours. We found no value in repeated ED CK measures. Disposition should not be based solely on CK levels. [West J Emerg Med. 2020;21(5)1195-1200.]

INTRODUCTION Behavioral emergencies are responsible for approximately 6% of all emergency department (ED) visits in the United States,1 where emergency physicians are frequently asked to exclude medical illnesses that may be causing or contributing to the patientâ&#x20AC;&#x2122;s acute psychiatric symptoms.2 While the policy of the American College of Emergency Physicians states that in adult ED patients with primary psychiatric complaints, Volume 21, no. 5: September 2020

diagnostic evaluation should be directed by the history and physical examination and that routine laboratory testing of all patients is of very low yield, considerable variation exists between physicians, departments, and institutions in what is generally considered necessary in the medical assessment of patients with acute psychiatric emergencies.3 Substance use is a common ED presentation, and alcohol, heroin, and cocaine have all been shown to cause

1195

Western Journal of Emergency Medicine

CK in ED Patients with Possible Substance-use Related Conditions Alzahri rhabdomyolysis. Rhabdomyolysis causes about 7-10% of all cases of acute kidney injury (AKI) annually.4,5 Previous studies have shown that the incidence of AKI is higher among patients who have rhabdomyolysis as a consequence of illicit drug or alcohol use.6-8 The same study found a 3.4% mortality among this cohort of patients with substance use and AKI. Thus, the importance of identifying and treating potential rhabdomyolysis or AKI prior to medically clearing patients with substance use is evident. A diagnosis of rhabdomyolysis is made by testing serum creatine kinase (CK) levels. The consensus definition has rather arbitrarily been chosen as five times the upper limit of normal, or approximately 1000 units per liter (U/L). More recently, proposed guidelines suggest that a diagnosis of rhabdomyolysis should be made only when the serum CK is higher than 50 times of the upper limit of normal or when CK elevation is accompanied by findings of AKI. This is because a mild to moderate elevation of serum CK above the normal reference range is expected in healthy adults after physical exertion. These mild to moderate elevations are often not clinically significant and do not require medical management, such as serial laboratory testing or intravenous (IV) hydration.9,10 Previously, weak correlations between the peak CK value and the incidence of AKI or peak serum creatinine have been reported.4 One study of 72 patients had shown that one-fourth of the patients with peak CK >10,000 U/L and positive drug screens for cocaine or heroin developed rhabdomyolysisassociated renal failure.11 Often, when AKI is seen at low CK levels around 5000 U/L, this occurs in the presence of several comorbidities such as sepsis, dehydration, and acidosis.12 It has been our experience that psychiatric facilities arbitrarily set low thresholds for serum CK above which they will not accept psychiatric patients in transfer until those levels normalize or show a decreasing trend. This practice typically leads to a delay in transfer of such patients to psychiatric facilities by way of serial laboratory testing, IV fluid therapy, and medical hospitalization—the benefit of which is not unequivocally proven. Given the above, here we set out to investigate the diagnostic utility and temporal kinetics of serum CK measurement as a predictor of AKI in ED patients who present with possible substance-use related conditions. Some of the patients we labeled as AKI could have had acute renal failure. This will be further detailed in the “Methods” section. Our purpose was to assess the diagnostic utility and temporal kinetics of serum CK measurement as a predictor of AKI in ED patients who present with possible substance-use related conditions. METHODS This was a retrospective cohort analysis of ED patients who presented to Ben Taub General Hospital (BTGH) and had a urine drug screen (UDS) ordered and resulted between 2009–2013. The UDS used was a quantitative, 10-panel immunoassay UDS that included amphetamine; barbiturate; benzodiazepines; cocaine; Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? There is conflicting information on how to use serum creatine kinase (CK) in evaluating patients with possible substance-use related conditions. What was the research question? In patients with possible substance use, what level of CK is associated with acute kidney injury? What was the major finding of the study? There was no value in repeated CK measures in the emergency department (ED). Disposition should not be based solely on CK level. How does this improve population health? This finding will help decrease unnecessary testing, lowering ED length of stay, and consequently ED waiting time.

methadone; opiates; oxycodone/oxymorphone; phencyclidine; propoxyphene; and tetrahydrocannabinol, provided by American Screening Corp. BTGH is a Level 1 trauma center and the largest county hospital in Houston, Texas. It is the only hospital in Houston with a psychiatric ED that is open 24 hours a day. In 2009 the BTGH ED starting using the EPIC ED module ASAP (Epic Systems, Verona, WI) , which allows for extraction of clinical data electronically. Data collection was completed by 2015, but statistical analysis was not done until 2018 as the author was matriculated in a full-time clinical training program. Data was extracted electronically from EPIC, populated into a Microsoft Excel (Microsoft Corporation, Redmond, WA) file, and includes demographics, chief complaints, vital signs, neuro-psychiatric physical examination findings, laboratory findings, psychiatric consult order time, ED medications given, orders, disposition and its time, and diagnosis. Inclusion criteria were either a positive UDS or serum ethanol >0.08 grams per deciliter (g/dL), with a chief complaint, diagnosis, or physical finding of intoxication, agitation, drug use, or confusion. Exclusion criteria included conditions that may alter CK results, including a history of chronic kidney disease, age >65 years, or a temperature >102 degrees Fahrenheit at any time. Statistical Analysis Patient characteristics were summarized by frequency with percentage and mean with standard deviation. AKI risk was defined as a creatinine of > 1.4 times baseline creatinine,

1196

Volume 21, no. 5: September 2020

Alzahri

CK in ED Patients with Possible Substance-use Related Conditions

as defined by RIFLE (Risk, Injury, Failure, Loss of kidney function, and End-stage kidney disease) classification. It is commonly labeled simply as AKI. We used the RIFLE term “AKI risk.”13 We labeled all patients with creatinine >1.4 x baseline as AKI risk with no subset analysis for patients with renal failure. As a result, these patients can be more accurately labeled as “at least AKI risk.” CK and creatinine were then summarized over time. A mixed model linear regression with discrete residuals was used to assess log transformed CK over time. Note that due to the skewed nature of the data, CK was log transformed. This study was approved by the Baylor College of Medicine (BCM) Institutional Review Board. This approval was maintained until data analysis was complete by 2018. Patient confidentiality was maintained. All data were stored on a password-protected computer accessible only to study investigators. The password-protected computers were managed and maintained by the BCM information technology department. Patients were not directly contacted. The original file included the patient’s CSN# (account number) as well as a running counter. The file was kept in the principal investigator (PI)’s password-protected BCM computer located in the PI’s .locked office. The PI deidentified the file (deleted CSN/medical record [MR]# columns) and saved the PHI redacted Excel file as a new password-protected file. The PHI (MR#/CSN) were deleted from the dataset before analysis. RESULTS There were 74,970 patients in the dataset, 52,869 with no CK measures and 22,101 (29%) with at least one CK measure. A total of 4272 patients had a chief complaint, diagnosis, or physical finding of intoxication, agitation or drug use, and 3085 of them had positive UDL or serum ethanol >0.08 g/dL. Eighty-eight patients were older than 65 years old; 72 of the remaining 2997 patients had history of chronic kidney disease; and 67 out of the remaining 2925 patients had a temperature more than 102º F at any time. Therefore, after applying the rest of the inclusion and exclusion criteria, 2858 (13%) patients remained. Mean age was 43.3 (standard deviation 12.5) years, 73% were male, 61% Black, 22% White, and 17% Hispanic. On average, CK was higher at 6-12 hours (p<0.001) and 12-18 hours (p = 0.016) compared to 6 hours. Additionally, CK was lower at 42-56 hours (p=0.011), 72 hours (p<0.001), and over 72 hours (p<0.001), compared to 6 hours. CK geometric mean over time is shown in Figure 1. The geometric mean is defined as the nth root of the product of n numbers for each time interval. Note that the y-axis of the figure is on the log scale. Fewer than 10 measures had a CK of zero that were dropped for the figure (since log of zero cannot be calculated). On average, CK is higher at 6-12 hours (p<0.001), and 12-18 hours (p = 0.016) compared to 6 hours. When compared to 6 hours on average, CK is lower at 42-56 hours (p = 0.011), Volume 21, no. 5: September 2020

Figure 1. Serum creatine kinase levels over time in patients presenting to the emergency department with substance use. CK, creatine kinase; ED, emergency department.

72 hours (p<0.001), and over 72 hours (p<0.001). This is available in supplemental materials as Table 1: The mixed model results for log CK over time. The average CK over time is shown with 95% confidence intervals in Figure 2. A figure of CK over time for those with at least three CK measures is available in supplemental materials. The geometric mean for CK is 335 U/L at 6 hours; 380 U/L at 6-12 hours; 358 U/L at 12-18 hours; 351 U/L at 18-24 hours; 324 U/L at 24-36 hours; 336 U/L at 36-42 hours; 299 U/L at 42-56 hours; 274 U/L at 72 hours; and 133 U/L at over 72 hours. When looking at maximum CK (Table 1), we only used patients with at least three CK measures, which was 888 U/L.

Figure 2. Log transformed serum creatine kinase level over time in patients presenting to the emergency department with substance use. The average CK over time is shown with 95% confidence intervals in Figure 2. The geometric mean for CK is 335 U/L at 6 hrs, 380 U/L at 6-12 hrs, 358 U/L at 12-18 hrs, 351 U/L at 18-24 hrs, 324 U/L at 24-36 hrs, 336 U/L at 36-42 hrs, 299 U/L at 42-56 hrs, 274 U/L at 72 hrs, and 133 U/L at over 72 hrs. CK, creatine kinase; ED, emergency department.

1197

Western Journal of Emergency Medicine

CK in ED Patients with Possible Substance-use Related Conditions Alzahri Table 1. Time interval associated with maximum serum creatine kinase (CK) level in patients with 3 CK measurements taken, n = 888. Time interval of Measurement (hrs) <6 6 – 12 12 – 18 18 – 24 24 – 36 36 – 42 42 – 56 72 >72

Frequency (n) of Maximum CK

218 168 153 124 96 51 31 22 25

24.6 18.9 17.2 14.0 10.8 5.7 3.5 2.5 2.8

In patients presenting to the ED with at least 3 serum CK measurements, the maximum CK was most frequently seen at the initial time interval prior to 6 hours. The frequency in which the maximum CK level was seen decreased as the time after 6 hours increased. CK, creatine kinase; hrs, hours; ED, emergency department.

AKI risk was calculated among those with at least two creatinine values, or 522 patients. There were five (1%) patients identified as having AKI risk. The odds of AKI risk were not associated with increase in CK over time. Table 2 shows the logistic regression results for AKI risk. Note that each row is a separate model for logistic regression and that times 6-12 hours and 72 hours could not converge. When looking at the full population, mean (SD) ED length of stay was 10.4 (5.8) hours and 56.7% were hospitalized (Table 3). Admissions were to either medical or psychiatric wards within the hospital. All the transferred patients were transferred to another psychiatric facility when no bed was available in the psychiatric ward. DISCUSSION Most labs set upper normal limit of serum CK as 198 U/L.4 Although a variety of conditions can contribute to serum CK above that level, most of them are not clinically significant. The clinical significance of rhabdomyolysis lies when there is associated acute renal failure and, subsequently, electrolyte disturbance.14 Ruling out clinically significant rhabdomyolysis in the ED often can be challenging. Incidence of acute renal failure secondary to rhabdomyolysis varies according to the cause of the disease and can range between 1-45%.4 While limited studies reported the incidence of renal failure in patients with substance use, most were done in a chronic substance use setting. These studies were also limited by sample size, which ranged between 16 to 716 patients.15-18 Moreover, some results contradicted each other. While most studies report AKI incidence of less than 5% , a study by Akmal et al reported 40% of acute renal failure from rhabdoyolysis cause by phencyclidine.19 Using the electronic health record, we were able to extract data for a relatively large sample size, or 74,970 patients who Western Journal of Emergency Medicine

Table 2. Univariate logistic regression for acute kidney injury based on 500-unit Increase in serum creatine kinase level. Time interval of measurement (hrs) Odds ratio 95% CI N <6 0.01 0.00-0.52 234 12 – 18 0.71 0.47-1.07 173 18 – 24 0.86 0.64-1.17 167 24 – 36 1.00 1.00-1.01 168 36 – 42 0.92 0.89-0.96 84 42 – 56 1.01 1.00-1.01 101 >72 1.06 0.99-1.12 110 Max 1.00 1.00-1.01 205 Note that each row is a separate model for logistic regression and that times 6-12 hrs and 72 hrs could not converge. The odds of acute ikidney injury (AKI) risk are decreased as creatine kinase (CK) increases for 6 hrs (p=0.021) and 36-24 hrs (p<0.001). No other CK measures are statistically associated with AKI. hrs, hours; CI, confidence interval; ED, emergency department.

had UDS results and 22,101 (29%) who had at least one CK measured. With this large sample size, we were able to better assess early temporal kinetics of serum CK measurement and its diagnostic utility in ED. The finding that serum CK levels continued to trend up and did not fall significantly below arrival measurements until after 42 hours supports that there is limited value in repeated measurements in the ED and a patient’s disposition should not be based solely on CK levels. Accordingly, when patients need to be transferred to a psychiatric facility, the accepting facility may not benefit from a repeated serum CK level and decision whether patient can be medically cleared should be based on other factors. We recommend discussing this finding with psychiatric departments’ decision-makers through interdepartmental meetings or in the form of a letter to psychiatric hospitals’ medical directors. As in previous studies,20 only 1% of patients had AKI risk. It is worth noting that many of these patients were brought in by emergency medical services, where prehospital care could have been started that may have included IV fluid administration and, subsequently, low incidence of AKI risk. Although a similar incidence was reported in prior studies, our extracted data did not indicate whether such prehospital intervention was done. Unlike some prior studies, we did not find an identifiable CK level associated with AKI, which could have been attributed to the small number of this subset group. Nevertheless, the finding that the odds of AKI are not associated with an increase in CK over time was limited by the small sample of this subset group analysis. LIMITATIONS One of the limitations in this retrospective cohort study

1198

Volume 21, no. 5: September 2020

Alzahri

CK in ED Patients with Possible Substance-use Related Conditions

Table 3. Creatine kinase measurement status and final disposition effect on mean ED length of stay, n = 74,970. N

Mean ED LOS in hours (SD)

P-value

52,818

10.7 (6.0)

reference

Yes

22,101

10.4 (5.8)

<0.001

Home

32,438

10.9 (5.7)

reference

Other

42,532

10.5 (6.1)

<0.001

Patient characterisitics CK measurement

Disposition

When looking at the full initial patient population presenting to the ED with a chief complaint of intoxication, agitation, drug use, or confusion, and either a positive urine drug screen or serum ethanol > 0.08 g/dl, the mean ED LOS was higher in those patients who had no CK measurements done (p < .001) and those patients who were discharged home (p < .001). CK, creatine kinase; ED, emergency department; LOS, length of stay; SD, standard deviation.

was that patients included in the study only had possible substance abuse. Inclusion criteria was either legally intoxicated with alcohol, as per the State of Texas serum alcohol level > 0.08, or had positive substance with a UDS and deemed intoxicated by the clinician (physician, physician assistant or nurse practitioner). Due to the fact that a positive predictive value of current UDS testing can be very low for some substances,21 only patients who were deemed intoxicated by the clinician were included in the study, for both alcohol and UDS. In patients who were included, the clinician indicated in the chart (either as patient self-reported chief complaint, clinical impression diagnosed as an International Classification of Diseases, Editions 9/10 diagnosis, or physical exam finding – usually predefined check boxes – that a patient had one of the following: intoxication, agitation, drug use, or confusion. While exclusion criteria included conditions that could affect CK,4 these are predefined boxes in the history section of the ED chart that are often skipped by emergency clinicians. Another limitation is that patients who reported drug use by history and had negative UDS were not included in the study. Some of these patients could have had false negative results. We defined AKI risk according to RIFLE criteria, which classifies elevation of serum creatinine >2x baseline as renal failure. We labeled all patients with creatinine >1.4x baseline as AKI risk with no subset analysis for patients with renal failure. This was due to the small sample size in this group. Although we excluded patients with end stage renal disease, we did not included glomerular filtration rate or urine output, both of which are significant factors of RIFLE.13 However, it would have been helpful to have carried out a subset analysis to each substance and alcohol separately, as some prior studies reported varying level of morbidity in certain substances.22 Volume 21, no. 5: September 2020

Further, it is possible that some substances had a diluting effect, as a previously mentioned study by Akmal et al. reported a 40% incidence of acute renal failure from rhabdomyolysis in the setting of phencyclidine use.18 Another limitation was the fact that we did not perform prior power analysis. Therefore, it is possible that this study was not powered enough to detect an association between AKI risk and an increase in CK over time, if there was any. Although we did not derive valuable information in regard to ED length of stay, one may notice the relatively long average ED stay. In this county/teaching hospital, both waiting time and boarding time are relatively greater than other community hospitals. This is more evident in this patient population. Due to the fact that BTGH is the only psychiatric ED open 24 hours in Houston, it is possible that many of these patients needed inpatient psychiatric admission or transfer to another Harris county psychiatric facility, both of which can further delay disposition. CONCLUSION In this population of patients presenting to the ED with substance use, we found that there was no identifiable CK level associated with increased AKI risk. We found that CK levels continued to trend up and did not fall significantly below arrival measurements until after 42 hours. Our findings support that there is limited value in repeated ED CK measurements, and disposition should not be based solely on CK levels. Further research is needed to expand understanding of the risk relationship between AKI risk, creatinine, and CK levels with the potential for more judicious ordering of diagnostic lab tests.

Address for Correspondence: Mohammad S. Alzahri, MD, King Saud University, Department of Emergency Medicine, Riyadh, Saudi Arabia, 2111 Westheimer Road #2406, Houston, TX 77098. Email: mohammadalzahri@gmail.com. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Alzahri. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1. Larkin GL, Claassen CA, Emond JA, et al. Trends in U.S. emergency

1199

department visits for mental health conditions, 1992-2001. Psychiatr

Western Journal of Emergency Medicine

CK in ED Patients with Possible Substance-use Related Conditions Alzahri Serv. 2005;56(6):671-7.

13. Bellomo R, Ronco C., Kellum JA, et al. Acute renal failure – defini-

2. Dolan JG, Mushlin AI. Routine laboratory testing for medical disorders in

tion, outcome measures, animal models, fluid therapy and information

psychiatric inpatients. Arch Intern Med. 1985;145(11):2085-8.

technology needs: The Second International Consensus Confer-

3. Lukens TW, Wolf SJ, Edlow JA, et al. Clinical policy: Critical issues in

ence of the Acute Dialysis Quality Initiative (ADQI) Group. Crit Care.

the diagnosis and management of the adult psychiatric patient in the

2004;8(4):R204-12.

emergency department. Ann Emerg Med. 2006;47(1):79-99.

14. Chen CY, Lin YR, Zhao LL, et al. Clinical factors in predict-

4. Bosch X, Poch E, Grau JM. Rhabdomyolysis and acute kidney injury. N

ing acute renal failure caused by rhabdomyolysis in the ED. Am J Emerg

Engl J Med. 2009;361(14):1411-3.

Med. 2013;31(7):1062-6.

5. Hall RC, Gardner ER, Stickney SK, et al. Physical illness manifesting as

15. Fine DM, Gelber AC, Melamed ML, et al. Risk factors for renal failure

psychiatric disease. II. Analysis of a state hospital inpatient population.

among 72 consecutive patients with rhabdomyolysis related to illicit drug

Arch Gen Psychiatry. 1980;37:989-95.

use. Am J Med. 2004;117(8):607-10.

6. Hall RC, Gardner ER, Popkin MK, et al. Unrecognized physical illness

16. Lombard J, Wong B, Young JH. Acute renal failure due to rhabdomyoly-

prompting psychiatric admission: a prospective study. Am J Psychiatry.

sis associated with cocaine toxicity. West J Med. 1988;148(4):466-8.

1981;138(5):629-35.

17. Merigian KS, Roberts JR. Cocaine intoxication: hyperpyrexia, rhabdomy-

7. Olshaker JS, Browne B, Jerrard DA, et al. Medical clearance and

olysis and acute renal failure. J Toxicol Clin Toxicol. 1987;25(1-2):135-48.

screening of psychiatric patients in the emergency department. Acad

18. Chhabra N, Gimbar RP, Walla LM, Thompson TM. Emergency depart-

Emerg Med.1997;4(2):124-8.

ment patient burden from an electronic dance music festival. J Emerg

8. Sood TR, Mcstay CM. Evaluation of the psychiatric patient. Emerg Med

Med. 2018;54(4):435-9.

Clin North Am. 2009;27(4):669-83.. 9.

19. Akmal M, Valdin JR, McCarron MM, et al. Rhabdomyolysis with and

Ward MM. Factors predictive of acute renal failure in rhabdomyolysis.

without acute renal failure in patients with phencyclidine intoxication. Am

Arch Intern Med 1988;148(7):1553-7.

J Nephrol. 1981;1(2):91–6.

10. Kenney K, Landau ME, Gonzalez RS, et al. Serum creatine kinase after

20. Safari S, Yousefifard M, Hashemi B, et al. The value of serum creatine

exercise: drawing the line between physiological response and exer-

kinase in predicting the risk of rhabdomyolysis-induced acute kidney

tional rhabdomyolysis. Muscle Nerve. 2012;45(3):356-62.

injury: a systematic review and meta-analysis. Clin Exp Nephrol.

11. Fine DM. Risk Factors for Renal Failure among 72 Consecutive

2016;20(2):153-61.

Patients with Rhabdomyolysis Related to Illicit Drug Use. Am J Med.

21. Bertholf RL, Sharma R, Reisfield GM. Predictive value of positive drug

2004;117(8):607-10

screening results in an urban outpatient population. J Anal Toxicol.

12. Hoffman MD, Ingwerson JL, Rogers IR, et al. Increasing creatine kinase concentrations at the 161-km Western States Endurance Run. Wilder-

2016;40(9):726-31. 22. Mokhlesi B, Garimella PS, Joffe A, Velho V. Street drug abuse leading to

ness Environ Med. 2012;23(1):56-60.

Western Journal of Emergency Medicine

critical illness. Intensive Care Med. 2004;30(8):1526-36.

1200

Volume 21, no. 5: September 2020

Review Article

Sepsis Alerts in Emergency Departments: A Systematic Review of Accuracy and Quality Measure Impact Matthew I. Hwang, BS* William F. Bond MD, MS† Emilie S. Powell MD, MBA, MS‡

*University of Illinois College of Medicine at Peoria, Peoria, Illinois † University of Illinois College of Medicine at Peoria, OSF HealthCare, Jump Simulation and Department of Emergency Medicine, Peoria, Illinois ‡ Northwestern University Feinberg School of Medicine, Northwestern Memorial Hospital, Department of Emergency Medicine, Chicago, Illinois

Section Editor: Gabriel Wardi, MD Submission history: Submitted November 29, 2020; Revision received May 5, 2020; Accepted May 17, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46010

Introduction: For early detection of sepsis, automated systems within the electronic health record have evolved to alert emergency department (ED) personnel to the possibility of sepsis, and in some cases link them to suggested care pathways. We conducted a systematic review of automated sepsis-alert detection systems in the ED. Methods: We searched multiple health literature databases from the earliest available dates to August 2018. Articles were screened based on abstract, again via manuscript, and further narrowed with set inclusion criteria: 1) adult patients in the ED diagnosed with sepsis, severe sepsis, or septic shock; 2) an electronic system that alerts a healthcare provider of sepsis in real or near-real time; and 3) measures of diagnostic accuracy or quality of sepsis alerts. The final, detailed review was guided by QUADAS-2 and GRADE criteria. We tracked all articles using an online tool (Covidence), and the review was registered with PROSPERO registry of reviews. A two-author consensus was reached at the article choice stage and final review stage. Due to the variation in alert criteria and methods of sepsis diagnosis confirmation, the data were not combined for meta-analysis. Results: We screened 693 articles by title and abstract and 20 by full text; we then selected 10 for the study. The articles were published between 2009–2018. Two studies had algorithm-based alert systems, while eight had rule-based alert systems. All systems used different criteria based on systemic inflammatory response syndrome (SIRS) to define sepsis. Sensitivities ranged from 10100%, specificities from 78-99%, and positive predictive value from 5.8-54%. Negative predictive value was consistently high at 99-100%. Studies showed some evidence for improved process-ofcare markers, including improved time to antibiotics. Length of stay improved in two studies. One low quality study showed improved mortality. Conclusion: The limited evidence available suggests that sepsis alerts in the ED setting can be set to high sensitivity. No high-quality studies showed a difference in mortality, but evidence exists for improvements in process of care. Significant further work is needed to understand the consequences of alert fatigue and sensitivity set points. [West J Emerg Med. 2020;21(5)1201-1210.]

INTRODUCTION Sepsis is defined as life-threatening organ dysfunction due to a dysregulated inflammatory response to infection.1 It is implicated in an estimated 1.7 million hospitalizations each year and is among the most costly conditions for hospitals.2,3 Delays Volume 21, no. 5: September 2020

in diagnosis of sepsis can lead to delay in treatment,4,5 which can lead to increased morbidity and mortality.6 Quality measures now track time to these treatments as process markers of successful care.7 While studies have questioned some of the interventions, such as protocol-driven fluid resuscitation,8 there

1201

Western Journal of Emergency Medicine

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact is general agreement that early antibiotic administration reduces mortality from sepsis.6,9-11 Risk for delays in diagnosis led to the development of automatic electronic sepsis alerts built into electronic health record (EHR) systems.10,12,13 Some of these systems were created for use in the inpatient ward,14,15 intensive care unit (ICU),16,17 and emergency department (ED),18,19 and some stretch across settings within a healthcare system.20,21 One study demonstrated that over 75% of sepsis hospitalizations presented in the ED, warranting a focused study of this population.22 The challenge of demonstrating the marginal impact of these systems is that they act alongside existing sepsis care processes in a very ill population whose incremental change in mortality may be difficult to detect. In addition, thanks to education campaigns for staff,10 the drive toward improvement in quality measures,23 and increasing board certification of emergency providers,24 ED personnel have become better trained and are likely better at detecting sepsis. Thus, in the highly visually and electronically monitored ED setting, the benefit of these systems over clinician gestalt may diminish over time. The possibility still exists that automated sepsis alerts may be an important method to detect more subtle cases or earlier presentations and may have greater value in less monitored settings. The value of these alert systems is measured based on their detection accuracy, with a goal of high sensitivity and, more importantly, their impact on process or outcome measures. However, alert systems carry a risk of alarm fatigue and distraction.25,26 Sepsis alerts add to already increasing alarms with the EHR, including those for physiology monitors, pharmacy checking, and infectious disease isolation. The positive impact of these automated sepsis alerts and their alarm methods on sepsis care, specific to the ED, remains an open question, and drove the desire for this systematic review. Alert systems vary in their criteria. Early systems were often rule-based using the Centers for Medicare and Medicaid Services (CMS) Sepsis-1 definition of sepsis: two of four systemic inflammatory response syndrome (SIRS) criteria with a suspected or identified infection source. SIRS is defined as at least two of the following four findings: temperature >38° Celsius (C) (100.4° Fahrenheit [F]) or <36°C (96.8°F); heart rate >90 beats/minute; respiratory rate >20 breaths/minute; or white blood count >12,000 per microliter (µL) or <4000/µL or 10% band forms.1 CMS with sepsis-2 set elevated temperature at >38.3°C (100.9°F).27 More advanced systems are using algorithms, which expand on the limited criteria of rule-based systems. Such criteria may include past medical history and lab values or vitals with near-real time updating. Evaluation of the success of these systems is complicated by difficulty establishing consensus28 and evolving definitions for the sepsis spectrum, including the 2016 update to sepsis-3.1 Thus, the diagnostic criteria are both evolving and in most cases based on discharge diagnosis, rather than information available in the ED. The ability to accurately diagnose and treat a specific disease may be measured by studying discharge Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? The use of automated clinical alerts is increasing, and complex algorithmic models are now being implemented. What was the research question? How do sepsis alert systems in the emergency department perform based on accuracy and quality measures? What was the major finding of the study? Process measures moderately improved. One low-quality study showed mortality benefit, while no high-quality studies did. How does this improve population health? Further research of alert system elements is needed. Our goal is to guide the development of sepsis alerts to improve outcome measures.

diagnosis, but it may not account for clinician decisions made with limited information, as is often encountered in ED settings. Discharge diagnosis as a standard does not account for a clinician’s ability to risk stratify and exclude lifethreatening conditions, which is valuable for stabilizing patients and completing the diagnostic workup. Although using chief complaint for quality evaluation or diagnostic criteria has been proposed, it has yet to be standardized.29,30 Due to evolving systems and definitions, we systematically reviewed studies assessing the effectiveness of these alerts. Our objectives were to determine whether automated electronic sepsis alerts in the ED are accurate and whether they have an impact on quality measures and/or mortality. METHODS This review followed guidelines presented by the Preferred Reporting Items for a Systematic Review and Meta-analysis of Diagnostic Test Accuracy Studies (PRISMA-DTA) and PRISMA-P.31,32 This review was registered with PROSPERO (Prospective Register of Systematic Reviews) . Search Strategy Databases for the search included PubMed MEDLINE, Embase, the Cochrane Library, and the Cumulative Index of Nursing and Allied Health Literature (CINAHL), from the earliest available dates to August 2, 2018. We defined the search according to four fields: emergency department; sepsis; electronic health record; and alerts/alarms. Details of the search strategy are described in Appendix A.

1202

Volume 21, no. 5: September 2020

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact

Eligibility Criteria Randomized trials, performance improvement trials (including before and after studies), and cohort studies were included in the screening.Â Eligible studies included published articles with the following: 1) adult patients in the ED, diagnosed with sepsis, severe sepsis, or septic shock (hereafter referred to as sepsis); 2) an electronic system that alerts a healthcare provider of sepsis in real or near-real time; and 3) measures of diagnostic accuracy or impact on quality of care measures. Exclusion criteria included the following: 1) primary data based on non-ED settings, such as prehospital, ICU, or the general wards; 2) articles studying medical conditions that can present with sepsis, such as specific infections (eg, influenza), pregnancy-related issues, and bacteremia, without assessing sepsis independently; 3) alert systems that screen only at triage, as opposed to reaching an alert trigger threshold at any point in the ED visit; and 4) non-English language articles lacking translation. We ensured chosen articles came from peerreviewed sources based on the presence of a peer-review process description on the journal homepage. Study Records We collected citations in a reference manager software Zotero (Corporation of Digital Scholarship, George Mason University, Fairfax, VA). Article screening was completed through the online software Covidence systematic review software (Veritas Health Innovation, Melbourne, Australia). Two independent reviewers (authors WB and MH), selected for articles based on the inclusion and exclusion criteria in the title and abstract screenings. At the next stage, two independent reviewers (authors WB and EP) selected articles in the full-text screening. Conflicts were resolved through regular meetings or conference calls. Data was collected by WB and MH, and then extracted with Covidence to be stored as a secure Microsoft Excel file (Microsoft Corporation, Redmond, WA). Data Items Qualitative data items for extraction included clinical setting, study design, age group, type of alert system, definition/ threshold for the alert, method of alert notification, treatment recommendation, and reference standard. The implemented alert system was considered the index test. We classified the alert systems as rule based or algorithm based. Among the eligible studies, the rule-based alerts used SIRS criteria. The algorithmic alerts had unique measures such as vitals, Glasgow Coma Scale, and creatinine. Variations for either system are described in Table 1. Quantitative data items included sample size, population size, accuracy, and outcome measures. Outcomes and Prioritization and Diagnostic Accuracy Measures We extracted data from articles on sepsis alerts for both diagnostic accuracy and impact on quality measures. Diagnostic accuracy assesses the ability of the alert to accurately detect Volume 21, no. 5: September 2020

sepsis. Measurements included positive and negative predictive values, sensitivity, and specificity. Quality measures of interest were process and outcome measures. Examples of process markers included compliance or time to antibiotic administration, fluid resuscitation, and lactate measurement. Outcome measures included mortality and length of ICU stay, although various additional markers were captured by different authors. When reported by the authors, we used confidence intervals for the given estimates. Data Synthesis A qualitative analysis of each study was used. The variation of sepsis definition for the alerts, the set points, methods of alerting, response processes, etc prevented an aggregated quantitative analysis. Bias and Applicability Covidence included a bias rating system based on the Cochrane standard of quality assessment. We added criteria from the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) to effectively assess diagnostic accuracy of the articles, per the recommendation of PRISMA-DTA, Leeflang, and Cochrane.31,33,34 We rated quality measure articles following the guidance of GRADE (Grades of Recommendation Assessment, Development and Evaluation).35 Each article was rated for bias regarding blinding of participants and personnel to the alert, blinding of outcome assessors, incomplete outcome data, selective outcome reporting, the index test, gold standard, and flow and timing. Once each component was finalized, a consensus overall quality rating was decided based on the risk of biases. The overall quality was scaled relative to the cohort study design. No articles had strong experimental designs (ie, randomized controlled trials); therefore, quality was ranked based on comparison within this cohort of articles. Details are recorded in Appendix B. RESULTS Study Selection and Characteristics We imported 731 articles into Covidence. After duplicate removal, 693 were screened by title and abstract. Twenty articles underwent full-text assessment, and 10 were selected for the study (Figure). Eight of these studies assessed diagnostic accuracy and six assessed quality measures. All studies were prospective or retrospective cohorts and were conducted in urban, tertiary and/ or academic medical centers (Table 1). Publishing years ranged from 2009â&#x20AC;&#x201C;2018. Two studies had algorithm-based alert systems, while eight had rule sets. All systems used different criteria based on SIRS to define sepsis. There was significant variability in the criteria used for activation of the sepsis alert, the threshold definitions that activated the alert, the presence or absence of triggering links to care order sets, and the degree and type of interventions triggered by the alert. Likewise, there were variations in the diagnostic criteria standards against which the

1203

Western Journal of Emergency Medicine

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact Table 1. Characteristics of included studies.

Source (the article)

Study design

Demographic

Type of alert system (the index test)

Alsolamy 201418

Prospective cohort

>14 years old

Rule based

Austrian 201836

Retrospective ≥18 years old cohort

Bansal 201837

Prospective cohort

Berger 201038

Brown 201639

Method of alert notification

Treatment recommended

≥2 SIRS criteria and organ dysfunction, or 2 organ dysfunction criteria^

Notification to nurse who pages the physician

Rule based

1st alert is SIRS based. 2nd and 3rd alerts are sepsis based, which is the SIRS alert plus systolic blood pressure <90mmHg or lactate ≥4 mg/dL

Electronic Yes (to all 3 notifications to alerts) the following, Alert 1: nurse Alert 2: nurse Alert 3: provider

ICD-9 coding for severe sepsis or septic shock on admission only

Adult patients (though not clearly specified)

Rule based

1st alert is SIRS based. 2nd alert is a sepsis alert, which is the SIRS alert plus WBC ≥12K or ≤4K Blood cultures ordered OR Lactate >4 mg/dL alone

Team leader paged

Yes, a sepsis response team in the post alert group

2 physician reviewers using standardized sepsis criteria, approved by Mayo Clinic enterprise subspecialty councils for EM and critical care

Prospective cohort

≥19 years old

Rule based

>2 SIRS criteria plus infection source

Electronic notification to clinician

Yes, lactate ≥2 SIRS criteria recommended and clinical suspicion, retrospectively

Prospective cohort

≥14 years old

Algorithm based

75 parameters including demographics, encounter details, lab tests, SIRS criteria, and other clinical measurements

Page and email Not specified to charge nurse

Admitted from ED to ICU and either 1) ICD9 discharge diagnosis relating to sepsis or infection or 2) identification by a QI coordinator in the ICU.

Martin Rico Prospective 201740 cohort

≥14 years old

Algorithm based

Series of parameters including lab tests, SIRS criteria, vitals, and Glasgow coma scale score

Electronic notification to clinician

Chart review with “clinical experts” with ICD-9 CM discharge diagnosis of sepsis

Meurer 200941

≥70 years old

Rule based

≥2 SIRS criteria

Prospective cohort

Definition/Threshold for the alert

Reference standard (compared to the index test)

Yes, with an e-order set

Clinical evaluation by an EM or ICU physician following 2012 surviving sepsis campaign guidelines

Page to study No Chart reviewers coordinator (3) confirmed who confirms or excluded the a source of diagnosis infection from the physician ^Systolic blood pressure <90 to 86 mmHg with intravenous fluids or <86 mm Hg regardless of fluids, blood oxygen saturation <90% to 85% with supplemental oxygen or <85% without oxygen, or lactate >2 mmol/L. SIRS, systemic inflammatory response syndrome; ICU, intensive care unit; ICD-9, International Classification of Diseases, 9th ed; mmHG, millimeters of mercury; mg/dL, milligram per deciliter; mmol/L, millimole per liter; WBC, white blood count; K, thousand; EM, emergency medicine; ED, emergency department; QI, quality improvement.

Western Journal of Emergency Medicine

1204

Volume 21, no. 5: September 2020

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact

Table 1. Continued. Source (the article)

Study design

Demographic

Type of alert system (the index test)

Definition/Threshold for the alert

Method of alert notification

Treatment recommended

Reference standard (compared to the index test)

Narayanan Retrospective ≥18 years old 201642 cohort

Rule based

1st alert is SIRS based. 2nd alert is a sepsis alert, which is the SIRS alert plus end organ dysfunction or fluid nonresponsive hypotension

Electronic notification to clinician

Chart review with ICD-9 code diagnosis of severe sepsis and septic shock

Nelson 201143

Prospective cohort

Rule based

≥2 SIRS criteria and 2 systolic blood pressure measurements less than 90mmHg

All caregivers notified with a page

Yes

Chart review with the same SIRS and hypotension criteria

Nguyen 201444

Retrospective All ED cohort patients*

≥18 years old

Rule based

≥2 SIRS criteria, Not specified Not specified 300 patients for and systolic blood which the alert pressure ≤90mmHg or did not fire were lactic acid ≥2.0mg/dL. randomly selected *”While children have different ranges for SIRS criteria, <1% of emergency department (ED) patients were <18 years old…” SIRS, systemic inflammatory response syndrome; ICD-9, International Classification of Diseases, 9th ed; mmHg, millimeters of mercury; mg/dL milligram per deciliter; ED, emergency department.

alerts were weighed, with most studies using chart review confirmation, while some used clinician confirmation. Only Nguyen et al had a control group of 300 randomly selected patients during a study period when the alert did not fire.44 All of the other articles were either prospective or retrospective cohort designs without control groups. Diagnostic Accuracy Diagnostic accuracy was recorded in Table 2 below. Specificity ranged from 78-99%, and positive predictive value (PPV) from 5.8% to 54%. Negative predictive value (NPV) was consistently high at 99-100%. Excluding Meurer et al,41 sensitivity ranged from 64-100%. Meurer et al had a sensitivity of 33.3% for the electronic alert alone, and 10.7% for the electronic alert and attending confirmation. With attending confirmation, specificity increased from 78.0% to 97.6%. The study had a low activation threshold of ≥2 SIRS criteria, the smallest sample size of 84, and an age range of 70 years or older. Patients were only included if they presented between 3 am and 9 pm on weekdays. This study also only included patients admitted from the ED, instead of all ED patients, risking selection bias. The notification system sent a page to the study coordinator before confirming with a physician. In contrast, other studies directly notified a member of the clinical team, excluding Nguyen et al, which did not describe the notification method.44 Five rule-based studies were of high quality.18,36,37,43,44 Two studies had systems with high accuracy. Alsolamy et al18 had a sensitivity of 93.2%, specificity 98.4%, and PPV 21.0%. Bansal et al37 had a sensitivity of 100%, Volume 21, no. 5: September 2020

specificity 96.2%, and PPV 29.3%. Highest PPV was achieved by Nelson et al43 at 54.0% and Nguyen et al44 at 44.7%. Austrian et al36 shared the number of total alerts fired, for any of three criteria sets including SIRS, nurse alert, and physician alert that included progressively more ill criteria. They report sensitivities of 73.0%, 23.8%, and 23.0%, respectively, and PPV of 13.0%, 22.4%, and 26.6% as expected for the more progressively stringent criteria. They did not share the denominator of all ED presenting patients for the retrospective period under study but report the total number of hospitalized

Figure. PRIMSA flow diagram.

1205

Western Journal of Emergency Medicine

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact Table 2. Diagnostic accuracy.

Sample size (n)*

Population size (N)^

Sensitivity (95% CI)

Specificity (95% CI)

Positive predictive value (95% CI)

Alsolamy 201418

205

49,838

93.18 (88.78-96.00)

98.44 (98.33-98.55)

20.98 (18.50-23.70)

Austrian 201836

1306

Not specified

Bansal 201837

419

27106

100 (99.12-100)

96.21 (95.97-96.43)

29.3

100

High

Brown 201639

352

93,733

76.4

95.3

5.8

99.9

Low

Martin Rico 201740

178

37,323

85 (67.2-99.5)

89 (88.8-89.7)

Low

Meurer 200941

Alert alone: 26 Alert and attending confirmation: 9

583

Nelson 201143

Sens. and Spec.: 1375 PPV and NPV: 1386

33460

Source

Negative predictive value (95% CI) 99.97 (99.95-99.98)

14.6

High High

Alert alone: Alert alone: 33.3 (23.3-43.4) 78.0 (71.7-84.4) Alert and attending Alert and attending confirmation: confirmation: 10.7 (4.1-17.3) 97.6 (95.2-99.9) 64

Overall Quality

Low

High

Nguyen 795 Not specified 44.7 High 201444 (41.2-48.2) *Alerts for sepsis meeting the diagnostic criterion standard of the individual article. ^Patients presenting to the emergency department (ED). CI, confidence interval; PPV, positive predictive value; NPV, negative predictive value; Sens. and Spec., sensitivity and specificity.

sepsis patients based on discharge diagnosis. Septic patients may have been sent home, but if we assume they captured all true positives and false negatives through final diagnosis of sepsis, this allows for calculation of sensitivity and PPV and does not allow the calculation of specificity or NPV. With 2144 patients with a final diagnosis of severe sepsis or septic shock, and 97,216 alerts (any of the three levels included), they had the largest retrospective sample size. Two studies assessing algorithm-based alerts were deemed low quality. Brown et al39 measured a sensitivity of 76.4%, specificity of 95.3%, and a low PPV of 5.8%. Martin Rico et al40 measured a sensitivity of 85%, specificity of 89%, and a PPV of 19%. Prevalence of sepsis compared to total ED patients was 0.3-2% in five studies.18,37,39,40,43 Meurer et al had a prevalence of 14.4%, but this was among patients â&#x2030;Ľ70 years old and it was the sole study with only SIRS criteria (a low threshold) for its sepsis definition. Quality Measures Quality measures are described in Table 3. Two studies evaluating quality measures were high quality: Austrian et al36 and Nelson et al.43 In Austrian et al, process markers of time to first lactate and vasopressor use significantly improved. Statistically insignificant findings included blood cultures drawn before antibiotic administration and whether a lactate was drawn. Antibiotic timing was not reported. For Nelson et al, process Western Journal of Emergency Medicine

markers of blood culture collection and chest radiograph before admission improved. Statistically insignificant findings included lactate collection and antibiotics given in the ED. Outcome measures of ICU transfer, ICU length of stay (LOS), and total LOS significantly improved for Austrian et al. Mortality did not improve significantly for either study. Four studies (Bansal, Berger, Martin Rico, Narayanan)37,38,40,42 were judged to be of low quality. Berger et al had significant improvement in lactate testing. Narayanan et al improved antibiotics within 60 minutes, time to antibiotics, and LOS. Bansal et al37 had nearly 100% sensitivity and specificity with a modest PPV of 29.3%, and had no significant improvements in survival rate. Of note, we established the article to be high quality in regard to diagnostic accuracy, while outcome measures were low quality. In contrast, Austrian et al36 had improvements in LOS and Martin Rico et al40 had improvements in mortality, while both exhibited moderate accuracy. None of the rule-based studies showed statistically significant improvements in mortality.36-38,42,43 The only outcome reported by an algorithm-based study (Martin Rico et al)40 was mortality, which showed significant improvement, although the study was judged to be of low quality. The alert system Narayanan et al42 studied did not recommend treatment as other systems did. For this rule-based system â&#x20AC;&#x153;antibiotics in 60 minutesâ&#x20AC;? meant time to antibiotics, and LOS significantly improved.

1206

Volume 21, no. 5: September 2020

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact

Table 3. Quality Measures. Source Austrian 201836

Sample size (n)* Before sepsis alert: 838

Population size (N)^

Significant results (process/outcome marker: prior vs. after)

Insignificant results

2144#

ICU transfer: 36.9% vs. 25.8%, p<0.001 ICU length of stay (days): 1.8 (3.7) vs. 1.2 (3.1), p<0.001 Length of stay (days): 10.1 (SD 10.1) vs. 8.6 (SD 7.9), p<0.001 Time to first lactate (days): 0.19 (0.94) vs. 0.16 (0.58), p<0.001 Vasopressor used: 28.8% vs. 22.7%, p<0.01

Blood cultures drawn prior to antibiotics: 79.0% vs 79.2%, p=0.92 In-hospital mortality: 8.5% vs. 7%, p=0.22 Lactate drawn (excluding â&#x2030;Ľ24hrs after ED arrival): 90.7% vs. 91.3%, p=0.65

High

In-hospital survival rate with SSRT activation in full cohort: 0.69 (95% CI, 0.31 to 1.56) In-hospital survival rate with SSRT activation among severe sepsis/septic shock patients: 0.53 (95% CI, 0.26 to 1.11)

Low

After Sepsis alert: 1306

Bansal 201837

Whole cohort: n=419

27106

Severe sepsis and septic shock: n=252

Berger 201038

Before sepsis alert: Lactate-151, Hyperlactatemia-33, Mortality-908. After sepsis alert: Lactate-366, Hyperlactatemia-54, Mortality-890

Martin Rico 1190 201740

Narayanan Prior to sepsis alert: 201642 n=111

Before alert: 2903

Lactate testing: 5.2% vs. 12.7% (95% CI, 6.0 Change in frequency of to 9.0%) absolute increase p<0.001 hyperlactatemia if lactate was tested: 21.9% vs. 14.8% (95% CI, -0.4 to 14.6) Mortality: 5.7% vs. 5.2% (95% After alert: CI, -1.6 to 2.6%, p=0.64) 2893&

37,323

Mortality: 36.3% vs. 26.1% Adjusted risk reduction for mortality: 36% (0.43-0.97) Incidence Rate Ratio: 0.64, p=0.036

not specified

Antibiotics in 60 minutes: 48.6% vs. 76.7%, p<.001 Length of stay odds ratio: [0.66 (0.53-0.82)] Mean time to antibiotics (minutes): 61.5 (33-171) vs. 29 (2-59), p<.001

After sepsis alert: n=103 Nelson 201143

184

Overall quality

Low

Mortality odds ratio: 0.64 (0.26-1.57)

Low

33460

Blood culture collected odds ratio: Antibiotic given in ED odds High [2.9 (1.1-7.7)] ratio: [2.8 (0.9-8.6)] Chest radiograph before admission odds Lactate collected odds ratio: ratio: [3.2 (1.1-9.9)] [1.7 (0.9-3.2)] *Alerts for sepsis meeting the diagnostic criterion standard of the individual article. ^Patients presenting to the emergency department. #All hospitalizations with severe sepsis or septic shock &All patients with sepsis. Total ED presentations not specified. ICU, intensive care unit; vs, versus; SD, standard deviation; ED, emergency department; CI, confidence interval; SSRT, sepsis and shock response team.

DISCUSSION Overall, most of the study designs used to assess the impact of sepsis alerts were weak, and the review authors had difficulty isolating the impact of the automated sepsis alert itself from broader interventions such as response teams or order set bundles. Thus, our review conclusions must be couched within the strength of the overall low-quality evidence. The limited evidence available suggests that sepsis alerts in Volume 21, no. 5: September 2020

the ED setting can be tuned to a high sensitivity for the detection of sepsis. Evidence from both low- and high-quality studies showed some improved process-of-care markers, including time to antibiotics, with the use of automated sepsis alerts.36,38,40,42,43 Lactate testing was studied by four groups with two producing significant results. Other than lactate measurement, no single measure consistently improved across studies. A lack of consistency of measured items and

1207

Western Journal of Emergency Medicine

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact measurement methods creates a challenge in forming a conclusion. For example, one study examined whether blood cultures were collected, as opposed to blood cultures collected before antibiotic administration. No high-quality studies showed a difference in mortality, and only one high-quality study showed impacts on ICU LOS and vasopressor use.36 Our findings are in keeping with a review by Makam in 2015 that covered alerts both inside and outside of the ED environment.45 Our review added recently published articles, including those that now use an algorithmic as opposed to simple rules-based approaches, and was focused on patients presenting to the ED. The strongest study designs we reviewed for inclusion were prospective cohort studies, but we would call attention to a well-executed performance improvement study conducted by Gatewood et al.46 They included a computerized alert with a multipronged intervention and showed a substantial improvement in sepsis bundle of care compliance. However, they did not show differences in mortality in part due to the inclusion of lower risk patients on the sepsis spectrum. Sepsis alerts represent a difficult area to study with traditional randomized methods. One challenge is that in the course of operational improvement, sepsis alert criteria and/or alert thresholds may be subtly changed in the background. This may be done by information technology, analytics, or EHR personnel to address PPV or safety concerns, usually with a clinicianâ&#x20AC;&#x2122;s input, but often without alerting all ED staff to the change. Moving to a more rigorous study design requires holding the alert constant and ethical approval for a non-alert or clinician gestalt arm. Thus, success will likely be found in future studies that use time series, or perhaps cluster randomized rollout methods across healthcare systems. Likewise, future areas for study could include comparisons of the method of alert, and the presence or absence of treatment recommendations. None of the studies addressed potential harms. Harm may include the alarm issues impacting staff, missing alternative diagnoses due to early anchoring on sepsis, and the follow-on effects of early, aggressive fluid intervention, which has been questioned more broadly in the sepsis literature.8 Significant further work is needed on the alarm consequences of the sensitivity set points, and if possible, such work should incorporate influences from other nonsepsis alarms in alarm fatigue. Although low quality, one algorithmic system showed significant mortality improvement, potentially validating its further development.40 Systems such as this are being developed to improve accuracy and PPV, and may include risk factors such as comorbid conditions and past medical history. These systems can effectively insert multiple variables into an equation using current and past patient data as regression coefficients, running the calculation repeatedly over the course of a patient stay as more predictor variables become available. The data creating the coefficients of such a regression-based equation would influence the predictorâ&#x20AC;&#x2122;s value. For example, a sepsis predictor tool based on the elderly would likely not be predictive for Western Journal of Emergency Medicine

children. The newest models of sepsis alerts include machine learning. Complex algorithmic models may use well over 50 variables, and a machine-learning program may be integrated into them. Machine learning uses computer programming to identify patterns and significant predictors beyond the reasonable capabilities of humans. With continual analysis, it can fine-tune coefficients and thresholds of the algorithm. Initial studies show promise,47-49 and additional research is required to assess its impact on clinical outcomes. LIMITATIONS Our limitations include a risk of publication bias because we did not search the gray literature or clinical trials for studies in progress. There are likely many hospital systems that have implemented sepsis alerts, collected data, and did not report it. Our consensus group was small in number, but we followed a rigorous process using review rubrics guided by well-accepted grading criteria. CONCLUSION Automated sepsis alerts in the ED may be set to a high sensitivity. Process measures show moderate benefit; however, no single measure has consistently improved, and high-quality studies have yet to demonstrate, a mortality benefit. Specific components of these systems, alarm fatigue, and sensitivity set points should be examined further. Sepsis alerts demonstrate utility and future research is indicated to build a more ideal alert system. Address for Correspondence: Matthew Hwang, BS, University of Illinois College of Medicine at Peoria, Jump Simulation, 1306 N Berkeley Ave., Peoria, IL 61603. Email: mihwang2@uic.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: ÂŠ 2020 Hwang et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

REFERENCES 1. Singer M, Deutschman CS, Seymour CW, et al. The Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3). JAMA. 2016;315(8):801-10. 2. Torio CM, Andrews RM. National Inpatient hospital costs: the most

1208

expensive conditions by payer, 2011. 2013. Available at: https://www. hcup-us.ahrq.gov/reports/statbriefs/sb160.pdf. Accessed February 2, 2020.

Volume 21, no. 5: September 2020

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact

3. Rhee C, Dantes R, Epstein L, et al. Incidence and trends of sepsis

2014;14:105.

in US hospitals using clinical vs claims data, 2009-2014. JAMA.

19. Balamuth F, Alpern ER, Abaddessa MK, et al. Improving recognition

2017;318(13):1241-9.

of pediatric severe sepsis in the emergency department:

4. Candel FJ, Sá MB, Belda S, et al. Current aspects in sepsis

contributions of a vital sign based electronic alert and bedside

approach. Turning things around. Rev Esp Quimioter.

clinician identification. Ann Emerg Med. 2017;70(6):759-68.e2.

2018;31(4):298-315.

20. Guirgis FW, Jones L, Esma R, et al. Managing sepsis: electronic

5. Kassyap CK, Abraham SV, Krishnan SV, et al. Factors affecting

recognition, rapid response teams, and standardized care save lives.

early treatment goals of sepsis patients presenting to the emergency department. Indian J Crit Care Med. 2018;22(11):797-800.

J Crit Care. 2017;40:296-302. 21. Westphal GA, Pereira AB, Fachin SM, et al. An electronic warning

6. Andersson M, Östholm-Balkhed Å, Fredrikson M, et al. Delay of

system helps reduce the time to diagnosis of sepsis. Rev Bras Ter

appropriate antibiotic treatment is associated with high mortality in

Intensiva. 2018;30(4):414-22.

patients with community-onset sepsis in a Swedish setting. Eur J Clin

22. Angel C, Leisman DE, Schneider SM, et al. Sepsis presenting in

Microbiol Infect Dis. 2019;38(7):1223-34.

emergency department versus inpatient settings: divergences in

7. Centers for Medicare and Medicaid Services. Severe sepsis and

prevalence, patient characteristics, initial resuscitation, outcomes,

septic shock: management bundle. Centers for Medicare and Medicaid

and costs. Ann Emerg Med. 2018;72(4, Supplement):S149.

Services (CMS). 2014. Available at: https://cmit.cms.gov/CMIT_public/

23. Ramsdell TH, Smith AN, Kerkhove E. Compliance with updated

ViewMeasure?MeasureId=1017. Accessed April 13, 2020.

sepsis bundles to meet new sepsis core measure in a tertiary care

8. The ARISE Investigators, ANZICS Clinical Trials Group, Peake SL, et al. Goal-directed resuscitation for patients with early septic shock. N

hospital. Hosp Pharm. 2017;52(3):177-86. 24. Reiter M, Wen LS, Allen BW. The emergency medicine workforce:

Engl J Med. 2014;371(16):1496-1506.

profile and projections. J Emerg Med. 2016;50(4):690-3.

9. Ballester L, Martínez R, Méndez J, et al. Differences in hypotensive

25. Guardia-LaBar LM, Scruth EA, Edworthy J, et al. Alarm fatigue: the

vs. non-hypotensive sepsis management in the emergency

human-system interface. Clin Nurse Spec. 2014;28(3):135-7.

department: door-to-antibiotic time impact on sepsis survival. Med

26. Curry JP, Jungquist CR. A critical assessment of monitoring practices,

Sci. 2018;6(4):91.

patient deterioration, and alarm fatigue on inpatient wards: a review.

10. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign: International Guidelines for Management of Sepsis and Septic

Patient Saf Surg. 2014;8(1):29. 27. Sartelli M, Kluger Y, Ansaloni L, et al. Raising concerns about the

Shock: 2016. Intensive Care Med. 2017;43(3):304-77.

sepsis-3 definitions. World J Emerg Surg. 2018;13:6.

11. Johnston ANB, Park J, Doi SA, et al. Effect of immediate

28. Kalantari A, Mallemat H, Weingart SD. Sepsis definitions: the

administration of antibiotics in patients with sepsis in tertiary

search for gold and what CMS got wrong. West J Emerg Med.

care: a systematic review and meta-analysis. Clin Ther.

2017;18(5):951-6.

2017;39(1):190-202.e6.

29. Griffey RT, Pines JM, Farley HL, et al. Chief complaint-based

12. Tafelski S, Nachtigall I, Deja M, et al. Computer-assisted decision

performance measures: a new focus for acute care quality

support for changing practice in severe sepsis and septic shock. J Int Med Res. 2010;38(5):1605-16.

measurement. Ann Emerg Med. 2015;65(4):387-95. 30. National Quality Forum. Advancing chief complaint-based quality

13. Herasevich V, Pieper MS, Pulido J, et al. Enrollment Into a time

measurement. 2019. Available at: http://www.qualityforum.org/

sensitive clinical study in the critical care setting: results from

Publications/2019/06/Advancing_Chief_Complaint-Based_Quality_

computerized septic shock sniffer implementation. J Am Med Inform

Measurement_Final_Report.aspx. Accessed April 13, 2020.

Assoc. 2011;18(5):639-44.

31. McInnes MDF, Moher D, Thombs BD, et al. Preferred Reporting

14. McRee L, Thanavaro JL, Moore K, et al. The impact of an electronic

Items for a Systematic Review and Meta-analysis of Diagnostic

medical record surveillance program on outcomes for patients with

Test Accuracy Studies: the PRISMA-DTA statement. JAMA.

sepsis. Heart Lung. 2014;43(6):546-9.

2018;319(4):388-96.

15. Sawyer AM, Deal EN, Labelle AJ, et al. Implementation of a real-time

32. Moher D, Shamseer L, Clarke M, et al. Preferred Reporting Items for

computerized sepsis alert in non-intensive care unit patients. Crit

Systematic Review and Meta-Analysis Protocols (PRISMA-P) 2015

Care Med. 2011;39(3):469-73.

Statement. Syst Rev. 2015;4(1):1.

16. Hooper MH, Weavind L, Wheeler AP, et al. Randomized trial of

33. Leeflang MMG, Deeks JJ, Gatsonis C, et al. Systematic reviews of

automated, electronic monitoring to facilitate early detection of sepsis in the intensive care unit. Crit Care Med. 2012;40(7):2096-101.

diagnostic test accuracy. Ann Intern Med. 2008;149(12):889-97 34. Deeks JJ, Wisniewski S, Davenport C. Guide to the contents of

17. Brandt BN, Gartner AB, Moncure M, et al. Identifying severe sepsis

a Cochrane diagnostic test accuracy protocol. 2013. Available at:

via electronic surveillance. Am J Med Qual. 2015;30(6):559-65.

https://methods.cochrane.org/sites/methods.cochrane.org.sdt/files/

18. Alsolamy S, Al Salamah M, Al Thagafi M, et al. Diagnostic accuracy of a screening electronic alert tool for severe sepsis and septic

public/uploads/Ch04_Sep2013.pdf. Accessed March 2, 2020. 35. Dijkers M. Introducing GRADE: a systematic approach to rating

shock in the emergency department. BMC Med Inform Decis Mak.

Volume 21, no. 5: September 2020

evidence in systematic reviews and to guideline development. KT

1209

Western Journal of Emergency Medicine

Hwang et al.

Sepsis Alerts in EDs: A Systematic Review of Accuracy and Quality Measure Impact Update. 2013;1(5):9.

Emerg Med. 2016;34(2):185-8.

36. Austrian JS, Jamin CT, Doty GR, et al. Impact of an emergency

43. Nelson JL, Smith BL, Jared JD, et al. Prospective trial of real-time

department electronic sepsis surveillance system on patient mortality

electronic surveillance to expedite early care of severe sepsis. Ann

and length of stay. J Am Med Inform Assoc. 2018;25(5):523-9.

Emerg Med. 2011;57(5):500-4.

37. Bansal V, FestiÄ&#x2021; E, Mangi MA, et al. Early machine-human interface

44. Nguyen SQ, Mwakalindile E, Booth JS, et al. Automated electronic

around sepsis severity identification: from diagnosis to improved

medical record sepsis detection in the emergency department. PeerJ.

management? Acta Med Acad. 2018;42 (1):27-38.

2014;2:e343.

38. Berger T, Birnbaum A, Bijur P, et al. A computerized alert screening

45. Makam AN, Nguyen OK, Auerbach AD. Diagnostic accuracy and

for severe sepsis in emergency department patients increases lactate

effectiveness of automated electronic sepsis alert systems: a

testing but does not improve inpatient mortality. Appl Clin Inform.

systematic review. J Hosp Med. 2015;10(6):396-402.

2010;1(4):394-407.

46. Gatewood MO, Wemple M, Greco S, et al. A quality improvement

39. Brown SM, Jones J, Kuttler KG, et al. Prospective evaluation of an

project to improve early sepsis care in the emergency department.

automated method to identify patients with severe sepsis or septic

BMJ Qual Saf. 2015;24(12):787-95.

shock in the emergency department. BMC Emerg Med. 2016;16(1):31.

47. Delahanty RJ, Alvarez J, Flynn LM, et al. Development and

40. Martin Rico P, Valdivia Perez A, and Lacalle Martinez JM. Electronic

evaluation of a machine learning model for the early identification of

alerting and decision support for early sepsis detection and

patients at risk for sepsis. Ann Emerg Med. 2019;73(4):334-44.

management: impact on clinical outcomes. Eur J Clin Pharmacol.

48. Shimabukuro DW, Barton CW, Feldman MD, et al. Effect of a

2017;19(1):33-40.

machine learning-based severe sepsis prediction algorithm on patient

41. Meurer WJ, Smith BL, Losman ED, et al. Real-time identification of

survival and hospital length of stay: a randomised clinical trial. BMJ

serious infection in geriatric patients using clinical information system surveillance. J Am Geriatr Soc. 2009;57(1):40-5.

Open Respir Res. 2017;4(1):e000234. 49. Nemati S, Holder A, Razmi F, et al. An interpretable machine learning

42. Narayanan N, Gross AK, Pintens M, et al. Effect of an electronic

model for accurate prediction of sepsis in the ICU. Crit Care Med.

medical record alert for severe sepsis among ED patients. Am J

Western Journal of Emergency Medicine

2018;46(4):547-53.

1210

Volume 21, no. 5: September 2020

Original Research

Dumpster Diving in the Emergency Department: Quantity and Characteristics of Waste at a Level I Trauma Center Sarah Hsu, BA* Cassandra L. Thiel, PhD† Michael J. Mello, MD, MPH‡ Jonathan E. Slutzman, MD§

*Warren Alpert Medical School of Brown University, Providence, Rhode Island † New York University Grossman School of Medicine, Department of Population Health, New York, New York ‡ Warren Alpert Medical School of Brown University, Department of Emergency Medicine, Providence, Rhode Island § Massachusetts General Hospital and Harvard Medical School, Department of Emergency Medicine, Boston, Massachusetts

Section Editor: Brian J. Yun, MD Submission history: Submitted April 27, 2020; Accepted June 25, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.47900

Introduction: Healthcare contributes 10% of greenhouse gases in the United States and generates two milion tons of waste each year. Reducing healthcare waste can reduce the environmental impact of healthcare and lower hospitals’ waste disposal costs. However, no literature to date has examined US emergency department (ED) waste management. The purpose of this study was to quantify and describe the amount of waste generated by an ED, identify deviations from waste policy, and explore areas for waste reduction. Methods: We conducted a 24-hour (weekday) ED waste audit in an urban, tertiary-care academic medical center. All waste generated in the ED during the study period was collected, manually sorted into separate categories based on its predominant material, and weighed. We tracked deviations from hospital waste policy using the hospital’s Infection Control Manual, state regulations, and Health Insurance Portability and Accountability Act standards. Lastly, we calculated direct pollutant emissions from ED waste disposal activities using the M+WasteCare Calculator. Results: The ED generated 671.8 kilograms (kg) total waste during a 24-hour collection period. On a per-patient basis, the ED generated 1.99 kg of total waste per encounter. The majority was plastic (64.6%), with paper-derived products (18.4%) the next largest category. Only 14.9% of waste disposed of in red bags met the criteria for regulated medical waste. We identified several deviations from waste policy, including loose sharps not placed in sharps containers, as well as re-processable items and protected health information thrown in medical and solid waste. We also identified over 200 unused items. Pollutant emissions resulting per day from ED waste disposal include 3110 kg carbon dioxide equivalent and 576 grams of other criteria pollutants, heavy metals, and toxins. Conclusion: The ED generates significant amounts of waste. Current ED waste disposal practices reveal several opportunities to reduce total waste generated, increase adherence to waste policy, and reduce environmental impact. While our results will likely be similar to other urban tertiary EDs that serve as Level I trauma centers, future studies are needed to compare results across EDs with different patient volumes or waste generation rates. [West J Emerg Med. 2020;21(5)1211-1217.]

INTRODUCTION Healthcare facilities create significant amounts of waste, with a majority of medical supplies disposed after a single use. Healthcare facilities in the United States (U.S.) generate over Volume 21, no. 5: September 2020

6600 metric tons of waste each day, which is approximately 19 kg per patient per day and 2 million tons of waste each year.1,2 This makes healthcare the second largest industry to contribute to landfill waste (only the food industry generates 1211

Western Journal of Emergency Medicine

Hsu et al.

Quantifying Waste in the Emergency Department more).3 Overall, the US healthcare industry contributes nearly 10% of all US greenhouse gas emissions as well as additional other pollutants that adversely affect human health.4 Healthcare waste is a direct contributor to these emissions and an important indicator of the impact of procurement practices. Improper sorting of medical wastes can increase healthcare costs. Overuse of hazardous waste disposal increases the costs and footprint of hauling and treating normal solid wastes. However, underuse of hazardous waste disposal can pose a public health risk and may incur fines. In some clinical spaces, even unused items must be thrown away after a patient has been treated in the space, adding not only to the footprints and costs of disposal, but also to the purchase of unnecessary, wasted supplies. Prior studies have shown that there is poor segregation of healthcare waste into proper waste streams, and there are many opportunities to divert waste from landfills, such as recycling and single-use device reprocessing for reuse.5-7 Efforts to reduce healthcare waste generation and improve sorting practices have the potential to reduce the environmental impact of healthcare and significantly reduce hospital costs for waste hauling and supply procurement.8 A waste audit is an effective way to visualize the categories of waste generated in a healthcare facility, locate where and how different types of waste are processed, and identify areas for waste diversion or waste management improvement. While waste audits of entire hospitals, intensive care units, operating rooms, and specific surgical procedures have been conducted previously, there is no literature surrounding waste management of the emergency department (ED) in the US.6-10 As a result, little is known about the quantity and characteristics of ED waste. However, the ED represents a significant portion of US healthcare, generating approximately 140 million visits annually, nearly 5% of all healthcare expenditures, and 12% of all outpatient visits.11,12 Understanding the characteristics of ED waste allows hospitals to find opportunities to reduce waste disposal costs and reduce the environmental impact of emergency care. The objective of this study was to quantify and describe the amount of waste generated by an ED, identify deviations from local waste management policies and guidelines, calculate direct pollutant emissions from waste disposal practices, and explore areas for waste diversion and reduction. METHODS Study Design and Setting We conducted a 24-hour physical waste audit in the ED of an urban, tertiary-care academic medical center. The medical center is a Level I trauma center with an annual ED volume of approximately 110,000 patient encounters per year. Following standard practice, Environmental Services staff collected all municipal solid waste (MSW), regulated medical waste (RMW), and recycling waste between 11 pm July 25 and 11 pm July 26, 2019. RMW included all items thrown in red biohazard bags as well as filled sharps containers. Recycling waste included Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? Healthcare contributes 10% of United States’ greenhouse gases and two million tons of waste each year. No literature on emergency department (ED) waste management currently exists. What was the research question? What are the quantities, characteristics, and carbon footprint (CO2e) of ED waste? What was the major finding of the study? The ED produced 672 kilograms (kg) waste/day and 1.99 kg waste/patient. Waste disposal itself emitted 3 metric tons CO2-e/day. How does this improve population health? Pollution and climate change harm human health. Reducing ED waste may reduce upstream and downstream pollution, mitigate climate change effects, and save money.

all items in recycling bins and all paper with protected health information (PHI) disposed of in secure bins. All waste described during the study period was stored in a designated collection space. The waste was then sorted into separate categories based on predominant material and subsequently weighed. The waste categories we selected were the following: hard plastic; soft plastic; paper products; food waste; textiles; glass; metal; electronic waste; and unused items/mixed materials. All unused items (defined as unopened items or opened but unused items) and uneaten food were indexed and counted. Any loose sharps found were also segregated and weighed as mixed material, but due to safety reasons were not counted. All pulse-oximetry probes found in the waste were also counted due to the institutions’ ability to send them for single-use device reprocessing, a process regulated by the US Food and Drug Administration to allow for single-use devices to be cleaned, repaired (if needed), re-sterilized (when indicated), inspected, and re-packaged for clinical use.13 Pharmaceutical wastes are handled by the hospital’s pharmacy department and were excluded. No universal wastes were collected during the study period. We identified deviations from hospital waste policy by using definitions from the hospital’s Infection Control Manual and Health Insurance Portability and Accountability Act standards. In addition, since RMW has been regulated on a state level since the US Medical Waste Tracking Act of 1988 expired in 1991, we defined RMW using state regulations. State regulations define medical waste as blood and blood products (including draining, liquid state, and materials saturated or dripping with

1212

Volume 21, no. 5: September 2020

Hsu et al.

Quantifying Waste in the Emergency Department

blood); pathological waste (including human anatomical parts and specimens of body fluids, excluding urine, nasal secretions, sweat, sputum, vomit, or fecal matter that don’t contain visible blood or confirmed diagnosis of infectious disease); cultures and stocks of infectious agents and associated biologicals; contaminated animal waste; sharps (including medical items that can cause punctures or cuts); and biotechnology by-product effluents.14 Such waste must be rendered safe (for example, via autoclave and shredding sharps) and then may be disposed with MSW. Otherwise, it must be handled by certified haulers to be treated off-site. The primary author (SH) was present and supervised all waste sorting and weighing, which was completed with the assistance of the senior author (JES) and three research assistants. Any disagreements regarding appropriate waste category were resolved by consensus. All study personnel wore strict isolation personal protective equipment throughout the waste audit, and sharps containers were weighed “asis” without opening and sorting any of their contents. Upon completion of the waste sorting, all waste was disposed in compliance with hospital policies. This project was undertaken as a quality improvement initiative at our subject hospital, and as such was not formally supervised by the institutional review board per its policies. Measurements All categories and types of waste were weighed using an Edlund ERS-60 Digital Receiving Scale with a sensitivity of 0.005 kilograms (kg). ED administrative staff provided aggregate data on patient volume and total length of stay during the 24-hour period of study and for total fiscal year 2019 for normalization purposes.

Data Analysis Data were entered into and analyzed using Excel (Microsoft, Redmond, WA), with univariate analysis listing frequency counts and percentages. To obtain estimates of annual waste generation rates, we normalized data collected by number of patient encounters, number of patient-hours in the ED, and by time, and we subsequently extrapolated by totals of those values for the fiscal year. We estimated direct pollutant emissions from waste disposal activities using the M+WasteCare Calculator (Mazzetti, San Francisco, CA.), specifying that MSW was landfilled, RMW was autoclaved and then landfilled, and recyclables were sent for recycling. M+ Wastecare calculates the approximate pollutant load associated with each step in the waste’s journey for each pollutant.15 These pollutant loads are added to give a final amount, in carbon dioxide equivalent (CO2e), the standard unit for carbon footprints. In all cases, emissions factors are used to perform the calculation. This includes (as applicable), emissions associated with transportation to the disposal facility, emissions associated with energy used for disposal (for autoclaving and alternative sources), emissions associated with transportation of any residuals to landfill, and emissions associated with landfill. RESULTS Over the 24-hour period of our study, we collected a total of 671.785 kg of waste (Table 1), or 1.999 kg/patient encounter. Of this total, 84% (567.38 kg) was collected in MSW (clear) bags, 11% (71.665 kg) in RMW (red biohazard) bags, and 5% (32.74 kg) in recycling bins. Excluding sharps containers, which were not individually audited for safety reasons, only 15% (7.45 kg) of the waste disposed in red bags met the criteria for RMW. Assuming all contents of sharps

Table 1. Composition of all wastes produced in emergency department in 24-hour period. Category of waste MSW

RMW

Recycling

Total

Mass (kg)

567.38

71.665

32.74

671.785

% of total

84.46%

10.67%

4.87%

100.00%

Hard plastic

110.615

17.79

2.525

130.93 (19.5%)

Soft plastic

Material 289.775

13.305

303.08 (45.1%)

Paper

92.43

3.105

28.011

123.546 (18.4%)

Food

40.865

0.62

41.485 (6.2%)

Textiles

18.695

4.72

23.415 (3.5%)

Glass

6.74

0.175

1.02

7.935 (1.2%)

Unused/mixed

5.065

8.02

0.94

14.025 (2.1%)

Metal

2.415

0.04

0.19

2.645 (0.4%)

Electronic waste

0.78

0.04

0.82 (0.1%)

Sharps 23.85 23.85 (3.6%) MSW, municipal solid waste (landfill); RMW, regulated medical waste (includes red bag or hazardous solid waste and sharps); kg, kilogram.

Volume 21, no. 5: September 2020

1213

Western Journal of Emergency Medicine

Hsu et al.

Quantifying Waste in the Emergency Department containers were correctly disposed, less than 5% (31.015 kg) of total waste was true RMW. Similarly, less than 5% (32.74 kg) of all waste was disposed in recycling bins. The majority, 86% (28.011 kg), of recycling waste consisted of paper records with PHI thrown in secure bins. Excluding the paper containing PHI, 20% (0.95 kg) of waste thrown into the recycling bins was not recyclable. The predominant material found in both MSW and RMW was plastic, at 65% of total waste, 71% (400 kg) of MSW, and 43% (31 kg) of RMW. The second most abundant waste category in MSW and RMW was paper products: 16% (92.43 kg) of MSW and 4% (3.105 kg) of RMW. Of all paper product waste, only 23% (28.011 kg) was shredded and recycled through the PHI paper secure bin. The third largest category in MSW and RMW was food waste, totaling over 41 kg, or 6% of the total. Within food waste, 19% (over 8 kg) was unopened or uneaten food, such as diet cranberry juice, bananas, and milk cartons, most of which are food items found in brownbag meals given to patients. Several large-quantity items in MSW and RMW were also sorted and weighed. There were 6.35 kg of emesis basins; 2.96 kg of tourniquets, of which 420 grams (g) or 76 tourniquets were still bundled and unused; 43.63 kg of gloves; and 24.395 kg of disposable cups. We found 201 unused items (5.92 kg), such as normal saline syringes, intravenous (IV) catheters, electrocardiogram and monitor electrode packets, and IV fluid bags in both MSW and RMW bags. Additional data regarding the breakdown of solid, medical, and recycling waste is in the supplementary appendix. Base case pollutant emissions resulting per day from waste disposal include 3110 kg CO2e (71% from RMW, 29% from MSW, and <1% from recycling) and 576 g of other criteria pollutants, heavy metals, and toxins (84% RMW, 13% MSW, and 3% recycling). These greenhouse gas emissions are equivalent to driving a car 7700 miles and only represent the pollution from the disposal of waste, not including the upstream environmental costs of their production, distribution, and use.16 We identified several deviations from institutional waste policy. We found paper products with PHI in MSW and RMW, which should have been placed in the PHI-paper secure bin. There were 285 g of loose sharps in standard red bags rather than being placed in sharps containers, which would have accounted for 1.9% of total sharps waste

(assuming all contents of sharps containers were actual sharps). In addition, 29 pulse-oximeter probes that should have been diverted and sent for re-processing were found in both MSW and RMW. Extrapolating our one-day data to a full year, our subject ED is estimated to generate 194,163 - 245,202 kg of waste annually (see Table 2). DISCUSSION In this 24-hour waste audit of an academic, tertiary-care ED, we collected, sorted, characterized, and weighed 672 kg of waste, representing 1.999 kg/patient. To the best of our knowledge, this is the first documented waste audit of an ED in the US and represents an important start in describing and improving upstream and downstream environmental impacts of the emergency care we provide. Little is known about the quantity and characteristics of ED waste in the US, and the only prior studies of ED waste have been conducted in Jordan and Australia. Two audits of EDs in Jordan published in 2004 and 2007 revealed that the daily generation rate per patient ranges from 0.289 â&#x20AC;&#x201C; 0.479 kg/patient/day, lower than the findings of our study.17,18 Comparing to our institution, though, is challenging given the likely large differences in operations in a nonOrganization for Economic Cooperation and Development country. In 2019, a study detailing a pilot program to reduce ED waste in a regional Australian hospital did not audit the waste, but found that efforts to improve waste segregation and recycling failed due to poor compliance.19 Staff felt that the process was time consuming and complicated and environmental services staff were seen mixing different waste bins together to simplify the process. In our study, 85% of all waste thrown into RMW did not meet the criteria for RMW. Given that RMW costs 5-10 times as much to dispose of compared to solid waste, diverting non-RMW from the red biohazard bags is a significant opportunity for cost savings. While our waste audit revealed that over 10% of total ED waste was disposed of as RMW, the Centers for Disease Control and Prevention (CDC) suggests that only 3-5% of hospital waste requires disposal as RMW.20 However, if all nonmedical waste were diverted from the RMW bags, the percentage of true RMW, including sharps, would be 4.6%, within the range of the CDC criteria.

Table 2. Estimated annual rate of ED waste generation for Fiscal Year 2019. Measured daily rate

Measured FY2019 stats

Estimated annual rate

Waste (Kg) /day (d)

671.785 kg/d

x 365 d/y

245,202 kg/y

Waste (Kg) /patient

1.999 kg/patient

x 113,297 patients/y

226,522 kg/y

0.244 kg/patient-h

x 853,397 patient-h/y

194,163 kg/y

Waste (Kg) /patient hours (h)

MSW, municipal solid waste (landfill); RMW, regulated medical waste (includes red bag or hazardous solid waste and sharps); kg, kilogram; y, year.

Western Journal of Emergency Medicine

1214

Volume 21, no. 5: September 2020

Hsu et al.

Quantifying Waste in the Emergency Department

The Healthcare Plastics Recycling Council estimates that approximately 20-25% of healthcare waste is plastic.21 Another study published in 2003 that looked at waste in a Massachusetts hospital revealed that only 20% of solid waste was plastic.22 However, a total of 65% of ED waste in our study was plastic, higher than both estimates. This is likely due to the fact that plastics production has been increasing exponentially over the past few decades.23 Other studies have shown that plastic composition is highest in the ED and that locations where there is high turnover of patients and poor set-up of bin locations reduces proper waste disposal as a priority.17,24 This discrepancy is also likely heightened due to the prevalence of single-use disposable devices in the ED. Efforts to explore reusable alternatives can also lead to waste reduction and supply savings.25 There is potential for increased recycling in the ED. Assuming all waste made of metal, glass, paper, and hard plastic (that do not meet RMW criteria) are able to be diverted and recycled, up to 258 kg or 38% of all ED waste could be recycled and diverted from landfill waste. Given that nearly 20% of items thrown into existing recycling bins (excluding secure paper bins) was non-recyclable, any efforts to increase recycling in the ED would need to be accompanied by training and other system changes to improve accessibility to recycling bins. Loose sharps not contained in sharps containers and paper containing PHI were found in both MSW and RMW. In an ED with hundreds of healthcare workers, hundreds of environmental services staff, and thousands of patients, it shouldnâ&#x20AC;&#x2122;t be surprising to find occasional waste handling deficiencies. Unfortunately, such events can pose a significant health hazard to staff and privacy risk to patients. These findings could expose a healthcare institution to regulatory agency action. Waste audits of entire hospitals in Turkey, Iran, and Brazil revealed that 17.1-31% of total hospital waste constitutes food or organic waste.26-28 This fraction of food waste is higher than in our study of the ED likely due to the fact that audits of entire hospitals include cafeteria or kitchen waste. However, the amount of food waste is still significant. Given that the average person eats 905 kg of food a year, one year of food waste from this ED could feed roughly 17 people for a year.29 In addition to the upstream pollution embedded in our supply chains, waste disposal itself directly generates pollutant emissions. Looking specifically at greenhouse gas emissions, which lead to climate change, and extrapolating from a oneday sample, our EDâ&#x20AC;&#x2122;s waste contributes over 1000 tons CO2e per year. This is equivalent to the greenhouse gas emissions of driving 200 passenger cars for one year in the US.16 There are additional emissions of toxins, criteria air pollutants, and heavy metals totaling over 200 kg annually. These pollutants, including arsenic, cadmium, dioxins, mercury, nitrogen oxides, sulfur dioxide, particulate matter, and volatile organic compounds, all harm human health. And this does not account for the upstream emissions resulting from the manufacture, transport, and use of materials. Volume 21, no. 5: September 2020

Our results indicate that significant improvements can be made to optimize ED waste management in order to reduce total waste generated, emissions from treatment, and waste hauling and treatment costs. If all the metal, glass, and hard plastic were recycled, if all the pulse-oximeter probes were reprocessed, all batteries went to electronic waste, all food waste was composted or diverted (or better yet, reduced), and all unused items were restocked or donated, approximately 305 kg or 45% of waste could have been diverted. Maximally optimizing the waste stream has the potential to divert over 100 tons of ED waste from the landfill each year. Optimizing the waste stream is only one part of the solution. One of the benefits of a waste audit is understanding the supply chain of the ED, as simple mass balance would dictate that nearly everything that enters the ED as a supply leaves the department with the patient or as waste. Upstream changes, such as switching disposable items to reusable items and researching opportunities for single-use device reprocessing have the potential to reduce the volume of disposables purchased. Given that the single largest category of waste in the ED is soft plastic, most of which is packaging, efforts to order items that use less packaging or purchase commonly used items separately from kits, has a high potential for waste reduction. LIMITATIONS This study has several limitations to its study design. First, our audit was conducted on a single day and the results may not be representative of the full year. During this 24hour period, 336 new patients were seen in the ED, which is higher than the daily average for fiscal year 2019 of 310 patients per day. As a result, an annual waste generation rate based simply on multiplying our one-day total by 365 days may be an over-estimate. We therefore generated two separate estimates of annual waste generation rates using kg/patient encounter and kg/patient-hour for comparison (see Table 2). This study was also conducted at a single site. While our results will likely be similar to other urban tertiary EDs that serve as Level I trauma centers, future studies are needed to compare results across EDs in other settings, which may have different patient volumes, waste generation rates, waste sorting practices and policies, and waste hauling and treatment contracts. For logistical and safety reasons, we limited some of our measurement capabilities. Waste items with liquid contents were classified without regard for the liquid components (i.e., full or incompletely-emptied IV fluid bags were classified as soft plastic). Sharps containers were weighed as-is, with the assumption that all contents were correctly sorted. A visual review of items through the plastic containers confirms this not to be true but could not be quantified due to safety concerns of opening and sorting sharps. Similarly, all true medical waste found in red

1215

Western Journal of Emergency Medicine

Hsu et al.

Quantifying Waste in the Emergency Department biohazard bags were not sorted into predominant categories and were calculated as mixed under “unused/mixed” materials in Table 1. All loose sharps in red biohazard bags were weighed in total. PHI-containing paper products incorrectly disposed in MSW were not segregated and weighed out of respect for patient privacy. Pharmaceutical waste was specifically excluded, as it is handled by the pharmacy department and not environmental services at our facility; however, the quantity was not expected to significantly alter the results presented here.

total knee arthroplasties. Can J Surg. 2013;56(2):97-102. 4. Eckelman MJ, Sherman JD. Estimated global disease burden from US health care sector greenhouse gas emissions. AM J Public Health. 2018;1008(S2):S120-2. 5. DiConsiglio J. Reprocessing SUDs reduces waste, costs. Mater Manag Health Care. 2008;17(9):40-2. 6. Kubicki MA, McGain F, O’Shea CJ, et al. Auditing an intensive care unit recycling program. Crit Care Resusc. 7. McGain F, Jarosz KM, Nguyen MNHH, et al. Auditing operating room recycling: a management case report. A&A Case Rep.

CONCLUSION Overall, the ED generated 2 kg of waste per patient encounter, 672 kg of waste per day, and an estimated 194,000 – 245,000 kg of waste per year. We also found poor segregation of MSW and RMW, and several deviations from institutional waste policies. Our study reveals opportunities to reduce total waste generated, decrease hospital waste costs, and reduce the environmental impact of emergency care. ACKNOWLEDGMENTS We thank all EVS staff involved in executing the logistics of this study, along with Michael Dykstra, Hargun Khanna, and Simone Sasse for assisting with the waste audit.

2015;5(3):47-50. 8. Reidel LM. Environmental and financial impact of a hospital recycling program. AANA J. 2011;79(4):S8-14. 9. Hames K. Healthcare waste disposal: an analysis of the effect of education on improving waste disposal. Healthcare Infection. 2013;18(3):110-4. 10. de Sa D, Stephens K, Kuang M, et al. The direct environmental impact of hip arthroscopy for femoroacetabular impingement: a surgical waste audit of five cases. J Hip Preserv Surg. 2016;3(2):132-7. 11. Lee MH, Schuur JD, Zink BJ. Owning the cost of emergency medicine: beyond 2%. Ann Emerg Med. 2013;62(5):498-505.e3. 12. Overview of the Nationwide Emergency Department Sample (NEDS). Healthcare Cost and Utilization Project. 2017. Available at: https://www.hcup-us.ahrq.gov/nedsoverview.jsp. Accessed December 6, 2019.

Address for Correspondence: Sarah Hsu, Warren Alpert Medical School of Brown University, Box G-9290, Providence, RI 02912. Email: sarah_hsu@brown.edu.

13. Center for Devices and Radiological Health. Reprocessing and reuse of single-use devices. Available at: http://www.fda. gov/regulatory-information/search-fda-guidance-documents/

reprocessing-and-reuse-single-use-devices. Accessed March 17, 2019. 14. Minimum Requirements for the management of medical or biological waste (State sanitary code chapter VIII), 105 CMR §480 (2010). Available at: https://www.mass.gov/regulations/105-CMR48000-minimum-requirements-for-the-management-of-medical-orbiological-waste. Accessed March 17, 2019. 15. Vernon W, Hartman J, Rothleder K, et al. The complexities of waste management. IFHE Digest. 2016:53-6. 16. United States Environmental Protection Agency. Greenhouse Gas Equivalencies Calculator. Available at: https://www.epa.gov/energy/ greenhouse-gas-equivalencies-calculator. Accessed March 31, 2020. 17. Bdour A, Altrabsheh B, Hadadin N, et al. Assessment of medical wastes management practice: a case study of the northern part of

REFERENCES

Jordan. Waste Manag. 2007;27(6):746-59.

1. Kwakye G, Brat GA, Makary MA. Green surgical practices for health

18. Awad AR, Obeidat D, Al-Shareef M. Mathematical-statistical models

care. Arch Surg. 2011;146(2):131-6.

of generated hazardous hospitals solid waste. J Environ Sci Heal A.

2. Howard J. Less waste how-to guide. Practice Greenhealth. 2019. Available at https://practicegreenhealth.org/sites/default/

2004;39(2):315-27. 19. Church R, Briggs D, Tran V. Climate emergency: towards a greener

files/2019-03/hhi-booklet-waste-bu_v3.0_full.pdf. Accessed November 17, 2020.

emergency department. Emerg Med Australas. 2019;31(2):274-5. 20. Sustainability Roadmap for Hospitals. Reduce Regulated

3. Stall NM, Kagoma YK, Bondy JN, et al. Surgical waste audit of 5

Western Journal of Emergency Medicine

Medical Waste (RMW) Generation. Available at: http://www.

1216

Volume 21, no. 5: September 2020

Hsu et al.

Quantifying Waste in the Emergency Department

sustainabilityroadmap.org/pims/42#.XepxwuhKg2w. Accessed

and costing methods for device procurement: comparing reusable

December 6, 2019

and single-use disposable laryngoscopes. Anesth Analg.

21. Healthcare plastics recycling solutions for hospitals | HPRC.

2018;127(2):434-43.

Available at: https://www.hprc.org/hospitals. Accessed December

26. Altin S, Altin A, Elevli B, et al. Determination of hospital waste

5, 2019.

composition and disposal methods: a case study. Pol J Environ

22. Lee BK, Ellebecker MJ, Moure-Ersaso R. Alternatives for treatment and disposal cost reduction of regulated medical wastes. Waste

Stud. 2003;12(2):251-5. 27. Dehghani MH, Azam K, Changani F, et al. Assessment of medical

Manag. 2004;24(2):143-51.

waste management in educational hospitals of Tehran University

23. Ritchie H, Roser M. Plastic Pollution. 2020. Available at: https://

Medical Sciences. J Environ Health Sci. 2008;5(2):131-6.

ourworldindata.org/plastic-pollution. Accessed February 16, 2020.

28. Ferreira AP, Veiga MM. Hospital waste operational procedures: a

24. Grose J, Bennallick M, Nichols A, et al. How can behaviour change

case study in Brazil. Waste Manage Res. 2003;21(4):377-82.

theory contribute to a reduce, re-use and recycle approach to waste

29. United States Department of Agriculture Economic Research

management in the UK NHS: a feasibility study. Sustainability.

Service. Food consumption and nutrient intakes. 2014. Available

2011:1-8.

at: https://www.ers.usda.gov/Data/FoodConsumption/. Accessed

25. Sherman JD, Raibley LA, Eckelman MJ. Life cycle assessment

Volume 21, no. 5: September 2020

December 5, 2019.

1217

Western Journal of Emergency Medicine

Original Research

Patient Characteristics and Clinical Process Predictors of Patients Leaving Without Being Seen from the Emergency Department Niels K. Rathlev, MD* Paul Visintainer, PhD† Joseph Schmidt, MD* Joeli Hettler, MD* Vanna Albert, MD, MBA* Haiping Li, MD*

*University of Massachusetts Medical School – Baystate, Department of Emergency Medicine, Springfield, Massachusetts † University of Massachusetts Medical School – Baystate, Department of Epidemiology and Biostatistics Core, Springfield, Massachusetts

Section Editor: David Lee, MD Submission history: Submitted February 26, 2020; Revision received June 29, 2020; Accepted June 29, 2020 Electronically published August 25, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.47084

Introduction: Delays in patient flow in the emergency department (ED) result in patients leaving without being seen (LWBS). This compromises patient experience and quality of care. Our primary goal was to develop a predictive model by evaluating associations between patients LWBS and ED process measures and patient characteristics. Methods: This was a cross-sectional study in a 95,000 annual visit adult ED comparing patients LWBS, with controls. Data were drawn from four seasonally adjusted four-week periods (30,679 total visits). Process measures included 1) arrivals per hour; 2) “door-to-provider” time; and the numbers of 3) patients in the waiting room; 4) boarding ED patients waiting for an inpatient bed; 5) providers and nurses (RN); and 6) patients per RN. Patient characteristics collected included 1) age; 2) gender; 3) race/ethnicity; 4) arrival mode (walk-in or via emergency medical services [EMS]); and 5) acuity based on Emergency Severity Index (ESI). Univariable analyses included t-tests and Pearson’s chi-square tests. We split the data randomly into derivation and validation cohorts. We used backward selection to develop the final derivation model, and factors with a p-value ≤ 0.05 were retained. Estimates were applied to the validation cohort and measures of discrimination (receiver operating characteristic) and model fit were assessed. Results: In the final model, the odds of LWBS increased with the number of patients in the waiting room (odds ratio [OR] 1.05; 95% confidence interval [CI], 1.03 to 1.06); number of boarding patients (OR 1.02; 95% CI, 1.01 to 1.03); arrival rate (OR 1.04; 95% CI, 1.02 to 1.05) and longer “door-to-provider” times (test of linear trend in the adjusted OR was p = 0.002). Patient characteristics associated with LWBS included younger age (OR 0.98; 95% CI, 0.98 to 0.99), and lower acuity (higher ESI category) (OR 2.01; 95% CI, 1.84 to 2.20). Arrival by EMS was inversely associated with LWBS (OR 0.29; 0.23 to 0.36). The area under the curve for the final model in the validation cohort was 0.85 (95% CI, 0.84 to 0.86). There was good agreement between the observed and predicted risk. Conclusion: Arrival rate, “door-to-provider time,” and the numbers of patients in the waiting room and ED boarders are all associated with patients LWBS. [West J Emerg Med. 2020;21(5)1218-1226.]

Western Journal of Emergency Medicine

1218

Volume 21, no. 5: September 2020

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED

INTRODUCTION Delays in care in the emergency department (ED) lead to a higher number of patients who leave without being seen (LWBS), ie, who are triaged and register for care but subsequently leave without any evaluation by a provider.1 On occasion, these patients suffer from significant illnesses or injuries and would, in hindsight, have benefited from time-sensitive medical interventions including emergency care. This compromises not only patient experience, but also safety, quality of care, and risk management. Many of the patients who LWBS do so because of delays in being seen, and up to 70% seek attention within 24 hours of leaving, either by returning to the ED or by presenting to alternative sites of medical care.2-3 Finally, high rates of LWBS negatively impact institutional revenue and may present a significant financial loss to the institution.4-5 High LWBS rates are predictably a challenge in large teaching institutions in metropolitan areas. High acuity and volume are associated with increased ED length of stay and rates of ED boarding and “left before completing treatment”; 6-7 non-profit institutions often compare unfavorably with forprofit competitors on these measures.8-9 Consequently, it is important to appreciate that hospitals have different baselines of performance that may be tied to volume and capacity, rather than quality of care. Notably, the current study was performed in a Level I trauma center teaching institution, which is the only tertiary-care referral center in a large, four-county area in western Massachusetts. It is one of the busiest EDs in New England based on annual volumes and has exceptionally high acuity based on the 2018 Association of Academic Chairs of Emergency Medicine Annual Survey. Our ED ranked above the 75%ile in both annual volume and rate of LWBS compared with national medians on this survey. The associations between LWBS rates, crowding, boarding of admitted patients in the ED, and delays in care have been described but continue to resist solution.10 Our aim was to evaluate associations between individuals who LWBS and ED process measures and patient characteristics, and to derive and validate a predictive model. We undertook this endeavor by performing a cross-sectional comparative study with the goal of developing a highly predictive model for discriminating patients who LWBS from those who initiate evaluation and treatment by a provider. METHODS Setting The study was performed at x Medical Center in y, z in the medical-surgical (non-psychiatric) adult ED, which is comprised of 66 licensed bays. Pediatric patients were excluded from the study. In 2015, 95,000 annual visits were seen in the adult ED with a baseline LWBS rate of 7.0%. The “provider in triage” model was not implemented during the study period. We defined a patient as LWBS if the individual received, at minimum, an abbreviated triage consisting of 1) reason for ED visit and age; 2) registered for care but Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Specific patient characteristics, new patient arrivals, and boarding hours are associated with patients leaving without being seen (LWBS). What was the research question? Can we derive and validate a highly predictive model of a patient LWBS? What was the major finding of the study? We validated a model with “very good” discrimination that included both patient characteristics and clinical process indicators. How does this improve population health? The model can be used in real time to predict whether a patient presenting for emergency care is likely to LWBS.

subsequently left without full registration or evaluation, ie, history or physical exam, by an advanced practitioner (AP) or physician. The patient was documented as LWBS after being called in the waiting room or treatment area with no response to overhead announcement on three separate occasions at 15-minute intervals. We typically did not know the exact time of LWBS unless the patient specifically informed the staff of his or her intent, but this was the exception and not the rule. Data was collected from our electronic health record and tracking system Cerner (North Kansas City, MO) on an hourly basis since process measures can vary significantly over longer time periods. We documented the relevant data at the beginning of every hourly interval starting with 00:00. Our goal was to evaluate the associations between individuals LWBS and 1) patient characteristics, and 2) process measures related to throughput and staffing. Study Design We employed a cross-sectional study design in which patients who LWBS were compared with controls who initiated evaluation and treatment by an AP or physician. To control for seasonal variability, data were drawn from four four-week periods (30,679 total visits) in September– December 2015 and March–June 2016. Patients were eligible for inclusion if they registered in the ED at any time during the indicated periods. Patients were excluded from analysis if they arrived under police escort or died in the ED.

1219

Western Journal of Emergency Medicine

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED Measured Variables Patient characteristics collected included age, gender, race/ethnicity (White, Hispanic, Black, Asian, or other/not indicated), arrival mode (walk-in or via emergency medical services [EMS]), acuity based on Emergency Severity Index (ESI) level, month of presentation (ie, March, June, September or January) and time of registration (categorized into four six-hour time periods). Insurance status was not included as a variable because patients had not completed the registration process, and their insurance status was not documented at the time they LWBS. Process measures of ED utilization and resource allocation were routinely collected every hour.11 These measures included the numbers of 1) patients in the waiting room; 2) patients in treatment bays (licensed ED bays plus hallway beds); and 3) boarders, ie, admitted patients waiting for an in-patient bed. The staff members on duty included the following: 1) attending physicians; 2) advanced practioners (AP); 3) emergency medicine residents; 4) registered nurses (RN) – our ED does not employ licensed practical nurses; and 8) patient care technicians who perform vital signs, obtain laboratory samples including blood draws and electrocardiograms, etc. The ratio of the numbers of patients per RN was computed based on patients in ED treatment bays only. The arrival rate of patients was measured for the 60-minute period in which study subjects presented. Finally, we measured the “door-to-provider time” (attending physician, resident, or AP) in 30-minute increments starting with the initial ED presentation. We chose 30-minute increments because previous literature has concluded that delays of 30 or 60 minutes appear to be critical time periods for patients when deciding to LWBS.12 Statistical Analysis Preliminary descriptive analyses included means and standard deviations, medians and ranges for continuous variables. We described categorical variables using frequency distributions. Univariable approaches included t-tests for continuous data and Pearson’s chi-square for categorical data. Since our sample size was large, we split the data into derivation (n = 14,937) and validation cohorts (n = 14,445) in order to assess the fit of the model. The data were split randomly and were approximately balanced on the number of days from each month. We used generalized estimating equations (GEE) with a binomial family and logit link to derive parameter estimates. Further, the GEE model clustered on day of registration (to account for day of the week) and employed robust standard errors. We used backward selection to develop a final model based on the derivation cohort (n = 14,937). Beginning with a model that included all variables, the least significant of those remaining was removed in an iterative fashion. Any process measures – related to utilization and resource allocation – with a Western Journal of Emergency Medicine

Rathlev et al.

p-value ≤ 0.05 were retained in the final model. The same was true of patient characteristics that met this criterion. The model that emerged from the backward selection process was compared to other model configurations of utilization variables that were considered to potentially capture LWBS risk. The final model was selected from among these comparisons using significance testing of variables for nested models or by way of a modified Akaike information criterion (AIC) for GEE models, as recommended by Pan and implemented in Stata by Cui.13-14 We then evaluated the final model from the derivation cohort in the validation cohort (n = 14,445). Discrimination was evaluated using a receiver operating characteristic (ROC) curve with 95% confidence intervals. Calibration was represented using a plot of the observed vs predicted risk of LWBS over deciles of categories. We also assessed calibration fit by computing the integrated calibration index (ICI).15 The ICI computes the difference between the observed and predicted probabilities over the range of predicted probabilities. Estimates of the mean, median, and the maximum absolute difference, Emax, are provided.16 Statistical analyses were conducted in Stata v15.1 (StataCorp, College Station, TX) and R (https://www.R-project.org/; Foundation for Statistical Computing, Vienna, Austria). The study was approved by the institutional review board of Baystate Medical Center. RESULTS A total of 30,679 patients visited the ED during the fourmonth study period. We calculated the following mean data for the study population: 1) 82 admissions per day; 2) 251 patients presenting per day; 3) 7.2% LWBS rate; and 4) 2.9% ESI 1; 40.8% ESI 2; 38.2% ESI 3; 17% ESI 4; 1.1% ESI 5. After removing 1297 observations due to exclusion criteria and missing data, 29,382 patients (95.8%) were available for study. In this cohort of 29,382 individuals, a total of 2,213 patients (7.5%) LWBS. Tables 1 and 2 show the description of the derivation cohort (n = 14,937) and differences between patients who did and did not LWBS. A total of 1122 (7.5%) patients LWBS. Although p-values for the comparisons of the two groups were statistically significant, absolute differences between the groups were generally small. There was a significant increase in the proportion of patients LWBS as the “door-to-provider” time increased in 30-minute increments. Table 3 shows the adjusted odds ratios (OR) for the variables that were retained in the final regression model based on the derivation cohort. Patient characteristics associated with LWBS included younger age (OR 0.98; 95% CI, 0.98 to 0.99) and lower acuity (higher ESI category) (OR 2.01; 95% CI, 1.84 to 2.20). Arrival by EMS was inversely associated with LWBS (OR 0.29; 0.23 to 0.36). In general, the odds of LBWS increased as clinical demand increased, as measured by number of patients in the waiting room (OR

1220

Volume 21, no. 5: September 2020

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED

Table 1. Univariable analysis of patient characteristics – derivation model. LWBS Patient Characteristics

Total

Yes

(N = 14,937)

(N = 13,815)

(N = 1,122)

P-value

49.5 (20.5)

50.4 (20.5)

38.1 (15.6)

< 0.001

Female

7,976 (53.4)

7,330 (53.1)

646 (57.6)

Male

6,961 (46.6)

6,485 (46.9)

476 (42.4)

2.65 (0.78)

2.60 (0.77)

3.18 (0.65)

Age, mean (SD) Gender, n (%)

Acuity (ESI score), mean (SD)

0.004

Race/ethnicity, n (%)

<0.0001 < 0.001

White

7,893 (52.8)

7,507 (54.3)

386 (34.4)

Hispanic

4,731 (31.7)

4,242 (30.7)

489 (43.6)

Black

1,852 (12.4)

1,697 (12.3)

155 (13.8)

Asian

178 (1.2)

162 (1.2)

16 (1.4)

Other/unknown

283 (1.9)

207 (1.5)

76 (6.8)

Arrival Mode, n (%)

< 0.001

Walk-in

8,198 (54.9)

7,234 (52.4)

964 (85.9)

EMS

6,739 (45.1)

6,581 (47.6)

158 (14.1)

Month, n (%)

< 0.001

September

3,647 (24.4)

3,396 (24.6)

251 (22.4)

December

3,585 (24.0)

3,401 (24.6)

184 (16.4)

March

4,020 (26.9)

3,558 (25.7)

462 (41.2)

June

3,685 (24.7)

3,460 (25.1)

225 (20.1)

6-hour time period, n (%)

<0.001

0001 – 6 am

1,858 (12.4)

1,754 (12.7)

104 (9.3)

6 am – 12 pm

4,614 (30.9)

4,444 (32.2)

170 (15.2)

12 pm – 6 pm

5,383 (36.0)

4,848 (35.1)

535 (47.7)

6 pm – midnight

3,082 (20.6)

2,769 (20.0)

313 (27.9)

LWBS, leaving without being seen; SD, standard deviation; EMS, emergency medical services; ESI, Emergency Severity Index.

1.05; 95% CI, 1.03 to 1.06), number of patients in treatment bays (OR 1.02, 95% CI, 1.01 to 1.02), number of boarding patients (OR 1.02; 95% CI, 1.01 to 1.03), and arrival rate (OR 1.03; 95% CI, 1.02 to 1.05). Adjusting for all other factors in the model, the odds of LWBS increased with longer “doorto-provider” times (measured in 30-minute increments). For this measure, a test of a linear trend in the adjusted ORs was significant at p < 0.002. Parameter estimates from the derivation cohort were applied to the validation cohort and measures of discrimination and model fit were assessed. Figure 1 shows the receiver operating characteristic (ROC) curve for discriminating patients who LWBS from those for whom evaluation and treatment by a provider was initiated. The model has “very good” discrimination as indicated by an area under the curve (AUC) of 0.85 (95% CI, 0.84 to 0.86). Figure 2 presents a plot of the observed vs predicted risk over deciles of the predicted risk. The plot shows good agreement between the observed risk and what was predicted Volume 21, no. 5: September 2020

by the model. To further assess model calibration, we also computed the integrated calculation index (ICI) (mean absolute difference) and associated measures.1-2 The ICI and median (E50) absolute difference between the observed and predicted probabilities over the range of predicted probabilities were 0.009 and .005, respectively. These estimates indicate that on average model predictions are nearly identical to observed probabilities. The 90th percentile (E90) and maximum difference (Emax) were 0.03 and 0.12, respectively. Thus, 90% of the differences between the observed and predicted probabilities were no larger than about three absolute percentage points. The largest absolute difference between observed and expected probabilities was 12%. The mean (ICI) and median (E50) absolute difference between the observed and predicted probabilities were 0.009 and .005, respectively. The 90th percentile (E90) and maximum difference (Emax) were 0.03 and 0.12, respectively. The largest absolute difference between observed and expected probabilities was 12%.

1221

Western Journal of Emergency Medicine

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED Table 2. Clinical process variables – derivation model. ED LWBS Process measures

Total

Yes

(N = 14,937)

(N = 13,815)

(N = 1,122)

P-value

11.1 (7.7)

10.7 (7.6)

16.4 (7.5)

< 0.001

Treatment bays

83.8 (19.2)

83.2 (19.3)

91.9 (15.6)

< 0.001

Boarders

16.1 (7.8)

15.8 (7.7)

19.9 (7.8)

< 0.001

Attending physicians

4.7 (1.3)

5.0 (1.2)

< 0.001

Advanced practitioners

2.9 (0.7)

0.825

Number, mean (SD) Waiting room

EM residents

4.0 (1.9)

4.2 (1.9)

< 0.001

Registered nurses

22.0 (3.2)

22.8 (2.7)

< 0.001

Patient care technicians

9.8 (2.3)

10.6 (2.0)

< 0.001

Patient - RN ratio, mean (SD)

4.2 (0.9)

4.7 (0.8)

< 0.001

Arrival rate/hour, mean (SD)

18.1 (6.7)

18.0 (6.7)

19.9 (6.4)

< 0.001

ED occupancy rate, mean (SD)

1.0 (0.2)

1.2 (0.2)

< 0.001

“Door-to-provider” time n (%)

<0.001

<30 mins

1912 (12.8)

1885 (13.6)

27 (2.4)

30 mins – 59 mins

3628 (24.3)

3510 (25.4)

118 (10.5)

60 mins – 89 mins

3527 (23.6)

3280 (23.7)

247 (22.0)

90 mins – 119 mins

2674 (17.9)

2406 (17.4)

268 (23.9)

120+ mins

3196 (21.4)

2734 (19.8)

462 (41.2)

ED, emergency department; LWBS, leaving without being seen; SD, standard deviation; EM, emergency medicine; RN, registered nurse; mins, minutes.

DISCUSSION The unique contribution of the study was the simultaneous focus on both patient characteristics and ED process measures, and the subsequent development of a validated model by analyzing the predictors most associated with patients LWBS. The final model demonstrated “very good” discrimination with an AUC of 0.85, which suggests that the model can add significant value in “real time” in distinguishing between patients who LWBS vs patients who stay for treatment. We used a comparative cross-sectional design to study an ED population of large sample size linking patient factors and ED processes to the rate of LWBS. Moreover, we validated our model in a separate cohort after adjusting for seasonality. Several previous comparative studies have been performed focusing solely on patient characteristics associated with LWBS.17-18 The results of these studies corroborate our finding that younger age is associated with a higher number of patients LWBS. We also found significant associations with lower acuity (higher ESI level) and arrival as a “walk-in” rather than by EMS. In terms of ED process measures, the number of boarders, patients in the waiting room and in treatment bays, arrival rate and “door-to-provider” times emerged as independent Western Journal of Emergency Medicine

predictors in our study. Despite the fact that these measures appear to be closely correlated, we limited multicollinearity by studying a large sample of patients and using a regression model with backward selection. This methodology removed many of the process measures from the final model. Consequently, we believe that our final model represents stable and precise estimates of measures associated with LWBS. The significance is that real-time modification of any of the measures, independently of the rest, may be associated with a reduction in the number of patients LWBS. Moreover, identifying the key ED process measures from our model can lead to targeted hospital-wide strategies for improving day-today operations. Hospital inefficiency and lack of patient flow result in an increase in the number of ED boarders, which emerged as a significant predictor of LWBS in our study. Optimized systems design and focused attention on the problem of boarding are required in the ED as well as on an institutional level in order to effect positive change.19 The “provider in triage” model was not implemented during the study period (nor has it been since completion of the study) since we believe the model is a resource intensive “work-around” of the true problem of ED boarding and poor hospital throughput. LWBS continues to be

1222

Volume 21, no. 5: September 2020

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED

Table 3. Final model: regression coefficients – derivation sample. Explanatory variable

Odds ratio

95% CI

P-value

Age

0.98

[0.98 - 0.99]

< 0.001

Acuity

2.02

[1.85 - 2.21]

< 0.001

Arrival mode

0.29

[0.23 - 0.36]

< 0.001

Arrival rate/hour

1.03

[1.02- 1.05]

< 0.001

Hour (linear spline) 0001 – 0600

1.0 (reference)

0601 – 1200

0.28

[0.18 - 0.42]

< 0.001

1201 - 1800

0.4

[0.26 - 0.62]

<0.001

1801 - 0000

0.56

[0.38 – 0.81]

0.002

Race/ethnicity

White (reference)

1.00

Hispanic

1.24

[1.04- 1.48]

Black

1.19

[0.96- 1.49]

0.11

Asian

1.22

[0.63- 2.37]

0.55

Other/unknown

4.86

[3.42- 6.92]

< 0.001

0.02

Month Sep 2015

1.0 (reference)

Dec 2015

0.78

[0.55 - 1.11]

0.18

Mar 2016

1.34

[0.98 - 1.84]

0.06

Jun 2016

1.02

[0.74 - 1.40]

0.90

No. in waiting room

1.05

[1.03- 1.06]

< 0.001

No. in treatment bays

1.01

[1.01 – 1.02]

< 0.001

No. of boarders

1.02

[1.01- 1.03]

0.001

[0.95 - 1.89]

0.09

Mean “door-to-provider” time <30 minutes

1.0 (reference)

30 mins – 59 mins

1.34

60 mins – 89 mins

1.69

[1.20 - 2.39]

0.003

90 mins – 119 mins

1.87

[1.28 – 2.73]

0.001

120+ mins

1.99

[1.34 – 2.96]

0.001

test of linear trend in odd ratios: p = 0.0002 CI, confidence interval, mins, minutes. a

a challenge even in EDs that have implemented the provider in triage model albeit at a lower level. This model is far from ubiquitous and results in a net expense to the organization since the provider cannot bill for the service on the professional side (at least not in the state of Massachusetts). We continue to prefer to address the “real” problem rather than providing a less optimal and more expensive approach to emergency care. While in-patient occupancy and LOS are important measures of patient flow in the hospital, we were not able to obtain these data in one-hour increments in our institution; we therefore could not include these measures in our model. Using an alternative method based on queueing theory principles, Wiler et al also determined that reducing the Volume 21, no. 5: September 2020

number of patients boarding in the ED reduces the rate of LWBS.10 A regression analysis model focused exclusively on ED process measures determined that the total number of patients cared for in the ED, number of resuscitation and trauma patients, and the number of observation admissions explained only 52.8% of the variability in LWBS.20 ED occupancy (the number of registered patients divided by the number of licensed ED beds) of greater than 140% was shown to be an important contributor by other investigators.21 These results clearly speak to the importance of managing in-patient and ED flow and LOS as priorities when attempting to reduce the number of patients who LWBS. We found that a “door-to-provider” time of greater than one hour appeared to be a point in time beyond which the

1223

Western Journal of Emergency Medicine

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED

Figure 1. Validation cohort – receiver operating characteristic curve. Area under the curve = 0.854. ROC, receiver operating characteristic; AUC, area under curve. -Asymptotic NormalROC

Obs

Area

Std. Err.

[95% Conf. Interval]

14,445

0.8542

0.0049

0.84324

0.86240

Figure 2. Calibration plot of observed vs predicted risk of patients leaving without being seen.

LWBS rate significantly increased (p < 0.003). While other investigators have found a range of delays of 30 minutes to two hours to be critical points in time, longer durations of the ED “front-end” process – from initial patient presentation to placement in an exam room – consistently predict that patients will LWBS at a higher rate. 12, 22- 26 “Door-to-provider” times are increasingly important and have been greatly modified Western Journal of Emergency Medicine

Rathlev et al.

by administrative designs including fast-track care and providers in triage.27-29 Based on our results, we emphasize the importance of identifying critical “door-to-provider” times associated with LWBS, as this may guide current and future strategies. The acuity level of some patients who LWBS may actually have prompted admission had they decided to stay and complete a full evaluation. This is of particular concern for higher risk patients who occasionally experience adverse outcomes after LWBS from the ED.30 Accordingly, more of these patients re-present to the ED within 48 hours for care compared with patients who receive a complete evaluation and management at their initial ED presentation.31 Fortunately, patients with time-sensitive emergency conditions are typically assigned ESI levels that justifiably lead to early provider evaluation.26 The patient to RN ratios and the RN, attending physician, and emergency medicine resident staffing numbers reached statistical significance in univariable analysis but were not found to contribute significantly to our final model; moreover, these measures did not contribute to improvements in discriminating patients who LWBS vs those who were evaluated by a provider. Using a 24-hour rather than one-hour period as the unit of measure, investigators have previously found that, after controlling for ED volume, hospital occupancy and admission rate, fewer RN staffing hours are associated with a statistically significant increase in the number of patients who LWBS.32 Considerable variability in process measures occur in the ED over 24-hour periods, which is the reason we chose to collect data in one-hour increments.33 We emphasize that our results should not be interpreted to mean that patient to RN ratios and other measures of staffing are unimportant. Rather, they suggest that one or more other variables in the final model were more strongly correlated with the outcome and explained much of the association between the outcome and staffing measures. Moreover, physician and RN staffing may simply not demonstrate sufficient variability, compared with other measures, to be statistically significantly associated with the observed variability in the rate of LWBS; greater variability in a predictor will reduce the variability in the estimated beta coefficient.34 Ultimately, measures such as the number of boarders, patients in the waiting room, arrival rate and “door-to-provider” times, demonstrated stronger associations with patients LWBS in our study. The ability to identify patients who are more likely to LWBS can highlight avenues for recovering potential lost revenue. Using this predictive model can help influence hospital administrators regarding the need to address boarding as a hospital-wide issue as opposed to an isolated ED problem. Moreover, the findings in this study can be used to advocate for additional staffing and creative workspace during hours when the arrival rates per hour are highest and when a surge in volume occurs. As mentioned earlier, this study highlights areas in which real-time modifications can result in significant changes in the rates of patients who LWBS. Strategies focused

1224

Volume 21, no. 5: September 2020

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED

on reducing boarding, reducing the number of patients in the waiting room and treatment bays, arrival rate, and door-toprovider times have the opportunity to result in increased revenue and improved care and patient satisfaction.

Address for Correspondence: Rathlev NK, MD, Department of Emergency Medicine, Baystate Medical Center, 759 Chestnut St, Springfield, MA 01199. Email: Niels.Rathlev@tufts.edu.

LIMITATIONS Limitations apply to our study including that this was a single-center, cross-sectional study with separate derivation and validation cohorts that were collected from the same institution and time frame. We captured data for four seasonally adjusted months, rather than an entire 12-month period, but believe that our large sample sizes are representative of the overall annual experience. Moreover, we were able to obtain ED process measures at one-hour intervals, but not in smaller increments of time. It is theoretically possible, but not likely in our experience, that these measures vary significantly over smaller time periods. Important hospital-wide measures, such as in-patient occupancy, are calculated only once a day at midnight in our institution, thus rendering them relatively meaningless for our purpose. We recognize that many of the variables assessed during model development are correlated with one another, which conceivably may induce multicollinearity and affect estimated standard errors of model coefficients. In severe cases, multicollinearity can produce very unstable and imprecise estimates of the standard errors, which may lead to unstable estimates of effect, wide CIs and misleading p-values. Multicollinearity, however, does not affect the utility of the regression model in estimating mean responses or making predictions.35 We applied remedies suggested by Vatcheva et al that focus on stabilizing the variance estimates.36 These include increasing the sample size, if possible, and removing one or more of the less important correlated variables. For the model development, our sample size was extremely large (n = 14,937) and we are therefore confident that our parameter estimates and standard errors are stable and precise. Secondly, our backward selection process removed many of the process and utilization variables, thus reducing the likelihood of severe multicollinearity. Third, we compared our final model with other possible models that may potentially capture LWBS risk. As such, we believe that our final model represents stable and precise estimates of factors associated with LWBS. CONCLUSION The rate with which patients LWBS from the ED is frequently cited as a measure of operational efficiency. Based on our results, the numbers of patients in the waiting room and boarding inside the treatment area are positively associated with patients LWBS. Moreover, the arrival rate of new patients per hour is also associated with this outcome. We found that “doorto-provider” time plays an important role and can, at least in some measure, be reduced through administrative design. Not surprisingly, patients who LWBS tend to be younger in age, lower in acuity with a higher ESI score, and arrive ambulatory rather than by EMS. Volume 21, no. 5: September 2020

REFERENCES 1. Pham JC, Ho GK, Hill PM, et al. National study of patient, visit, and hospital characteristics associated with leaving without being seen: predicting LWBS. Acad Emerg Med. 2009;16:949-55. 2. Fry M, Thompson J, Chan A. Patients regularly leave emergency departments before medical assessment: a study of did not wait patients, medical profile and outcome characteristics. Aust Emerg Nurs J. 2004;6:21-6. 3. Baker DW, Steven CD, Brook RH. Patient who leave a public hospital ED without being seen: causes and consequences. JAMA 1991; 266:1085-90. 4. Olanrewaju A, Soremekun MD, Biddinger PD, et al. Operational and financial impact of physician screening in the ED. Am J Emerg Med. 2012;30:532-9. 5. Falvo T, Grove L, Stachura R, et al. The financial impact of ambulance diversions and patient elopements. Acad Emerg Med. 2007;14:58-62. 6. Handel, DA, Fu R, Vu E, et al. Association of emergency department and hospital characteristics with elopements and length of stay. J Emerg Med. 2014;46:839–46. 7. Pitts SR, Vaughns FL, Gautreau MA, et al. A cross-sectional study of emergency department boarding practices in the United States. Acad Emerg Med. 2014;21:497-503. 8. Hsiah RY, Asch SM, Weiss RE, et al. Hospital determinants of emergency department left without being seen rates. Ann Emerg Med. 2011 Jul; 58(1): 24–32.e3. 9. Stock LM, Bradley GE, Lewis RJ, et al. Patients who leave emergency departments without being seen by a physician: magnitude of the problem in Los Angeles County. Ann Emerg Med 1994;23:294-8. 10. Wiler JL, Bolandifar E, Griffey RT, et al. An emergency department flow model based on queueing theory principles. Acad Emerg Med. 2013;20:939-46. 11. McCarthy ML, Ding R, Pines JM, et al. Comparison of methods

1225

Western Journal of Emergency Medicine

Rathlev et al.

Characteristics and Predictors of Patients Leaving Without Being Seen from the ED for measuring crowding and its effects on length of stay in the

25. Shaik SB, Jerrard DA, Witting MD, et al. How long are patients willing

emergency department. Acad Emerg Med. 2011;18:1269-77.

to wait before leaving without being seen? West J Emerg Med.

12. Pielsticker S, Whelan L, Arthur AO, et al. Identifying patient door-to-

2012;4:217-20.

room goals to minimize left-without-being-seen rates. West J Emerg

26. Jones P, LeFevre J, Harper A, et al. Effect of the shorter stays in

Med. 2015;16:611-8.

emergency departments time target policy on key indicators of quality

13. Pan W. Akaike’s information criterion in generalized estimating

of care. NZ Med J. 2017;130:35-44.

equations. Biometrics. 2001;57(1):120-5.

27. Yousefi M, Yousefi M, Fogliatto FS, et al. Simulating the behavior

14. Cui J. QIC program and model selection in GEE analyses. Stata J.

of patients who leave a public hospital emergency department

2007;7(2):209-20.

without being seen by a physician: a cellular automaton and

15. Austin PC, Steyerberg EW. The integrated calibration index (ICI) and

agent-based framework. Brazilian J Med and Biol Research.

related metrics for quantifying the calibration of logistic regression

2018;3:1-10.

models. Stat Med. 2019;38(21):4051-65.

28. Rowe BH, Guo X, Villa-Roel C, et al. The role of triage liaison

16. Harrell FE. (2015). Multivariate modeling strategies. Regression

physicians on mitigating overcrowding in emergency departments: a systematic review. Acad Emerg Med. 2011;18:111-20.

Modeling Strategies: With Applications to Linear Models, Logistic Regression and Survival Analysis, 2nd ed. (pages 63-103). New

29. Holroyd BR, Bullard MJ, Latoszek K, et al. Impact of a triage liaison

York: Springer.

physician on emergency department overcrowding and throughput: a

17. Sun BC, Binstadt ES, Pelletier A, et al. Characteristics and

randomized controlled trial. Acad Emerg Med. 2007;14:702-4.

temporal trends of “left before being seen” visits in US emergency

30. Rowe BH, Channan P, Bullard M, et al. Characteristics of patients

departments, 1995-2002. J Emerg Med. 2007;32(2):211-5.

who leave emergency departments without being seen. Acad Emerg

18. Ding R, McCarthy ML, Li G, et al. Patient who leave without being

Med. 2006;18:848-52.

seen: their characteristics and history of emergency department use.

31. Tropea J, Sundarajan V, Gorelik A, et al. Patient who leave without

Ann Emerg Med 2006; 48(6):686-93.

being seen in emergency departments: an analysis of predictive

19. Sharieff GQ, Burnell L, Cantonis M, et al. Improving emergency

factors and outcomes. Acad Emerg Med. 2012;19:439-47.

department time to provider, left-without-treatment rates and average

32. Ramsey Z, Palter JS, Hardwick J, et al. Decreased nursing staffing

length of stay. J Emerg Med. 2013;45:426-32.

adversely affects emergency department throughput metrics. West J

20. Hobbs D, Kunzman SC, Tandberg D, et al. Hospital factors

Emerg Med. 2018;19:496-500.

associated with emergency department patients leaving without

33. Anderson D, Pimentel L, Golden B, et al. Drivers of ED efficiency: a

being seen. Am J Emerg Med. 2000;18:767-72.

statistical and cluster analysis of volume, staffing and operations. Am

21. Polevoi SK, Quinn JV, Kramer NR. Factors associated with patients

J Emerg Med. 2016;34:155-61.

who leave without being seen. Acad Emerg Med. 2005;12:232-6.

34. Vittinghoff, E, Glidden DV, Shiboski SC, et al. (2012). Predictor

22. Arendt KW, Sadosty AT, Weaver AL, et al. The left-without-being-

selection. Regression Methods in Biostatistics: Linear, Logistic,

seen patients: What would keep them from leaving? Ann Emerg Med.

Survival, and Repeated Measures Models, 2nd ed. (pages 395-430).

2003;42:317-23.

New York: Springer.

23. McNamara KJ. Patients leaving the emergency department without

35. Kutner MH, Nachtsheim CJ, Neter J, et al. (2013). Multiple regression

being seen by a physician: Is same-day follow-up indicated? Am J

II. Applied Linear Statistical Models, 5th ed. (pages 214-293). New

Emerg Med 1995;13:136-41.

York, NY: McGraw-Hill.

24. Spechbach H, Rochat J, Gaspoz JM et al. Patient’s time perception

36. Vatcheva KP, Lee, M, McCormick JB, et al. Multicollinearity in

in the waiting room of an ambulatory emergency unit: a cross-

regression analyses conducted in epidemiologic studies. Epidemiol.

sectional study. BMC Emerg Med. 2019;19:41.

2016;6(2):227.

Western Journal of Emergency Medicine

1226

Volume 21, no. 5: September 2020

Original Research

Development and Implementation of a Community Paramedicine Program in Rural United States Lucas A. Myers, BAH, NRP* Peter N. Carlson, MBA, NRP* Paul W. Krantz, DO† Hannah L. Johnson, MHA‡ Matthew D. Will, MBA, NRP* Tasha M. Bjork, MBA‡ Marlene Dirkes, NRP* Justin E. Bowe, NRP* Kirk A. Gunderson, NRP* Christopher S. Russi, DO§

*Mayo Clinic, Mayo Clinic Ambulance Service, Rochester, Minnesota † Mayo Clinic Health System, Department of Emergency Medicine, Eau Claire, Wisconsin ‡ Mayo Clinic Health System, Department of Administration, Eau Claire, Wisconsin § Mayo Clinic, Department of Emergency Medicine, Rochester, Minnesota

Section Editor: Shira A. Schlesinger, MD, MPH Submission history: Submitted July 17, 2019; Revision received June 26, 2020; Accepted July 3, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.44571

Introduction: Community paramedicine (CP) is an innovative care model focused on medical management for patients suffering from chronic diseases or other conditions that result in overutilization of healthcare services. Despite their value, CP care models are not widely used in United States healthcare settings. More research is needed to understand the feasibility and effectiveness of implementing CP programs. Our objective was to develop a CP program to better meet the needs of complex, high-utilizer patients in a rural setting. Methods: We conducted an observational descriptive case series in a community, 25-bed, critical access hospital and primary care clinic in a rural Wisconsin county. Multiple stakeholders from the local health system and associated ambulance service were active participants in program development and implementation. Eligible patients receiving the intervention were identified as complex or high need by a referring physician. Primary outcomes included measures of emergency department, hospital, and clinic utilization. Secondary measures included provider and patient satisfaction. Results: We characterized 32 unique patients as high utilizers requiring assistance in medical management. These patients were enrolled into the program and categorized as high utilizers requiring assistance in medical management. The median age was 76 years, and 68.8% were female. After six months, we found a statistically significant decline in patient utilization for primary care (53.3%, p = .006) and ED visits (59.3%, p = .007), but not for hospitalizations (60%, p = .13, non-significant (NS), compared to the six months preceding enrollment. Overall, the total number of healthcare contacts was increased after implementation (623 before vs 790 after, + 167, +26.8%). Implementation of the CP program resulted in increased overall use of local healthcare resources in patients referred by physicians as high utilizers. Conclusion: The implementation of an in-home CP program targeting high users of healthcare resources resulted in a decrease in utilization in the hospital, ED, and primary care settings; however, it was balanced and exceeded by the number of CP visits. CP programs align well with population health strategies and could be better leveraged to fill gaps in care and promote appropriate access to healthcare services. Further study is required to determine whether the shift in type of healthcare access reduces or increases cost. [West J Emerg Med. 2020;21(5)1227-1233.]

Volume 21, no. 5: September 2020

1227

Western Journal of Emergency Medicine

Myers et al.

Community Paramedicine Program in Rural U.S. INTRODUCTION Community paramedic (CP) programs are an evolving practice of non-emergency, community-based care. The traditional model of emergency medical services (EMS) in the United States focuses on the response to acute injury and illness. Conversely, the primary foci of CP programs are preventive, with an emphasis on primary care delivery, prevention, screening, and wellness. CP programs target several areas of emphasis, including patients who have difficulty managing single or multiple chronic diseases, who have high risk for readmission after discharge, and in general overuse healthcare services.1 In 2015 more than 100 agencies established CP programs in 33 states2 with variable state regulatory environments. Some states, such as Minnesota, have established a level of care and Medicaid reimbursement through legislation. However, at the time of this study, the State of Wisconsin had not passed legislation allowing for reimbursement. Therefore, the value of a CP program may best be demonstrated through reduced use of services. The aim of this study was to describe a CP program developed by a health system medical provider and an associated ambulance service and to analyze its impact on healthcare use and the subsequent financial ramifications. METHODS Setting The primary service area for this study was a rural area in northwestern Wisconsin. The county has a population of 45,5633 and covers approximately 890 square miles. The area ambulance service employs nine full-time, ground-ambulance emergency medical technicians and paramedics and receives approximately 1500 combined emergent and non-emergent requests for service annually. The regional health system hospital is a 25-bed, critical access hospital and primary care clinic with 445 employees. This project was approved by the Mayo Clinic Institutional Review Board. Multidisciplinary Program Creation A multidisciplinary team from the health system site and the associated ambulance service was formed in July 2015 to develop the CP program, which included the following features: emergency physician leadership; health system administration; EMS leadership and research coordination; nursing, home health and hospice care, the Office of Population Health, palliative care; quality resources (management engineering and internal consulting); information technology; compliance; and local paramedics. The multidisciplinary team identified gaps of care within the community through quantitative data analysis and stakeholder interviews and then identified opportunities in the emergency department (ED) and the primary care department, with a focus on high-use patients. Through development of the program infrastructure, process, and procedures, the team consulted with home health and hospice services staff. Over 12 Western Journal of Emergency Medicine

Population Health Research Capsule What do we already know about this issue? Community paramedicine (CP) is a flexible, relatively low cost way to extend primary care outside of the clinic and hospital setting and into the community. What was the research question? Does implementation of a CP program decrease utilization of traditional healthcare resources? What was the major finding of the study? CP decreased utilization, but was balanced and exceeded by the number of visits. How does this improve population health? CP is an important tool in decreasing highcost healthcare utilization such as emergency department visits and hospitalizations.

months, the team focused on the development of care processes and procedures, paramedic communication with physicians for patient care plans, an appropriate referral and scheduling process, and assessment of quality outcomes. Routine reports were presented to ambulance service and health system leaders. The group met weekly with CP personnel to discuss patient volume and issues with the program. Medical Director The physician who was the medical director for the CP program also served as the medical director for two local ambulance services and a local paramedic training institution in addition to working as an emergency physician within the same health system as the one in this project. The CP program medical director (PMO) assisted in all aspects of development and implementation and focused specifically on educating and interacting with referring physicians, developing medical guidelines, and reviewing medical records for quality purposes. Community Paramedicine Education Two local CPs, chosen by the site manager, attended a CP training program at Hennepin Technical College in Eden Prairie, Minnesota.4 The one-semester, distance education course consisted of 72 hours of classroom time, 72 hours of online content, and 196 hours of clinical time conducted locally. The standardized curriculum was created by the North Central EMS Institute.5 Both CPs attended all clinical hours at the regional hospital that participated in this program, and

1228

Volume 21, no. 5: September 2020

Myers et al.

Community Paramedicine Program in Rural U.S.

spent considerable time with primary care physicians enrolling patients into the program. Beyond the CPs’ clinical education, the secondary intention was to create a working relationship and rapport between the CPs and the referring physicians. Additionally, clinical time was spent with staff involved with home health and hospice, wound care, the ED, respiratory therapy, and mental health. Both CPs received CP certification through the Minnesota Emergency Medical Services Regulatory Board before the start of the program. During this project, the state of Wisconsin did not offer CP certification, but it did allow CP projects to occur within the state with prior approval. Education for Referring Physicians The CP PMO provided formal presentations to primary care physicians and hospitalists and informal presentations to emergency physicians. The broad inclusion criteria were intended to recruit patients who were high users of the ED and the clinic, who had a high risk for readmission or falls, who had chronic illness or needed postsurgical wound care, or who required frequent international normalized ratio (INR) monitoring or other blood tests. The only exclusion criteria were patients younger than 21 years and patients in skilled nursing facilities. An online survey to evaluate physicians’ perceptions of the program was distributed to all physicians who had referred at least one patient.

Medical Guidelines Development Medical guidelines were developed from existing CP program guidelines, with permission, from Eagle County Paramedic Services in Edwards, Colorado.6 The medical guidelines were adapted and used to address the specifics of a home visitation, including the history and examination, medication reconciliation, home assessment, and specific procedures such as drawing blood, point-of-care testing, and wound care. The state of Wisconsin required receipt of these protocols before the program began to ensure that all medical care was within the current, state-defined paramedic scope of care.

Volume 21, no. 5: September 2020

Scheduling Physicians identified potential candidates for the CP program during clinic appointments, ED visits, or hospital stays. If the patient agreed to enroll, the physician completed an order for a CP visit and documented objectives and a care plan in the EHR for the CP to review. The order was automatically printed in the paramedic office, and one of the CPs would schedule the visit. The physician determined the frequency of visits (eg, once weekly or twice weekly). Patient scheduling was shared with the ambulance dispatch center to allow for safety checks every 30 minutes during on-scene time. Visits occurred Monday through Friday from 8am to 5pm, with each visit lasting approximately one hour. Vehicle and Equipment The CPs used a clearly marked passenger car that required no additional modification. The vehicle was equipped with a response bag containing equipment for assessment. Additional equipment that was not already carried by the ambulance service included the following: a scale and measuring tape (to measure the patient’s weight and height); an INR testing machine; an otoscope; laboratory blood vials from the hospital; and a cooler for transporting specimens. Each CP was given a laptop computer (for in-home documentation into the EHR) and a cell phone.

Staffing Model The two CPs were each allocated 20 hours weekly for a total of 40 hours per week: 10 hours for patient scheduling, administrative duties, and visit planning, and 30 hours for in-home visits. The model allotted one hour per in-home visit and 30 minutes of travel time per visit. With visits scheduled Monday through Friday, the maximum number of patient visits per week was 20 (four per day).

Integrated Health Record The regional health system hospital used an electronic health record (EHR), while the ambulance service used an EMS-specific product. The hospital EHR allowed for scheduling of patient visits and direct messaging between the ordering physician and the CP. Additionally, the CP required

access to the hospital EHR to review the medical order and pertinent clinical history. The CPs received training to perform all documentation in the hospital EHR. Information technology specialists developed the new documents and templates within the EHR. No documentation was done within the EMS patient-record system. Each CP received a companyissued smartphone for business relating to the CP program and to support EHR documentation while in the patient’s home.

Patient Visit 1. Before the patient visit, the CP accessed the patient’s EHR to review and confirm the physician’s order, care plan, history, visit notes, laboratory test results, and current medications and doses. CPs arrived at the patient’s home at the scheduled time and called the dispatch center to confirm their arrival. While meeting with the patient, the CPs focused on six key areas: 2. Present health status: evaluation of activity level; patient perception of health; and current medications. 3. Past health history: review of allergies; illnesses, surgical procedures; hospitalizations, immunizations, most recent evaluation by a physician, and family medical history. 4. Physical examination: review of general health status and specific systems. 5. Medication reconciliation: review of current medications, including dosages, daily schedule, and adherence to therapy; identification of medications that might have been prescribed by another physician or another medical

1229

Western Journal of Emergency Medicine

Myers et al.

Community Paramedicine Program in Rural U.S. provider; and assisting with sorting of medications if a sorting system was used. 6. Environmental assessment: use of the Physical Environment Assessment Tool (PEAT) scale at the first visit and at subsequent visits when necessary.7 7. Specific physician orders: review of specific orders, such as providing wound care; monitoring INR; testing blood glucose; or drawing blood. The CP documented information from the assessment, including the PEAT scale, into the EHR through mobile remote access while still at the patient’s home or after the visit. If the CP had any concerns, the CP called the ordering physician or the oncall physician for direction. If any assessment finding indicated the need for urgent assessment by a physician (eg, chest pain or stroke symptoms), the CP was instructed to call 911. Physician Review of the Visit After the CP completed documentation in the EHR, an automated alert was sent to the ordering physician and the CP program medical director. If a single visit was ordered, the physician could order subsequent visits, revise the care plan, or discharge the patient from the program. The CP PMO reviewed all the CP’s documentation and provided feedback for quality improvement. Referring Physicians An online, 10-item survey was created and distributed to physicians who referred at least one patient to the CP program. The aim of the survey was to evaluate the physicians’ impressions of the program, communication with CPs, and overall satisfaction with the program. Data Analysis We included for analysis patients enrolled from March 1–September 30, 2016. As part of data abstraction, the CP PMO eviewed patient health records to determine the primary medical reason for referral. Patients were grouped into one of three categories: high users needing medical management; high risk for readmission; and post-discharge follow-up. The study team exported all patient visits to the ED, all hospitalizations, and all primary clinic uses because primary care charges could result from an ED visit or hospitalization rather than from only a visit to the primary care physician’s office. The PMO evaluated all visits and clinic uses to determine whether they were related to the referring reason six months before enrollment and six months after enrollment. Although all patients were categorized with a single referring reason, most patients had comorbidities noted by the referring physician and were often referred with more than one reason. All visits and clinic uses were included for analysis if they were related to a referring reason. The Mayo Clinic Institutional Review Board approved this study. Western Journal of Emergency Medicine

Objectives The purpose of this study was to describe and evaluate the creation of a rural CP program and determine the change in healthcare utilization resulting from community paramedic in-home visits. The primary end-point was to analyze whether change in utilization type (ED, primary care, and hospitalizations) occurred by implementing CP visits. Further, we analyzed the number of CP visits required to create such reduction in utilization types. RESULTS During the seven-month study period, 42 unique patients were enrolled in the program: 32 were classified as high users with medical management, six as high risk for readmission, and four as post-discharge follow-up. High Users Needing Medical Management The median age of the 32 high users was 76 years; 22 (68.8%) were women. The total number of in-home CP visits for the six months after each patient’s enrollment was 412 (range, 1-47 per patient). Primary referral reasons are shown in Table 1. Primary care physicians referred seven patients (21.9%), emergency physicians referred 15 (46.9%), and hospitalists referred 10 (31.2%) as part of discharge from an admission. Individual patient use of health services decreased from the six months before enrollment to the six months after enrollment (Table 2). The total number of visits and clinic uses decreased in the six months after enrollment (Table 3). In the six months before enrollment, 10 patients required 911 services a total of 16 times. During the six months following enrollment, 10 patients had a total of 14 requests for 911 services. The payer mix for these 32 patients was 94% (30/32) government insurance (Medicare or Medicaid)/ and 6% (2/32) private insurance. High Risk for Readmission and Post-discharge Follow-up Six patients were categorized in the high-risk readmission group, but one patient was enrolled twice during the study

Table 1. Primary referral reason for patients categorized as high users of medical resources. Primary referral reason

Patients, No. (%)

Falls

11 (34)

Chronic pain

6 (19)

Hypertension

4 (12)

Diabetes mellitus

3 (9)

Respiratory condition

3 (9)

Mental health

2 (6)

Multiple comorbidities

2 (6)

Congestive heart failure

1 (3)

1230

Volume 21, no. 5: September 2020

Myers et al.

Community Paramedicine Program in Rural U.S.

Table 2. Individual patient use of health services before and after enrollment. 6-month period, number of patients Health service

Differencea

Before enrollment

After enrollment

Patients, No.

Decrease, %

Primary care

-16 (p=.006)

53.3

Emergency department

-16 (p=.007)

59.3

Hospitalization 10 4 -6 (p=0.13) 60.0 Difference = After enrollment - Before enrollment Statistical Test: McNemarâ&#x20AC;&#x2122;s test of paired proportions was used to determine if there was a difference in the proportion of health services before and after enrollment. A continuity correction was applied to approximate the Chi-Square distribution. a

period. Results of 72-hour and 30-day readmissions are shown in Table 4. The study team used the same outcome measures for the four patients enrolled for post-discharge follow-up. All patients were referred by a hospitalist before discharge or at discharge from a hospitalization. Referring Physicians The survey for referring physicians garnered a response rate of 86% (18/21) (Table 5). DISCUSSION Healthcare organizations, nationally, are looking for safe, high-quality mechanisms to get the right patient to the right place of care. In some cases patients can be managed in their homes with the right support and resources in place to avoid costly and potentially risky hospital admission. In the era of payment reform and many systems moving to an accountable care organization model, healthcare organizations look to mitigate readmission penalties and develop programs to manage patients remotely in their homes when possible. Approximately 80% of older adults have at least one chronic disease, and over two-thirds of all healthcare costs are attributed to treating those diseases.8 The ability of these patients to self-manage varies. Patients who have difficulty with self-management may benefit from a CP program that is integrated with the patientâ&#x20AC;&#x2122;s primary care provider. A CP program can supplement clinic visits

with physician-ordered vital sign monitoring, point-ofcare testing, medication reconciliation, assistance in diet planning, and other areas of wellness. The national, acute care 30-day readmission rate for Medicare beneficiaries is nearly 20%. Readmission rates greater than the national average put healthcare systems at risk for financial penalties from the Centers for Medicare & Medicaid Services.9 To decrease readmission rates, healthcare systems are developing methods to identify patients who have the greatest readmission risk. Organizations are turning to CP programs to help reduce that risk. A readmission can occur for various reasons, including adverse drug reaction, incorrect use of prescription medication, increased risk of fall, exacerbation of the primary cause for hospitalization, and poor wound care after a surgical procedure. Regardless of the reason, an integrated CP program can address these issues and more specific issues as identified and ordered by the physician. While there was a reduction in primary care visits (n = 221), ED visits (n = 15) and hospitalizations (n = 9), there were a total 412 CP visits conducted to achieve these results. There was an increase in utilization when considering the addition of the CP visits. However, the cost of the CP visit compared to other visit types (primary care, ED, hospital) must be considered. In the case of primary care visit reduction, it is likely that these visits were merely replaced by the CP visit, which may have a cost benefit, especially to the patient, when considering patient travel and time away from work. Careful

Table 3. Aggregate use of health services before and after enrollment by the patient population (n=32). 6-month period, number of events Health service

Differencea

Before enrollment

After enrollment

Events, No.

Change, %

Primary care

547

326

-221 (p<.001)

-40.4

Emergency department

-15 (p=.17)

-25.0

Hospitalization

-9 (p=.095)

-56.2

Community paramedic visits

412

623

790

+167

+26.8%

Total healthcare contacts

Difference = After enrollment - Before enrollment Statistical Test: For each health service, a z-test of proportions was used to test whether the number of tests before and after enrollment were the same. The z-test statistic was computed by comparing the proportion of tests for a given service that occurred after enrollment, and comparing to 0.5. a

Volume 21, no. 5: September 2020

1231

Western Journal of Emergency Medicine

Myers et al.

Community Paramedicine Program in Rural U.S. Table 4. Emergency department (ED) visit and readmission rates within 72 hours and 30 days after hospital discharge. â&#x2030;¤72 Hours Patient category

â&#x2030;¤30 Days

ED visit

Readmission

ED visit

Readmission

High-risk discharge (n=7), No. (%)

0 (0)

1 (14.3)

Postdischarge follow-up (n=4), No. (%)

0 (0)

1 (25.0)

measurement of the cost of developing and deploying a CP program vs the savings from patient and payor expenses will be important for future research and a cost-benefit analysis. Overuse of the ED can stress healthcare resources by increasing ED wait times, delaying ambulance response times, and diverting ambulances because of hospital crowding. Frequent patient use of the ED has been a long-standing issue,10-15 and patients who overuse the ED may also overuse other medical services, such as primary and inpatient care.16 Patients enrolled in the CP program for assistance in medical management and previous overuse of healthcare resources realized a decrease in primary care use, ED visits, and hospitalizations of 53.3%, 59.3%, and 60.0%, respectively. The decrease in use implies smaller charges to the patient and, given the primary payer sources of this population, a reduction in unreimbursed expenses to the health system. This analysis was observational; however, several areas of future quality improvement were identified. Referring documentation lacked clearly defined patient care objectives, making it difficult to establish patient care and outcome goals to successfully discharge patients from the program. Future work will include implementing a care-planning process where the CP will create and document goals and objectives in conjunction with the patient and the primary care physician to create a plan for successful discharge from the program in the fewest visits necessary. LIMITATIONS It was not possible to identify whether or when enrolled patients pursued medical care outside the health system.

Accordingly, such use would not be represented in these findings. Further, healthcare providers were aware of this study, inherently introducing selection bias. While efforts were made to apply risk-assessment tools consistently, we could not control for a potential selection bias within the cohort. While we did observe decreases in clinic and hospital resource utilization, not all were statistically significant. It is important to continue to evaluate CP programs and publish results of large and diverse sample sizes. It is also necessary to account for the cost of startup and maintenance of a CP program in comparison to the cost avoidance from ED and hospital utilization reduction. In this experience, 412 CP visits were conducted. Future programs will benefit from measuring and improving upon efficiencies where possible to provide the greatest impact with the fewest encounters. While decrease in utilization is described here, it must be acknowledged that CP visits themselves are a form of healthcare utilization and the number of visits conducted for this small sample size was extensive. The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper. CONCLUSION The implementation of a CP program targeting high users of healthcare resources resulted in a decrease in healthcare use in the hospital, ED, and primary care settings. This program may also reduce readmission rates for high-risk patients discharged from the hospital. Referring physicians generally agreed that the program benefited their patients.

Table 5. Physician survey results. Agree

Disagree

Undecided

I am comfortable with the community paramedic referral process

Survey item

Patients I refer benefit from the community paramedic visit(s)

My expectations of the community paramedic visit(s) are met

Following a community paramedic visit, I see improvements in the patientsâ&#x20AC;&#x2122; health/wellness

Patients are satisfied with the care delivered by the community paramedic

I am satisfied with the ability to communicate with the community paramedic about care plans

The community paramedic is responsive to changes in the plan of care

The community paramedic provides quality care to the patients I refer

I would recommend this process to other clinicians

The community paramedic program should be expanded in my region

Western Journal of Emergency Medicine

1232

Volume 21, no. 5: September 2020

Myers et al.

Community Paramedicine Program in Rural U.S.

Address for Correspondence: Lucas A. Myers, BAH, NRP, Mayo Clinic, Mayo Clinic Ambulance Service, 200 First St SW, Rochester, MN 55905. Email: myers.lucas3@mayo.edu.

6. ECPS. Eagle County Paramedic Services. 2017. Available at: http:// eaglecountyparamedics.com/#&panel1-1. Accessed June 27, 2017. 7. EMSWORLD. The PEAT Scale: an EMS tool. 2017. Available at:

http://www.emsworld.com/article/10324367/the-peat-scale-an-emstool. Accessed June 27, 2017. 8. National Council on Aging. Chronic disease self-management facts. 2017. Available at: https://www.ncoa.org/news/resources-for-reporters/ get-the-facts/chronic-disease-facts/. Accessed June 27, 2017. 9. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360(14):1418-28. 10. Schull MJ, Morrison LJ, Vermeulen M, et al. Emergency department overcrowding and ambulance transport delays for patients with chest pain. CMAJ. 2003;168(3):277-83. 11. Schneider KC and Dove HG. High users of VA emergency room facilities: are outpatients abusing the system or is the system abusing them?. Inquiry. 1983;20(1):57-64.

REFERENCES

12. Bond TK, Stearns S, Peters M. Analysis of chronic emergency

1. Patterson DG, Coulthard C, Garberson LA, et al. What is the

department use. Nurs Econ. 1999;17(4):207-13, 237.

potential of community paramedicine to fill rural health care gaps? J

13. Mandelberg JH, Kuhn RE, Kohn MA. Epidemiologic analysis of an

Health Care Poor U. 2016;27(4A):144-58.

urban, public emergency departmentâ&#x20AC;&#x2122;s frequent users. Acad Emerg

2. Mobile integrated healthcare and community paramedicine (MIH-CP): a national survey. EMS World. 2015;Supp:5-16.

Med. 2000;7(6):637-46. 14. Sandoval E, Smith S, Walter J, et al. A comparison of frequent and

3. United States Census Bureau. Quick Facts, Barron County, Wisconsin.

infrequent visitors to an urban emergency department. J Emerg Med.

2016. Available at: https://www.census.gov/quickfacts/fact/table/ barroncountywisconsin/PST045216. Accessed June 27, 2017.

2010;38(2):115-21. 15. Cunningham A, Mautner D, Ku B, et al. Frequent emergency

4. Hennepin Technical College. Community Paramedic Program.

department visitors are frequent primary care visitors and report

2017. Available at: https://www.hennepintech.edu/cts/pages/1224. Accessed June 27, 2017.

unmet primary care needs. J Eval Clin Pract. 2017;23(3):567-73. 16. Hansagi H, Olsson M, Sjoberg S, et al. Frequent use of the hospital

5. The SAVVIK Foundation. North Central EMS Institute. 2017.

emergency department is indicative of high use of other health care

Available at: www.ncemsi.org. Accessed June 27, 2017.

Volume 21, no. 5: September 2020

services. Ann Emerg Med. 2001;37(6):561-7.

1233

Western Journal of Emergency Medicine

Original Research

Paramedic Pain Management Practice with Introduction of a Non-opiate Treatment Protocol Laurel O’Connor, MD* Julianne Dugas, MPH† Jeffrey Brady, MD* Andrew Kamilaris, MD* Steven K. Shiba, MS‡ Ricky C. Kue, MD, MPH§ John P. Broach, MD*

* University of Massachusetts Medical School, Department of Emergency Medicine, Worcester, Massachusetts † Boston Medical Center, Department of Emergency Medicine, Boston, Massachusetts ‡ Charles E. Schmidt College of Medicine at Florida Atlantic University, Boca Raton, Florida § South Shore Health, Department of Emergency Medicine, South Weymouth, Massachusetts

Section Editor: Joshua B. Gaither, MD Submission history: Submitted February 20, 2020; Revision received April 15, 2020; Accepted June 10, 2020 Electronically published August 21, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.6.47032

Introduction: There is concern about the initiation of opiates in healthcare settings due to the risk of future misuse. Although opiate medications have historically been at the core of prehospital pain management, several states are introducing non-opiate alternatives to prehospital care. Prior studies suggest that non-opiate analgesics are non-inferior to opiates for many acute complaints, yet there is little literature describing practice patterns of pain management in prehospital care. Our goal was to describe the practice patterns and attitudes of paramedics toward pain management after the introduction of non-opiates to a statewide protocol. Methods: This study was two-armed. The first arm employed a pre/post retrospective chart review model examining medication administrations reported to the Massachusetts Ambulance Trip Information System between January 1, 2017–December 31, 2018. We abstracted instances of opiate and non-opiate utilizations along with patients’ clinical course. The second arm consisted of a survey administered to paramedics one year after implementation of non-opiates in the state protocol, which used binary questions and Likert scales to describe beliefs pertaining to prehospital analgesia. Results: Pain medications were administered in 1.6% of emergency medical services incidents in 2017 and 1.7% of incidents in 2018. The rate of opiate analgesic use was reduced by 9.4% in 2018 compared to 2017 (90.6% vs 100.0%). The absolute reduction in opiate use in 2018 was 3.6%. Women were less likely (odds ratio [OR] = 0.78, 95% confidence interval [CI], 0.69-0.89) and trauma patients were more likely to receive opiates (OR = 2.36, CI, 1.96-2.84). Mean transport times were longer in opiate administration incidents (36.97 vs 29.35 minutes, t = 17.34, p<0.0001). We surveyed 100 paramedics (mean age 41.98, 84% male). Compositely, 85% of paramedics planned to use nonopiates and 35% reported having done so. Participants planning to use non-opiates were younger and less experienced. Participants indicated that concern about adverse effects, efficacy, and time to effect impacted their practice patterns. Conclusion: The introduction of non-opiate pain medication to state protocols led to reduced opiate administration. Men and trauma patients were more likely to receive opiates. Paramedics reported enthusiasm for non-opiate medications. Beliefs about non-opioid analgesics pertaining to adverse effects, onset time, and efficacy may influence their utilization. [West J Emerg Med. 2020;21(5)1234-1241.]

Western Journal of Emergency Medicine

1234

Volume 21, no. 5: September 2020

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol

INTRODUCTION The management of acute pain is central to emergency care in both the hospital and prehospital settings. Pain prevalence in the prehospital setting has been reported to be between 42-53%.1-2 Both the National Association of Emergency Medical Services Physicians and the American College of Emergency Physicians have emphasized the importance of addressing the high incidence of pain reported by emergency medical services (EMS) utilizers.3-4 However, recent literature suggests that many prehospital providers undertreat prehospital pain complaints.5-7 The availability of specific types of analgesics varies by state, but traditionally opiates have been at the core of prehospital analgesia protocols. This precedent is complicated by the ongoing national opiate crisis and the concern that utilization of opiates in the acute setting may engender longterm misuse and addiction. The opioid epidemic has been prominent in the minds of healthcare providers and has had major impacts on how analgesics are dispensed.8 While there has been some research linking administration of opiates in the acute care setting with recurrent opiate use, this phenomenon has not been well described in the prehospital setting.9 In Massachusetts, acetaminophen, ketorolac, and ibuprofen were introduced to the pain management protocol for paramedics on January 1, 2018.10 While these medications have a favorable safety profile and do not have the addictive or sedative qualities that render opiates dangerous, they do have the potential to cause harm in some patients. Ibuprofen and ketorolac have an adverse effect profile that includes gastric bleeding, renal dysfunction, and platelet derangement; acetaminophen may contribute to hepatic toxicity.11 While those adverse effects are relatively uncommon, there is some concern about administering these medications in the prehospital setting in which the patient is undifferentiated and diagnostic testing is extremely limited. There is also the barrier of widespread belief that non-opiates are less efficacious than opiates for acute pain and have an unacceptable time to effect. However, there is a considerable body of literature demonstrating non-inferiority in non-opiate medication administration as compared to opiates for many conditions commonly encountered in the prehospital setting including renal colic, long bone fracture, and other minor traumatic limb injuries.12-15 While most clinicians are in agreement that prehospital providers should treat pain, there remains equipoise as to how to guide prehospital pain management. Providers must attempt to consolidate information pertaining to patients’ self-reported levels of pain, clinical characteristics, and the possible adverse effects of available analgesics, and then make an expeditious decision about which medication to administer. Despite the more widespread availability of non-opiate analgesics and their proven efficacy, there are few data available regarding practice patterns of prehospital Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? In the midst of the opioid-use epidemic, the judicious use of opiate analgesics for acute pain is paramount to mitigate the risk of future misuse. What was the research question? We asked if there were changes in prehospital pain management after introduction of nonopiate analgesics. What was the major finding of the study? Prehospital opiate administration by paramedics was reduced when non-opiate options were available. How does this improve population health? The introduction of non-opiate medications to prehospital protocols enables paramedics to avoid opiates when appropriate.

providers or about their perceptions of prehospital pain management. This study sought to describe trends in prehospital analgesic use and providers’ attitudes toward pain management one year after the introduction of nonopiate options to the state protocol in Massachusetts. METHODS Setting and participants This project used data from Advanced Life Support (ALS) EMS agencies across Massachusetts. All data for the retrospective chart review arm of the study were derived from the Massachusetts Ambulance Trip Recording System (MATRIS) for ambulance trip-sheets ranging from January 1, 2017–December 31, 2018.8,16 This standardized database contains data uploaded from 224 ALS-capable services in the state (with 97.9% reporting compliance), and it is National Emergency Medical Services Information System compliant.16 MATRIS is maintained by the Massachusetts Department of Public Health. Paramedics for the perceptions survey arm were recruited from 16 departments representing urban, suburban, and rural services in Massachusetts. All participants were recruited at scheduled training and administrative meetings. Agencies included two hospital-based services, five fire-based services, and nine private services. Participants were considered eligible for inclusion if they were at least 18 years old, fluent in English, and currently nationally registered, licensed, and field-active paramedics. In total, 104 participants were

1235

Western Journal of Emergency Medicine

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol approached and 100 completed the administered survey (96% enrollment rate). Procedure We queried the MATRIS database for all administrations of morphine or fentanyl documented in prehospital trip-sheets in 2017 and all administrations of morphine, fentanyl, ketorolac, acetaminophen, or ibuprofen in 2018. Patients who received morphine or fentanyl comprised the “opiate cohort,” whereas patients who received ketorolac, acetaminophen and ibuprofen comprised the “non-opiate” cohort. After being implemented in the state protocol, non-opiate medications became available to all agencies simultaneously. We excluded all encounters where the primary and/or secondary impression was cardiac arrest or obvious death. Encounters where neither the primary impression nor secondary impression contained a pain complaint (eg, respiratory distress, respiratory arrest, etc.) were excluded in an effort to eliminate instances in which opiates were used for sedation and not as analgesics. At the time of this project, ketamine was not available for prehospital analgesia; it was used only for induction in medicationassisted intubation or for agitated delirium. Encounters where only ketamine was administered were excluded. We placed patients who received both opiate and non-opiate medications in the opiate cohort. Records in which data was incompatible with MATRIS parameters or obviously erroneous were excluded. We included encounters for which there was no primary or secondary impression listed but the dispatch chief complaint was pain related. The inclusion decision-making parameters are depicted in Figure. For the survey portion of the project, participants were asked to complete an anonymous, 10-minute pencil-and-paper survey. The survey was conducted in the spring following the first full year of the implementation of non-opiate medications in the state protocol. No additional discussion or education regarding prehospital pain management occurred in the setting of the survey administration. Data were collected and managed using REDCap (Research Electronic Data Capture) tools (V9.1.0) hosted by the primary study site. REDCap is a secure, web-based software platform designed to support data capture for research studies.17-18 The institutional review boards at all participating institutions, as well as the Massachusetts Department of Public Health, approved this project. Measures Parameters extracted from MATRIS for the data arm of this study included medication administered, dispatch complaint, subject age, gender, initial systolic blood pressure (SBP), initial heart rate (HR), transport time, primary and secondary impression, the EMS agency providing service, and the location of the EMS call. The perceptions survey collected demographic information from participants and then utilized a series of binary question and Likert scales to assess subjects’ attitudes about the benefits and barriers to using opiate and Western Journal of Emergency Medicine

Figure. Inclusion decision-making parameters for paramedic administration of opiate and non-opiate medications. EMS, emergency medical services; SBP, systolic blood pressure; HR, heart rate.

non-opiate medications for prehospital analgesia. The full language of the survey is depicted in supplemental Table 2. Analysis For the MATRIS data, we calculated descriptive statistics on all measures for the three cohorts (2017 opiate patients, 2018 opiate patients, and 2018 non-opiate patients). Comparative statistics were performed for all measured patient factors including age, gender, mean initial SBP, and mean initial HR. We completed all statistical computations using SAS v9.4 (SAS Institute, Cary, NC). P-values were obtained using Welch-Satterthwaite t-testing for continuous variables such as age, SBP and HR, and chi-square testing of independence for categorical variables. We used Pearson correlation values to assess the association between county median income and the proportion of medication administration incidents involving opiates. For the survey arm, we calculated descriptive data for all participants, and subsequently for the subgroups of participants that planned to administer non-opiates and those that did not. P-values were calculated using chi-square

1236

Volume 21, no. 5: September 2020

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol

testing for binary variables and Wilcoxon rank-sum testing for continuous variables. The Likert scale data for perceptions data were reported descriptively for all three cohorts (all participants, participants planning to use nonopiates and participants not planning to use non-opiates). Finally, planning to use non-opiates was analyzed by demographics and Likert scale responses, using chi-square tests of independence for categorical variables and Pearson correlations for continuous variables. RESULTS MATRIS Data Descriptive data for all chart review subjects is summarized in Table 1. Subject demographics between the pre- and postintervention cohorts were not significantly different. In total there were 677,364 emergent field EMS responses in 2017 and 673,561 in 2018; the rate of pain medication administration was 1.6% in 2017 and 1.7% in 2018. Total medication administrations are reported in Table 2. Overall, the rate of opiate analgesic use was reduced by 9.4% in 2018. The absolute reduction in opiate use in 2018 was 3.6% (385/10809) compared with 2017. Once non-opiate options were introduced, women were more likely than men to receive an opiate medication (OR = 0.78, 95% CI, 0.69-0.89). There were no statistically significant differences in mean age between opiate and non-opiate recipients. There were small but statistically significant differences between mean initial SBP and HR (-1.93 milligrams of mercury and +1.81 beats per minute, respectively, in the opiate cohort). In cases where a primary impression was available, most non-opiates were administered for medical chief complaints. Patients with traumatic complaints were significantly more likely to receive opiate medications (OR = 2.36, 95% CI, 1.96-2.84). In both years, and within the opiate and non-opiate cohorts, abdominal pain was the most common clinical

impression for which pain medication was administered. The ratio of opiates to total pain medication administrations by individual EMS services in 2018 ranged from 0.39 to 1.00. In total, 120 out of 224 (54%) of reporting services administered at least one non-opiate during the post-intervention year. Three services had an opiate administration ratio less than 50%, 27 had an opiate administration rate under 75%, and 74 had opiate administration rates under 90%. When the EMS services included in the survey arm were examined separately, the proportion of opiates administered in 2018 ranged from 0.81-1.00; there was no significant deviation in their administration patterns as compared to the rest of the state. We calculated Pearson correlation between proportion of opiate administration and median county income of service location and it was not significant (R = 0.25, p = 0.41). Paramedics Perceptions Data In total, we surveyed 100 participants (mean age 42 years, 95% CI, 40.19-43.77; 84% male). All participant demographics and those of cohorts planning and not planning to use non-opiates are summarized in Table 3. Participants who reported planning to use non-opiates were younger (mean age 39) and less experienced (mean 11.15 years of experience) than those who did not (41.4 years, p<0.05 and 15 years of experience, p=0.01, respectively). There was no significant difference in the pain scale number at which cohorts reported as benchmarks for administering opiate medications (p = 0.238). Paramedics with greater experience and older age were more likely to administer opiates at a lower patient-reported pain scale (R= 0.32, p < 0.05 and R = 0.27, p <0.05, respectively). Responses to Likert scale-based perceptions questions are described in supplemental Table 1. The majority of paramedics (76%) reported agreeing or strongly agreeing that there was a duty to treat pain in the prehospital setting and

Table 1. MATRIS+ patient demographics. 2017 (n = 10809) Opiate

2018 (n = 11502)

Non-opiate*

Opiate§

Non-opiate*

P-value

53.7 (22.193)

N/A

54.01 (22.34)

55.31 (21.31)

0.218

Male

5237 (48.6)

N/A

5027 (48.4)

454 (42.3)

0.266

Female

5537 (51.4)

N/A

5352 (51.6)

619 (57.7)

Mean transport time minutes (SD)

36.12 (18.89)

N/A

36.98 (20.33)

29.35 (29.35)

Mean initial SBP mm hg, (SD )

143.46 (26.91)

N/A

143.75 (26.55)

145.68 (25.66)

Age, mean (SD)

Gender, n (%)

Mean initial HR 88.22 (19.24) N/A 88.38 (19.42) 86.64 (18.49) bpm, (SD) + Massachusetts Ambulance Trip Recording System § Opiate category includes morphine and fentanyl. *Non-opiate category includes ibuprofen, acetaminophen and ketorolac. SD, standard deviation; SBP; systolic blood pressure; mm hg, millimeters of mercury; bpm, beats per minute.

Volume 21, no. 5: September 2020

1237

Western Journal of Emergency Medicine

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol Table 2. Medication administration by year. Annual totals 2017

2018

Total EMS calls

677,364

673,561

Total administrations

10,809

11,502

<0.001

10,809 (100%, 95% CI, 99.96-100)

10,424 (90.63%, 95% CI, 90.08-91.15)

0.002

N/A

1,078

N/A

Opiates

Non-opiates

P-value

Medication Administration Demographics 2018 Opiate , n

Non-opiate*, n

Female

5,352

619

0.781, 95% CI 0.688-0.887

Male

5,027

454

Mean age (SD)

54.01 (22.34)

55.31 (21.31)

Mean transport time (minutes, SD)

36.98 (20.33)

29.35 (29.35)

-7.631 (-8.494,-6.768)

< 0.001

Mean initial SBP (mm Hg, SD)

143.75 (26.55)

29.35 (29.35)

-1.930 (0.201,3.659)

0.029

Mean initial HR, bpm (SD)

88.38 (19.42)

86.64 (18.49)

1.808 (-3.004,-0.613)

0.003

Δ mean (95% CI)

P-value < 0.001

T= 1.90

0.058

Opiate category includes morphine and fentanyl. *Non-opiate category includes ibuprofen, acetaminophen and ketorolac. EMS, emergency medical services; OR, odds ratio; SD, standard deviation; SBP, systolic blood pressure; mm hg, millimeters of mercury; HR, heart rate; bpm, beats per minute; CI, confidence interval. §

90% reported believing that prehospital pain management was effective. Participants not planning to give non-opiate medications were more likely to agree that pain was difficult to assess in the prehospital setting, more likely to be concerned about the adverse effects of both opiates and non-opiates, and more likely to believe that non-opiates were not effective in managing pain and took too long to work. Participants who reported that they were planning to give non-opiate medications were more likely to be concerned that administering pain medications would change patients’ clinical presentation for providers in the ED. Concerns about drug-seeking behavior and opiate tolerance were not different between cohorts. Few participants responded affirmatively to concerns regarding adverse effects (11%), efficacy (12%), and time to effect (21%) impacting their decision to administer nonopiates. Globally, participants also reported agreement that the non-opiate ketamine should be available for prehospital analgesia (72% agreed or strongly agreed) although there was less support for lidocaine nerve block (33% agreed or strongly agreed). There was no consensus on support for implementation of more structured protocols for selecting prehospital analgesics (26% agreed or strongly agreed; 50% disagreed or strongly disagreed). Western Journal of Emergency Medicine

DISCUSSION Analysis of one state’s data a year after the advent of non-opiate options demonstrates a modest but statistically significant absolute reduction in the use of opiates. Although more work must be done, this is cautiously encouraging; the rate of opiate administration has dropped while the rate of pain medication administration has increased slightly. Although limited demographic and clinical data are available, there are some significant patterns in how medications are administered. Trauma patients and men are more likely to receive opiate medications, and women are more likely to receive nonopiates. Possible explanations for the utilization of opiates in trauma patients include the likelihood that they have more severe or apparent pathology as opposed to the undifferentiated medical patient, a higher concern for hemorrhage, or heightened concern that a trauma patient may be an operative candidate. Previous literature has shown a significant gender disparity in acute pain management.2,19 There are not enough data from this study to determine whether the biases that have created this discrepancy factor into prehospital pain management; however, the demonstration of gender inequality in medication administration is

1238

Volume 21, no. 5: September 2020

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol

Table 3. Survey respondent demographics. All Subjects

Plan to give non-opiates

Do not plan to give non-opiates

P-value

41.98

38.91

41.4

< 0.01

Median

Range

24,62

26,53

Male

84 (84)

69 (82)

14 (93)

Female

16 (16)

15 (18)

1 (7)

11.73

11.15

Median

Range

1,34

1,26

Yes

95 (95)

80 (94.1)

15 (100)

5 (5)

5 (5.9)

0 (0)

Yes

35 (35)

35 (41.2)

0 (0)

65 (65)

50 (58.8)

15 (100)

Yes

85 (85)

85 (100)

0 (0)

15 (15)

0 (0)

15 (100)

7.02

6.94

7.5

Age Mean

Gender, n (%) 0.45

Years of Experience Mean

0.01

Have given fentanyl to a patient, n (%) 0.954

Have given non-opiate to a patient, n (%) N/A

Plan to give acetaminophen, ketorolac or ibuprofen, n (%) N/A

Pain scale at which opiate given, n (%) Mean Median

7.5

Range

2,10

5,10

4.21

N/A N/A

0.238

Pain scale at which non-opiate given, n (%) Mean Median

Range

1,10

consistent with known inequalities. Clinical parameters such as blood pressure, heart rate, and transport time do not have clinically significant differences in values with regard to the chosen analgesic. The state’s slow incorporation of non-opiate medications may be related to unfamiliarity and some initial discomfort with adverse-effect profiles. There may also be some uncertainty as to the appropriate use of a non-opiate vs an opiate for varying levels of reported pain and degree of pathology. The overall proportion of EMS patients who receive pain medication is very low – less than 2% per year – which brings into question whether the introduction of prehospital opiate medications is a significant contributor to later opiate misuse. Although more research is needed and the use of non-opiate medications should be encouraged Volume 21, no. 5: September 2020

N/A

when appropriate, it may be that prehospital pain management is still largely inadequate and that targeting prehospital opiate use may not be the most fruitful use of resources for misuse prevention. Perceptions Data Prehospital providers largely reported believing that pain management was part of their duty in the prehospital setting; however, there was controversy among respondents regarding gauging pain levels. While many prehospital providers employ the common 0-10 pain scale, there is no strict protocol requirement correlating a certain number with choice of analgesic and there was considerable range in the numbers that providers reporting being their “cut-off” for deciding to administer an opiate medication (2-10, mean 7).

1239

Western Journal of Emergency Medicine

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol Notably, the majority of respondents did report a difference in their threshold to initiate an opiate vs a non-opiate with a higher number correlating with initiating an opiate. This suggests that providers are individually using an internal decision-making framework that involves stratifying pain medication choice to the level of pain reported by patients. Methods of assessing pain level and correlating this with a particular analgesic are beyond the scope of this study, but this variety demonstrates a lack of standardization in pain management and suggests that there is significant variation among providers. Globally, apprehension about the possible adverse effects of the non-opiates was of lesser concern to the surveyed prehospital providers. Some providers expressed a concern for giving non-steroidal medications to patients who may require operative management or patients who are suspected to have internal hemorrhage. The literature, however, largely refutes the concern that one-time use leads to significant hemorrhagic complications.20- 21 The other concerns significant to providers with regard to non-opiate use include the belief that non-opiates are not as efficacious as opiates and that they take too long to work. While most providers agree that there are some conditions for which an opiate medication would be considered more appropriate, there is conclusive evidence that there are many conditions common to EMS where non-opiate medications are equally efficacious with regard to both patient safety and satisfaction and therein might be considered more appropriate for use.15 The Advent of Non-Opiate Options There are additional practical considerations in the use of non-opiates in EMS. There is value in initiating nonopiate pain management immediately rather than delaying administration of the same medication after a patient’s in-hospital evaluation. A patient who has not been administered an opiate medication may be able to have a shorter ED course because there is less concern about sedation, and if applicable, he or she would be able to operate machinery and return to activity sooner. Patients who have adequate pain management with a non-opiate in the field are less likely to expect opiate-based management in the ED whereas a patient who immediately receives opiate may be more likely to expect the same in the ED, even if the diagnosis is not one that would normally require opiate medication. Finally, non-opiate medications permit patients who cannot receive opiates to attain pain management in the field. In Massachusetts, among other states, patients with a history of substance use disorder or other reasons not to receive opiate medication have access to a voluntary “nonopioid directive form,” which signals to providers that they must receive alternative medications; having a robust arsenal of other options increases the feasibility and desirability of this directive.22 Western Journal of Emergency Medicine

Future Work and Study Future studies may seek to describe whether there is an association between prehospital opiate use, ED opiate use, and long-term opiate use. More surveillance of prehospital practice patterns as providers become more familiar with non-opiate analgesics is needed. There have been studies demonstrating a reduction in overall opiate utilization in the acute care setting when non-opiate pain management options are made first line in pain management protocol; an extension of this type of trial to the prehospital setting is an important avenue of exploration.23,-24 Finally, there are a number of additional pain management adjuncts including ketamine, lidocaine nerve blocks, and nitrous oxide that have not been universally implemented in the prehospital setting; these may be validated as highly efficacious, prehospital pain management options. LIMITATIONS Both arms of this study had multiple limitations. The data collected were in a single state with its own protocols and therefore have limited generalizability to the rest of the country. There were relatively limited demographic and clinical data available for patient subjects, and the doses of the medications administered were not available. ED data and final diagnoses were not available for subjects. As with the introduction of many protocols, there may be lag time between the implementation phase of the intervention and the prevalence of provider use of the intervention, so it is possible that data from coming years will yield a more representative depiction of pain management practice patterns. The survey arm of the study was limited to 100 providers and may not be representative of all licensed paramedics in the state. The proportion of each agency that participated in the survey was not recorded due to concerns about anonymity, and therefore one agency may have been relatively over-represented. Subjects were recruited as a convenience sample, which may have biased the results. Finally, a Hawthorne effect may have created bias given the current cultural environment pertaining to opiates. CONCLUSION Non-opiate medications have been modestly incorporated into one state’s practice a year after introduction. Limited data are available on providers’ patterns of pain management, but there are some trends that may inform future educational opportunities for the medical director. Paramedics largely report enthusiasm for the non-opiate analgesic. The prehospital setting would benefit from more literature describing the efficacy of prehospital pain management and its contribution to the clinical course of acute care patients. ACKNOWLEDGMENTS The 2018 Massachusetts College of Emergency Physicians Resident Research Grant supported this project. The authors are grateful for this invaluable support.

1240

Volume 21, no. 5: September 2020

O’Connor et al.

Paramedic Pain Management with a Non-opiate Treatment Protocol

Address for Correspondence: Laurel O’Connor, MD, University of Massachusetts Medical School, Department of Emergency Medicine, 55 Lake Avenue North, Worcester, MA 01655. Email: laurel.o’connor@umassmemorial.org.

Tintinalli’s Emergency Medicine: A Comprehensive Study Guide (8th ed.). New York, NY: McGraw-Hill Education. 12. Al B, Sunar MM, Zengin S et al. Comparison of IV dexketoprofen trometamol, fentanyl, and paracetamol in the treatment of renal colic in the ED: a randomized controlled trial. Am J Emerg Med.

2018;36(4):571–6. 13. Pathan, SA, Mitra B, Straney LD, et al. Delivering safe and effective analgesia for management of renal colic in the emergency department: a double-blind, multigroup, randomised controlled trial.

Copyright: © 2020 O’Connor et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

Lancet. 2016;387(10032):1999–2007. 14. Serinken M, Eken C, Turkcuer I, et al. Intravenous paracetamol versus morphine for renal colic in the emergency department: a randomised double-blind controlled trial. Emerg Med J. 2012;29(11):902–5. 15. Sobieraj DM, Baker WL, Martinez BK, et al. (2019). Comparative Effectiveness of Analgesics To Reduce Acute Pain in the Prehospital

REFERENCES

Setting. Comparative Effectiveness Review No. 220. Rockville, MD: Agency for Healthcare Research and Quality.

1. Galinski M, Ruscev M, Gonzalez G, et al. Prevalence and

16. Massachusetts Department of Public Health Office of Emergency

management of acute pain in prehospital emergency medicine.

Medical Services. Massachusetts Ambulance Trip Recording

Prehosp Emerg Care. 2010;14(3):334–9.

System. 2019. Available at: https://www.mass.gov/info-details/

2. Lord B, Cui J, Kelly A. The impact of patient sex on paramedic

massachusetts-ambulance-trip-record-information-system-matris.

pain management in the prehospital setting. Am J Emerg Med.

Accessed October 10, 2019.

2009;27(5):525–9.

17. Harris PA, Taylor R, Thielke R, et al. Research electronic data

3. American College of Emergency Physicians. Out-of-Hospital Use of

capture (REDCap) – A metadata-driven methodology and workflow

Analgesia and Sedation. Ann Emerg Med. 2016;67(2):305–6.

process for providing translational research informatics support, J

4. Alonso-Serra HM, Wesley K, NAEMSP. Prehospital pain

Biomed Inform. 2009;42(2):377-81.

management. Prehosp Emerg Care. 2003;7(4):482-8.

18. Harris PA, Taylor R, Minor BL, et al. The REDCap consortium:

5. Albrecht E, Taffe P, Yersin B, et al. Undertreatment of acute pain (oligoanalgesia) and medical practice variation in prehospital

Building an international community of software partners, J Biomed

analgesia of adult trauma patients: a 10 yr retrospective study. Brt J

Inform. 2019;95:103208. 19. Hirsh AT, Hollingshead NA, Matthias MS, et al. The influence of

Anaes. 2013;110(1):96–106.

patient sex, provider sex, and sexist attitudes on pain treatment

6. Brennan F, Carr D, Cousins M. Access to pain management—still

decisions. J Pain. 2014;15(5):551-9.

very much a human right. Pain Med. 2016;17(10):1785–9.

20. Gobble, RM, Hoang HL, Kachniarz B, et al. Ketorolac does not

7. Campbell JN. APS 1995 Presidential address. Pain Forum.

increase perioperative bleeding: a meta-analyiss of randomized

1996;5(1):85–8.

control trials. Plast Reconstr Surg. 2014;133(3):741–55.

8. U.S. Department of Health and Human Services Centers for Disease Control and Prevention. Understanding the Epidemic. 2018. Available

21. Strom BL, Berlin JA, Kinman JL, et al. Parenteral ketorolac and risk of gastrointestinal and operative site bleeding: a postmarketing

at: https://www.cdc.gov/drugoverdose/epidemic/index.html. Accessed

surveillance study. JAMA. 1996;275(5):376–82.

February 25, 2019.

22. Massachusetts Department of Public Health Office of Emergency

9. Butler MM, Ancona RM, Beauchamp GA, et al. Emergency department prescription opioids as an initial exposure preceding

Medical Services. 2017. Implementation of the STEP Law:

addiction. Ann Emerg Med. 2016;68(2):202–8.

Voluntary Non-Opioid Directive Form. 2017. Available at: https:// www.mass.gov/files/documents/2017/01/xd/non-opioid-directive_0.

10. Massachusetts Department of Public Health Office of Emergency

pdf. Accessed November 6, 2019.

Medical Services. Emergency update to 2.13 Pain and Nausea

23. Cohen V, Motov S, Rockoff B, et al. Development of an opioid

Management Protocol. 2017. Available at: https://www.mass. gov/lists/emergency-medical-services-statewide-treatment-

reduction protocol in an emergency department. Am J Health Syst

protocols#emergency-statewide-treatment-protocol-updates-.

Pharm. 2015;72(23),2080–6. 24. Duncan RW, Smith KL, Maguire M, et al. Alternatives to opioids for pain

Accessed February 25, 2019.

management in the emergency department decreases opioid usage and

11. Ducharme J. (2016). Acute pain management. In JE Tintinalli JS, Stapczynski OJ, Ma DM, Yealy GD, Meckler, and DM Cline (Eds.),

Volume 21, no. 5: September 2020

1241

maintains patient satisfaction. Am J Emerg Med. 2019;37(1),38–44.

Western Journal of Emergency Medicine

Original Research

Insurance Does Not Affect Adverse Events While Awaiting Surgery for Ankle Trauma in One System Adam B. Dobbins, APRN, PhD§ John Krumme, MD* Monica Gaddis, PhD†‡ Shin Hye Park, RN, PhD§ Manna Varghese¶ Michael R. Brancato¶ Christopher M. Shaw, MD|| Karen Wambach, RN, PhD§

*Washington Cancer Institute, Department of Musculoskeletal Oncology, Washington DC, Maryland † University of Missouri Kansas City, Department of Emergency Medicine, Kansas City, Missouri ‡ University of Missouri Kansas City, Department of Biomedical and Health Informatics, Kansas City, Missouri § University of Kansas Medical Center, School of Nursing, Department, Kansas City, Kansas ¶ University of Missouri Kansas City, School of Medicine, Department, Kansas City, Missouri || University of Missouri Kansas City, Department of Orthopaedic Surgery, Kansas City, Missouri

Section Editor: Gary Johnson, MD Submission history: Submitted February 6, 2020; Revision received May 4, 2020; Accepted May 20, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46861

Introduction: Ankle injuries that are not properly cared for can have devastating effects on a patient’s health and ability to maintain an active lifestyle. Recommended outpatient surgery may be difficult to obtain for many groups of patients, including those without insurance or minority races. Patients who are of low socioeconomic status also have worse outcomes following trauma. The purpose of this study was to examine whether insurance status impacts the number of adverse events that patients face prior to receiving surgical treatment following an emergency department (ED) visit for an acute ankle injury. Methods: We conducted a retrospective chart review at two medical centers within the same healthcare system. The sample included 192 patients presenting to the ED with an unstable ankle injury between October 1, 2015– May 1, 2018. We used chi-square and t-test analysis to determine differences in rates of adverse events occurring while awaiting surgery. Results: Few (4%) patients presented as being self-pay. Neither Medicare (χ2 (1) (N = 192) = 2.389, p = .122), Medicaid (χ2 (1), (N = 192) = .084, p = .772), other insurances (χ2 (1) (N = 192) = .567, p = .452), or private insurance (χ2 (1) (N=192) = .000, p = .982) was associated with a difference in rates of adverse events. Likewise, gender (χ2 (1) (N = 192) = .402, p = .526), race (χ2 (3) (N = 192) = 2.504, p = .475), and all other demographic variables failed to show a difference in occurrence of adverse events. Those admitted to the hospital did show a lower rate of adverse events compared to those sent home from the ED (χ2 (1) (N = 192) = 5.452, p = .020). Sampled patients were admitted to the hospital at a high rate (49%). Conclusion: The sampled facilities did not have adverse event rates that differed based on insurance status or demographic features. These facilities, with hospital-based subsidy programs and higher than expected admission rates, may manage their vulnerable populations well and may indicate their efforts to eliminate health disparity are effective. [West J Emerg Med. 2020;21(5)1242-1248.]

Western Journal of Emergency Medicine

1242

Volume 21, no. 5: September 2020

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery

INTRODUCTION Current standard of care for treatment of unstable ankle fractures in the emergency department (ED) is to evaluate and splint, and then have the patient present for outpatient orthopedic follow-up.1 However, those who face barriers to obtaining outpatient treatment may have poorer outcomes than others, indicating this standard of practice may not be optimal for all patients. Navigating outpatient follow-up and outpatient surgery in the face of socioeconomic and payer-source differences may result in significant health disparity in acute ankle-injury patients. Researchers have identified barriers to ED patients getting follow-up. Health systems often do not maintain accurate telephone numbers,2 and making followup appointments can be difficult or appointments may not be available.3 Patients relying on Medicaid or those without insurance4,5 and minority race populations6 have increased difficulty securing follow-up. Acute ankle injuries can have long-term sequela including recurrent sprains of the injured ankle, instability with sensations of “giving way,” stiffness and swelling, or other symptoms that prevent patients from participating in everyday activities, even with sound treatment.7 For individuals who enjoy being active or whose livelihoods depend on standing or moving, failure to return to health following this type of injury can cause significant harm. Along with prolonged instability and potential permanent loss of or decrease in mobility, ankle fractures that do not heal in proper alignment are seven times more likely to develop ankle arthritis, which can cause pain and stiffness requiring long-term treatments.8 Trauma patients without insurance have increased rates of mortality and complications,9-11 indicating that there may be disparities in accessible care for trauma patients. Understanding barriers to proper care may provide information that could lead to achieving more health equality as dictated by Healthy People 202012 and other groups.13,14 To our knowledge, no studies have looked at follow-up rates or disparities that affect the surgical ankle-fracture patient. There is a lack of research that explores whether or not the current practice of stabilizing acute ankle injuries in the emergency department (ED) and instructing patients to follow up with a specialist for further evaluation and surgical treatment leads to health disparity among the non-insured. The purpose of this study was to explore whether, among patients who suffer an unstable ankle injury, insurance status is associated with an increased incidence of adverse events experienced prior to surgical correction. Secondary purposes, including whether demographic factors such as gender or race, being homeless, or intoxicated at the time of injury, were also explored. METHODS Design We conducted a retrospective chart review with data abstracted from the electronic health records (EHR) at two EDs within a single institution to examine the relationship Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Improperly treated, unstable ankle injuries, which generally require surgical fixation, can lead to a plethora of adverse sequela, including chronic pain and immobility. What was the research question? Among patients who suffer an unstable ankle injury, is insurance status associated with an increased incidence of adverse events prior to surgical correction? What was the major finding of the study? We found no difference in rates of adverse events prior to obtaining surgery in patients with acute ankle injuries, regardless of insurance type. How does this improve population health? This data indicates it is possible to take steps to reduce barriers to optimal surgical treatment of unstable ankle injuries and decrease disparity in healthcare delivery

between payer sources and adverse events while awaiting surgery in patients suffering acute, unstable ankle fractures. Sampling and Setting We collected data from the EHRs of two EDs within a single health system where the same orthopedic team serves as consultant for both EDs. One ED is an urban, safety-net, nonprofit hospital near the downtown area of a large Midwest, US city. It serves as the primary teaching hospital for an adjacent medical college and its mission speaks to providing accessible healthcare regardless of a patient’s ability to pay. As such, it treats many vulnerable populations including the homeless and those without health insurance. The second site lies in a suburban area and focuses on primary care services and provides easy access for acute and well-care needs for all ages. The two facilities share an EHR system. Subsidized care is available at both facilities for qualifying patients who live within the same county as the hospitals and meet income requirements. International Classification of Diseases, 10th edition (ICD-10) codes were identified to capture patients who presented to the EDs with a closed ankle injury for which the standard of care is typically surgical fixation. Table 1 shows a full list of codes used. We obtained all EHRs from patients presenting to either of the two EDs between October 1, 2015–

1243

Western Journal of Emergency Medicine

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery Table 1. ICD-10* codes used to capture patients with an unstable ankle injury.

Ankle fracture

Bimalleolar fracture

Lateral malleolus fracture

Medial malleolus fracture

Pilon fracture

Trimalleolar fracture

Distal tibial articular fracture

Syndesmotic injury

S82.843A

S82.841

S82.63XA

S82.53SA

S82.873

S82.842

S82.64XA

S82.51XA

S82.871

S82.851

S82.3

S93.439A

S82.852

S82.30

S93.431

S82.843

S82.65XA

S82.52XA

S82.872

S82.853

S82.301

S93.431A

S82.844

S82.66XA

S82.845

S82.61XA

S82.53XA

S82.873

S82.854

S82.301A

S93.432

S82.54XA

S82.874

S82.855

S82.302

S93.432A

S82.846

S82.61XA

S82.55XA

S82.875

S82.856

S82.302A

S93.439

S82.846

S82.63XA

S82.56XA

S82.876

S93.439A

S82.851A

S82.309

S82.842A

S82.852A

S82.309A

S82.844A

S82.853A

S82.39

S82.845A

S82.854A

S82.391

S82.855A

S82.391A

S82.856A

S82.392 S82.392A S82.399 S82.399A

*International Classification of Diseases, 10th edition.

May 1, 2018 and meeting one of the identified ICD-10 codes. We developed a master list of charts that included patient identifying information within a REDCapT database. All study data were collected and managed using REDCap electronic data capture tools.15 We assigned random codes to each chart and removed all identifying information. The master list with patient identifying information was stored separately from data collected. Using predetermined guidelines that indicate surgery would typically be recommended for the treatment of an ankle injury, a fifth-year orthopedic resident reviewed radiographs of each subject to determine whether surgical fixation would likely be recommended. Guidelines for surgical injuries included lateral malleolus injury with joint subluxation, lateral malleolus injury with medial clear space widening on stress or standing view radiograph, displaced medial malleolus fracture, bimalleolar fractures, trimalleolar fractures, or high fibular fractures with a positive stress exam. Those that were determined to be surgical were included. Data were abstracted from the selected charts by two researchers who were blinded to the purpose of the study. We calculated Cohen’s kappa scores to check inter-rater reliability, and the lead researcher trained abstractors to ensure as much consistency between the abstractors as possible. There were 552 medical records with ankle injuries per the selected ICD codes, of which 255 were identified as unstable after radiograph review. On chart review, 13 were not actually acute ankle injuries or EHR data were not available. An additional 20 patients presented directly to orthopedics or Western Journal of Emergency Medicine

podiatry and were not ED patients, three of whom suffered injuries while hospitalized. For 30 patients, surgery was not recommended, despite their injuries. The most common reasons for not having surgery recommended were co-morbid conditions that increased surgical risks or physician preference at the time of initial evaluation. A sample of 192 cases remained and were included in the study. Measures For this study we considered any ankle injury as found above that is expected to require surgical intervention to promote proper healing as an unstable ankle injury. The dependent variable was adverse events that served as an additional injury or problem with obtaining surgical intervention. Time of surgery served as the time that patient charts were no longer reviewed as they had begun terminal treatment for the injury. Adverse events included the following: re-injury at the original site; delay in surgery greater than three weeks; lost to follow-up where no records up to eight weeks post-injury were found to indicate surgery was ever performed; return ED visits prior to surgery; new traumatic injury; and new pressure ulcer at the site of injury or elsewhere on the body. The primary independent variable was insurance status and was grouped into the following categories: 1) private insurance; 2) Medicare; 3) Medicaid; 4) worker’s compensation/liability insurance; 5) self-pay/no-charge; and 6) other, for which the majority of “other” patients were included in the hospital-provided subsidy plan. It is important to note

1244

Volume 21, no. 5: September 2020

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery

that the subsidy plan can be applied retroactively, so many of these patients were likely self-pay at the time of the initial ED visit and retroactively converted to the subsidy plan. Other variables collected were the demographic data of age in years, biological gender, and race/ethnicity grouped as White, Black, Hispanic, or other. Residency information was collected and grouped as private home, nursing home, homeless, or other, and the county and state of residence was included. Alcohol and drug (excluding marijuana) intoxication at the time of injury was collected, identified by healthcare provider notes or a diagnosis code related to alcohol or drug intoxication within the ED chart during the same initial visit for injury. Data Analysis We used SPSS version 22 (IBM Corp, Armonk, NY) for data analysis. All data were imported from REDCap into SPSS for analysis. Prior to collecting data a power analysis identified 196 as a target sample size for this study. Descriptive statistics were examined individually and the chi-square (χ2) test of association was applied to categorical independent and dependent variables. The primary independent variable of insurance was examined on each variable in a 2x2 table to determine whether the dependent variable was statistically different when the independent variable of an adverse event occurring was present compared to when not present. We also examined secondary outcomes examined via χ2 techniques for categorical data and with the t-test statistic for continuous level data. Significance level was set at less than or equal to .05. We applied Bonferroni adjustments in levels of statistical significance when appropriate after comparing multiple variables against the dependent variable. Ethical Considerations The involved academic institutions’ institutional review boards reviewed all study protocols and permission was granted from the hospital’s privacy committee to use the EHR data. The study was granted exempt classification since only medical records were being used and risk to patients was small. All data were secured within REDCap and patient identifiers were stored separately from the data collected. Patient identifier information was only accessed when it was necessary to review information on the patient within the medical record and used only by researchers tasked with reviewing patient charts. Data collection that involved the use of patient identifying information was always conducted in a private location to prevent possible casual observation of patient information that could occur in a public venue. RESULTS There were 192 patients seen in one of two EDs within this single hospital system who sustained an acute ankle injury that needed surgical repair. The mean age of patients was 43.63 (standard deviation [SD] 14.1) years, and 55% were male. White race was predominant at 46%, with fewer Black (34%), Volume 21, no. 5: September 2020

Hispanic (11%), or other (9%) races represented. This reflects a sampling of the general ED population, which was 50% Black, 33% White, and 11% Hispanic. The majority resided in private homes (91%), and approximately 5% were homeless. Fifteen percent were identified as intoxicated with alcohol at the time of initial visit and 5% with other substance intoxication. The ankle injury was an isolated injury in 84% of patients and 49% were admitted to the hospital directly from the ED. Among the 38.3% of patients with “other insurance” listed, almost all had a hospital-specific subsidy applied either at the time of ED visit or applied to their account retroactively. Patients who presented to the ED as self-pay, and had the subsidy applied retroactively, were queried as “other insurance” and did not remain selfpay. Otherwise, insurance classifications were represented as 18.1% with private insurance, 12.4% with Medicare, 16.1% with Medicaid, 10.9% with workers’ compensation or liability insurance, and 4.2% remained self-pay. Fifteen percent of all patients sustained an adverse event prior to surgical treatment. Related to insurance status, the rate of adverse events ranged from 10% in the workers’ compensation/liability group to 25% in the Medicare group. There were no statistically significant differences in insurance types noted between those with adverse events and those without adverse events. There were no significant differences in any other demographic variables among those having and not having an adverse event, except for those “not admitted to the hospital” who had a 2.755 increased odds of having an adverse event compared to those admitted directly to the hospital during their initial ED visit (χ2 (1) (N = 192) = 5.452, p = .020). Reasons for admission were frequently not clear on chart review with 33% of charts not giving any indication of reason for admission. Most, 42%, were admitted by trauma services following a dangerous mechanism of injury, often for observation. Other reasons included 12% admitted due to their comorbidities, 7% for pain control, 4% for social or economic reasons, and 2% for mobility concerns. Those individuals who sustained multiple injuries at the time of ED visit had 5.814 increased odds of having a presurgical adverse event compared to those having an isolated injury, although this was not statistically significant (χ2 (1) (N = 192) = 3.613, p = .057). All demographic variables, as well as the results of the comparisons by complication/no complication, are shown in Table 2. DISCUSSION This study, conducted at two EDs within a single hospital system, failed to identify any differences in rates of adverse events prior to obtaining surgery in patients with acute ankle injuries requiring surgical correction regardless of type of insurance coverage. This is in contrast to previous studies in which acute trauma patients had increased rates of mortality and complications when they did not have insurance.9-11 Furthermore, previous research showed that obtaining follow-

1245

Western Journal of Emergency Medicine

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery up care can be difficult,2,3 which seems paramount to patients who are often discharged with the intent to secure outpatient surgical services. Previous research also indicated follow-up was particularly difficult to obtain for those on Medicaid and

without insurance.4-6 The current standard of care for ankle fractures such as those focused on in this study is to treat patients on an outpatient basis.1 However, among the patients sampled at

Table 2. Demographics and chi-square calculated p-values for subjects with and without adverse events prior to obtaining terminal (surgical) treatment for acute ankle injuries.

All n(%)

Patients with adverse events n (%)

Patients without adverse events n (%)

Ď&#x2021;2 statistic

P-value

Odds ratio

Private

35 (18)

5 (15)*

29 (85)

0.000

.982

1.012

Medicare

24 (12)

6 (25)*

18 (75)

2.389

.122

2.212

Medicaid

31 (16)

4 (13)

27 (87)*

0.084

.772

1.182

Workers comp/ liability

21 (11)

2 (10)

19 (90)*

0.485

.486

1.704

Variable Payer source

Self/no pay

8 (4)

2 (15)*

6 (75)

0.727

.394

2.012

74 (38)

9 (12)*

65 (88)

0.567

.452

1.386

Male

107 (55)

17 (16)*

89 (84)

0.402

.526

1.302

Female

86 (45)

11 (13)

75 (87)

White

88 (46)

12 (14)

76 (86)*

0.117

.732

1.152

Black

66 (34)

12 (18)*

54 (82)

1.045

.307

1.529

Hispanic

21 (11)

1 (5)

20 (95)*

1.826

.177

3.75

Other

17 (9)

3 (18)*

14 (82)

0.141

.708

1.285

Private home

176 (91)

24 (14)

151(86)*

1.520

.218

2.111

Nursing home

Other Gender

Race

Residence

Homeless

10 (5)

2 (20)*

8 (80)

0.248

.618

1.499

Other

6 (3)

4 (67)*

2 (33)

1.748

.186

3.077

Yes

29 (15)

3 (10)

26 (90)*

0.493

.483

1.570

163 (85)

25 (15)

138 (85)

Yes

9 (5)

2 (22)*

7 (78)

0.442

.506

1.724

183 (95)

26 (14)

157 (86)

Yes

163 (84)

27 (17)

135 (83)*

3.613

.057

5.814

30 (16)

1 (3)

29 (97)

95 (49)

8 (9)

86 (91)*

5.452

.020**

2.755

ETOH intoxication

Drug intoxication

Isolated injury

Admitted hospital Yes No Age

98 (51)

20 (20)

78 (80)

Mean (SD)

t-test statistic

P-value

43.77 (14.0)

44.57(11.8)

43.62 (14.4)

-.330

.742

Note: * Higher odds of event occurring, ** Statistically significant with p<.05, #indicates t-statistic. ETOH, ethyl alcohol; SD, standard deviation.

Western Journal of Emergency Medicine

1246

Volume 21, no. 5: September 2020

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery

this facility, nearly half (49%) were admitted to the hospital at the time of their initial ED visit. This is in stark contrast to previously reported admission rates of 17% for ankle fractures in Finland16 and 31% in Italy.17 Although this facility is a major, inner-city, trauma center, 84% of patients had isolated ankle injuries; thus, severity of illness does not readily explain the high admission rate. Many patients were admitted for observation following a dangerous mechanism of injury. Chart abstraction was attempted to determine the cause of admission for patients; however, in a majority of charts admission decisions were not clear. This facility serves a high volume of patients considered vulnerable; thus, healthcare providers here may be more likely to admit patients for social reasons or to prevent adverse events in comparison to other institutions. Indeed, being admitted at the time of the ED visit was the only statistically significant finding in this study, showing fewer adverse events occurred when patients were directly admitted from the ED. This healthcare system and the orthopedic group that ultimately makes admission decisions for these patients treat a large number of low-income, racially diverse, and other vulnerable patient populations. These healthcare providers may proactively and aggressively treat these patients, thereby decreasing the odds of the patients receiving disparate care. The orthopedic clinic has also committed to following up with all patients that present through the facilities’ EDs to assist patients to get insurance coverage or hospital-based subsidy, or even making the exception to provide surgery to those who cannot pay. Anecdotally, patients frequently report that other local facilities will not provide them surgical or follow-up services due to their financial/insurance status, despite identifying that their injury needs additional care. This study also sampled a lower number of self-pay patients than was expected. This study found only about 4% were listed as self-pay compared to national database reports of about 16% in 2010.18 This is likely because the institution has a subsidy program. Patients who live within the same county and qualify may obtain reduced or no-cost services despite a lack of insurance. This subsidy program can be applied to ED visits retroactively; thus, a large number of patients who would be self-pay at other facilities were likely marked as “other insurance” in this instance. The EHR does not allow users to separate patients identified initially as self-pay from those who had the subsidy applied after the ED visit. Despite this, neither the remaining self-pay patients nor the “other” insurance category, which includes the subsidy program patients, had a statistically different rate of having adverse events. Although not statistically significant, Medicare patients had 2.389 increased odds of having an adverse event prior to receiving surgical treatment. This may be a reflection of agerelated decreased ability to heal following injury, rather than related to insurance coverage. Patients with an isolated injury had 5.814 decreased odds of having an adverse event. Again, although this finding is not statistically significant, it may Volume 21, no. 5: September 2020

suggest that multitrauma patients may be at higher risk than those with isolated ankle injuries. Currently there is a widespread call to reduce healthcare disparities.12-14 The findings of this study indicate that this single hospital system may provide appropriate care for vulnerable populations and may be meeting goals to minimize healthcare disparity based on patient insurance status and patient demographics. LIMITATIONS Examination of this data failed to support the primary outcome that insurance status at a single facility correlated with difficulty obtaining surgical correction of an unstable ankle injury. While there were no significant differences, type II error is always a possibility, especially with this small sample size. This study was also limited in its ability to generalize beyond this health system. Given that only a single system was used for data collection in this study, along with the unexpected rates of patients admitted to the hospital from the ED and those with self-pay status, these results may be difficult to extrapolate to any larger population. This may be the result of efforts within this health system to decrease disparity and may well be unlike many other facilities. This was a retrospective chart review. Data in EHRs are collected by healthcare providers as part of their routine care for patients and are not collected with the methodological rigor that researchers use in collecting data. Therefore, it must be understood that the information gained from these records may contain inaccuracies or information recorded in a way that does not translate well into the research data-collection procedure. Abstractors were trained prior to reviewing charts and were updated if problems arose along the way (eg, properly identifying patients as self-pay or those with hospitalsubsidized discount plans); they used standardized forms with precise definitions, and were blinded to the purpose of the study – all methods recommended to strengthen the chart review process.19,20 The sample of 192 records did not meet the pre-study power estimated need of 196. Including other facilities, or using a national database may help to strengthen future research in this area and provide for increased generalizability. This study was limited in scope by examining only outcomes prior to surgical intervention. Another question of concern to patients would be adverse event occurrence until complete healing of the injury. Factors such as surgical complications, poor wound healing after surgery, hardware failure, and acute or chronic pain are important patient-centered outcomes not examined in this study. The research could also be expanded to include other common, surgically treated fractures such as upper extremity, vertebral, or hip fractures. CONCLUSION This retrospective chart review shows that patients who present to one of two EDs within the same hospital

1247

Western Journal of Emergency Medicine

Dobbins et al.

Insurance Does Not Affect Adverse Events While Awaiting Ankle Surgery system did not show differences in sustaining adverse events prior to receiving surgical treatment based on insurance status or demographic variables. This is not consistent with other research and may indicate that this facility has implemented progressive policies and procedures to decrease health disparities among patients who fall into vulnerable population categories.

7. Kaikkonen A, Lehtonen H, Kannus P, et al. Long-term functional outcome after surgery of chronic ankle instability: a 5-year follow-up study of the modified Evans procedure. Scand J Med Sci Sports. 1999;9(4):239-44. 8. American Academy of Orthopaedic Surgeons. Arthritis of the foot and ankle. 2015. Available at: http://orthoinfo.aaos.org/topic. cfm?topic=a00209. Accessed June 14, 2017. 9. Baraga MG, Smith MK, Tanner JP, et al. Anterior cruciate ligament

Address for Correspondence: Adam B. Dobbins, APRN, PhD, Truman Medical Center, Department of Emergency Medicine, 2301 Holmes Rd, Kansas City, MO 64108. Email: Adam.Dobbins@ tmcmed.org.

injury and access to care in South Florida: Does insurance status play a role? J Bone Joint Surg Am. 2012;94(24):e183. 10. Chikani V, Brophy M, Vossbrink A, et al. Association of insurance status with health outcomes following traumatic injury: statewide multicenter analysis. West J Emerg Med. 2015;16(3):408-13.

11. Downing SR, Oyentunji TA, Greene WR, et al. The impact of insurance status on actuarial survival in hospitalized trauma patients: When do they die? J Trauma. 2011;70(1):130-4. 12. Office of Disease Prevention and Health Promotion. Disparities. 2014. Available at: https://www.healthypeople.gov/2020/about/

Copyright: ÂŠ 2020 Dobbins et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

foundation-health-measures/Disparities. Accessed June 10, 2017. 13. Ubri P, Artiga S. (2016). Disparities in health and health care: five key questions and answers. In: The Kaiser Family Foundation. 14. U.S. Department of Health & Human Services. Elimination of Health Disparities. Available at: https://www.surgeongeneral.gov/priorities/ prevention/strategy/elimination-of-health-disparities.html. Accessed

REFERENCES

June 10, 2017.

1. Koehler SM and Eiff P. Overview of ankle fractures in adults.

15. Harris PA, Taylor R, Thielke R, et al. Research electronic data

Available at: https://www.uptodate.com/contents/overview-of-ankle-

capture (REDCap) - a metadata-driven methodology and workflow

fractures-in-adults?source=search_result&search=unstable%20

process for providing translational research informatics support. J

bimaleolar%20ankle%20fracture&selectedTitle=1~150. Accessed

Biomed Inform. 2009;42(2):377-81.

June 10, 2017.

16. Somersalo A, Paloneva J, Kautiainen H, et al. Incidence of fractures

2. Aaland MO, Marose K, Zhu TH. The lost to trauma patient follow-

requiring inpatient care. Acta Orthop. 2014;85(5):525-30.

up: a system or patient problem. J Trauma Acute Care Surg.

17. Tarantino U, Capone A, Planta M, et al. The incidence of hip,

2012;73(6):1507-11.

forearm, humeral, ankle, and vetebral fragility fractures in Italy:

3. Vieth TL and Rhodes KV. Nonprice barriers to ambulatory care after

results from a 3-year multicenter study. Arthritis Res Ther.

an emergency department visit. Ann Emerg Med. 2007;51(5):607-13. 4. Asplin BR, Rhodes KV, Levy H, et al. Insurance status and

2010;12(6):R226. 18. Watts SH, Bryan ED, Tarwater PM. Changes in insurance status and

access to urgent ambulatory care follow-up appointments. JAMA.

emergency department visits after the 2008 economic downturn.

2005;294(10):1248-54.

Acad Emerg Med. 2014;22(1):73-80.

5. Magnusson AR, Hedges JR, Vanko M, et al. Follow-up compliance

19. Gilbert EH, Lowenstein SR, Koziol-McLain J, et al. Chart reviews in

after emergency department evaluation. Ann Emerg Med.

emergency medicine research: Where are the methods? Ann Emerg

1993;22(3):560-7.

Med. 1996;27(3):305-8.

6. Lee BW, Murakami Y, Ducan MT, et al. Patient-related and system-

20. Kaji AH, Schriger D, Green S. Looking through the retrospectoscope:

related barriers to glaucoma follow-up in a county hospital population.

reducing bias in emergency medicine chart review studies. Ann

Invest Ophthalomol Vis Sci. 2013;54(10):6542-8.

Emerg Med. 2014;64(3):292-8.

Western Journal of Emergency Medicine

1248

Volume 21, no. 5: September 2020

Original Research

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the Emergency Department Chad M. Cannon, MD* Ross T. Miller, MD, MS* Krista L. Grow, MD* Seth Purcell, MD* Niaman Nazir, MD†

*University of Kansas Medical Center, Department of Emergency Medicine, Kansas City, Kansas † University of Kansas Medical Center, Department of Preventative Medicine and Public Health, Kansas City, Kansas

Section Editor: Gabriel Wardi, MD Submission history: Submitted February 3, 2020; Revision received May 4, 2020; Accepted May 5, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46811

Introduction: While numerous studies have found emergency department (ED) lactate levels to be associated with increased in-hospital mortality, little information is available on the role age plays in this association. This study investigates whether age is a necessary variable to consider when using lactate levels as a marker of prognosis and a guide for management decisions in the ED. Methods: This was a retrospective cohort study in an urban, tertiary-care teaching hospital. A total of 13,506 lactate levels were obtained over a 4.5-year period. All adult patients who had a lactate level obtained by the treating provider in the ED were screened for inclusion. The main outcome measure was in-hospital mortality using age-adjusted cohorts and expanded lactate thresholds with secondary outcomes comparing mortality based on the primary clinical impression. Results: Of the 8796 patients in this analysis, there were 474 (5.4%) deaths. Mortality rates increased with both increasing lactate levels and increasing age. For all ages, mortality rates increased from 2.8% in the less than 2.0 millimoles per liter (mmol/L) lactate level, to 5.6% in the 2.02.9 mmol/L lactate level, to 8.0% in the 3.0-3.9 mmol/L lactate level, to 13.9% in the 4.0-4.9 mmol/L lactate level, to 13.7% in the 5.0-5.9 mmol/L lactate level, and to 39.1% in the 6.0 mmol/L or greater lactate level (p <0.0001). Survivors, regardless of age, had a mean lactate level <2.0 whereas nonsurvivors had mean lactate levels of 6.5, 4.5, and 3.7 mmol/L for age cohorts 18-39, 40-64, and ≥ 65 years, respectively. Conclusion: Our findings suggest that although lactate levels can be used as a prognostic tool to risk stratify ED patients, the traditional lactate level thresholds may need to be adjusted to account for varying risk based on age and clinical impressions. [West J Emerg Med. 2020;21(5)1249-1257.]

INTRODUCTION Lactate has been studied as a marker of critical illness for over a half century.1 Lactate levels can be used as a surrogate of tissue hypoperfusion in critically ill patients presenting to the emergency department (ED). Lactate production and metabolism are critical to the ability of the body to respond to metabolic stressors and varying shock states.2 However, lactate may also be elevated due to varying conditions in the absence of tissue hypoxia through a variety of mechanisms.3,4 Lactate levels can Volume 21, no. 5: September 2020

readily be obtained and used to identify patients at high risk of death, even prior to the development of hemodynamic instability. Poor organ perfusion, if not reversed, ultimately leads to organ dysfunction and failure, shock, and potentially death. The use of lactate levels has been shown to be a predictor of prognosis in diverse populations of critically ill patients ranging from trauma to septic shock.5–10 In the last decade, there has been increased use and evaluation of lactate in the ED and these studies demonstrate that elevated lactate levels are

1249

Western Journal of Emergency Medicine

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED associated with increased mortality.11–14 Prior studies have already demonstrated the utility of lactate levels to predict mortality in patients admitted to the hospital who presented from the ED with infection11,15,16 and severe sepsis,17,18 as well as trauma.19 With an increasing focus on guidelines and quality performance measures, initiatives such as the Surviving Sepsis Campaign, the Centers for Medicare and Medicaid Services (CMS) core measures, the Medicare Access and CHIP Reauthorization Act of 2015 and the Merit Based Incentive Payments System quality measures have all incorporated the measurement of serum lactate levels into their most current guidelines, likely contributing to the increasing the number of lactate levels being ordered in the ED.20 Since the beginning of early goal-directed therapy (EGDT) lactate levels have been increasingly used as diagnostic, therapeutic, and prognostic markers in ED patients.13,21 As there are numerous causes for an elevated lactate, it is important for emergency physicians to consider both sepsis and alternative diagnoses, because the prognostic value of lactate can vary depending on the underlying cause.22 While lactate has been shown to be sensitive for occult sepsis, the current literature supports the notion that while it is highly sensitive, it is not specific.23,35 Historically, the same lactate stratification levels have been used regardless of patient age or disease state: low levels, < 2.0 millimoles per liter (mmol/L); intermediate levels, 2.0-3.9 mmol/L; and high levels, ≥ 4.0 mmol/L.2 Despite this cutoff of ≥ 4.0 mmol/L being used in models such as EGDT21 and CMS criteria for septic shock,24 the intermediate lactate level has also proven to be a high-risk group, as previously noted by Mikkelsen and Howell.15,17 In addition to investigating expansion of lactate threshold levels, adjusting these lactate thresholds for age may also contribute to additional discrimination. Portal et al showed that higher ED lactate values are associated with greater mortality in adults over 65 years of age, with or without the presence of infection.9 It has also been shown that there is a higher mortality with increasing age in patients with lactic acidosis.9 Therefore, in this study we hypothesized that older patients within the same lactate threshold level would have higher mortality rates, regardless of ED diagnosis. Finally, we hypothesized that additional lactate groupings (six) would provide greater mortality rate discrimination than the three traditionally used groupings for “low,” “intermediate,”, and “high.” Hence, the primary objective of this study was to examine the combination of lactate level and age as predictors of mortality in adult patients who had a lactate level drawn in the ED as part of their initial work-up. Secondary objectives were 1) to compare traditional vs expanded lactate thresholds as predictors of mortality in three separate age cohorts; and 2) to identify the most common ED clinical impressions associated with an elevated lactate level. METHODS Study Design This was a retrospective, observational, cohort study of adult Western Journal of Emergency Medicine

Cannon et al.

Population Health Research Capsule What do we already know about this issue? Lactate levels are a predictor of mortality among emergency department (ED) patients. What was the research question? This study investigates the effect of age across lactate levels as a predictor of mortality in ED patients. What was the major finding of the study? Increases in lactate or age, individually or in combination, were significantly associated with an increasing mortality risk. How does this improve population health? These results suggest that lactate levels and age together can be used to guide clinical practice, and traditional lactate thresholds may need to be both expanded and adjusted for age.

patients (aged 18 years or older) who presented to the ED at an urban, tertiary-care teaching hospital from October 2009– May 2013. The average annual census for the ED is approximately 60,000 visits. The patients included were identified via Epic (Epic Systems, Verona, WI), the electronic health record for the hospital. The study is reported in accordance to the STROBE guidelines.25 It was reviewed by the institutional review board (the University of Kansas Medical Center Human Subjects Committee) and a waiver of informed consent was granted. Adult patients (aged 18 years or older) who had a lactate level drawn in the ED were included in the study. The ED in which this study was performed obtained a lactate level on individuals at the discretion of the emergency provider. At our institution, a sepsis protocol has existed since 2005, based upon recommendations from the Surviving Sepsis Campaign.26 This protocol educated all emergency providers on practice patterns and early recognition of sepsis, including the early obtainment of lactate levels. However, it was ultimately up to the emergency provider to determine whether or not a lactate level was necessary, and often lactate levels were ordered for many reasons other than suspected sepsis.2 Lactate levels were measured at the bedside with the Abbott point-of-care I-STAT (43%) (Abbott Laboratories, Chicago, IL), and in the hospital laboratory using a Beckman Coulter instrument (57%) (Beckman Coulter, Inc. Brea, CA). Bedside point-of-care lactate measurements have been shown to have excellent correlation with lab-reported lactate levels.27 We included only the first lactate level and clinical impression obtained during any ED visit in the dataset, and for individuals with multiple ED visits during the study period, only

1250

Volume 21, no. 5: September 2020

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED

the first lactate level and clinical impression of the most recent encounter was included. We excluded patients with a diagnosis of seizure because associated high lactate levels carry a very low mortality risk in that subset and prior lactate research studies have excluded patients with seizures2,12 (Figure 1). Transfer patients from outside hospitals are directly admitted to in-patient services at our institution; thus, there were no transfer patients included in this cohort. Demographics, including age, gender, race, vital signs, and diagnosis codes were obtained from the hospital discharge database and linked to lactate levels. The diagnosis codes included for acute infection and acute organ dysfunction, in Table 1 and 2, were defined by Angus et al in 2001.28 The primary outcome measure was in-hospital mortality, defined as patients who were admitted and died during the same encounter. Secondary outcome measures included hospital admission and admission to the intensive care unit (Table 1). Statistical Methods We calculated descriptive statistics for age, lactate levels, vital signs, admission rates, mortality rates, and diagnoses. The patients were stratified into three age cohorts determined a priori: 18-39 years; 40-64 years; and 65 years or older. Patients were stratified into one of six lactate level cohorts that were also determined a priori: less than 2.0 mmol/L; 2.0-2.9 mmol/L; 3.03.9 mmol/L; 4.0-4.9 mmol/L; 5.0-5.9 mmol/L; and 6.0 mmol/L

or greater. We performed analysis using SAS software version 9.3 (SAS Institute, Inc., Cary, NC). Chi-squared and Kruskal-Wallis tests were performed to compare age cohorts, lactate levels, and diagnoses to demographics, vital signs, and outcomes. We used logistic regression to estimate mortality odds ratios. Logistic models predicting mortality included either age groups (18-39, 40-64 and â&#x2030;Ľ 65) or lactate level groupings (< 2, 2-2.9, 3-3.9, 4-4.9, 5-5.9, and â&#x2030;Ľ 6), or a combination of both age and lactate level groupings as predictors. Logistic model using < 2 lactate level as the reference was stratified for each age group. Similarly, logistic model using the 18-39 age group as the reference was stratified for each lactate level grouping. We reported all results using an alpha level of 0.05. When applicable, 95% confidence intervals and standard error of the mean (SEM) were reported. RESULTS Lactate levels were obtained on 13,506 patients, or 6.17% of the total patients seen in the ED over a 4.5-year period. Of these, we excluded 4710: 18 had lab error lactate values; 213 were younger than 18 years old; 4084 had multiple ED encounters; and 395 had a diagnosis of seizure (Figure 1). Thus, a total of 8796 patients were included in our analysis. A total of 474 (5.4%) in-hospital deaths occurred. Mortality rates generally increased with increasing lactate level and age (Tables 1 and 2). As lactate and/or age rose, patients were noted to have increased incidence

Figure 1. Study flowchart depicting the inclusion and exclusion criteria for ED patients with lactate levels. ED, emergency department.

Volume 21, no. 5: September 2020

1251

Western Journal of Emergency Medicine

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED Table 1. Population characteristics by lactate levels. Initial lactate level (mmol/L)

Demographics

Clinical variables

<2 (N=5634)

2 - 2.9 (N=1802)

3 - 3.9 (N=659)

4 - 4.9 (N=296)

5 - 5.9 (N=139)

≥6 (N=266)

55, 54.3 (± 18.5)

57, 56.5 (± 18.0)

57, 56.8 (± 17.3)

56, 56 (± 18.0)

55, 54.6 (± 18.6)

57, 56.4 (± 17.7)

<0.0001*

Male, n (%)

2,256 (45.4)

932 (51.7)

365 (55.4)

157 (53.0)

84 (60.4)

156 (58.6)

<0.0001

White, n (%)

3,593 (63.8)

1147 (63.7)

423 (64.3)

186 (62.8)

76 (54.7)

148 (55.6)

0.02

Lactate, mmol/L (median, mean ± SD)

1.3, 1.3 (± 0.4)

2.3, 2.4 (± 0.3)

3.4, 3.4 (± 0.3)

4.4, 4.4 (± 0.3)

5.3, 5.4 (± 0.3)

8.3, 9.4 (± 3.4)

<0.0001*

SBP, mmHg (median, mean ± SD)

134, 135 (± 27)

132, 134 (± 29)

127, 130 (± 31)

125, 126 (± 31)

127, 126 (± 30)

121, 126 (± 36)

<0.0001*

Hypotensive (SBP<90) , n (%)

171 (3.2)

82 (4.8)

47 (7.5)

33 (11.4)

14 (10.7)

33 (13.3)

<0.0001

Sepsis, n (%)

327 (5.8)

187 (10.4)

101 (15.3)

57 (19.3)

24 (17.3)

33 (12.4)

<0.0001

Severe sepsis, n (%)

141 (2.5)

123 (6.8)

65 (9.9)

45 (15.2)

20 (14.4)

29 (10.9)

<0.0001

Septic shock, n (%)

44 (0.8)

31 (1.7)

18 (2.7)

19 (6.4)

9 (6.5)

20 (7.5)

<0.0001

1,456 (25.8)

471 (26.1)

179 (27.2)

75 (25.3)

31 (22.3)

45 (16.9)

0.03

791 (14.0)

317 (17.6)

156 (23.7)

86 (29.1)

33 (23.7)

56 (21.1)

<0.0001

3,858 (68.5)

1,420 (78.8)

567 (86.0)

274 (92.6)

124 (89.2)

237 (89.1)

<0.0001

Admitted to ICU, n (%)

362 (6.4)

209 (11.6)

140 (21.2)

93 (31.4)

47 (33.8)

121 (45.5)

<0.0001

Mortality, n (%)

157 (2.8)

100 (5.6)

53 (8.0)

41 (13.9)

19 (13.7)

104 (39.1)

<0.0001

Age (median, mean ± SD)

Acute infection, n (%) Acute organ dysfunction, n (%) Outcome

Admitted, n (%)

P-value

*Based on Kruskal-Wallis (non-parametric) test. SBP, Systolic Blood Pressure; ICU, Intensive Care Unit; mmol/L, millimoles per liter; mmHg, millimeters of mercury; SD, standard deviation.

Table 2. Population characteristics by age. Age

Demographics

Clinical variables

Age (median, mean ± SD)

40 - 64 (N=4086)

≥ 65 (N=2778)

P-value

30, 29.4 (± 6.1)

53, 53.1 (± 6.7)

75, 76 (± 8.1)

<0.0001*

Male, n (%)

893 (46)

2,034 (50)

1,323 (48)

0.02

White, n (%)

1,055 (54.7)

2,621 (64.2)

1,897 (68.3)

<0.0001

Lactate, mmol/L (median, mean ± SD)

1.5, 2.0 (± 1.7)

1.6, 2.1 (± 1.7)

<0.0001*

SBP, mmHg (median, mean ± SD)

131, 132 (± 24)

133, 134 (± 29)

133, 135 (± 30)

0.09*

Hypotensive (SBP<90) , n (%)

39 (2.1)

207 (5.3)

134 (5.1)

<0.0001

Sepsis, n (%)

97 (5.0)

345 (8.4)

287 (10.3)

<0.0001

Severe sepsis, n (%)

51 (2.6)

196 (4.8)

176 (6.3)

<0.0001

Septic Shock, n (%)

13 (0.7)

73 (1.8)

55 (2.0)

<0.001

418 (21.6)

1,002 (24.5)

834 (30.1)

<0.0001

Acute infection, n (%) Acute organ dysfunction, n (%) Outcome

18 - 39 (N=1932)

148 (7.7)

670 (16.4)

621 (22.4)

<0.0001

1,132 (58.6)

3,002 (73.5)

2,346 (84.5)

<0.0001

Admitted to ICU, n (%)

164 (8.5)

444 (10.9)

364 (13.1)

<0.0001

Mortality, n (%)

40 (2.1)

216 (5.3)

218 (7.9)

<0.0001

Admitted, n (%)

*Based on Kruskal-Wallis (non-parametric) test. SBP, Systolic Blood Pressure; ICU, Intensive Care Unit; mmol/L, millimoles per liter; mmHg, millimeters of mercury; SD, standard deviation.

Western Journal of Emergency Medicine

1252

Volume 21, no. 5: September 2020

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED

of hypotension, septic shock, hospital admission, and ICU admission (Tables 1 and 2, p <0.05). Mortality generally increased within each age cohort with increasing lactate levels (Figure 2, p <0.0001). Figure 2 shows that patients with mean lactate levels less than 2.0 mmol/L had relatively low mortality rates in each defined age cohort: 0.7% in the 18-39 year old cohort; 2.7% in the 40-64 year old cohort; and 4.5% in the 65 years and older cohort, respectively. Mortality rates were higher in each age cohort in patients with lactate levels of 4.0 mmol/Lâ&#x20AC;&#x201C;4.9 mmol/L: 8.2% in the 18-39 year old cohort; 12.6% in the 40-64 year old cohort; and 19.0% in the 65 years and older cohort (Figure 2). Mortality rates continued to increase, and at 6.0 mmol/L

higher mortality rates were observed across all age cohorts: 28.3% in the 18-39 year old cohort; 41.2% in the 40-64 year old cohort; and 41.6% in the 65 years and older cohort (Figure 2, all p values <0.0001). For adults aged 18-39 years, a lactate level of 4.0 mmol/L or higher was associated with a mortality of 5% or greater; for adults aged 40-64 years, this threshold decreased to â&#x2030;Ľ 3.0 mmol/L; for adults aged 65 years or older, a lactate level of â&#x2030;Ľ 2.0 mmol/L was associated with a 5% or greater mortality rate (Figure 2, p <0.0001). Mean lactate levels were consistently higher within each age cohort in non-survivors as compared to survivors, and mean lactate levels in non-survivors decreased as age increased (Figure 3, p <0.0001).

Figure 2. In-hospital mortality rate of ED patients by age and lactate level. *The mortality and lactate association within each age group row is significant with a p <0.0001. mmol/L, millimoles per liter; ED, emergency department.

Figure 3. ED mean lactate level of survivors and non-survivors. mmol/L, millimoles per liter, ED, emergency department.

Volume 21, no. 5: September 2020

1253

Western Journal of Emergency Medicine

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED Table 3. Top clinical impressions of lactate >2 mmol/L by frequency. Clinical impression

Frequency

Lactate (Mean)

Age (Mean)

Admitted ICU (%)

Mortality (%)

Abdominal pain

264

2.8

51.9

6.1

3.0

Pneumonia

228

3.1

63.7

20.6

12.3

Respiratory distress

200

3.5

59.5

32.5

14.5

Sepsis

190

4.3

58.4

40.5

21.1

UTI

152

2.9

62.5

5.3

3.3

AMS

142

3.9

57.1

21.1

7.0

N/V/D

120

3.2

53.6

4.2

5.0

Fever

102

2.7

54.8

5.9

DKA

3.5

45.7

23.2

1.0

Dehydration

3.1

60.5

8.9

3.8

GI bleed

4.0

59.6

42.9

15.7

Cellulitis

2.8

50.6

4.4

Cardiac arrest

10.5

62.2

43.1

70.8

Liver failure

3.2

54.4

17.3

11.5

Infectious abdominal diseases

3.9

57.7

8.3

6.3

Alcohol intoxication

3.3

51.1

10.5

0.0

Substance abuse* 35 4.4 38.0 17.1 0.0 *Includes PCP, cocaine, and unknown ingestion. ICU, intensive care unit; UTI, urinary tract infection; AMS, altered mental status; N/V/D, nausea/vomitting/diarrhea; DKA, diabetic ketoacidosis; GI, gastrointestinal; mmol/L, millimoles per liter.

The mean lactate level of non-survivors was 6.5 mmol/L (SEM = 0.98) in the 18-39 year-old cohort, 4.5 mmol/L (SEM = 0.26) in the 40-64 year-old cohort, and 3.7 mmol/L (SEM = 0.24) in the 65 years or older cohort (p <0.0001). Mean lactate levels of survivors appeared consistent across the three age cohorts at approximately 2.0 mmol/L (SEM = 0.03, 0.02, 0.03, respectively). Mean lactate levels were different both overall and within the three age cohorts for gender (male vs female). Overall, males had a mean lactate of 2.22 as compared to 1.95 for females (p <0.0001). Logistic modeling showed that both age and lactate were significant predictors of mortality. The odds ratios of mortality showed the same trend as the raw mortality rates in Figure 2: they generally increased with increasing lactate within each age cohort and with increasing age within each lactate-level cohort. When controlled for age, an increasing lactate level was still significantly associated with the outcome of in-hospital mortality (Appendix). Outside of cardiac arrest, the primary clinical impressions resulting in the highest lactate levels were substance abuse, sepsis, and gastrointestinal (GI) bleed (Table 3). GI bleed had the highest ICU admission rate at 42.9%, followed by sepsis at 40.5%. While the most frequent clinical impression was abdominal pain, it had the second lowest mortality rate behind substance abuse and alcohol intoxication. Cardiac arrest had the highest mortality rate but only a marginal ICU admission rate, likely because patients were deceased prior to admission. Cardiac arrest, sepsis, GI bleed, respiratory distress, and pneumonia had Western Journal of Emergency Medicine

the highest mortality rates while diabetic ketoacidosis (DKA), alcohol intoxication, and substance abuse were the clinical impressions associated with the lowest mortality rates. DISCUSSION The use of lactate as a diagnostic indicator, prognostic marker, and/or resuscitation endpoint in patients with various disease states has been well described in the literature and has become routine in ED clinical practice.11,15,23,29–35 However, traditional lactate-level thresholds (low < 2.0 mmol/L; intermediate 2.0 to 3.9 mmol/L; and high ≥ 4.0 mmol/L) have been used to guide care without regard to patient age or underlying disease state.11 Expansion of these lactate-level thresholds and considerations of age and underlying disease states may prove useful in risk stratification and management decisions. In our study, patients 65 years of age or older with lactate levels between 2.0-2.9 mmol/L had an in-hospital mortality rate of 9.4%. The 40-64 year-old cohort had a similar mortality rate (8.2%) with lactate levels between 3.0-3.9. The 18-39 year old cohort did not have a similar mortality rate (>8.2%) until their lactate levels exceeded 4.0 mmol/L, the level traditionally considered as “high.” The similar prognosis seen in varying lactate-level thresholds across age cohorts should raise caution in applying a simple “one size fits all” threshold approach to using lactate in clinical decisions. It is important to address the individual and possible combined effects that age and lactate have on mortality, and what is driving and contributing to the increasing mortality rates

1254

Volume 21, no. 5: September 2020

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED

noted in Figure 2. In an effort to examine this more closely, we used logistic regression modeling to evaluate the significance of lactate on mortality, using lactate <2 and age 18-39 separately and then combined as reference groups. Individually and within the age cohorts of 18-39, 40-64, and ≥ 65, lactate was found to be a significant predictor of mortality. Logistic regression modeling output, as noted in the appendix, supports the primary unadjusted findings of Figure 2. Either increases in lactate, age, or both, when compared to their respective reference cohort categorizations, does generally increase the odds of death. Using lactate levels clinically in the ED patient is complex and a provider should not be reassured by a seemingly “non-high (i.e., 2-3.9 mmol/L)” lactate level. Our results suggest that both variables, lactate and age, are important to consider in this patient population when assessing risk. The existing literature evaluating lactate as a risk stratification tool for in-hospital mortality has predominantly looked at patients with the diagnosis of sepsis or trauma, raising the question of potential value of lactate use on the ED patient.35,36 Our study is the first in a large population to show that there is a clinically relevant difference in mortality across both expanded lactate cohorts and age cohorts. A statistical difference in lactate levels between genders was also noted; however, this finding was beyond the scope of this paper and could be explored in future research. We observed a rise in mortality in each age cohort as lactate levels increased except for the 5.0-5.9 mmol/L lactatelevel cohort, which actually had a slight decrease in mortality compared to the 4.0-4.9 mmol/L cohort in our 18-39 year-old and 65 years and older patients. This may be due to the lower number of patients (n = 139) in this lactate level cohort. With the increasing number of lactate tests being ordered in EDs, it has become more important than ever for the clinician not to associate an elevated lactate solely with sepsis. It is well known that there are numerous causes for an elevated lactate and it is important to not narrow the differential diagnoses prematurely.18,37 Clinicians also need to be aware of diagnoses associated with high lactate levels but low mortality rates. Similar to the findings in our study (Table 3), DKA patients commonly present with elevated lactate levels and it has been shown that lactic acidosis in DKA is not associated with increased morbidity or mortality.38 Substance-abuse patients are another example of those who can have elevated lactate levels but low associated mortality rates, which was also consistent with our study (Table 3).39,40 LIMITATIONS This study, which analyzed a large number of patients within a single hospital, has several limitations. This was a retrospective analysis and carries the disadvantage of potential selection bias. Similar to Porter et al,12 the exclusion of multiple visits may overestimate the mortality rate; however, this avoids oversampling. Data abstracted for this study was from years 2009–2013; there could be more variation in practice patterns Volume 21, no. 5: September 2020

now. This study examined all patients with a lactate level measured in the ED, at the discretion of the treating provider and although protocols for sepsis screening are in place institutionally, it is difficult to extrapolate, retrospectively, a clinician’s rationale for deciding to order a lactate or not. Of note, the dataset did not capture specific causes for lab error and did not distinguish between arterial or venous samples; however, prior research has observed a strong correlation between arterial and venous concentrations.41 This data may not be generalizable to specific ED patients with presumed or known conditions and may vary depending on provider practice patterns and/or institutional guidelines. It is important to recognize that certain clinical disease processes and medications may also cause elevated lactate levels, and that a decision based upon a lactate level needs to be taken into context with the overall clinical picture. The clinical impressions used in this study were made by the treating emergency provider after the initial workup of the patient was completed, and there was no attempt to determine the etiology of the lactate levels associated with different clinical impressions and what effect the clinical impression may have. We only attempted to analyze associations, not causations. Additionally, patients may have had multiple clinical impressions associated with their encounter; however, only the primary clinical impression was ultimately included in this study. Not all conditions that are associated with an elevated lactate level portend significant risk for mortality. We excluded patients with a diagnosis of seizure because this is a common ED presentation that is associated with an elevation of lactate but confers a known low risk of mortality.2,12 There may be other conditions associated with elevated lactate levels that also carry low risk for mortality that we did not exclude; these lowrisk conditions have the potential to dilute the overall mortality risk shown in our results. On the contrary, including conditions such as cardiac arrest with an obvious high risk of death has the potential to overestimate the overall mortality risk in this study. Even if the included 65 patients with cardiac arrest (45 deaths) were removed, overall mortality in the study would only have dropped from 5.4% to 4.9%. There may be significant differences in lactate prognostic ability based upon gender and race that could be explored in future research. Physiologic characteristics of lactate metabolism and clearance, such as body mass index, diet, and medication use, are confounding factors that could account for differences in lactate level and prognostic ability that could not be controlled for in this study. CONCLUSION This study suggests that the combination of increasing lactate levels and/or age are associated with increasing inhospital mortality. Our findings suggest that lactate levels may be used as a prognostic tool to help risk stratify ED patients. These findings suggest that the traditional lactate level thresholds currently used to guide clinical practice may need to be both

1255

Western Journal of Emergency Medicine

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED expanded and adjusted for age. Clinicians need to be aware of the many potential causes of lactate elevation as the clinical and prognostic importance of an elevated lactate also varies widely by disease state.

Emerg Med. 2005;45(5):524-8. 12. Del Portal DA, Shofer F, Mikkelsen ME, et al. Emergency department lactate is associated with mortality in older adults admitted with and without infections. Acad Emerg Med. 2010;17(3):260-8. 13. Walker CA, Griffith DM, Gray AJ, et al. Early lactate clearance

ACKNOWLEDGMENTS We would like to thank Eric Howser for his help in data extraction for the project.

in septic patients with elevated lactate levels admitted from the emergency department to intensive care: time to aim higher? J Crit Care. 2013;28(5):832-7. 14. Barfod C, Lundstrøm LH, Lauritzen MMP, et al. Peripheral venous

Address for Correspondence: Ross T. Miller, MD, MS, University of Kansas Medical Center, Department of Emergency Medicine, 3901 Rainbow Blvd, Kansas City, Kansas 66160. Email: rmiller7@ kumc.edu.

lactate at admission is associated with in-hospital mortality: a prospective cohort study. Acta Anaesthesiol Scand. 2015;59(4):514-23. 15. Howell MD, Donnino M, Clardy P, et al. Occult hypoperfusion and mortality in patients with suspected infection. Intensive Care Med. 2007;33(11):1892-9.

16. Trzeciak S, Dellinger RP, Chansky ME, et al. Serum lactate as a predictor of mortality in patients with infection. Intensive Care Med. 2007;33(6):970-7. 17. Mikkelsen ME, Miltiades AN, Gaieski DF, et al. Serum lactate is associated with mortality in severe sepsis independent of organ

Copyright: © 2020 Cannon et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

failure and shock. Crit Care Med. 2009;37(5):1670-7. 18. Bakker J, Coffernils M, Leon M, et al. Blood lactate levels are superior to oxygen-derived variables in predicting outcome in human septic shock. Chest. 1991;99(4):956-62. 19. Hung KKC. BET 2. Serum lactate as a marker for mortality in patients

REFERENCES

presenting to the emergency department with trauma. Emerg Med J.

1. Broder G, Weil MH. Excess lactate: an index of reversibility of shock

2009;26(2):118-9.

in human patients. Science. 1964;143(3613):1457-9.

20. Oedorf K, Day DE, Lior Y, et al. Serum lactate predicts adverse

2. Andersen LW, Mackenhauer J, Roberts JC, et al. Etiology and

outcomes in emergency department patients with and without

therapeutic approach to elevated lactate levels. Mayo Clin Proc.

infection. West J Emerg Med. 2017;18(2):258-66.

2013;88(10):1127-40.

21. Rivers E, Nguyen B, Havstad S, et al. Early goal-directed therapy

3. Wardi G, Brice J, Correia M, et al. Demystifying lactate in the

in the treatment of severe sepsis and septic shock. N Engl J Med.

emergency department. Ann Emerg Med. 2020;75(2):287-98.

2001;345(19):1368-77.

4. Brooks GA. Cell-cell and intracellular lactate shuttles. J Physiol.

22. van den Nouland DPA, Brouwers MCGJ, Stassen PM. Prognostic

2009;587(Pt 23):5591-600.

value of plasma lactate levels in a retrospective cohort presenting

5. Rashkin MC, Bosken C, Baughman RP. Oxygen delivery in critically

at a university hospital emergency department. BMJ Open.

ill patients.:relationship to blood lactate and survival. Chest.

2017;7(1):e011450.

1985;87(5):580-4.

23. Pedersen M, Brandt V, Holler JG, et al. Lactate level, etiology, and

6. Abramson D, Scalea TM, Hitchcock R, et al. Lactate clearance and

mortality of adult patients in an emergency department: a cohort study.

survival following injury. J Trauma. 1993;35(4):584-8; discussion 588-9. 7. Bakker J, Gris P, Coffernils M, et al. Serial blood lactate levels can

Scand J Trauma Resusc Emerg Med. 2015;23 (Supplement 1):A33. 24. Kalantari A, Mallemat H, Weingart SD. Sepsis definitions: the

predict the development of multiple organ failure following septic

search for gold and what CMS got wrong. West J Emerg Med.

shock. Am J Surg. 1996;171(2):221-6.

2017;18(5):951-6.

8. Vincent JL, Dufaye P, Berré J, et al. Serial lactate determinations

25. von Elm E, Altman DG, Egger M, et al. The Strengthening the

during circulatory shock. Crit Care Med. 1983;11(6):449-51.

Reporting of Observational Studies in Epidemiology (STROBE)

9. Manikis P, Jankowski S, Zhang H, et al. Correlation of serial blood

Statement: guidelines for reporting observational studies. Int J Surg.

lactate levels to organ failure and mortality after trauma. Am J Emerg Med. 1995;13(6):619-22.

2014;12(12):1495-9. 26. Dellinger RP, Levy MM, Rhodes A, et al. Surviving Sepsis Campaign.

10. Schulman AM, Claridge JA, Carr G, et al. Predictors of patients who

Crit Care Med. 2013;41(2):580-637.

will develop prolonged occult hypoperfusion following blunt trauma. J

27. van Horssen R, Schuurman TN, de Groot MJM, et al. Lactate point-

Trauma. 2004;57(4):795-800.

of-care testing for acidosis: cross-comparison of two devices with

11. Shapiro NI, Howell MD, Talmor D, et al. Serum lactate as a predictor of mortality in emergency department patients with infection. Ann

Western Journal of Emergency Medicine

routine laboratory results. Pract Lab Med. 2016;4:41-9. 28. Angus DC, Linde-Zwirble WT, Lidicker J, et al. Epidemiology of

1256

Volume 21, no. 5: September 2020

Cannon et al.

Age-adjusted and Expanded Lactate Thresholds as Predictors of All-Cause Mortality in the ED

severe sepsis in the united states: analysis of incidence, outcome,

35. Shetty AL, Thompson K, Byth K, et al. Serum lactate cut-offs as a

and associated costs of care. Crit Care Med. 2001;29(7):1303-10.

risk stratification tool for in-hospital adverse outcomes in emergency

29. Browning R, Datta D, Gray AJ, et al. Peripheral venous and arterial

department patients screened for suspected sepsis. BMJ Open.

lactate agreement in septic patients in the emergency department.

2018;8(1):e015492.

Eur J Emerg Med. 2014;21(2):139-41.

36. Park YJ, Kim DH, Kim SC, et al. Serum lactate upon emergency

30. Gatien M, Stiell I, Wielgosz A, Ooi D, et al. Diagnostic performance of

department arrival as a predictor of 30-day in-hospital mortality in an

venous lactate on arrival at the emergency department for myocardial infarction. Acad Emerg Med. 2005;12(2):106-13.

unselected population. PLoS One. 2018;13(1):1-14. 37. Rhodes A, Evans LE, Alhazzani W, et al. Surviving Sepsis Campaign:

31. Puskarich MA, Kline JA, Summers RL, et al. Prognostic value of

International Guidelines for Management of Sepsis and Septic

incremental lactate elevations in emergency department patients with suspected infection. Acad Emerg Med. 2012;19(8):983-5.

Shock: 2016. Intensive Care Med. 2017;43(3):304-77. 38. Cox K, Cocchi MN, Salciccioli JD, et al. Prevalence and significance of

32. Puskarich MA, Illich BM, Jones AE. Prognosis of emergency

lactic acidosis in diabetic ketoacidosis. J Crit Care. 2012;27(2):132-7.

department patients with suspected infection and intermediate lactate

39. Verelst S, Vermeersch P, Desmet K. Ethylene glycol poisoning

levels: a systematic review. J Crit Care. 2014;29(3):334-9.

presenting with a falsely elevated lactate level. Clin Toxicol (Phil).

33. Reed L, Carroll J, Cummings A, et al. Serum lactate as a screening

2009;47(3):236-8.

tool and predictor of outcome in pediatric patients presenting to the

40. Pernet P, BĂŠnĂŠteau-Burnat B, Vaubourdolle M, et al. False elevation

emergency department with suspected infection. Pediatr Emerg

of blood lactate reveals ethylene glycol poisoning. Am J Emerg Med.

Care. 2013;29(7):787-91.

2009;27(1):132.e1-132.e2.

34. Singer AJ, Taylor M, Domingo A, et al. Diagnostic characteristics of a

41. Velissaris D, Karamouzos V, Pantzaris ND, et al. Relation between

clinical screening tool in combination with measuring bedside lactate

central venous, peripheral venous and arterial lactate levels in

level in emergency department patients with suspected sepsis. Acad

patients with sepsis in the emergency department. J Clin Med Res.

Emerg Med. 2014;21(8):853-7.

2019;11(9):629-34.

Volume 21, no. 5: September 2020

1257

Western Journal of Emergency Medicine

Original Research

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy Danielle T. Miller, MD* Hashim Q. Zaidi, MD† Priyanka Sista, MD¶ Sarah S. Dhake, MD§ Matthew J. Pirotte, MD¶ Abra L. Fant MD, MS‡ David H. Salzman, MD, MEd‡

*Stanford University School of Medicine, Department of Emergency Medicine, Palo Alto, California † University of Chicago School of Medicine, Department of Emergency Medicine, Chicago, Illinois ‡ Northwestern University Feinberg School of Medicine, Department of Emergency Medicine, Chicago, Illinois § NorthShore University Health System, Department of Emergency Medicine, Chicago, Illinois ¶ Vanderbilt University School of Medicine, Department of Emergency Medicine, Nashville, Tennessee

Section Editor: Jeffrey Druck, MD Submission history: Submitted January 12, 2020; Revision received May 20, 2020; Accepted May 20, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46207

Introduction: Emergency department thoracotomy (EDT) is a lifesaving procedure within the scope of practice of emergency physicians. Because EDT is infrequently performed, emergency medicine (EM) residents lack opportunities to develop procedural competency. There is no current mastery learning curriculum for residents to learn EDT. The purpose of this study was to develop and implement a simulation-based mastery learning curriculum to teach and assess EM residents’ performance of the EDT. Methods: We developed an EDT curriculum using a mastery learning framework. The minimum passing standard (MPS) for a previously developed 22-item checklist was determined using the Mastery Angoff approach. EM residents at a four-year academic EM residency program underwent baseline testing in performing an EDT on a simulation trainer. Performance was scored by two raters using the checklist. Learners then participated in a novel mastery learning EDT curriculum that included an educational video, hands-on instruction, and deliberate practice. After a three-month period, residents then completed initial post testing. Residents who did not meet the minimum passing standard after post testing participated in additional deliberate practice until mastery was obtained. Baseline and post-test scores, and time to completion of the procedure were compared with paired t-tests. Results: Of 56 eligible EM residents, 54 completed baseline testing. Fifty-two residents completed post-testing until mastery was reached. The minimum passing standard was 91.1%, (21/22 items correct on the checklist). No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS, with deliberate practice sessions not exceeding 40 minutes. Scores from baseline testing to post-testing significantly improved across all postgraduate years from a mean score of 10.2/22 to 21.4/22 (p <0.001). Mean time to complete the procedure improved from baseline testing (6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) to post-testing (5 min 19 seconds, interquartile range 4 min 17sec - 6 min 15 sec; p = 0.001). Conclusion: This simulation-based mastery learning curriculum resulted in all residents performing an EDT at a level that met or exceeded the MPS with an overall decrease in time needed to perform the procedure. [West J Emerg Med. 2020;21(5)1258-1265.]

Western Journal of Emergency Medicine

1258

Volume 21, no. 5: September 2020

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy

INTRODUCTION The emergency department thoracotomy (EDT) is a rare, lifesaving procedure that is within the scope of practice of emergency physicians.1,2 EDT is a complex procedure that involves opening the thoracic cavity to intervene on critical injuries to the heart and other thoracic structures. Due to the infrequency of clinical exposure, studies of emergency medicine (EM) residents suggest minimal opportunities to develop procedural competency in EDT.3-7 Despite the infrequent presentation, EM residents must have adequate training to achieve the skills required to competently perform this emergent procedure. Unfortunately, there is a paucity of literature describing the ideal teaching approach. Early teaching modalities have included written and computer modules to teach and assess trainees.8,9 However, studies of these teaching modalities have concluded that tactile performance is a necessary component of developing procedural competency.8 Cadaveric or porcine models have been explored for establishing proficiency; however, the expense of the models and the need for repetitive deliberate practice to ensure competency have made these modalities cost-prohibitive for education on a widespread basis.3,5,7 Highfidelity simulation models are increasingly being employed to allow for repetitive practice; however, no competency-based curriculum currently exists for EDT.10-13 Mastery learning is a well-regarded, reliable, and highly effective competency-based education approach within health professions education. Its core tenants dictate that trainees must achieve an a priori-defined level of high proficiency in a given instructional unit with little to no variation prior to proceeding to the next unit.14,15 Simulationbased mastery learning involves repetitive simulated performance of the intended cognitive or psychomotor skills coupled with rigorous personalized and informative feedback, with the goal of achieving mastery of the subject. This process entails establishing a minimum passing standard (MPS), baseline testing of the target skills on simulated models, deliberate practice of target skills, and continued practice with further testing until the MPS is reached.16-18 Simulation-based mastery learning has been used in graduate medical education training to provide procedural exposure in a safe environment, allow for deliberate practice, and evaluate procedural competency.19-22 Simulation-based mastery learning has been found to be superior to non-mastery instruction in procedural success rates, procedure time, and complication rates.17,18 This study had several objectives. The first was to develop and implement a simulation-based mastery learning curriculum for EDT. Second was to establish a MPS for a previously developed 22-item checklist for use in this mastery curriculum.23 Third was to determine whether this mastery learning curriculum could result in achievement of the MPS by all participants. Fourth was to compare baseline and final Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Emergency department thoracotomy (EDT) is a lifesaving procedure, but emergency medicine (EM) residents lack opportunities to develop procedural competency. What was the research question? Can a simulation-based mastery learning curriculum on EDT improve resident procedural skills? What was the major finding of the study? The simulation-based mastery learning curriculum resulted in all residents performing an EDT at mastery level. How does this improve population health? The simulation-based mastery learning curriculum can be used for EM residents to gain competency in this rare but life-saving procedure.

post-test performance on checklist items and time to perform an EDT in a simulated environment. Fifth was to determine participant satisfaction with the curriculum. METHODS Study Setting and Population This was a prospective cohort study of EM residents participating in a mastery learning curriculum for EDT. The study was conducted at a four-year academic EM residency training program from July 2018–June 2019. All participants were EM residents. Residents included postgraduate year one (PGY1) to PGY4 levels. Four residents involved in the study design, checklist creation, and session facilitation were excluded from participation. Participants were informed about the study and invited to participate voluntarily. This study was reviewed by the institutional review board at Northwestern University and deemed to be exempt. Standard Setting A MPS for the previously published checklist23 was established by an expert panel using the mastery Angoff method24. A panel of 17 physicians – 15 emergency physicians and two trauma surgeons with experience performing and teaching the procedure – were recruited to serve as judges for the standard-setting process. Judges were asked to estimate the percentage of well-prepared learners who would perform each

1259

Western Journal of Emergency Medicine

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy checklist item correct at the completion of training. A “wellprepared” learner was defined as a resident who could safely and successfully perform the procedure without supervision in clinical practice. Judges completed an electronic form between May–July 2018 using Qualtrics (Qualtrics LLC, Provo, UT). Baseline Assessment All 56 eligible PGY1-4 EM residents were invited to participate in the curriculum. EM residents were asked to quantify the number of EDTs performed in the clinical environment and the simulated environment prior to baseline assessment. During a three-month period, participants were scheduled to complete a baseline assessment. A novel simulated thoracotomy model created by Northwestern Simulation was used in assessment and teaching (Figure 1). This model featured realistic, three-dimensional printed anatomical features including skin and subcutaneous tissue, ribs, lungs, heart, pericardium, phrenic nerve, blood, aorta, esophagus, and spine. In baseline testing, each resident was presented with a clinical scenario of a patient with a penetrating trauma who had just lost vital signs immediately prior to arrival to the ED and was asked to perform an EDT on the simulated model. Performance was recorded using the checklist and was completed by two raters. A total of seven raters (four women, three men) were trained for these sessions. All raters were EM trained and included three EM faculty and four EM senior residents (PGY3 or PGY4). Residents who participated as raters were not included in mastery learning data outcomes. Sessions

were audio and video recorded and made available to the raters if needed for review. Sessions were timed. Neither the raters nor the participants knew the MPS. Curriculum Intervention In the subsequent three months after completion of the baseline simulation assessments, residents participated in educational sessions. Components of the curriculum included a detailed instructional procedure video, individualized instruction through skills stations, and deliberate practice performing a simulated EDT with feedback. Learners did not see or have access to the checklist throughout the curriculum. An EDT procedural video was created specifically for this curriculum by the Northwestern Simulation Lab in conjunction with the Northwestern Innovations Lab (Appendix 2). The educational video was created by a team of emergency and trauma physicians and contained 11 sections: Overview; Indications; Contraindications; Anatomy; Equipment; Preparation; Procedure; Troubleshooting; Aftercare; Complications; and References. Learners were assigned to watch the video individually prior to the practice sessions. After watching the video, residents participated in a 20-minute, individual hands-on practice session. Of the seven checklist raters, six (three women, three men) were trained as facilitators for these sessions. The 20-minute sessions were divided into three stations, relating to specific actions to perform an EDT. In the first station, the facilitator reviewed the instruments

Figure 1. Image of the thoracotomy simulation trainer, with views of the interior chest structures: A) heart with cardiac wound; B) incised pericardium; C) inferior left lung; D) posterior ribs; E) esophagus; and (F) aorta.

Western Journal of Emergency Medicine

1260

Volume 21, no. 5: September 2020

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy

used in performing an EDT. This included both a review of all instruments and practice with assembling the rib spreader. The second station included identification of anatomic structures on the simulated model, along with instruction on the control of cardiac hemorrhage via Foley catheter insertion, staple, or suturing with pledgets. The third station included identification of the aorta and esophagus on the simulated model, an explanation on how to identify each structure respectively, and a demonstration of how to cross-clamp the aorta. Trainees were then given the opportunity to practice the entire EDT procedure with real-time feedback on their performance and were allowed to come to additional sessions for deliberate practice as desired. To accommodate resident scheduling, the deliberate practice sessions occurred during a three-month period from completion of baseline testing. Post-testing During a three-month period following completion of the deliberate practice sessions, residents underwent initial posttesting. Each resident was asked to perform an EDT on the simulated model. Performance was assessed using the same checklist and was completed by two raters. Sessions were video recorded and made available to the raters if needed for review. The same seven raters for baseline testing completed post-testing sessions. Residents who did not reach MPS at initial post-testing returned for additional deliberate practice at a later date. During the subsequent session, residents were informed of missed or incorrect steps during their initial assessment to direct their additional deliberate practice. Any participant not meeting or exceeding MPS continued with deliberate practice and testing until the MPS was met. After achievement of mastery, residents completed a postcurriculum survey. The post-curriculum survey used a Likert scale 1-5 for the estimation of self-efficacy in performing EDT after the curriculum intervention and desirability of future inclusion of the curriculum in residency training. Statistical Analysis We analyzed score differences on the baseline performance and post-curriculum intervention using paired t-tests, Stata version 14 (StataCorp LLC, College Station, TX). Within-group differences for PGY1-4 from baseline performance to post-testing were also analyzed using paired t-tests. We analyzed time to completion of the procedure from baseline testing to post-testing using paired t-tests. The precurriculum and post-curriculum surveys were analyzed using central tendency metrics. RESULTS The minimum passing standard was calculated to be 91.1%. To meet or exceed this threshold, the learner needed to perform 21 of 22 checklist items correctly. Of 56 eligible residents, 54 completed baseline testing (Table 1). Two residents were unable to complete baseline testing due to Volume 21, no. 5: September 2020

Table 1. Demographic data of emergency medicine resident participants in thoracotomy simulation training. Characteristic

Residents (n = 54)

Male

36 (66.7%)

Female

18 (33.3%)

PGY1

15 (27.8%)

PGY2

15 (27.8%)

PGY3

14 (25.9%)

PGY4

10 (18.5%)

PGY, postgraduate year.

scheduling conflicts. Fifty-two residents completed posttesting until mastery was reached (Table 2). In pre-curriculum survey data, 9.6% of participants had performed an EDT in the clinical environment and 22.6% in the simulated environment. No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS (Figure 2). Of the 52 residents who completed post-testing, 31 passed on initial post-curriculum testing with the remaining 21 achieving the MPS after additional deliberate practice (Table 2). The amount of deliberate practice time did not exceed 40 minutes. Comparison of mean scores from baseline testing to final post-testing across all PGY years significantly improved from average raw score of 10.2/22 (standard deviation [SD] = 4.8), to 21.4/22 (SD = 0.6, t(52) = 16.7, p <0.001). Comparison of the mean percentage of items correct on the checklist from baseline to initial post-testing was also significant (average raw score of 10.1/22, SD = 4.8 to 20.2/22, SD = 1.7, t[52] = 15.5, p <0.001). Average time to complete the procedure in baseline testing (M = 6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) compared to final post-testing (M = 5 min 19 sec, IQR = 4 min 17 sec - 6 min 15 sec) was significant (t [52] =3.4, p = 0.001). Participants reported an improvement in confidence for performing the procedure (median grade of 4 on 5-point Likert scale). Participants reported the desire for this curriculum to be included in the future curriculum for the residency (median grade of 5 on 5-point Likert scale). DISCUSSION Our study demonstrates a simulated-based mastery learning curriculum can effectively develop EDT skills in EM residents. To our knowledge, this is the first mastery learning curriculum to teach EDT. This curriculum adds another procedure to the list where mastery learning can function as an educational strategy to improve baseline procedural skills in residents, as seen with other mastery learning curricula, such as central venous cannulation, lumbar puncture, and thoracentesis.19,21,22 The data obtained

1261

Western Journal of Emergency Medicine

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy

Figure 2. Baseline testing, post-testing, and additional post-testing scores of emergency medicine residents on the emergency department thoracotomy procedural checklist with a line demarcating the minimum passing standard.

Table 2. Baseline testing, post-testing, and additional post-testing scores on a 22-item checklist of emergency medicine residents on the procedure of emergency department thoracotomy. Baseline testing n=54

Post-testing

Mean checklist score out of 22 (SD)

Number met MPS (n)

n=52

Mean checklist score out of 22 (SD)

Additional post-testing Number met MPS (n)

n=21

Mean checklist score out of 22 (SD)

Number met MPS (n)

PGY1

7.3(3.9)

19.5(1.8)

21.5(0.5)

PGY2

8.2 (5.1)

20.5(2.2)

21.6(0.6)

PGY3

13.7(3.1)

20.2(1.5)

21.8 (0.4)

PGY4

13.4 (2.5)

20.9(0.6)

21 (-)

SD, standard deviation; MPS, minimum passing standard; PGY, postgraduate year.

during the baseline assessment, where no resident was able to achieve the MPS, provides supporting evidence that residents have limited experience and instruction on this procedure and that current educational approaches are insufficient to ensure graduating residents are able to perform this critically important emergent procedure. All residents who participated in the curriculum were able to achieve the MPS within one or two 20-minute sessions of deliberate practice. The mean score on the procedural checklist after the curriculum intervention improved for all PGY levels (p<0.001). Similar to other mastery learning curricula, the outcomes demonstrate a uniform high performance with minimal variability of performance.Â The Western Journal of Emergency Medicine

average time to perform the procedure also improved by an average of 62 sec (from 6 min 21 secÂ to 5 min 19 sec, p = 0.001). Considering that EDT is performed on patients in or very near cardiac arrest, the improvement in time to perform this procedure was an important outcome. The significant improvement of scores from baseline to initial post-testing also demonstrates that improvement in skills can be achieved, although progression to ensure all participants meet mastery standards requires additional deliberate practice. Our analysis also shows that self-reported resident confidence in performing an EDT after the curriculum intervention was high (median 4 on 5-point Likert scale). These findings are similar to previous findings of increased confidence in residents after

1262

Volume 21, no. 5: September 2020

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy

mastery learning training in other procedures.19,21 Steps in baseline testing that were rarely performed correctly included ensuring all instruments were present (mean percent of residents performing correctly 9.3%); maintaining sterility (22.2%); gathering equipment (29.6%); controlling cardiac hemorrhage (29.6%); and cross-clamping the aorta (29.6%). These results are similar to our previous study in which a pilot group of general surgery and EM residents and attendings performed an EDT on the simulation trainer and were evaluated with the checklist; those who had not performed an EDT in the clinical environment had lower mean scores on ensuring all instruments were present, maintaining sterility, and gathering equipment.23 In the pilot study, those who had not performed an EDT in the clinical environment also on average performed worse on all steps involved in controlling cardiac hemorrhage from incising the pericardium, to delivering the heart, to controlling hemorrhage via Foley catheter, suture, or pledgets. Mastery learning is an ideal educational strategy for teaching EDT, as clinical experience alone is clearly not sufficient for training. In addition, our data shows that previous experience alone does not predict procedural competency as none of these residents achieved MPS in baseline testing. While previous studies have created curricula to teach this procedure, none have been mastery based. Bohnen et al created an EDT curriculum for surgical residents.12 This pilot study included eight expert and six novice surgeons performing an EDT on a simulation model. While this study created a checklist, it focused on five broad tasks for performing the procedure: 1) opening chest/rib spreader utilization; (2) pericardiotomy/cardiac repair; (3) open cardiac massage; (4) clamping aorta; and (5) control of pulmonary hilum. Residents were evaluated using a surgical assessment tool, the Objective Structured Assessment of Technical Skills, which has not been validated in EM and focuses on (1) surgical technique, (2) general skills, and (3) global rating. This checklist and curriculum are not easily translated to EM. Our approach provides a more detailed checklist for the procedure, and while initially designed with a focus for EM residents, the checklist and curriculum could be used by any learner who needs to learn how to perform an EDT. Additionally, while two previous studies have created curriculum to teach EDT, neither has assessed for competence or mastery learning.12,13 Both have been small pilot studies showing improvement in confidence performing the procedure after a curriculum intervention but not mastery of the procedure. The video created by this curriculum is also an additional resource for procedural teaching. Previous research has shown that videos for procedural teaching can be an effective modality for learning. For example, a previous study by Saun et al demonstrated that a New England Journal of Medicine video on the procedure of chest Volume 21, no. 5: September 2020

tube insertion was as effective as a video-recorded didactic for teaching the knowledge and technical skills for chest tube insertion, with participants expressing high satisfaction with the new modality.25 Current videos on the procedure of EDT often have poor visualization of anatomic structures, or often have limited instruction on when and how to perform the procedure.26-28 The EDT video that we created for this curriculum allows for proper visualization of the anatomic components of the procedure. Additionally, this video contains key instruction on indications, contraindications, anatomy, equipment, troubleshooting, complications, and aftercare, which, to our knowledge, current videos do not fully encompass. Arguments against mastery learning have often noted that mastery learning compared to non-mastery learning requires more time.17 The estimated time requirement for this program included 18 four-hour sessions. This time was divided into five days of baseline testing, eight days of deliberate practice, and five days of post- testing. For those who did not meet MPS on initial post-testing, an average of 20 of additional deliberate practice and 10 minutes of retesting were required, with no learner exceeding 40 minutes. While our curriculum included individualized instruction for 20 minutes with a facilitator leading a learner through three stations, the curriculum could be altered to decrease time required of facilitators by grouping residents during these stations. Additionally, during analysis of the baseline assessment, we found several steps with particularly low correct performance (ensuring all instruments were present, maintaining sterility, gathering equipment, controlling cardiac hemorrhage, and cross-clamping the aorta). If residency programs have limited time and resources to perform this mastery learning curriculum, these experiences could guide resource allocation for practice sessions. LIMITATIONS This study was conducted at a large urban, academic four-year EM residency program in the United States, and thus may not be generalizable. This study also was conducted at an institution with access to a simulated model to provide this educational intervention to residents. We did not assess for resident performance of EDT in a patient care environment, and thus we cannot comment on translation of skills into the clinical environment. Future work could potentially use the checklist in a video-recorded clinical environment to assess for competency. Additionally, given this is a rare procedure with high mortality rates, we were unable to assess patient-centered outcomes for this educational intervention, including patient morbidity and mortality. Furthermore, we were unable to determine retention of this skill due to limitations of funding and academic calendar scheduling. Ideally, we would have completed retention assessment six months to one year following achievement of mastery to inform whether additional practice is needed to maintain skills necessary to

1263

Western Journal of Emergency Medicine

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy perform an EDT to a mastery level. Finally, more studies are needed to explore whether additional teaching modalities are as effective in teaching this procedure.

Med. 1994;1(4):373-81. 4. Hayden SR and Panacek EA. Procedural competency in emergency medicine: the current range of resident experience. Acad Emerg Med. 1999;6(7):728-35.

CONCLUSION In conclusion, this study demonstrates that a simulationbased, mastery learning curriculum improves performance of residents in simulated EDT. This curriculum can be used for residents to gain competency in this rare but life-saving procedure.

5. Sanchez LD, DelaPena J, Kelly SP, et al. Procedure lab used to improve confidence in the performance of rarely performed procedures: Eur J Emerg Med. 2006;13(1):29-31. 6. Fitzgerald M, Tan G, Gruen R, et al. Emergency physician credentialing for resuscitative thoracotomy for trauma. Emerg Med Australas. 2010;22(4):332-6.

ACKNOWLEDGMENTS We would like to acknowledge Nabil M. Issa MD, Associate Professor of Trauma and Critical Care Surgery and Medical Education, Joseph A. Posluszny MD, Assistant Professor of Trauma and Critical Care Surgery, and Michael B. Shapiro MD, Chief of Trauma and Critical Care Surgery, Professor of Trauma and Critical Surgery, at Northwestern University, Feinberg School of Medicine for their work on the checklist and minimum passing standard committees. We would like to acknowledge Stefanie Sebok-Syer PhD, Instructor of Emergency Medicine, Stanford University School of Medicine for her work on the statistical analysis.

7. Puchwein P, Sommerauer F, Clement HG, et al. Clamshell thoracotomy and open heart massage--A potential life-saving procedure can be taught to emergency physicians: an educational cadaveric pilot study. Injury. 2015;46(9):1738-42. 8. Chapman DM, Rhee KJ, Marx JA, et al. Open thoracotomy procedural competency: validity study of teaching and assessment modalities. Ann Emerg Med. 1996;28(6):641-47. 9. Davis JS, Garcia GD, Wyckoff MM, et al. Knowledge and usability of a trauma training system for general surgery residents. Am J Surg. 2013;205(6):681-4. 10. Medwid K, Rosenwald R, Gang M. Simulated thoracotomy model. West J Emerg Med. 2014;15(5.1):86. 11. Hamilton AJ, Prescher H, Biffar DE, et al. Simulation trainer for practicing emergent open thoracotomy procedures. J Surg Res.

Address for Correspondence: Danielle T. Miller, MD, Stanford University School of Medicine, Department of Emergency Medicine, 900 Welch Road, Suite 300, Palo Alto, CA 94304. Email: dmiller8@stanford.edu.

2015;197(1):78-84. 12. Bohnen JD, Demetri L, Fuentes E, et al. High-fidelity emergency department thoracotomy simulator with beating-heart technology and OSATS tool improves trainee confidence and distinguishes level of

skill. J Surg Educ. 2018;75(5):1357-66. 13. Yates E, Chirurgi R, Adamakos F, et al. Development and utilization of 3D printed material for thoracotomy simulation. Emerg Med Int. 2018;2018:9712647. 14. McGaghie WC, Miller GE, Sajid AW, et al. Competency-based

Copyright: ÂŠ 2020 Miller et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

curriculum development on medical education: an introduction. Public Health Pap. 1978;(68):11-91. 15. McGaghie WC, Barsuk JH, Wayne DB. Mastery learning with deliberate practice in medical education. Acad Med. 2015;90(11):1575. 16. McGaghie WC, Siddall VJ, Mazmanian PE, et al. Lessons for continuing medical education from simulation research in

REFERENCES

undergraduate and graduate medical education: effectiveness of

1. Seamon MJ, Haut ER, Van Arendonk K, et al. An evidence-

continuing medical education: American College of Chest Physicians

based approach to patient selection for emergency department

Evidence-Based Educational Guidelines. Chest. 2009;135(3

thoracotomy: a practice management guideline from the Eastern

Suppl):62S-8S.

Association for the Surgery of Trauma. J Trauma Acute Care Surg.

17. Cook DA, Brydges R, Zendejas B, et al. Mastery learning for health

2015;79(1):159-73.

professionals using technology-enhanced simulation: a systematic

2. Burlew CC, Moore EE, Moore FA, et al. Western Trauma Association Critical Decisions in Trauma: resuscitative thoracotomy. J Trauma

review and meta-analysis. Acad Med. 2013;88(8):1178-86. 18. Griswold-Theodorson S, Ponnuru S, Dong C, et al. Beyond

Acute Care Surg. 2012;73(6):1359-63.

the simulation laboratory: a realist synthesis review of clinical

3. Chapman DM, Marx JA, Honigman B, et al. Emergency thoracotomy:

outcomes of simulation-based mastery learning. Acad Med.

comparison of medical student, resident, and faculty performances on written, computer, and animal-model assessments. Acad Emerg

Western Journal of Emergency Medicine

2015;90(11):1553-60. 19. Barsuk JH, McGaghie WC, Cohen ER, et al. Use of simulation-

1264

Volume 21, no. 5: September 2020

Miller et al.

Mastery Learning Curriculum for ED Thoracotomy

based mastery learning to improve the quality of central venous

procedural training. AEM Educ Train. 2019;4(2):139-46.

catheter placement in a medical intensive care unit. J Hosp Med.

24. Yudkowsky R, Park YS, Lineberry M, et al. Setting mastery learning

2009;4(7):397-403.

standards. Acad Med. 2015;90(11):1495-500.

20. Barsuk JH, McGaghie WC, Cohen ER, et al. Simulation-based

25. Saun TJ, Odorizzi S, Yeung C, et al. A peer-reviewed instructional

mastery learning reduces complications during central venous

video is as effective as a standard recorded didactic lecture in

catheter insertion in a medical intensive care unit: Crit Care Med.

medical trainees performing chest tube insertion: a randomized

2009;37(10):2697-2701.

control trial. J Surg Educ. 2017;74(3):437-42.

21. Barsuk JH, Cohen ER, Caprio T, et al. Simulation-based education

26. Mellick LB. Open Thoracotomy. Available at: https://www.youtube.

with mastery learning improves residentsâ&#x20AC;&#x2122; lumbar puncture skills.

com/watch?v=8BlPxQI2C90. Accessed December 3, 2019.

Neurology. 2012;79(2):132-7.

27. Moayedi S, Lemkin D. Resuscitative thoracotomy. Available at:

22. Barsuk JH, Cohen ER, Williams MV, et al. Simulation-based mastery

https://www.youtube.com/watch?v=A57ZB_J4FuY. Accessed

learning for thoracentesis skills improves patient outcomes: a randomized trial. Acad Med. 2018;93(5):729-35.

December 3, 2019. 28. Essentials of Emergency Medicine/EMCrit Conference. EEMCrit

23. Zaidi HZ, Dhake SS, Miller DT, et al. Emergency department

2018 017 Live Thoracotomy Demonstration. Available at: https://www.

thoracotomy: development of a reliable, validated checklist for

Volume 21, no. 5: September 2020

youtube.com/watch?v=7CfSToBqgAY. Accessed December 3, 2019.

1265

Western Journal of Emergency Medicine

Brief Research Report

Effect of Resident Physicians in a Supervisory Role on Efficiency in the Emergency Department Aaron S. Kraut, MD* Lauren Sheehy, MD* Benjamin H. Schnapp, MD, MEd* Brian Patterson, MD*†

*University of Wisconsin School of Medicine and Public Health-BerbeeWalsh, Department of Emergency Medicine, Madison, Wisconsin † University of Wisconsin-Madison, Department of Industrial and Systems Engineering and Department of Biostatistics and Medical Informatics, Madison, Wisconsin

Section Editor: Tehreem Rehman, MD Submission history: Submitted Janurary 17, 2020; Revision received June 27, 2020; Accepted July 2, 2020 Electronically published August 24, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.46587

Introduction: While patient throughput and emergency department (ED) length of stay (LOS) are recognized as important metrics in the delivery of efficient care, they must be balanced with the educational mission of academic centers. Prior studies examining the impact of learners on throughput and LOS when staffing directly with attending physicians have yielded mixed results. Herein we sought to examine the impact of a staffing model involving a supervisory resident “pre-attending” (PAT) on ED throughput and LOS, as this model offers a valuable educational experience for residents, but may do so at the expense of operational efficiency. Methods: We retrospectively analyzed 26,702 unique patient encounters at a university-affiliated community ED between July 1, 2017–January 1,2019. The experimental group was comprised of patients seen primarily by midlevel providers, who staffed with a PAT, who subsequently staffed with an attending physician. The control group was comprised of patients seen by midlevel providers and staffed directly with attendings without a PAT. We used a parametric hazard model to analyze the effect of the presence of a PAT on service time, controlling for potential confounders including timing of presentation and patient demographics. Results: The presence of a PAT is associated with a statistically significant increase in service time of five minutes (p = 0.006). Holding other variables equal, predicted service time in the experimental group was 173 minutes (95% confidence interval (CI), 171-176), while that for controls was 168 minutes (95% CI, 165-171). Conclusion: The presence of a PAT is associated with a statistically significant increase in service time, but the magnitude (five minutes) is likely operationally insignificant. The negligible increase in service time is offset by the benefit to residents’ training. The results of this study may be helpful for residency programs considering the addition of a PAT shift structure. [West J Emerg Med. 2020;21(5)1266-1269.]

INTRODUCTION Patient throughput and emergency department (ED) length of stay (LOS) are recognized as important metrics in the delivery of efficient care in emergency medicine (EM).1 However, academic centers must balance expeditious care delivery with the educational mission of training the next generation of emergency physicians.2 Often, educational leaders

Western Journal of Emergency Medicine

must overcome operational resistance to learning initiatives that may threaten clinical efficiency without sufficient data to justify the implementation of their educational strategies. This is particularly germane to discussions about teaching and supervisory structures in the ED. Prior studies have reported conflicting evidence on the effect of learners on ED patient throughput. Bhat et al

1266

Volume 21, no. 5: September 2020

Kraut et al.

Resident Physicians in a Supervisory Role

demonstrated that attending physicians’ patients per hour were increased when working with a resident learner compared to working alone, suggesting increased efficiency.3 Conversely, several recent studies have reported positive correlations between the presence of residents and ED LOS.4,5,6 However, multiple additional studies have demonstrated that ED LOS is unaffected by the presence of residents or medical students.7,8 The addition of a supervisory “pre-attending” role (PAT) provides a unique and valuable educational experience for residents late in their training and addresses the Accreditation Council for Graduate Medical Education (ACGME) directive to incorporate graduated responsibility into residency training.9 Under a standard patient care model, a patient is evaluated by a resident or an advanced practice provider (APP), who is supervised directly by an attending physician. With the supervisory PAT model, a patient is evaluated by an APP, who is supervised by a PAT resident, who is then supervised by an attending physician.10 This provides senior residents the opportunity to supervise care in a controlled setting that mimics the environment in which they will practice upon graduation from residency. In addition, this care delivery model is recommended by the Society of Emergency Medicine Physician Assistants and endorsed by the American College of Emergency Physicians as a means of preparing residents to be leaders of physician-APP teams in clinical practice.11 Our previous work in developing the PAT experience using mastery learning principles suggests that resident participants feel it is highly educationally valuable.12 However, the effect of this care delivery model on ED patient throughput has not yet been examined. Given the staffing model’s educational value, understanding how the presence of a PAT affects patient throughput is critical for educators in EM seeking to justify this implementation of a graduated responsibility model for their trainees. Including an additional provider in a patient’s evaluation has the potential to increase ED LOS by adding another individual who must interview and examine the patient, but it could also expedite patient workups if the PAT is able to provide attending-level oversight to APPs, essentially doubling the “attending” coverage in the ED. We sought to determine the effect of a supervisory resident PAT on the clinical efficiency of a university-affiliated community ED. In addition, we endeavored to quantify and qualify the educational value of the PAT experience for resident physicians. METHODS Study Design We conducted a retrospective observational study using a dataset of consecutive patients from a single, universityaffiliated, community ED with approximately 18,000 visits per year. Relevant variables were extracted from the electronic health record (EHR) (Epic, Verona, WI) via data query. All patients who presented from July 1, 2017–January 1, 2019, during the days of the week and hours when the PAT may have Volume 21, no. 5: September 2020

been working were included in the analysis. This study was reviewed by the institutional review board and declared exempt. Study Setting and Population The study ED is covered by attending physicians in 12-hour, single covered shifts from 7 am-7 pm. APPs covered three shifts from 9 am-5 pm, 12 pm-9 pm, and 5 pm-2 am. Additionally, during weeks when a PAT resident was scheduled, that resident would work from 9 am-7 pm Monday, Tuesday, Wednesday and Friday. In this way, we abstracted data on patients who presented from 9-5 on these days in our analysis. The experimental group was comprised of patients seen by APPs and attendings with a supervisory PAT resident. These PAT supervisory residents were third-year EM residents in a three-year academic EM residency program. The control group consisted of those patients seen by APPs and attendings without a PAT supervisory resident. Measurements Analysis was conducted on data abstracted from the EHR. We determined PAT status by presence or absence of a PAT assigned to the patient’s treatment team. Self-assignment to the treatment team is a standard part of the PAT workflow for all residents. The EHR records the time of patient rooming as well as the time when the patient is dispositioned (as determined by an order to admit, transfer, or discharge the patient). LOS, our primary outcome, was calculated as the difference between these two times. The following variables were abstracted for each patient encounter: age; gender; hour of day; day of week; and disposition. These variables were preselected for analysis in advance based on both likelihood of potentially affecting patient LOS and potential to vary between PAT and non-PAT shifts. Data Analysis We used a parametric hazard model to examine the association between the explanatory variables and the pickup time. Analyses were conducted using Stata 14 (Stata Corp, College Station, TX). We analyzed LOS as a time-to-event outcome using a parametric proportional hazard model.13 This model assumes an underlying functional form of the duration distribution and then estimates the multiplicative or proportional effect of each explanatory variable on the underlying distribution.14 We tested six underlying distributions (exponential, Weibull, gompertz, lognormal, log logistic, and generalized gamma), and while the results were qualitatively similar between the models we found the gamma distribution provided the best model fit based on both Akaike and Bayesian information criteria.15 All model results presented are based on the gamma distribution model. RESULTS We analyzed a total of 26,702 patient encounters that occurred within the specified date range. Of these encounters,

1267

Western Journal of Emergency Medicine

Kraut et al.

Resident Physicians in a Supervisory Role two were dropped for missing data (no roomed time), 328 were dropped based on nonstandard disposition (ie, discharge against medical advice; left without being seen), and a further 18,424 were removed due to taking place on nights or weekends when a PAT is never scheduled. The remaining 7948 encounters were divided into PAT and non-PAT. Control variables are displayed by PAT status in the Table. For the 4527 PAT encounters, the unadjusted mean LOS was 190 minutes (95% confidence interval [CI], 187-193). For the 3421 non-PAT encounters, the unadjusted LOS was 180 minutes (95% CI, 177-183). In the parametric hazard model, presence of a PAT was significantly associated with a marginal LOS increase of five minutes: the adjusted mean LOS for PAT encounters was 173 minutes (95% CI, 171-176) and the adjusted mean LOS for non-PAT encounters was 168 minutes (95% CI, 165-171; Table). DISCUSSION Our study demonstrated that resident PATs supervising APPs do have a small but significant effect on time to disposition for patients in the ED. With all EM residency programs balancing the dual mandate of resident education and care for patients, this data can be helpful as programs consider whether and how to implement supervising resident roles. While it may seem intuitively obvious that requiring an additional physician to evaluate each ED patient adds to the LOS, this is not necessarily the case. Previous studies have shown that consultant evaluation, imaging and laboratory tests are some of the most significant factors impacting time in the ED.16 If the PAT had been seeing patients while he or she was waiting for these tests, it is possible that we would have seen no effect on ED LOS. The effect of supervising residents would also likely be washed out in clinical environments with long wait times. As this study was conducted in a community ED with virtually no wait times and a relatively high percentage of simple complaints like ear infections, this likely was a contributing factor to our findings. However, while the increased LOS we found was statistically significant, it is not clear whether this was operationally important, as it represents only a 3% increase over the average LOS for encounters that do not involve the PAT. Further, previous operations literature

on ED LOS has described an improvement of 11 minutes as “modest,” suggesting that effects on ED crowding are likely to be minimal.17 The ACGME requires all residencies to implement graduated responsibility; trainees cannot simply see a greater number of patients as they progress in training but must be entrusted with more roles and tasks as they progress.9 While a PAT role is only one way of addressing this mandate, this role has high appeal to both residents and teaching faculty because it mimics as closely as possible the experience nearly all graduates will have when they leave residency and first become an attending: supervising APPs, residents, or medical students. In our previously published work in developing a mastery learning curriculum for the PAT role, we reported that 75% of participating residents felt more prepared to function as an attending because of their PAT experience, while 66% agreed that they learned things in the PAT role that they would not have otherwise. The majority of participants also reported that the feedback they received in the PAT role helped them to improve as physicians and aided in their ability to secure a job after completion of residency12 (Appendix A). Anecdotally, many residents identified the PAT experience as “highly valuable” in comments from their semi-annual evaluations and consistently highlight the role as one of our program’s greatest strengths on our Annual Program Evaluation survey. “A great improvement to third year,” wrote one graduate. “It was an eye opening experience staffing the APPs, and I was asked about it at every single one of my job interviews.” In addition to added educational value, the PAT role can assist educational leaders in the realm of resident assessment. The Emergency Medicine Milestones set out a variety of competencies in which programs are required to assess residents’ progress; a PAT role can allow the assessment of skills such as task switching and multitasking in a closely supervised setting.18 LIMITATIONS This was a retrospective, single-center study that occurred at a community ED (although affiliated with an academic site), where APPs and students were supervised by the PAT. Conclusions may not be generalizable to an academic setting where the supervising resident would supervise junior residents. It is possible that other variables, such as

Table. Patient demographics and mean length of stay for control (non-pre-attending encounters) and experimental (pre-attending encounters) groups. Pre-attending encounters N = 4,527 (95% CI)

Non-pre-attending encounters N = 3,421 (95% CI)

Age

47.7(47.1-48.4)

45.8(45.0-46.5)

Female gender

57.8(56.3-59.2)

54.4(52.7-56.0)

Proportion discharged

78.4(77.2-79.5)

80.5(79.1-81.8)

Unadjusted LOS (min)

190 (187-193)

180(177-183)

Adjusted LOS (min)

173 (171-176)

168(165-171)

CI, confidence interval; PAT, pre-attending; LOS, length of stay; min, minutes.

Western Journal of Emergency Medicine

1268

Volume 21, no. 5: September 2020

Kraut et al.

Resident Physicians in a Supervisory Role

increasing patient volumes over time, may have contributed to our findings. This study focused on LOS as a measure of quality of care and did not assess other patient care outcomes that may be affected by differing staffing structures, such as relative value units, number of tests ordered, or number of return visits. Importantly, this study did not rigorously assess objective learning outcomes associated with the PAT model of care delivery, such as the achievement of ACGME Milestone benchmarks, nor did it systematically evaluate APP satisfaction with this model. These both represent ideal outcome measures for future studies.

3. Bhat R, Dubin J, Maloy K. Impact of learners on emergency medicine attending physician productivity. West J Emerg Med. 2014;15(1):41-4. 4. Lammers RL, Roiger M, Rice L, et al. The effect of a new emergency medicine residency program on patient length of stay in a community hospital emergency department. Acad Emerg Med. 2013;10(7):725-30. 5. James C, Harper M, Johnston P, et al. Effect of trainees on length of stay in the pediatric emergency department. Acad Emerg Med. 2009;16(9):859-65. 6. French, D, Zwemer FL, Schneider S. The effects of the absence of emergency medicine residents in an academic emergency department. Acad Emerg Med. 2002;9(11):1205-10.

CONCLUSION The presence of a “pre-attending” is associated with an increase in time to disposition of 5 minutes. The downsides of this 3% increase in time to disposition are likely outweighed by the significant benefits to residents’ training, which, although subjectively significant, could be assessed by more objective measures in future studies. The results of this study may serve as critical justification for residency programs seeking to implement a graduated supervisory structure in the face of concerns about adverse effects on patient throughput and operational efficiency.

7. McGarry J, Krall SP, McLaughlin T. Impact of resident physicians on emergency department throughput. West J Emerg Med. 2010;11(4):333-5. 8. Dehon E, McLemore G, McKenzie LK. Impact of trainees on length of stay in the emergency department at an academic medical center. South Med J. 2015;108(5):245-8. 9. Accreditation Council for Graduate Medical Education. ACGME Common Program Requirements. 2017. Available at: https://www. acgme.org/Portals/0/PFAssets/ProgramRequirements/CPRs_201707-01.pdf. Accessed February 28, 2019. 10. Ross TM, Wolfe RE, Murano T, et al. Three-vs. four-year emergency medicine training programs. J Emerg Med. 2019;57(5):160999. 11. American College of Emergency Physicians Clinical and Practice Management Resources. Advanced practice providers (physician assistant and nurse practitioner) medical-legal issues. 2016.

Address for Correspondence: Aaron S. Kraut, MD, University of Wisconsin School of Medicine and Public Health – BerbeeWalsh, Department of Emergency Medicine, 800 University Bay Drive, Suite 310, Madison, WI 53705. Email: askraut@medicine.wisc.edu.

Available at: https://www.acep.org/globalassets/uploads/uploadedfiles/acep/clinical-and-practice-management/resources/medical-legal/ mlc_adv-prac-prov-ip_final_nov-2016.pdf. Accessed April 30, 2020. 12. Schnapp B, Kraut A, Barclay-Buchanan, C, et al. A graduated

Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. This project was supported by grant number K08HS024558 (author BP) from the Agency for Healthcare Research and Quality. The content is solely the esponsibility of the authors and does not necessarily represent the official views of the Agency for Healthcare Research and Quality.

responsibility supervising resident experience using mastery learning principles. MedEd Publish. 2019;54(3). 13. Greene WH. (2003). Limited Dependent Variable and Duration Models. In Greene. Econometric Analysis, 5th ed. (pages 756-803). New York: Pearson. 14. Cleves M, Gould W, Gutierrez R, et al. (2010). Parametric Models.

Copyright: © 2020 Kraut et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

An Introduction To Survival Analysis Using Stata, 3rd ed. (pages 231246). Texas: StataCorp. 15. Vrieze SI. Model selection and psychological theory: a discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychol Methods. 2012;17: 228-43. 16. Yoon P, Steiner I, Reinhardt G. Analysis of factors influencing length of stay in the emergency department. CJEM. 2003;5(3):155-61.

REFERENCES

17. Han JH, France DJ, Levin SR, et al. The effect of physician triage on

1. Wiler JL, Welch S, Pines J, et al. Emergency department

emergency department length of stay. J Emerg Med. 2010;39(2):227-33.

performance measures updates: proceedings of the 2014 Emergency

18. The Emergency Medicine Milestone Project. Accreditation

Department Benchmarking Alliance Consensus Summit. Acad Emerg

Council for Graduate Medical Education, & American Board of

Med. 2015;11(5):542-53.

Emergency Medicine. 2015. Available at: https://www.acgme.

2. Kesselheim JC and Cassel CK. Service: an essential component of

org/Portals/0/PDFs/Milestones/EmergencyMedicineMilestones.

graduate medical education. N Engl J Med. 2013;(368):500–1.

pdf?ver=2015-11-06-120531-877. Accessed October 11, 2019.

Volume 21, no. 5: September 2020

1269

Western Journal of Emergency Medicine

Original Research

Feasibility of Health Literacy Tools for Older Patients in the Emergency Department Matthew J. McGuinness, BS* Joshua Bucher, MD† James Karz, DO† Carla Pardee, PA-C† Laryssa Patti, MD† Pamela Ohman-Strickland, PhD‡ Jonathan V. McCoy, MD†

*Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey † Rutgers Robert Wood Johnson Medical School, Department of Emergency Medicine, New Brunswick, New Jersey ‡ Rutgers School of Public Health, Piscataway, New Jersey

Section Editor: Kathleen Walsh, MD Submission history: Submitted January 18, 2020; Revision received April 28, 2020; Accepted May 19, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46594

Introduction: This study evaluates the feasibility of using a volunteer research associate (RA) to administer two separate health literacy assessment tools in the emergency department (ED), specifically in an older population of patients. The outcomes measured were administration time and interruptions. Methods: Using a prospective, cross-sectional study with a convenience sample, adult patients over the age of 55 presenting between June–August 2018 to one urban, academic ED were evaluated by a volunteer RA using either the Newest Vital Sign (NVS) or the Short Assessment of Health Literacy (SAHL). All patients 55 years of age or older who consented to participate were included. We excluded from this study the following: patients with dementia or other disability involving reading, speech, or cognitive function, as noted in their medical record or by their attending physician; prisoners; and those subjectively deemed in extremis or too ill to participate by their attending physician. Results: Health literacy was assessed in 202 patients using either the NVS or SAHL. Mean time of administration was 214.0 seconds for the NVS, and 206.8 for the SAHL. The maximum time of administration for the NVS was 563 seconds, compared to 607 seconds for the SAHL. We found that 95.2% of NVS and 93.9% of SAHL tests incurred no interruptions during administration. Conclusion: No significant difference was found between the length of time needed to administer the NVS or SAHL to older patients in the ED. Both tools averaged an administration time of around three to four minutes, and neither incurred regular interruptions to its administration by a volunteer RA. Further study is needed to assess validity of these tools in an ED setting. [West J Emerg Med. 2020;21(5)12701274.]

INTRODUCTION In their extensive 2004 report, the Institute of Medicine defined health literacy as “the degree to which individuals can obtain, process, and understand the basic health information and services they need to make appropriate health decisions.”1 As healthcare systems change, patients frequently remain responsible for adhering to treatment Western Journal of Emergency Medicine

protocols and seeking proper follow-up. Adequate health literacy is key to achieving proper compliance to medication use and has been shown to improve healthcare outcomes.2,3 Health literacy is complex, and often described as consisting of a variety of components, including literacy and numeracy. The term “literacy” is used to explain aspects of language involving reading, writing, speaking, or listening, while 1270

Volume 21, no. 5: September 2020

McGuinness et al.

Feasibility of Health Literacy Tools for Older Patients in the ED

â&#x20AC;&#x153;numeracyâ&#x20AC;? refers to quantitative aspects of health literacy, including basic mathematical operations.1 Issues of health literacy may be further complicated by language barriers or other barriers to communication. For these reasons the American Medical Association has encouraged continued research in health literacy.4,5 Among the variety of healthcare settings, the emergency department (ED) is a focal point of health literacy research. The 2003 National Assessment of Adult Literacy estimated that around 36% of adults had basic or below basic health literacy levels.4 A review of ED patients showed that low health literacy may be present in up to 40% of patients.3 Care in the ED involves rapid decision-making and swift communication between providers and their patients. Low health literacy in the ED has been associated with worse healthcare outcomes and increased recidivism.3,6 One challenge of assessing health literacy in the ED is doing so effectively and efficiently. The ideal health literacy assessment tool for an ED is one that is easily understood by both the patient and those who may administer it (including volunteers, technicians, nurses, physician assistants, and physicians), takes little time to administer, is well studied, validated, and considers various demographic factors such as age or language spoken. Various tools have been developed, but many are primarily for or only available to English-speaking patients.7,8 Common assessment tools used for both English and Spanish speakers include the Newest Vital Sign (NVS) and Short Assessment of Health Literacy (SAHL).6, 9,10 Due to their relative ease of use, short time to administer, and availability in both English and Spanish, these tools are good prototypes for use in the ED and should be further evaluated. While these tools have been found to be effective in their ability to assess health literacy, concern has been raised about their efficiency, as well as their utility in measuring health literacy among the elderly.11 Therefore, further investigation is needed to determine the feasibility of administering the NVS and SAHL in this particular segment of the population in the ED. The purpose of this study was to determine the extent to which these two health literacy assessment tools (the NVS and SAHL) can feasibly be performed by a volunteer research associate (RA) to assess health literacy of older ED patients. This study included both English and Spanish speakers to analyze the performance of these tools in a diverse patient population. METHODS Study Design and Population This was a prospective, convenience sampling, crosssectional study. The 2017 American Community Survey estimates 49.3% of the population of the study location speaks Spanish.12 Verbal consent was obtained from all patients. This study was conducted at an urban, academic adult ED with approximately 80,000 total annual visits. Patient recruitment Volume 21, no. 5: September 2020

Population Health Research Capsule What do we already know about this issue? Low health literacy has been associated with worse health outcomes and increased recidivism in the emergency department (ED). Up to 40% of ED patients may have low health literacy. What was the research question? Can a volunteer research associate (RA) feasibly administer health literacy assessments to older patients in the ED? What was the major finding of the study? Administration of a health literacy tool by a volunteer associate takes 3-4 minutes, and is not regularly interrupted. How does this improve population health? Regular assessment of patient health literacy in the ED is feasible, and may be best achieved with the help of volunteer RAs.

occurred between 9 am and 5 pm, primarily on weekdays based on availability of a medical student acting as a volunteer RA. No direct recruitment of volunteer RAs occurred during this study. In preparation for this study, the RA had extensively reviewed and helped prepare the administration tools, a process that took no more than three hours, and practiced with colleagues before administering to patients. Patients enrolled were identified by the RA as being 55 years of age or older by electronic health record review and spoke English or Spanish. Exclusion criteria were as follows: patients 90 years of age or older; any patient deemed to be under significant distress by their attending physician; prisoners; and patients who had an altered mental status for any reason. Patients were also excluded if their primary language was not English or Spanish. Patients 90 years of age and older were excluded in order to maintain institutional review board (IRB) compliance and avoid collection of protected health information along given the demographic data being collected. IRB approval was obtained from the IRB board of Rutgers University. Protocol Upon identifying a patient eligible for participation in this study, the RA approached the care provider most closely associated with the patient (either a physician assistant or physician) to ask about enrolling the patient. The care provider

1271

Western Journal of Emergency Medicine

McGuinness et al.

Feasibility of Health Literacy Tools for Older Patients in the ED identified whether the patient met any exclusion criteria (critical illness, physical distress, or alteration of mental status, dementia, or other disability involving reading, speech, or cognitive function). If the patient was deemed eligible by the physician assistant (PA) or physician caring for the patient, the RA then administered the assessment during the patient’s ED visit. Data collection did not interfere with patient care. All staff in the ED were made aware of this study and encouraged to interrupt assessments or research activity if patient care was required. Results of the health literacy assessment were not reported to any physician and PA caring for the patient. Despite the limited hours of potential enrollment, it is believed that nearly every patient meeting potential inclusion criterion was identified and screened during these times, although we collected no data explicitly examining percentage of eligible patients recruited. Each encounter with an enrolled patient consisted of a survey of demographic information and an assessment of their health literacy using one of the standardized aforementioned tools (either the SAHL or NVS). A computerized, randomnumber generator, with 1 representing the NVS and 2 representing the SAHL, was used to randomly assign which tool would be administered to each patient. The survey was conducted in the patient’s preferred language. To standardize the experience for each patient, the RA attempted to minimize questions from patients about the test; however, this may have led to concerns in patient’s understandings of the test material. To assess the efficiency of each survey, we recorded the time elapsed to administer them, and the frequency of interruptions. Family or friends visiting the patient were advised not to assist the patient during the survey and health literacy test. Screening Tools and Outcomes The health literacy screening tools used included the SAHL10 and NVS,11 both of which were previously validated by comparison to the Test of Functional Health Literacy in Adults (TOFHLA), the most frequently used tool across outpatient settings. The TOFHLA is widely regarded as one of the most validated of all health literacy assessments. Both initial validation studies for these tools took place in outpatient, primary care settings.10,11

Western Journal of Emergency Medicine

Table 1. Demographic data of patients administered survey tools to assess health literacy. Demographic Total number administered Mean age of patients

NVS

SAHL

104

68.1

69.2

Spanish forms administered (%)

27 (26%)

24 (24.5%)

Number of female patients (%)

53 (51%)

51 (52%)

Number with 4-year degree or higher education (%)

24 (23.1%)

33 (33.7%)

NVS, newest vital sign; SAHL, short assessment of health literacy.

Table 2. Summary statistics for seconds to complete and time of interruptions for NVS and SAHL health literacy tools. Time

Data Analysis Distributions of age, number of seconds of interruptions, and time taken were investigated using histograms and means. Frequencies were reported for the distribution of categorical variables. We compared significance in time of administration differences between scoring groups using t-tests. RESULTS Of 202 patients enrolled in this study, 104 patients were randomly assigned to and administered the NVS while 98 were randomly assigned to and administered the SAHL. Table 1 demonstrates demographic data of each study group. The mean

age of patients who took the NVS was 68.1 years. The mean age of patients who took the SAHL was 69.2 years. Spanish speakers represented 19.2% of those administered the NVS, and 16.3% of those administered the SAHL. Table 2 includes all data associated with time of administration and interruptions to administration. The NVS averaged a mean time of administration of 214.0 seconds (3.57 minutes), while the SAHL averaged a mean time of 206.8 seconds (3.45 minutes). There was no significant difference in time of administration between the NVS and SAHL (t = 0.6379, p = 0.5242). The longest time needed to administer the NVS was 563 seconds (9.38 minutes), compared to 607 seconds (10.1 minutes) for the SAHL; 95.2% of all NVS tests and 93.9% of all SAHL tests incurred no interruptions during administration. For both the NVS and SAHL, interruptions lasted a mean of approximately five seconds (5.54 for NVS, and 4.96 for SAHL).

NVS

SAHL

Mean TOA

214.0 seconds

206.8 seconds

SD of TOA

75.97

84.4

Min TOA

106.0 seconds

Max TOA

563.0 seconds

607.0

199.2-288.8 seconds

189.9-223.7 seconds

95.2%

93.9%

5.54 seconds

4.96 seconds

CI for mean Percentage of tests with no interruption Mean time of interruptions per administration

p = 0.5242

NVS, newest vital sign; SAHL, short assessment of health literacy; TOA, time of administration; SD, standard deviation; CI, confidence interval.

1272

Volume 21, no. 5: September 2020

McGuinness et al.

Feasibility of Health Literacy Tools for Older Patients in the ED

DISCUSSION Concern continues to grow that health literacy issues may be an epidemic affecting health outcomes and healthcare costs.3,13-16 Low health literacy can impact a variety of issues in healthcare including patient decision-making and understanding discharge instructions.17 The best way to address these issues remains a serious discussion for healthcare providers and administrators, and a consistent means of measuring and analyzing patient health literacy is needed. Research interests regarding the use of health literacy tools in the ED has grown, but these studies may not adequately represent older adults, a population of obvious importance in the acute care setting.7,11 Ideally, a tool for evaluating the health literacy of older adults in the ED should be simple, efficient, and accessible to a diverse patient population. Having an awareness of a patient’s health literacy, or lack thereof, allows physicians, PAs, nurses, and all care providers to identify those who may need additional support with regard to decision-making and follow-up care. Few studies have evaluated the feasibility or efficiency of implementing health literacy tools within the ED. Carpenter et al examined feasibility of health literacy tools in the ED, but focused on the NVS, Rapid Assessment of Adult Literacy in Medicine (REALM), and the short version of TOFHLA tests among average-aged, English-speaking patients.8 Our study focused primarily on the health literacy of older patients, including both English and Spanish speakers, as well as use of a volunteer associate in administration of health literacy tools. Other studies of health literacy assessments in the ED have not exclusively featured an older population, and many have not used both English- and Spanish-speaking patient populations.8,11,13,18 In addition, our sample population represented diversity in level of education, which may be a more accurate depiction of ED patients. Given the complex, busy environment of ED care, health literacy tools are unlikely to be part of routine assessment by physicians, PAs, and other providers. Therefore, efficient, reliable tools may need to be performed by other support staff, with ED volunteers representing a potentially useful group to provide this role. This study demonstrates that use of a volunteer in the ED setting could allow for rapid assessment of health literacy of older patients. This information could prove useful for physicians and ED staff in their decision-making and communication with patients given health literacy’s impact on patient outcomes and recidivism.3,6,17 Both the SAHL and NVS are common tools for assessment of health literacy and show similar ease of use with regard to time of administration for older patients. Neither test was lengthy enough to incur significant interruptions, an important consideration concerning the environment of an ED. A lack of interruptions, despite encouragement of ED staff to interrupt testing for patient care, implies that administration of these tools by volunteer staff is unlikely to impede patient care. Although not actively measured, the physicians and PA involved with this study noted no impact in their ability to provide care because Volume 21, no. 5: September 2020

of these assessments. Both tests averaged a time between three and four minutes to administer, an implication that both have similar efficiency in an ED setting. Times of greater length could make either test inappropriate in the care of patients in the ED setting, as it would discourage routine assessment of health literacy, even by a volunteer or staff member not directly providing care. Of note, the longest administration time for both tests was approximately 10 minutes in length. In many EDs, 10 minutes is likely to be too lengthy for a health literacy assessment. This could reflect the need for a cutoff point among health literacy assessments in the ED – an area for future study or consideration in development of new tools. Defining the most efficient and clinically useful health literacy assessment tool remains an issue of importance to improving patient care.7 This study emphasizes the feasibility of these two assessments, the NVS and SAHL, in the ED setting, as well as administration of these tools by a volunteer RA. However, ease of use is only one important criterion for an appropriate health literacy tool. Future study in this area will need to emphasize validity of these tools in this setting, and among specific patient populations such as the elderly or Spanish-speaking patients. Further development may be needed to generate tools that will maximize efficiency for this setting, while retaining high sensitivity and specificity for adequate health literacy. Finally, the success of a volunteer administering tests in this limited setting presents a possibility for similar roles for medical students and volunteers in the ED setting. Additional development of programs such as this could involve these volunteers to help gather important information on social determinants of health to better patient care. LIMITATIONS Despite random assignments, selection bias was possible given our exclusion criteria. This study focused on older patients at a single center and excluded several groups frequently seen in the ED population. The most apparent of these are individuals who were deemed too critically ill to be interviewed. The determination of when a patient was “critically ill” was made by a patient’s physician or PA on a case-by-case basis, introducing the possibility that certain patient populations were improperly excluded. In addition, the study excluded individuals with dementia or other neurologic disability. If more individuals with unknown or unreported cognitive decline or dementia were more prevalent in receiving a certain tool, it could introduce bias that interferes with the scores or time taken to administer that tool. Sampling occurred via convenience sampling. While this carries some risk of selection bias, it is the most efficient method for studies based in the ED. Enrollment was restricted to a limited period based on the availability of a volunteer, and did not include all hours in which patients meeting inclusion criteria would present to the ED. The percentage of patients meeting criteria who were evaluated for potential enrollment was not assessed. No direct measures were used by research staff to determine ongoing

1273

Western Journal of Emergency Medicine

McGuinness et al.

Feasibility of Health Literacy Tools for Older Patients in the ED testing quality apart from initial training. Nor were any ED staff objectively surveyed regarding their views about how administration of tests in this study impacted care in the ED. While discouraged from contributing, visitors with the patient could have influenced patient responses. In addition, only a single RA was available to administer the health literacy tools, increasing the possibility to bias administration of the tools. This RA speaks Spanish, but was not certified as a medical interpreter in Spanish at the time of data collection, which may raise concern about patient understanding of instructions and tools. However, instructions given in Spanish are directly provided by the NVS and SAHL tools for the possibility of use by non-fluent personnel, and overall instructions about this study were written by a certified medical interpreter.

2. Downey LVA, Zun LS. Assessing adult health literacy in urban healthcare settings. J Natl Med Assoc. 2018;100(11):1304-8. 3. Herndon JB, Chaney M, Carden D. Health literacy and emergency department outcomes: a systematic review. Ann Emerg Med. 2010;57(4):334-45. 4. Health Literacy: Report of the Council on Scientific Affairs. Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs, American Medical Association. JAMA. 1999;281(6):552-7. 5. Powers BJ, Trinh JV, Bosworth HB. Can this patient read and understand written health information? JAMA. 2010;304(1):76-84. 6. Griffey RT, Kennedy SK, McGownan L, et al. Is low health literacy associated with increased emergency department utilization and recidivism? Acad Emerg Med. 2014;21(10):1109-15. 7. Carpenter CR, Kaphingst KA, Goodman MS, et al. Feasibility and

CONCLUSION This study examined feasibility of the NVS and SAHL as tools to examine health literacy of an older population of ED patients. This study successfully employed a volunteer research associate to administer these health literacy tests in an ED setting. No major differences were seen in the amount of time needed to administer these tools by a volunteer RA across the entire study population. In addition, there were instances in which both tools exceeded nine minutes to administer. This encourages continued study into finding more efficient tools in evaluating health literacy, especially for older patients. Further study of these tools, and programs to implement their use in the ED, must highlight their validity and overall effectiveness in assessing health literacy in an ED setting.

diagnostic accuracy of brief health literacy and numeracy screening instruments in an urban emergency department. Acad Emerg Med. 2013;21(2):137-46. 8. Stonbraker S, Schnall R, Larson E. Tools to measure health literacy among Spanish speakers: an integrative review of the literature. Patient Educ Couns. 2015;98(12):1513-23. 9. Patel PJ, Steinberg J, Goveas R, et al. Testing the utility of the Newest Vital Sign (NVS) health literacy assessment tool in older AfricanAmerican patients. Patient Educ Couns. 2011;85(3):505-7. 10. Lee SD, Stucky BD, Lee JY, et al. Short Assessment of Health Literacy – Spanish and English: a comparable test of health literacy for Spanish and English speakers. Health Serv Res. 2010;45(4):1105-20. 11. Weiss BD, Mays MZ, Castro KM, et al. Quick assessment of literacy in primary care: the Newest Vital Sign. Ann Fam Med. 2005;3(6):514-22. 12. United States Census Bureau. Languages spoken at home. 20132017 American Community Survey 5-year estimates. 2017. Available

Address for Correspondence: Joshua Bucher, MD, Rutgers Robert Wood Johnson Medical School, Robert Wood Johnson University Hospital, Department of Emergency Medicine, MEB 104, One Robert Johnson Place, New Brunswick, NJ 08903. Email: bucherjt@rwjms.rutgers.edu. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 McGuinness et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http:// creativecommons.org/licenses/by/4.0/

at: https://data.census.gov/cedsci/table?q=spanish%20speaking%20 new%20brunswick%20nj%202017&g=1600000US3451210&hidePrevie w=false&tid=ACSST5Y2017.S1601&layer=VT_2017_160_00_PY_D1& y=2017&vintage=2017&cid=S1601_C01_001E. Accessed April 7, 2019. 13. Marcus EN. The silent epidemic--the health effects of illiteracy. N Engl J Med. 2006;355(4):339-41. 14. Parker RM, Wolf MS, Kirsch I. Preparing for an epidemic of limited health literacy: weathering the perfect storm. J Gen Intern Med. 2008;23(8):1273-6. 15. Sørensen K, Van den Broucke S, Fullam J, et al. Health literacy and public health: a systematic review and integration of definitions and models. BMC Public Health. 2012;12:80. 16. Phillips AL, Arya M. Raising awareness: the overlapping epidemics of low health literacy and HIV. J Assoc Nurses AIDS Care. 2016;27(4):372-5. 17. Schenker Y, Fernandez A, Sudore R, et al. Interventions to improve

REFERENCES

patient comprehension in informed consent for medical and surgical

1. Institute of Medicine, Board on Neuroscience and Behavioral Health, Committee on Health Literacy. (2004). Health Literacy A Prescription to

procedures: a systematic review. Med Decis Making. 2011;31(1):151-73. 18. McNaughton C, Wallston KA, Rothman RL, et al. Short, subjective

End Confusion. In Kindig DA, Panzer AM, Nielsen-Bohlman L (Eds.).

measures of numeracy and general health literacy in an adult

Washington, DC: National Academies Press.

emergency department. Acad Emerg Med. 2011;18(11):1148-55.

Western Journal of Emergency Medicine

1274

Volume 21, no. 5: September 2020

Original Research

Two-Item Fall Screening Tool Identifies Older Adults at Increased Risk of Falling after Emergency Department Visit Christopher J. Solie, DO Morgan B. Swanson, BS Kari Harland, MPH, PhD Christopher Blum, BS Kevin Kin, BS Nicholas Mohr, MD, MS

University of Iowa Hospitals and Clinics, Department of Emergency Medicine, Iowa City, Iowa

Section Editor: Danya Khoujah, MBBS Submission history: Submitted February 17, 2020; Revision received May 20, 2020; Accepted May 22, 2020 Electronically published August 20, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.5.46991

Introduction: Few emergency department (ED)-specific fall-risk screening tools exist. The goals of this study were to externally validate Tiedemann et al’s two-item, ED-specific fall screening tool and test handgrip strength to determine their ability to predict future falls. We hypothesized that both the two-item fall screening and handgrip strength would identify older adults at increased risk of falling. Methods: A convenience sample of patients ages 65 and older presenting to a single-center academic ED were enrolled. Patients were asked screening questions and had their handgrip strength measured during their ED visit. Patients were given one point if they answered “yes” to “Are you taking six or more medications?” and two points for answering “yes” to “Have you had two or more falls in the past year?” to give a cumulative score from 0 to 3. Participants had monthly followups, via postcard questionnaires, for six months after their ED visit. We performed sensitivity and specificity analyses, and used likelihood ratios and frequencies to assess the relationship between risk factors and falls, fall-related injury, and death. Results: In this study, 247 participants were enrolled with 143 participants completing follow-up (58%). During the six-month follow-up period, 34% of participants had at least one fall and 30 patients died (12.1%). Fall rates for individual Tiedemann scores were 14.3%, 33.3%, 60.0% and 72.2% for scores of 0,1, 2 and 3, respectively. Low handgrip strength was associated with a higher proportion of falls (46.3%), but had poor sensitivity (52.1%). Conclusion: Handgrip strength was not sensitive in screening older adults for future falls. The Tiedemann rule differentiated older adults who were at high risk for future falls from low risk individuals, and can be considered by EDs wanting to screen older adults for future fall risk. [West J Emerg Med. 2020;21(5)1275-1282.]

INTRODUCTION Background Falls are the leading cause of traumatic mortality in older adults.1,2 Nearly three million older adults are seen annually in United States emergency departments (ED) for falls.3 Each year 33% of community-dwelling adults over the age of 65

Volume 21, no. 5: September 2020

experience a standing level fall.4 Of patients admitted to the hospital with a ground level fall, 44% are readmitted and 33% die within one year.5-7 Those who present to the ED with fallrelated injuries and are discharged have higher rates of future falls, functional decline, and additional ED visit within three months than other older adults.8,9

1275

Western Journal of Emergency Medicine

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit Goals A recent review looked at thresholds both for testing individuals for fall risk and treating with fall prevention therapies.10 The authors concluded that individuals having a 27% risk of falling in the next six months should receive fall prevention interventions. Our goal was to evaluate two tools, which can feasibly be performed in an ED, to determine whether they can appropriately risk stratify patients’ fall risk above the treatment threshold of 27% with good sensitivity. The primary objective of this study was to measure the ability of the two-item fall screening tool previously devised and internally validated by Tiedemann et al and handgrip strength to predict future six-month fall risk in adults 65 years and older presenting to the ED.11 Secondary objectives were to assess both tools’ ability to predict fall-related injury and/or death within six months.

Population Health Research Capsule What do we already know about this issue? Few ED-specific fall-risk screening tools exist, making it difficult to identify highrisk individuals who would benefit from fall prevention therapies. What was the research question? Would low handgrip strength and a 2-item fall screening tool be able to accurately predict future 6-month fall risk in older adults presenting to the ED? What was the major finding of the study? The two-item screening tool identified those at high risk of future falls with good sensitivity.

METHODS Study Design This study was a single-center, prospective observational cohort study of ED patients. It was approved by the local institutional review board, and written informed consent was obtained from all participants. Study Setting and Population All study enrollments were performed at a single Midwestern academic ED with an annual patient volume of about 65,000 patients annually. Patients were eligible for participation in the study if they were 65 years of age or older and were treated in the ED between 9 am and 11:59 pm on weekdays and 2 pm and 10 pm on weekends. We excluded from the study patients currently living in a nursing home, prisoners, patients with limited English-language skills, and those without the capacity to provide informed consent. If a participant moved to a nursing home after the time of consent, but during the study follow-up period, they remained in the study. No compensation was provided to study participants. The study is reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies.12 Study Protocol Qualifying patients presenting to the ED were consented for the study by a trained research assistant during their ED visit from March 2017–June 2017. Participants provided basic demographic information, completed self-administered fall screening surveys, and provided handgrip strength measurements. Following ED discharge, patients were mailed six follow-up postcards at consecutive monthly intervals from one month to six months after their ED visit. Participants who did not respond to the monthly postcard were then contacted by email or phone (based on patient preference). Western Journal of Emergency Medicine

How does this improve population health? Future falls for older adults identified by a screening tool during the ED visit could be decreased by referral to fall prevention therapies.

Measurements At the time of the ED visit, screening questions from Tiedemann et al were used to assess geriatric fall risk.11 Using Tiedemann’s rule, as done in the previous validation study, having two or more falls within the prior year was worth two points and taking six or more medications was worth one point (Table 1). Handgrip strength was measured using a handgrip dynamometer (Constant 200 lbs. Digital Hand Dynamometer Grip Strength Measurement Meter, Camry Electronic Ltd, Guangdong, China) on both hands in kilogram-force (kgf), and handedness was also reported by the patient.

Table 1. Overview of Tiedemann score calculation for identifying older adults at risk of future falls. Tiedemann score Components

Total score

1276

Survey question

Score

“Have you had 2 or more falls in the past 12 months?”

If yes, +2 points

“Are you taking 6 or more medications?”

If yes, +1 point

Sum of two scores

Score = 0 to 3

Volume 21, no. 5: September 2020

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit

Key Outcome Measures The primary outcome was any fall within six months. Secondary outcomes included fall-related injury and/or death during the six-month follow-up period. Falls and fall-related injuries were ascertained from participant self-report from monthly follow-up (postcard, email, or phone call). In the monthly follow-up, participants were asked, “Have you fallen within the past month?” for falls and, if they had fallen, “Were you injured?” for fall-related injuries. Death was determined from a combination of family or friend report during monthly follow-up, and participants lost to follow up were screened in the state death registry to identify cases where loss to followup was because of death. Data Analysis Demographic information was reported across fall status using descriptive statistics including the chi-square tests and Mann-Whitney U tests, as appropriate. Test charactersitics (sensitivity, specificity, likelihood ratios, and diagnostic odds ratios) for the Tiedemann rule were calculated for each Tiedemann score (0,1,2, and 3). We generated a receiver operating characteristic (ROC) curve to evaluate the predictive value of the Tiedemann score for six-month fall within the study population. To generate 95% confidence intervals (CI) for the area under the curve (AUC) estimates for the ROC curve, we randomly generated 1000 study samples (with replacement of observations) from the study data by bootstrapping. This provided an estimate of the variation in the AUC point estimate. Similar analyses were completed for the secondary outcomes of fall-related injury and composite fall or death. Handgrip strength was reported as mean and 95% CI by dominant hand and compared between the those who fell and those who did not. We used stratification by gender, as there were observed differences in the distributions of handgrip strength by gender. A ROC curve was generated using similar techniques to those described above, including internal validation of the AUC with bootstrapping, to evaluate the predictive value of handgrip strength for six-month fall. A threshold value for low handgrip strength was selected by identifying the handgrip value for each gender that maximized specificity and sensitivity. Low handgrip strength was defined as less than 16 kg for females and less than 25 kg for males. Using the dichotomized measure, we calculated sensitivity, specificity, likelihood ratios, and diagnostic odds ratios to determine the prognostic utility of low handgrip strength for six-month fall. Similar analysis was completed for both secondary outcomes. To account for high loss to follow-up, we conducted a post hoc survival analysis using interval censoring assessed for differences in fall-free survival by Tiedemann screening status. A composite outcome of death or fall was used to define the outcome. Participants were right-censored at the end of the six-month follow-up period or after loss to followVolume 21, no. 5: September 2020

up (i.e, a missed monthly survey). Survival function estimates curves were constructed to visualize fall-free survival, median survival time was computed for each group, and differences in fall-free survival rates tested for significance with the log-rank test, incorporating interval censoring. Sample Size Calculation Assuming an alpha of 0.05, power of 0.80, prevalence of a fall within six months of 17%13 and test sensitivity of 93% and specificity of 61% for a fall within six months,13 147 participants were needed for analysis. Assuming 40% lost to follow-up based upon a previous ED study with similar follow-up methods,13 245 participants were needed to have a final analysis sample of 147 participants. Sensitivity Analysis We performed a sensitivity analysis incorporating all participants, including those lost to follow-up at six months,. Participants lost to follow-up were assumed to not have fallen (outcome = 0) for the sensitivity analysis. Missing Data For the primary analysis describing the test characteristics of the fall screening tools, we used complete case analysis. The complete case analysis population included participants who responded to the monthly follow-up surveys at month six; deaths were included in the loss-to-follow-up population for this analysis. For the outcome of death or fall, participants who died during the study period were also included. For the survival analysis, we included participants from study enrollment until the first event (fall or death) or until rightcensoring occurred (ie, end of six-month study period or lost to follow-up). Data from participants who partially completed follow-up were compared to those completing all of followup to examine how those lost to follow-up might differ from those who completed six-month follow-up. We conducted all data analysis in SAS version 9.4 (SAS Institute, Cary, NC). RESULTS Decription of Study Population A total of 247 patient were enrolled, with 74 patients (30%) lost to complete six months of follow-up (Figure 1). Thirty participants died during the study. There were 194 participants who completed at least part of the follow up. The final six-month fall analysis included 143 participants. Demographics of our study participants can be found in Table 2. The median age was 74 years. Men made up 47% of the study population. There were no major differences in age, gender, dominant or non-dominant handgrip strength or fallrelated visits in those lost to follow-up (Supplemental Table S1). During initial ED evaluation 23 (16%) patients reported two or more falls in the past year and 96 (67%) reported taking six or more medications (Table 3). Low handgrip strength was found in 54 (38%) patients.

1277

Western Journal of Emergency Medicine

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit

Figure 1. Flowchart of enrollment in study of tool to predict future falls in older adults. *For survival analyses and composite outcome, 30 participants who died are included in analysis (n = 173).

Table 2. Subject demographics. Total population (N=173)

Fall (n=53)

No fall (n=120)

P-value

73.5 (69.0 – 80.1)

74.5 (69.5 – 81.0)

73.3 (68.7 – 80.1)

0.297

Male, n (%)

79 (45.7)

26 (49.1)

53 (44.2)

Female, n (%)

94 (54.3)

27 (50.9)

67 (55.8)

Dominant grip strength, mean (95% CI)

22.6 (21.0 – 24.2)

20.9 (18.7 – 23.1)

23.4 (21.3 – 25.5)

0.337

Non-dominant grip strength, mean (95%CI)

20.7 (19.1 – 22.3)

20.2 (17.3 – 23.1)

20.9 (19.1 – 22.8)

0.613

Dominant grip strength, mean (95% CI)

28.5 (25.9 – 31.1)

25.6 (22.6 – 29.0)

29.8 (26.3 – 33.4)

0.103

Non-dominant grip strength, mean (95% CI)

26.4 (24.0 – 28.9)

25.0 (20.4 – 29.5)

27.2 (24.2 – 30.1)

0.117

Dominant grip strength, mean (95% CI)

17.8 (16.4 – 19.2)

16.6 (14.4 – 18.7)

18.3 (16.5 – 20.1)

0.247

Non-dominant grip strength, mean (95% CI)

16.0 (14.5 – 17.5)

15.5 (12.7 – 18.4)

16.2 (14.4 – 18.0)

0.643

16 (9.3)

6 (5.0)

10 (18.9)

0.004

Age, median (IQR) Gender

0.552

Handgrip test (kg) Total

Male

Female

Fall-related current ED visit, n (%)

IQR, interquartile range; kg, kilogram; CI, confidence interval, ED, emergency department.

Western Journal of Emergency Medicine

1278

Volume 21, no. 5: September 2020

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit Secondary Outcomes: Death and Injury Of the participants who reported a fall, 54% had a fallrelated injury. The Tiedemann rule was able to distinguish low risk from high risk partipants for fall-related injuries (Table 4). The percentage of participants who fell and had a fall-related injury was nearly identical in those with a negative Tiedemann score vs those with a score of 1 or greater (50% vs 54%). There were 30 (12.1%) participants who died during their sixmonth follow-up period. The Tiedemann rule also performed well in risk stratifying those at risk of fall or death. Patients with a Tiedemann score of 0 had a significantly greater probability of a fall-free survival at six months (Figure 3). Patients with scores of 1 or greater had a significantly lower probability of a fall-free survival. Fall-related injuries were higher in the group that had low handgrip strength (27.8% vs 18.2%). The low handgrip strength group had a higher rate of fall and/or death at six months (40.0% vs 24.2%) and a lower rate of fall-free survival (Figure 3).

Primary Outcome: Falls There were 48 (34%) participants who had a fall within six months of their ED presentation with a total of 107 reported falls. A Tiedemann score of 0 had a 14.3% fall rate. Tiedemann scores of 1,2 and 3 had fall rates of 33.3%, 60.0% and 72.2%, respectively. A score of 1 or greater was sensitive (87.5%, 95% CI, 78.1 – 96.9 ) in identifying those who fell. Higher score thresholds were more specific for future fall risk; however, they were poorly sensitive (Table 4). There was no difference in handgrip strength between the group that fell and the group that did not (Table 2). Participants with low grip strength did have a high fall rate (46%), but this was poorly sensitive (52%, 95% CI, 38.0-66.2) and specific (69%, 95% CI, 60.2-78.7) (Table 4). The receiver operating area under the curve for handgrip strength was 0.645 (95% CI, 0.639 – 0.646) in men and 0.612 (95% CI, 0.610 – 0.617) in women (Figure 2). There was no threshold for hand grip strength found to identify those at greatest risk of falling.

Table 3. Questionnaire results by monthly fall status. Falls One month falls

Three month falls

Six month falls

Question

Yes

n (ROW%)

dOR (95% CI)

n (ROW%)

dOR (95% CI)

n (%)

dOR (95% CI)

Two or more falls in past year

9 (39.1)

10.38 (3.34 – 32.23)

7 (30.4)

7.06 (2.19 – 22.78)

16 (69.6)

6.29 (2.37 – 16.68)

Six or more medications

10 (10.4)

0.79 (0.27 – 2.34)

8 (8.3)

0.62 (0.20 – 1.91)

39 (40.6)

2.89 (1.26 – 6.64)

Low hand grip 54 11 (20.4) 4.30 (1.40 – 13.16) strength dOR, diagnostic odds ratio; CI, confidence interval.

9 (16.7)

3.36 (1.06 – 10.63)

25 (46.3)

2.47 (1.21 5.06)

Table 4. Test characteristics to predict 6-month fall outcomes. Injury Rate, n (%) (26/143, 18.2%)

Fall and/or death rate, n (%) (53/173, 30.6%)

2.47 (1.21 – 5.06)

15/54 (27.8%)

28/70 (40.0%)

0.77 (0.63 – 0.91)

6.69 (2.22 – 20.14)

7/18 (38.9%)

15/25 (60.0%)

2.97 (-2.29 – 8.22)

0.96 (0.88 – 1.03)

3.10 (0.50 – 3.91)

2/5 (40.0%)

3/10 (30.0%)

37.9% (28.1 – 47.7)

1.41 (1.14 – 1.68)

0.33 (0.07 – 0.59)

4.27 (1.65 – 11.05)

23/101 (22.8%)

46/128 (35.9%)

54.2% (40.0 – 68.3)

45.3% (35.3 – 55.3)

0.99 (0.67 – 1.31)

1.01 (0.63 – 1.40)

0.98 (0.49 – 1.96)

14/78 (17.9%)

28/93 (30.1%)

12.5% (3.1 – 21.9)

62.1% (52.4 – 71.9)

0.33 (0.07 – 0.59)

1.41 (1.14 – 1.68)

0.23 (0.09 – 0.61)

3/42 (11.5%)

7/45 (15.6%)

Fall rate, n (%) (48/143, 33.6%)

Sensitivity

Specificity

+LR

-LR

dOR

25/54 (46.3%)

52.1% (38.0 – 66.2)

69.5% (60.2 – 78.7)

1.71 (1.01 – 2.41)

0.69 (0.47 – 0.92)

13/18 (72.2%)

27.1% (14.5 – 39.7)

94.7% (90.3 – 99.2)

5.15 (0.11 – 10.19)

3/5 (60.0%)

6.3% (0.0 – 13.1)

97.9% (95.0 – 100.0)

≥1

42/101 (41.6%)

87.5% (78.1 – 96.9)

26/78 (33.3%)

6/42(14.3%)

Low hand grip strength* Tiedemann’s Screen Score

+LR, positive likelihood ratio; -LR, negative likelihood ratio; dOR, diagnostic odds ratio. *Low handgrip strength defined as a dominant handgrip strength of less than 18 kg (women) and 25 kg (men).

Volume 21, no. 5: September 2020

1279

Western Journal of Emergency Medicine

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit

Solie et al.

Figure 2. Receiver operating characteristic curve for handgrip strength. AUC, area under the curve.

Figure 3. Interval-censored Kaplan-Meier plots by screening test results.

DISCUSSION Falls are a major problem for older adults and our health system.1-9 Identifying older adults at risk of future falls is important as interventions have proven to decrease fall risk. Many of these interventions involve referral and home-based assessment, which can be coordinated through the ED. In the US, fall-prevention programs are often offered by senior community centers, YMCAs and physical therapists. Many programs focus on balance, strength Western Journal of Emergency Medicine

training, and environmental changes. The Prevention of Falls in the Elderly Trial proved that ED treatments can prevent future falls. This study randomly assigned high risk older adults to a fall-intervention program, and participation in the program decreased future falls from 52% to 32%.14 Recognizing the importance of identifying those at increased risk for future falls, the Society of Academic Emergency Medicine Geriatric Emergency Medicine

1280

Volume 21, no. 5: September 2020

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit

Residency Core Competencies and the “Geriatric Emergency Department Guidelines” recommend that geriatric patients be screened for fall risk, although there is currently no specific screening tool recommended.15-17 While many fall-screening tools exist, few have been evaluated in an ED setting. A review of ED-specific fallscreening tools identified only two studies that derived EDspecific fall-screening tools using individual risk factors with six-month falls as the primary outcome.10 Extensive fall risk evaluations are not feasible in EDs, but brief screening programs hold promise. Our study found that simply asking two questions can distinguish those at high risk of falling from those at lower risk. This is the first EDspecific fall-risk screening tool to be externally validated. In our study, the Tiedemann rule was able to distinguish those at high risk of falling from those at low risk. The tool also performed well in identifying those at increased risk of fall-related injury and fall or death. Asking two questions enabled care providers to distinguish those who would benefit from fall prevention interventions with good sensitivity (87.5%, 95% CI, 78.1 – 96.9). Older adults with a Tiedemann score of 0 had such a low fall rate (14.3%) that sending these patients to fall prevention therapies would likely have been of little benefit. A score of 1 or higher had a combined fall rate of 41.6%. Using the treatment threshold of 27% previously described by Carpenter et al, we recommend EDs using the Tiedemann two-question screening tool refer those with a score of 1 or greater to fallprevention interventions. Frailty is the state of vulnerability due to poor resolution of homeostasis as a response to a stressor event and has been found to put older adults at greater risk for falls.18-20 Of the many proposed ways to measure frailty, one of the simplest is handgrip strength, which has been shown to be a single marker for frailty, more than chronological age itself.21 Handgrip strength is measured with a hand dynamometer, which is a non-invasive, inexpensive device (approximately $25) that can perform the measurement in seconds. As decreased handgrip strength has been used to identify frailty and frailty has been associated with increased risk of falls, we predicted that decreased handgrip strength would be able to predict increased risk of future falls. In our study, low handgrip strength was associated with an increased risk of fall, fall-related injury and fall or death at six months, but did not perform well as a fall-risk screening tool as it failed to identify almost half of those who fell. While checking handgrip strength in the ED may not be useful as a fall-risk screening tool, there may be other benefits from checking handgrip strength as it did identify a more frail subgroup of older adults given increased rates of future injury and death. This could help in adding objective data for supporting an individual’s need for nursing home placement. Future studies are needed to evaluate its utility in the ED. Volume 21, no. 5: September 2020

LIMITATIONS This study has several limitations. The biggest limitation was our loss to follow-up. Although our loss to follow-up was high, it was lower than our predicted loss to follow-up of 40%; thus, we met our goal sample size. Those lost to follow-up had lower handgrip strength and had a higher incidence of falls contributing to their index visit (Supplemental Table S1). We had anticipated a lower fall rate, as reported in another USbased ED study,13 but our fall rate was similar to that found in Tiedemann et al’s study. Our study was performed at a single academic center that primarily serves a White, non-urban population. These findings may not reflect EDs that serve other demographic groups as our population may have different fall hazards than older adults in more urban locations. However, the consistency between our results and the Tiedemann study suggest that our findings would likely be similar in other EDs. While patients were prospectively enrolled, patients with less acute conditions were likely consented more often, causing healthier older adults to likely be over-represented and making patient enrollment not truly consecutive. Patients who declined to be in the study were not tracked, making it difficult to get a sense for any self-selection bias. This study relies on older adults’ self-reported fall. Recall bias has been reported in the past when measuring older adults’ reporting of falls.22 CONCLUSION Future falls and fall-related injuries are high in older adults presenting to the ED. Handgrip strength was not a sensitive screening tool for predicting future falls in older adults. In a validation of Tiedemann et al’s fall-risk screening tool, we found the two-item screening tool was useful to distinguish those at high risk of six- month fall from those at a low risk of falling. EDs may consider using the two-item screening tool developed by Tiedemann et al to assess older adults for future fall risk as it is externally validated and feasible to perform in the ED.

Address for Correspondence: Christopher J. Solie, DO, University of Iowa Hospitals and Clinics, Department of Emergency Medicine, 200 Hawkins Dr, Iowa City, IA 52242. Email: christophersolie@gmail.com. Conflicts of Interest: By the WestJEM article submission agreement, all authors are required to disclose all affiliations, funding sources and financial or management relationships that could be perceived as potential sources of bias. No author has professional or financial relationships with any companies that are relevant to this study. There are no conflicts of interest or sources of funding to declare. Copyright: © 2020 Solie et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

1281

Western Journal of Emergency Medicine

Solie et al.

Fall Screening Tool Identifies Older Adults at Increased Risk after ED Visit REFERENCES

2013;30(11):918‐22.

1. Yeo YY, Lee SK, Lim CY, et al. A review of elderly injuries seen

12. von Elm E, Altman DG, Egger M, et al. The Strengthening the

in a Singapore emergency department. Singapore Med J.

Reporting of Observational Studies in Epidemiology (STROBE)

2009;50(3):278‐83.

statement: guidelines for reporting observational studies. J Clin

2. Albert M, McCaig LF, Ashman JJ. Emergency department visits by

Epidemiol. 2008;61(4):344-9.

persons aged 65 and over: United States, 2009-2010. NCHS Data

13. Carpenter CR, Scheatzle MD, D’Antonio JA, et al. Identification of

Brief. 2013;(130):1‐8.

fall risk factors in older adult emergency department patients. Acad

3. Centers for Disease Control and Prevention. Injury Prevention &

Emerg Med. 2009;16(3):211‐9.

Control: Web-based Injury Statistics Query and Reporting System

14. Close J, Ellis M, Hooper R, et al. Prevention of Falls in the

(WISQARS). 2017. Available at: https://www.cdc.gov/injury/wisqars/

Elderly Trial (PROFET): a randomised controlled trial. Lancet.

overview/key_data.html. Accessed May, 16 2019.

1999;353(9147):93-7.

4. Stalenhoef PA, Crebolder HF, Knottnerus JA, et al. Incidence, risk

15. Hogan TM, Losman ED, Carpenter CR, et al. Development of

factors, and consequences of falls among elderly subjects living

geriatric competencies for emergency medicine residents using an

in the community: a criteria‐based analysis. Eur J Public Health.

expert consensus process. Acad Emerg Med. 2010;17(3):316-24.

1997;7(3):328–34

16. Carpenter CR, Bromley M, Caterino JM, et al. Optimal older adult

5. Downing A and Wilson R. Older people’s use of accident and

emergency care: introducing multidisciplinary geriatric emergency

emergency services. Age Ageing. 2005;34:24-30.

department guidelines from the American College of Emergency

6. Roberts DC, McKay MP, Shaffer A. Increasing rates of emergency

Physicians, American Geriatrics Society, Emergency Nurses

department visits for elderly patients in the United States, 1993 to

Association, and Society for Academic Emergency Medicine. Acad

2003. Ann Emerg Med. 2008;51(6):769-74.

Emerg Med. 2014;62(7):1360-3.

7. Ayoung-Chee P, McIntyre L, Ebel BE, et al. Long-term outcomes

17. American College of Emergency P, American Geriatrics S,

of ground-level falls in the elderly. J Trauma Acute Care Surg.

Emergency Nurses A, et al. Geriatric emergency department

2014;76(2):498‐503.

guidelines. Ann Emerg Med. 2014;63(5):e7-25.

8. Sirois MJ, Émond M, Ouellet MC, et al. Cumulative incidence of

18. Clegg A, Young J, Iliffe S, et al. Frailty in elderly people. Lancet.

functional decline after minor injuries in previously independent older Canadian individuals in the emergency department. J Am Geriatr

2013;381(9868):752–62. 19. Ensrud KE, Ewing SK, Cawthon PM, et al. A comparison of frailty

Soc. 2013;61(10):1661‐8.

indexes for the prediction of falls, disability, fractures, and mortality in

9. Carpenter CR. Deteriorating functional status in older adults after

older men. J Am Geriatr Soc. 2009;57(3):492‐8.

emergency department evaluation of minor trauma--opportunities and

20. Cheng MH and Chang SF. Frailty as a risk factor for falls among

pragmatic challenges. J Am Geriatr Soc. 2013;61(10):1806-7.

community dwelling people: evidence from a meta-analysis. J Nurs

10. Carpenter CR, Avidan MS, Wildes T, et al. Predicting geriatric falls following an episode of emergency department care: a systematic

Scholarsh. 2017;49(5):529‐36. 21. Syddall H, Cooper C, Martin F, et al. Is grip strength a useful single

review. Acad Emerg Med. 2014;21(10):1069‐82. 11. Tiedemann A, Sherrington C, Orr T, et al. Identifying older people

marker of frailty? Age Ageing. 2003;32(6):650-6. 22. Cummings SR, Nevitt MC, Kidd S. Forgetting falls. The limited

at high risk of future falls: development and validation of a

accuracy of recall of falls in the elderly. J Am Geriatr Soc.

screening tool for use in emergency departments. Emerg Med J.

1988;36(7):613-6.

Western Journal of Emergency Medicine

1282

Volume 21, no. 5: September 2020

Editorial

COVID-19: An Emerging Threat to Antibiotic Stewardship in the Emergency Department Michael S. Pulia, MD, MS* Ian Wolf, BS† Lucas T. Schulz, PharmD‡ Aurora Pop-Vicas, MD, MPH§ Rebecca J. Schwei, MPH* Peter K. Lindenauer, MD, MSc¶

*University of Wisconsin Madison, School of Medicine and Public Health, Department of Emergency Medicine, Madison, Wisconsin † University of Wisconsin Madison, School of Medicine and Public Health, Madison, Wisconsin ‡ University of Wisconsin Madison, School of Medicine and Public Health, Department of Pharmacy, Madison, Wisconsin § University of Wisconsin Madison, School of Medicine and Public Health, Department of Medicine, Madison, Wisconsin ¶ University of Massachusetts Medical School - Baystate, Department of Medicine, Springfield, Massachusetts

Section Editor: Ioannis Koutroulis, MD, PhD, MBA Submission history: Submitted June 22, 2020; Accepted July 7, 2020 Electronically published August 7, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.7.48848

While current research efforts focus primarily on identifying patient level interventions that mitigate the direct impact of COVID-19, it is important to consider the collateral effects of COVID-19 on antimicrobial resistance. Early reports suggest high rates of antibiotic utilization in COVID-19 patients despite their lack of direct activity against viral pathogens. The ongoing pandemic is exacerbating known barriers to optimal antibiotic stewardship in the ED, representing an additional direct threat to patient safety and public health. There is an urgent need for research analyzing overall and COVID-19 specific antibiotic prescribing trends in the ED. Optimizing ED stewardship during COVID-19 will likely require a combination of traditional stewardship approaches (e.g. academic detailing, provider education, care pathways) and effective implementation of host response biomarkers and rapid COVID-19 diagnostics. Antibiotic stewardship interventions with demonstrated efficacy in mitigating the impact of COVID-19 on ED prescribing should be widely disseminated and inform the ongoing pandemic response. [West J Emerg Med. 2020;21(5)1283-1286.]

Specifically, early reports suggest high rates of antibiotic utilization in COVID-19 patients despite their lack of direct activity against viral pathogens.1,2 Unnecessary use of antibiotics is a primary driver of antimicrobial resistance, a global public health crisis,3,4 and a significant risk to patient safety due to the risk of serious adverse drug events (ie, allergic reactions) and Clostridioides difficile infection. As the hospital entry point for most patients with potential COVID-19, the emergency department (ED) is a critical setting for stewardship efforts.5 The ED has unique, systemslevel barriers to quality improvement interventions that require customized approaches to antibiotic stewardship.6 It is important to consider how the ongoing pandemic exacerbates existing challenges to antibiotic stewardship for acute respiratory conditions in the ED. 1283

Western Journal of Emergency Medicine

COVID-19: An Emerging Threat to Antibiotic Stewardship in the ED ANTIBIOTIC USE IN PATIENTS WITH COVID-19 Most available reports on COVID-19 have focused on characterization of the disease and associated outcomes; so there is limited information available on antibiotic use patterns. Two recent systematic reviews identified that 72% of patients with COVID-19 receive antibiotic therapy despite only 7% having a bacterial co-infection.2,7 While these findings raise substantial concern about potential overuse of antibiotics in COVID-19, the underlying studies are lacking sufficient prescribing detail to fully characterize the dilemma. Specifically, they do not include critical details such as when/ where the antibiotics were initiated, indication for initiation (eg, empiric therapy vs confirmed co-infection), and duration/ spectrum of therapy. In the few studies that did report coinfections, they often fail to provide details on how the infection was diagnosed and type of infection (ie, viral, bacterial, or fungal). There is a need to obtain prescribing data from the ED due to its primacy in the initial evaluation of COVID-19 patients and its susceptibility to the stewardship challenges posed by the pandemic. Only by gathering this information can the magnitude and appropriateness of ED-based antibiotic prescribing related to COVID-19 be accurately evaluated. POTENTIAL ADVERSE EFFECTS OF COVID-19 ON EMERGENCY DEPARTMENT STEWARDSHIP Due to unique, system-level factors, optimizing antibiotic prescribing in the ED is challenging. Emergency care providers have specifically reported that time pressures, clinical inertia, perceived patient expectations, and diagnostic uncertainty can lead to overuse of antibiotics.8,9 Individual providers’ perceptions of the risk-to-benefit ratio for antibiotics may also drive the substantial inter-provider prescribing rate variability observed in the literature.10,11 In EDs experiencing a surge in volumes related to COVID-19 (eg, New York), the increased caseload may exacerbate the pre-existing barriers to stewardship. Additionally, fear of COVID-19 and a baseline lack of understanding among some patients of how antibiotics work may accelerate actual or perceived expectations for antibiotic therapy.12 These challenges in optimizing antibiotic prescribing in the ED are further exacerbated by diagnostic limitations inherent to the first pandemic wave. Patients with COVID-19 infection can present with a wide spectrum of illness severity and nonspecific clinical features (eg, cough, dyspnea) that overlap substantially with other common acute respiratory conditions such as asthma, congestive heart failure, and bronchitis.13 Given the lack of widespread access to accurate and rapid COVID-19 diagnostics, including the absence of point-of-care assays, it is incredibly difficult to differentiate COVID-19 from other acute respiratory conditions for which antibiotics are generally indicated (eg, community-acquired pneumonia and chronic obstructive pulmonary disease exacerbations). This is likely to drive further overuse of antibiotics, given that high rates of unnecessary antibiotic prescribing are already observed for respiratory conditions of less diagnostic uncertainty (eg, asthma, Western Journal of Emergency Medicine

Pulia et al.

influenza, and bronchitis), where therapeutic guidelines are clearly established and do not support antibiotic use.14-16 The increased prevalence of early hypoxia and progression to respiratory failure reported for COVID-19 compared to other infectious respiratory conditions is another potential factor underlying high rates of antibiotic utilization.17 For instance, Sepsis CMS Core Measure 1 (SEP-1) requires rapid administration of broad-spectrum antibiotics for all ED patients with two systemic inflammatory response syndrome criteria and a lactate > 2.0 mmol/L . This definition of severe sepsis may create antibiotic prescribing pressure for patients with minor vital sign or metabolic perturbations.18 Additionally, early guidelines for critically ill patients with COVID-19 recommended consideration of antibiotics due to the possibility of bacterial co-infection, despite a lack of evidence demonstrating improved outcomes.19,20 Although not directly related to COVID-19 infections, the pandemic public health response effort appears to have significantly altered patterns of ED utilization. Several reports confirm dramatic decreases in ED volumes for emergent conditions, potentially related to “stay at home” orders or fear of COVID-19 exposure in the hospital.21,22 Delayed presentations can cause more severe presentations of acute respiratory (eg, asthma) and infectious conditions (eg, sepsis), which may in turn lead to increased rates of antibiotic utilization and expanded spectrum of empiric therapy. While these effects are speculative, due to a lack of detailed reporting on antibiotic treatment of COVID-19 patients in the ED, they share similarities with known challenges to ED antibiotic prescribing and can be mitigated by established stewardship interventions. EMERGENCY DEPARTMENT ANTIBIOTIC STEWARDSHIP STRATEGES FOR COVID-19 There are several relevant stewardship interventions with established effectiveness in curbing inappropriate antibiotic usage for acute respiratory conditions in the ED. Academic detailing, care pathways/guidelines, and pharmacist review can improve empiric antibiotic selection for community-acquired pneumonia.23-27 The use of rapid, viral pathogen-detection assays has been proposed as a means to reduce antibiotic initiation and facilitate earlier discontinuation among ED patients with respiratory tract infections (eg, influenza).28-31 The ID NOW point-of-care testing platform by Abbott (Chicago, IL) that is used for rapid influenza, strep A, and respiratory syncytial virus now has a test for COVID-19 that can deliver a test result in 13 minutes with over 90% agreement with molecular polymerase chain reaction (PCR) assays for SARSCov-2.32,33 Given the role of diagnostic uncertainty in antibiotic prescribing, these assays will be an increasingly important tool for optimizing stewardship during the ongoing pandemic. Although the availability, diagnostic performance, and turnaround time of diagnostic tests for SARS-CoV-2 is likely to continue improving over time, we are over five months into this pandemic and rapid testing is still not widely available in the ED. There is an immediate need for research elucidating the

1284

Articles in Press

Pulia et al.

COVID-19: An Emerging Threat to Antibiotic Stewardship in the ED

role of host response biomarkers in helping clinicians identify bacterial infections in patients with acute respiratory illnesses. Procalcitonin (PCT) is a Food and Drug Administrationapproved biomarker that differentiates viral from bacterial infections and can safely guide antibiotic decision-making for stable patients with acute respiratory infections.34 However, rapid PCT is not widely used in the US and a recent trial demonstrated significant antibiotic use in low-risk pneumonia patients despite a negative PCT, suggesting the need for additional clinician education and decision support.35.36 Early reports indicate PCT remains negative in COVID-19 infection and may be useful in easing concerns of bacterial coinfection, although further research is needed to confirm these findings.1,37,38 Finally, although not yet available in the US, a point-of-care host response assay for respiratory tract infections that incorporates both a bacterial (C-reactive protein) and viral biomarker (myxovirus resistance protein A) has a reported 99% negative predictive value for bacterial infections.39 The fingerstick sample collection method and 10-minute turnaround time offer a promising alternative to PCR assays that require higher level personal protective equipment during nasal swab collection and are generally associated with turnaround times of several hours. This assay has been proposed as a potential triage tool in a tiered COVID-19 diagnostic strategy, but further validation of its performance in confirmed cases will be required before introduction into clinical practice.40 CONCLUSION The ongoing COVID-19 pandemic is exacerbating known challenges to optimal antibiotic stewardship in the ED, representing an additional direct threat to patient safety and public health via antibiotic overprescribing and promotion of bacterial resistance. There is an immediate need for research characterizing ED antibiotic prescribing patterns and the performance of host response biomarkers among patients with confirmed or suspected COVID-19. Antibiotic stewardship approaches shown to effectively mitigate the impact of COVID-19 in the ED should be widely disseminated and inform future pandemic responses. Address for Correspondence: Michael S. Pulia, MD, MS, University of Wisconsin Madison School of Medicine and Public Health, Deparment of Emergency Medicine, 800 University Bay Drive Suite 310, Madison, WI 53705. Email: mspulia@medicine.wisc.edu.

1. Zhou F, Yu T, Du R, et al. Clinical course and risk factors for mortality of adult inpatients with COVID-19 in Wuhan, China: a retrospective cohort study. Lancet. 2020;395(10229):1054-62. 2. Rawson TM, Moore LSP, Zhu N, et al. [Ahead of Print]. Bacterial and fungal co-infection in individuals with coronavirus: A rapid review to support COVID-19 antimicrobial prescribing. Clin Infect Dis. May 2, 2020. 3. Ventola CL. The antibiotic resistance crisis. Pharm Ther. 2015;40(4):277-83. 4. ONeill J. Tackling Drug-Resistant Infections Globally: Final Report and Recommendations. AMR. 2016;1-84. 5. May L, Cosgrove S, L’Archeveque M, et al. Antimicrobial stewardship in the emergency department and guidelines for development. Ann Emerg Med. 2013;62(1): 69-77. 6. Pulia M, Redwood R, May L. Antimicrobial stewardship in the emergency department. Emerg Med Clin North Am. 2018;36(4):853-72. 7. Lansbury L, Lim B, Baskaran V, et al. [Ahead of Print] Co-infections in people with COVID-19: a systematic review and meta-analysis. J Infect. May 27, 2020. 8. Almatar MA, Peterson GM, Thompson A, et al. Factors influencing ceftriaxone use in community-acquired pneumonia: emergency doctors’ perspectives. Emerg Med Australas. 2014;26(6):591-5. 9. May L, Gudger G, Armstrong P, et al. Multisite exploration of clinical decision-making for antibiotic use by emergency medicine providers using quantitative and qualitative methods. Infect Control Hosp Epidemiol. 2014;35(9):1114-25. 10. Klein EY, Martinez EM, May L, et al. Categorical risk perception drives variability in antibiotic prescribing in the emergency department: a mixed methods observational study. J Gen Intern Med. 2017;32(10):1083-9. 11. Jones BE, Sauer B, Jones MM, et al. Variation in outpatient antibiotic prescribing for acute respiratory infections in the veteran population: a cross-sectional study. Ann Intern Med. 2015;163(2):73-80. 12. Broniatowski DA, Klein EY, Reyna VF. Germs are germs, and why not take a risk? Patients’ expectations for prescribing antibiotics in an innercity emergency department. Med Decis Making. 2015;35(1):60-7. 13. CDC. Coronavirus Disease 2019 (COVID-19) – Symptoms. Centers for Disease Control and Prevention. Published May 13, 2020. Accessed June 5, 2020. https://www.cdc.gov/coronavirus/2019-ncov/symptomstesting/symptoms.html 14. Fleming-Dutra KE, Hersh AL, Shapiro DJ, et al. Prevalence of Inappropriate Antibiotic Prescriptions Among US Ambulatory Care Visits,

2010-2011. JAMA. 2016;315(17):1864-73. 15. Lindenauer PK, Stefan MS, Feemster LC, et al. Use of antibiotics among patients hospitalized for exacerbations of asthma. JAMA Intern Med. 2016;176(9):1397-1400. 16. Ghazi IM, Nicolau DP, Nailor MD, Aslanzadeh J, Ross JW, Kuti

Copyright: © 2020 Pulia et al. This is an open access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) License. See: http://creativecommons.org/ licenses/by/4.0/

Articles in Press

REFERENCES

JL. Antibiotic utilization and opportunities for stewardship among hospitalized patients with influenza respiratory tract infection. Infect Control Hosp Epidemiol. 2016;37(5):583-9. 17. Faust JS, Rio C del. [Ahead of Print]. Assessment of deaths from

1285

Western Journal of Emergency Medicine

COVID-19: An Emerging Threat to Antibiotic Stewardship in the ED COVID-19 and from seasonal influenza. JAMA Intern Med. May

Pulia et al.

controlled trial. Lancet Respir Med. 2017;5(5):401-11.

14, 2020.

30. Abbas S, Bernard S, Lee KB, et al. Rapid respiratory panel testing:

18. Pulia MS, Redwood R, Sharp B. Antimicrobial stewardship in the

Impact of active antimicrobial stewardship. Am J Infect Control.

management of sepsis. Emerg Med Clin North Am. 2017;35(1):199-217. 19. Poston JT, Patel BK, Davis AM. Management of critically ill adults with

2019;47(2):224-5. 31. Li J, Kang-Birken SL, Mathews SK, et al. Role of rapid diagnostics

COVID-19. JAMA. 2020;323(18):1839-41.

for viral respiratory infections in antibiotic prescribing decision

20. Buetti N, Mazzuchelli T, Priore EL, et al. [Ahead of Print] Early

in the emergency department. Infect Control Hosp Epidemiol.

administered antibiotics do not impact mortality in critically ill patients

2019;40(9):974-8.

with COVID-19. J Infect. June 4, 2020.

32. Abbott Launches Molecular Point-of-Care Test to Detect Novel

21. Hartnett KP, Kite-Powell A, DeVies J, et al. Impact of the COVID-19

Coronavirus in as Little as Five Minutes. 2020. Available at: https://

pandemic on emergency department visits-United States, January 1,

abbott.mediaroom.com/2020-03-27-Abbott-Launches-Molecular-Point-

2019â&#x20AC;&#x201C;May 30, 2020. MMWR-Morbid Mortal W. 2020;69(23):699-704..

of-Care-Test-to-Detect-Novel-Coronavirus-in-as-Little-as-Five-Minutes.

22. Baum A, Schwartz MD. Admissions to Veterans Affairs hospitals

Accessed June 20, 2020.

for emergency conditions during the COVID-19 pandemic. JAMA.

33. Abbott Releases Interim Clinical Study Data on ID NOW COVID-19

2020;324(1):96-9.

Rapid Test Showing Strong Agreement to Lab-Based Molecular PCR

23. Borde JP, Kern WV, Hug M, et al. Implementation of an intensified

Tests. 2020. Available at: https://abbott.mediaroom.com/2020-05-21-

antibiotic stewardship programme targeting third-generation

Abbott-Releases-Interim-Clinical-Study-Data-on-ID-NOW-COVID-19-

cephalosporin and fluoroquinolone use in an emergency medicine

Rapid-Test-Showing-Strong-Agreement-to-Lab-Based-Molecular-PCR-

department. Emerg Med J. 2015;32(7):509-15.

Tests. Accessed June 20, 2020.

24. Kulwicki BD, Brandt KL, Wolf LM, et al. Impact of an emergency

34. Schuetz P, Wirz Y, Sager R, et al. Procalcitonin to initiate or discontinue

medicine pharmacist on empiric antibiotic prescribing for pneumonia and

antibiotics in acute respiratory tract infections. Cochran Syst Rev.

intra-abdominal infections. Am J Emerg Med. 2019;37(5):839-44.

2017;10(10):CD007498.

25. Beavin L, Arnold F, Peyrani P, et al. Antimicrobial stewardship in

35. Huang DT, Yealy DM, Filbin MR, et al. Procalcitonin-guided use

hospitalized patients with respiratory infections: ten-year experience

of antibiotics for lower respiratory tract infection. N Engl J Med.

from the Robley Rex Louisville VA Medical Center. Univ Louisville J

2018;379(3):236-49.

Respir Infect. 2017;1(4):30-4.

36. Pulia MS, Schulz LT, Fox BC. Procalcitonin-guided antibiotic use. N Engl

26. Hortmann M, Heppner HJ, Popp S, et al. Reduction of mortality in

J Med. 2018;379(20):1971-2.

community-acquired pneumonia after implementing standardized

37. Liu F, Li L, Xu M Da, et al. Prognostic value of interleukin-6, C-reactive

care bundles in the emergency department. Eur J Emerg Med.

protein, and procalcitonin in patients with COVID-19. J Clin Virol.

2014;21(6):429-35.

2020;127:104370.

27. Almatar M, Peterson GM, Thompson A, et al. Clinical pathway

38. Metlay JP and Waterer GW. [Ahead of Print]. Treatment of community-

and monthly feedback improve adherence to antibiotic guideline

acquired pneumonia during the coronavirus disease 2019 (COVID-19)

recommendations for community-acquired pneumonia. PloS One.

pandemic. Ann Intern Med. May 7, 2020.

2016;11(7):e0159467.

39. Shapiro NI, Self WH, Rosen J, et al. A prospective, multi-centre US

28. Blaschke AJ, Shapiro DJ, Pavia AT, et al. A national study of the impact

clinical trial to determine accuracy of FebriDx point-of-care testing for

of rapid influenza testing on clinical care in the emergency department. J

acute upper respiratory infections with and without a confirmed fever.

Pediatr Infect Dis Soc. 2014;3(2):112-8.

Ann Med. 2018;50(5):420-29.

29. Brendish NJ, Malachira AK, Armstrong L, et al. Routine molecular point-

40. Pulia MS, Oâ&#x20AC;&#x2122;Brien TP, Hou PC, et al. Multi-tiered screening and

of-care testing for respiratory viruses in adults presenting to hospital with

diagnosis strategy for COVID-19: a model for sustainable testing

acute respiratory illness (ResPOC): a pragmatic, open-label, randomised

capacity in response to pandemic. Ann Med. 2020;52(5):207-14.

Western Journal of Emergency Medicine

1286

Articles in Press

Erratum

This Article Corrects: “Conference Didactic Planning and Structure: An Evidence-based Guide to Best Practices from the Council of Emergency Medicine Residency Directors” D. Brian Wood, MD* Jaime Jordan, MD† Rob Cooney, MD, MSMedEd‡ Katja Goldflam, MD§ Leah Bright, DO¶ Michael Gottlieb, MD||

*St. Joseph’s Medical Center, Department of Emergency Medicine, Stockton, California † Ronald Reagan UCLA Medical Center, David Geffen School of Medicine, Department of Emergency Medicine, Los Angeles, California ‡ Geisinger Commonwealth School of Medicine, Department of Emergency Medicine, Scranton, Pennsylvania § Yale University, Department of Emergency Medicine, New Haven, Connecticut ¶ Johns Hopkins University, Department of Emergency Medicine, Baltimore, Maryland || Rush University Medical Center, Department of Emergency Medicine, Chicago, Illinois

Electronically published August 10, 2020 Full text available through open access at http://escholarship.org/uc/uciem_westjem DOI: 10.5811/westjem.2020.8.49438

West J Emerg Med. 2020 July;21(4):999-1007. Conference Didactic Planning and Structure: An Evidence-based Guide to Best Practices from the Council of Emergency Medicine Residency Directors Wood DB, Jordan J, Cooney R, Goldflam K, Bright L, Gottlieb M Erratum in West J Emerg Med. 2020 September;21(5):1287. Author name misspellled. The fourth author, originally published as Katja Goldfam, MD is revised to Katja Goldflam, MD.

Abstract Emergency medicine residency programs around the country develop didactic conferences to prepare residents for board exams and independent practice. To our knowledge, there is not currently an evidence-based set of guidelines for programs to follow to ensure maximal benefit of didactics for learners. This paper offers expert guidelines for didactic instruction from members of the Council of Emergency Medicine Residency Directors Best Practices Subcommittee, based on best available evidence. Programs can use these recommendations to further optimize their resident conference structure and content. Recommendations in this manuscript include best practices in formatting didactics, selection of facilitators and instructors, and duration of individual sessions. Authors also recommend following the Model of Clinical Practice of Emergency Medicine when developing content, while incorporating sessions dedicated to morbidity and mortality, research methodology, journal article review, administration, wellness, and professionalism. PMCID: PMC7390555 [PubMed - indexed for MEDLINE]

Volume 21, no. 5: September 2020

1287

Western Journal of Emergency Medicine

CALL FOR SUBMISSIONS Team Based Learning Podcasts Lectures Small Group Learning and Workshops Oral Boards Simulation Curricula Innovations

CALL FOR REVIEWERS

JETem is an online, open access, peerreviewed journalrepository for EM educators. VISIT JETem.org to learn more about submissions or if youâ&#x20AC;&#x2122;re interested in being a JETem reviewer.

N M! I JO AE /A L CA

Championing individual physician rights and workplace fairness BENEFITS

Western Journal of Emergency Medicine Subscription CAL/AAEM News Service email updates Free and discounted registration to CAL/AAEM events And more!

CAL/AAEM NEWS SERVICE

Healthcare industry news Public policy Government issues Legal cases and court decisions

In collaboration with our official journal

Join the CAL/AAEM Facebook Group to stay up-to-date: www.facebook.com/groups/calaaem www.aaem.org/calaaem AAEM-0720-255

INSPIRED CALIFORNIA ACEPâ&#x20AC;&#x2122;S ANNUAL CONFERENCE 2020 Education is targeted to Medical Students and Residents, but all are welcome to attend. Friday, September 25, 2020 Vir tual