Friday, December 18, 2015

Creation of a Validated, Reliable Procedural Checklist, Global Skills Assessment, and Entrustable Professional Activities Assessment for Simulation-Based Education in Neonatal Intubation for Pediatric Trainees

Lindsay Johnston, MD, MEd
Yale University School of Medicine
 
Introduction: Development of a procedural skills checklist, global skills assessment (GSA), and Entrustable Professional Activities (EPA) assessment for NETI with validity evidence is critical for future simulation-based education and assessment in neonatal endotracheal intubation (NETI). 
Methods:  A convenience sample of 23 providers at Yale-New Haven Children’s Hospital, including beginner, intermediate, and expert performers, completed a single NETI attempt on a bench-top airway trainer.  Performance was recorded with a video camera and a videolaryngoscope (VL).  Blinded expert raters evaluated performance on videos using a procedural skills checklist, a GSA, and EPA assessment developed through the INSPIRE research network, as well as the Cormack-Lehane (CL) and Percent of Glottic Opening (POGO) scale. Evidence for five sources of validity is presented. 

Results: Evidence for content validity (literature review, Delphi review) was excellent.  Response process evidence (rater training and calibration) was very good.  Evidence for internal structure (inter-rater reliability) was very good/ excellent.  Relations to other variables (relation between the checklist, GSA, EPA scores, provider characteristics, and airway visualization) ranged from poor to excellent.  Evidence for consequential validity (standard setting procedures) was good.
Conclusion: Simulation-based assessment can provide validity evidence in the development of a procedural skills checklist for NETI.  Further work is needed to determine how these findings translate into clinical success rates.