Arielle Levy, MD, MEd
Hospital Sainte Justine
Background: Many scoring instruments have been described to measure clinical performance during resuscitation; however, the validity of these tools has yet to be proven in paediatric resuscitation.
Aims: To determine if an existing scoring instrument of clinical performance during simulated paediatric resuscitations using PALS algorithms is valid in a setting other than where it was developed.
Method(s): This was a prospective quasi-experimental non equivalent control group design, field experiment in a simulation lab of a paediatric tertiary care academic facility. A total of 13 PGY 1's and 11 PGY 3's were videotaped during 5 paediatric resuscitation simulated scenarios in February and March 2011. Raters, all paediatric emergency physicians, scored the scenarios on resident performance before and after a PALS (Paediatric Advanced Life Support) course using a standardized score sheet. Each video recording was viewed and scored independently by two different raters. Means and standard deviations of scores for trainee performance were calculated for each level of training/residency and for each scenario before and after PALS training. The validity of the score was evaluated by measuring the variation of the score for participants following formal training in resuscitation (PALS course). A priori, it was determined that participants should improve their score after participating to the PALS course. It was also stipulated a-priori that to be valid, the score should report better scores for senior resident in comparison to junior residents. Pre PALS and post PALS means for each level PGY1 and PGY3 were compared using an ANOVA. To investigate differences in the scores of the two groups over the 5 scenarios, a two factor ANOVA was used. Overall and individual inter-rater reliability was measured for the five scores using intraclass correlation coefficient (ICC).
Results: Following the PALS course, the score improved by 8.6% (3.8-13.3), 15.7% (8.6-22.7), 6.3% (-1.8-14.3), 18.2% (9.3-27), 4.1% (-3.0-11.2) for the pulseless non-shockable arrest, pulseless shockable arrest, dysrhythmia, respiratory, and shock scenarios respectively. There was no difference between PGY 1's and PGY 3's score before or after the PALS course. The ICC were 0.85 (0.74-0.92), 0.98 (0.96-0.99), 0.92 (0.87-0.96), 0.97 (0.95-0.98), 0.94 (0.90-0.97) for the pulseless non-shockable arrest, pulseless shockable arrest, dysrhythmia , respiratory, and shock scenarios respectively.
Conclusions: Existing scoring instruments are a valid measure of clinical performance of junior and senior paediatric residents after PALS training. The scoring instrument was able to demonstrate a statistically significant improvement following a PALS course for PGY 1 and PGY 3 paediatric residents for the pulseless non-shockable arrest, pulseless shockable and respiratory arrest scenarios. An increasing trend was observed for the shock and dysrhythmia scenarios. However it was not able to discriminate between PGY 1's and PGY 3's both before and after the PALS course for any scenarios. The scoring instrument showed very good inter-reliability for all scenarios.
