Kim Bjorklund
Nationwide Children's Hospital
Background: A standardized measure of operative performance is essential to successfully training plastic surgery residents. At present, no procedure-specific intra-operative resident assessment for which there is good validity evidence is routinely used across plastic surgery training programs in the United States. This study describes an operative performance rating system for plastic surgery residents using an index procedure and provides validity evidence for the instrument.
Methods: 3 separate audio/video recordings of residents at varying PGY levels performing a CTR procedure were recorded. The 3 videos were subsequently reviewed by a group of expert hand surgeons and by SIUSOM faculty using the OPRS instrument to assess resident operative performance.
Results: Validity evidence for the instrument in the form of content, relationship to other variables, response process and internal structure was provided. Inter-rater reliability was consistently fair to moderate and all assessment items were highly correlated. Local SIUSOM faculty routinely demonstrated higher overall scores for PGY 1 and PGY 6 residents compared to expert raters.
Conclusions: Although limited by small numbers, the study suggests potential assessment bias based upon PGY year, identity and performance history may exist and independent assessment by unbiased raters or comparison to national operative norms may be valuable. Our study provides baseline validity evidence for a resident operative performance rating assessment tool that can be integrated into practice.
