University of Cincinnati
Problem
Internal medicine residents and faculty perform invasive bedside procedures such as paracentesis. However, there is no standard for determination of competency. Most administrators rely on procedure logs to make such judgements, but procedural numbers alone are not surrogates for competent performance.
Approach
The authors aimed to develop and provide validity evidence for a novel assessment tool to evaluate paracentesis performance. They reviewed the literature and used a modified Delphi method to create a tool that combines a procedural checklist, global assessment scale, and entrustment scale. The assessment tool was piloted in the simulation environment with 19 faculty and residents. Three raters independently rated each paracentesis to evaluate the tool’s reliability and ability to discriminate between different levels of performance. The Angoff method was used with local and national experts to determine composite scores on the assessment tool that would signify both borderline and competent performance.
Outcomes
The Paracentesis Competency Assessment Tool (PCAT) development process and pilot study results provide evidence for content, response process, internal structure and consequence validity. The PCAT is the first paracentesis assessment tool used in practice to combine a skills checklist, global assessment scale and entrustment scale. Linear regression of pilot results show only weak correlation between checklist performance as compared to global and entrustment performance. This highlights the importance of using all three measures together to guide decisions of competency.
Next Steps
This pilot study was conducted in the simulation environment. Further data is needed on resident performance on live patient procedures using the PCAT, as well as how the tool can assess maintenance of skill over time.
