Douglas Larsen, MD, MEd
Washington University School of Medicine
Objective: We assessed the long-term retention of information learned through repeated testing with standardized patients, written testing, and studying on a final simulated clinical encounter and on a final written test after six months.
Background: Previous laboratory studies in cognitive psychology and applied studies in medical education have shown that repeated written testing produces superior long-term retention of information compared to repeated studying when assessed by a final written test. However, it is unknown if the mnemonic benefits of testing transfer to clinical encounters or if simulation testing with standardized patients produces superior retention.
Methods: Forty-one first-year medical students participated in the study. All students participated in a teaching session that covered three neurological topics. They then participated in four weekly learning sessions. In the learning sessions, they were randomized in a counter-balanced fashion to perform one of three learning activities for each topic: simulation testing with a standardized patient encounter, written testing, or studying a review sheet. Students repeated the same activity for each topic in all four sessions. Six months after the initial teaching session, they engaged in a standardized patient encounter for each of three topics to assess long-term retention of the information. One week later they took a final written test on each of the three topics.
Results: For the final simulation tests, a 3X3 ANOVA revealed a main effect of learning activity [F(2,76)=15.86, MSE=.2, p=.000002, 2=.29]. Simulation testing produced significantly better performance than both written testing [59% vs. 49%; t(40)=3.30, p=.002, d=.55] and studying [59% vs. 43%; t(40)=5.11, p=.00008, d=.84]. Written testing produced significantly better performance than studying [49% vs. 43%; t(40)=2.14, p=.04, d=.33]. However, there was a significant interaction between learning activity and topic [F(4,76)=3.67, MSE=.01, p=.009, 2=.16], indicating that the pattern of results varied between topics. For the final written tests, a 3X3 ANOVA demonstrated a significant main effect of learning activity [F(2,76)=16.69, MSE=.01, p=.000001, 2=.31].
Simulation and written test groups were equal (both 61%), and both were significantly better than repeated study [61% vs. 48%, for simulation vs. study, t(40)=4.44, p=.00007, d=.73; and 61% vs. 48%, for written testing vs. study, t(40)=4.62, p=.00004, d=.70]. However, as with the previous analyses, there was a significant interaction between learning activity and topic [F(2,76)=6.09, MSE=.01, p=.0003, 2=.24].
Conclusions: Overall, repeated testing by both standardized patient simulation appears to produce substantially better long-term retention than repeated study. Repeated simulation appears superior when the final outcome is assessed through clinical simulation, but when outcomes are evaluated in written format simulation and written testing essentially perform equivalently. There was some variation in the pattern of results between topics. Further research is needed to investigate how to optimally incorporate these under-utilized learning activities in medical education.
