Clinical rotations present some unique educational challenges for faculty and students. One of the main goals of the fourth year veterinary curriculum is to help students build strong clinical reasoning skills. However, the unstructured nature of the teaching hospital, including unpredictable caseload and limited time for formal instruction, makes it challenging for faculty to foster and assess students’ clinical reasoning skills. In this article, we will examine how one faculty member developed a formative assessment activity that helps students build their clinical reasoning skills.
Educational Challenge
Dr. Ashley Saunders, Associate Professor of Cardiology, wanted to help students on the Cardiology Rotation improve their ability to process patient data to make sound diagnostic and therapeutic decisions. Conventional methods, such as discussing cases during rounds, did not provide her with a clear picture of each student’s current thinking pattern. She observed that it was too easy for students to evade working through problems individually, relying on the more confident, vocal students in the group to provide all the answers. This situation left her unable to provide the targeted, individualized feedback that is necessary for students to gain expertise. She needed an assessment method that would allow her to determine each student’s individual thought process in order to correct errors in their clinical reasoning.
Educational Solution
Drs. Jodi Korich and Lisa Keefe at the Center for Educational Technologies worked with Dr. Saunders to develop a formative assessment instrument for use on the Cardiology Rotation. As you can see from the example provided below, the instrument presents students with a series of short clinical vignettes based on cardiac disorders frequently encountered in general practice. Students review the vignettes and make a clinical decision designed to probe their ability to process patient history and physical examination findings.
The team designed the assessment instrument based upon the Script Concordance Test, or SCT, devised by researchers working in human medical education (Fournier, Demeester, & Charlin, 2008). For over a decade, the SCT test has been used as a means to assess aspects of clinical reasoning in medical schools, especially for clinical decisions that are inherently ambiguous (i.e. there is not one correct answer). Often, the SCT test has been used for summative (graded) assessment. However, in our case, we were more interested in using the SCT test as a formative assessment tool to help students learn. To achieve this goal, the team made several modifications to the SCT instrument. First, we included a justification box with each question, where students must explain how they arrived at their answers. This provides Dr. Saunders with a window into each student’s thinking pattern. We also simplified the test by shrinking the Likert scale from five options to three. The team issued the test questions to a panel consisting of general practitioners and residents to determine how non-experts would respond to the clinical questions posed. The panel responses were used to help create a grading rubric, a key feature of the SCT where more than one correct answer is often possible. The panel also provided feedback on the questions, helping the team to identify confusing questions that needed further refinement.
During each Cardiology Rotation, Dr. Saunders administers the SCT activity to students via the Moodle learning management system. Student are asked to complete the activity independently before rounds. During rounds, Dr. Saunders uses the SCT questions as a platform for case-based discussion, using individual student’s responses to gauge understanding and correct errors in their thinking patterns.
Student Feedback
Believe it or not, students really enjoy taking this test! They claim the test helps them to self-identify problems in processing patient data and helps them refine their clinical reasoning skills.
“I like being placed in a scenario and having to weed through the information myself, and deciding what is useful information and which is spurious.”
— 4VM student
Students also like that they can think through the scenarios independently at their own pace, before discussing the best answers and rationale behind them with a larger group.
“It allowed us to consider our answers, and think through everything independently without being put on the spot.” — 4VM student
“[The SCT] was extremely helpful because it had me apply what I thought first, and then [afterward] talking it out to figure out the reasoning to why I was right or wrong really helped.” —4VM student
The Instructor’s Perspective
Dr. Saunders believes that the SCT activity is a great addition to her rotation. She was surprised how often students might provide the “correct” answer, but not always through sound reasoning practices. In other words, they got the right answer for the wrong reasons. The SCT provides a mechanism for her to better pinpoint those areas that a student can work on to help them refine their clinical reasoning skills.
After using the SCT for the past five months, Dr. Saunders thinks the SCT activity has also helped her to become a better teacher. It has given her a window into students’ brains, allowing her to better understand how students think and where they are likely to encounter problems in their reasoning process. Armed with this valuable knowledge, she is beginning to make adjustments in other courses that she teaches to help prepare students more effectively for their fourth year rotation.
“The students really do enjoy this test. They have the opportunity to think about the case and then we review their thought process together. They like the feedback and always ask for more questions.”
— Dr. Ashley Saunders
Conclusion
The modified SCT test was successfully utilized to assess students’ clinical reasoning skills, and to provide individualized student feedback on the fourth year cardiology rotation. Student perception of the SCT activity was positive and a survey indicated that they would like to have access to more SCT questions to help them refine their clinical reasoning skills on other clinical rotations.
Suggested reading:
Fournier, J.P., Demeester, A., & B. Charlin. 2008. Script Concordance Tests: Guidelines for Construction. BMC Medical Informatics and Decision Making. 8:18. http://dx.doi.org/10.1186/1472-6947-8-18
Roediger, H.L., & A.C. Butler. 2011. The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences. 15:1, 20-27. http://dx.doi.org/10.1016/j.tics.2010.09.003