Summative assessment
For many years, I (Greg) struggled with a common teaching dilemma: how can I help my undergraduate students develop transferrable knowledge and skills in courses dominated by high-stakes tests that traditionally reward memorization above all else?
In 2019, I and my mentor Lekelia (Kiki) Jenkins (Arizona State University) were grappling with the issue of alignment between active-learning exercises and summative assessments. Would students engage in activities that did not directly relate to problems they would face on subsequent exams? Working together via Sue Wick's PALM Network, Kiki and I were adding more active learning to my Human Anatomy & Physiology courses, but we were stymied by the alignment issue, for which the field of science education seemed to lack scalable solutions.
As we pondered this dilemma, I came across an essay by Ben Wiggins (then at the University of Washington, now at Shoreline Community College) on his Public Exam system. Public Exams' direct matching of learning goals with actual exam questions -- consistent with the general principle of backward design (McTighe & Thomas 2003) -- appealed strongly to me and Kiki, who, with help from Ben, then adapted some aspects of this approach to formulate a related but distinct assessment framework that we dubbed Test Question Templates, or TQTs for short. Early TQT dissemination efforts led two additional faculty -- Usha Sankar (then at Fordham University, now at Drexel University) and Marcus Lawrence (Southern Utah University) -- to seek my mentorship via the PALM Network, as I had previously done with Kiki. These new pairings have led to further explorations of TQTs and further interest from other members of the science education community.
This TQT origin story shows that the development of the TQT framework is not attributable to a single individual, but instead reflects the input of a community of science educators spanning a range of roles at a range of institutions.
A Test Question Template is a student-facing resource that explicitly connects a Lesson Learning Objective (LLO) with multiple specific examples of how that LLO might be assessed on a test. TQTs thus show students what they will need to do on tests, and how to practice, without revealing all details of the tests. From the instructor side, TQTs' pre-specification of test question formats makes test-writing more straightforward; by varying certain details within a consistent broader structure, we can generate additional new questions with relative ease.
One example of a TQT is shown below. Hundreds of additional examples are available in a Google Drive folder.
Beyond making tests less stressful and more rewarding for students and instructors, TQTs should generally promote students' transfer of knowledge to new contexts by encouraging practice on multiple examples with different surface features (Kaminske et al. 2020).
Since the TQT framework emphasizes (A) cognition above and beyond straight memorization, (B) transparent alignment of practice and testing, and (C) abundant opportunities for creative and collaborative student practice, it may be considered a cousin of approaches such as specs grading, Mary-Ann Winkelmes' TILT, and Deb Donovan's Learning Targets and Success Criteria.
Selected TQT-Related Publications
|
- Yoojin Choi, Lekelia D. Jenkins, and Gregory J. Crowther. Practice it to ace it: students writing practice questions for tests. Advances in Physiology Education: in review.
- Gregory J. Crowther, Merrill D. Funk, Kelly M. Hennessey, and Marcus M. Lawrence. Frontier-model chatbots can help instructors create, improve, and use Learning Objectives. Advances in Physiology Education: resubmission in progress.
- Gregory J. Crowther and Benjamin L. Wiggins (2024). Exam reform: an opportunity for the redistribution of academic power. Journal of Microbiology and Biology Education: in press.
- Gregory J. Crowther, Victoria L. VanHeel, Sasha D. Gradwell, Casey J. Self, and Krista L. Rompolski (2024). General skills amidst the details: alternative learning objectives and a framework of competencies for human anatomy. Advances in Physiology Education: in press.
- Gregory J. Crowther, Usha Sankar, Leena S. Knight, Debra L. Myers, Kevin T. Patton, Lekelia D. Jenkins, and Thomas A. Knight (2023). Chatbot responses suggest that hypothetical biology questions are harder than realistic ones. Journal of Microbiology and Biology Education 24(3): e00153-23.
- Dilan P. Evans, Lekelia D. Jenkins, and Gregory J. Crowther (2023). Student perceptions of a framework for facilitating transfer from lessons to exams, and the relevance of this framework to published lessons. Journal of Microbiology and Biology Education 24(1): e00200-22.
- Gregory J. Crowther and Thomas A. Knight (2023). Using Test Question Templates (TQTs) to teach physiology core concepts. Advances in Physiology Education 47(2): 202-214.
- Gregory J. Crowther (2021). How do kidneys make urine from blood? Qualitative and quantitative approaches to filtration, secretion, reabsorption, and excretion. CourseSource 8: 42.
- Gregory J. Crowther, Benjamin L. Wiggins, and Lekelia D. Jenkins (2020). Testing in the age of active learning: Test Question Templates help
to align activities and assessments. HAPS Educator 24(1): 74-81.
Selected TQT-Related Webinars & Podcasts
|
- Greg Crowther. Outcomes Assessment. New Faculty Institute, Washington State Board of Community and Technical Colleges, August 12, 2024.
- Kevin Patton, Greg Crowther, and Ben Wiggins. Examining the Anatomy & Physiology Exam. TAPP Radio, Episode 149, June 30, 2024.
- Greg Crowther, Hilary Kemp, and Tom Knight. Chatbot responses suggest that hypothetical biology questions are harder than realistic ones. JMBE Live!, May 17, 2024.
- Greg Crowther. Exams on Demand: Using Templates to Write Multiple Equivalent Problems. The Grading Conference, June 4, 2022.
- Hilary Kemp (performing as Joan Jett). I Love TQTs! Northwest PULSE, January 29, 2021.
- Kevin Patton and Greg Crowther. Test Question Templates Help Students Learn. TAPP Radio, Episode 70, June 2, 2020.
- Zachary Via and Greg Crowther. The PALM Network: All Hands on Deck! National Institute on Scientific Teaching, December 4, 2020.
Ongoing TQT-Related Projects
|
Different members of the "TQTeam" have been working on various TQT-related projects as summarized below.
- TQTs for Human A&P and Cell Biology
- While I continue to write and edit TQTs, my existing TQTs are available in Google Drive folders for Human Anatomy and Physiology and Cell Biology. (These are mostly intended for sophomore pre-nursing students, but some are better for more advanced students.)
- Effects of Pre-Releasing Exam Materials on Students and Faculty
- with Ben Wiggins and Robin Altman
- TQTs and Standards-Based/Specs Grading in Human A&P
- with Kiki Jenkins, Tom Knight, Karen Perell-Gerson, and Usha Sankar
- Using TQTs to Help Students Integrate Previous Content and New Content
- Do Animals and Aliens Belong in Human Physiology Courses?
- with Usha Sankar, Erik Silldorff, and Crystal Uminski
- TQTs and You and Me?
- Do you have an idea for a TQT-related project? Please get in touch with me (gcrowther at everettcc dot edu)!
Starting in October 2024, the work of Ben Wiggins (PI) and me (Co-PI) on Public Exams and TQTs will be funded by the National Science Foundation! Our $500K proposal was titled Pre-Released Exam Strategy Assessment Research.
We of the TQTeam have also been grateful for past seed money from ADInstruments (HAPS workshop sponsor, 2021), the American Physiological Society (Teaching Career Enhancement Award, 2021-2022), the Everett Community College Foundation (2021-2022), Sue Wick's PALM Network (2018-2022), and Jeff Morris's ROSE Network (2021-2022).
Dr. C sipping some TQTea.
|