When Jeff Sellen and several colleagues meet every other week for a peer review session, they aren’t poring over research papers; they are poring over their own class assignments and students’ responses to them.
Sellen and eight faculty members who all teach World Civilizations meet regularly to discuss assignments, grading criteria and student papers. Or more specifically, these members of the “Raters Group” work backward from a critical thinking rubric to help each other create assignments and design classroom exercises that elicit evidence of the skills they are trying to teach. And then they evaluate student papers and responses to determine if they were successful.
“The idea is to rate our own performances as well as student performances,” Sellen said.
Program faculty identified broad course objectives for World Civilization several years ago, he said, but the critical task is helping students achieve those goals — and for instructors to become more effective in providing that assistance.
More confident
“My teaching has changed, I think fundamentally, because of my participation in the group,” said Theresa Jordan, a history instructor who usually teaches one section of World Civilizations each semester.
Before, she said, she always wanted to see critical thinking from her students. But because of the way her assignments were designed, it was possible for students who were thinking critically to get poor grades, while students who weren’t thinking critically, but had a good grasp of content, could score well.
Now, she said, she feels much more confident that her assignments and her objectives are aligned so that when a student receives an A, it truly reflects superior critical thinking skills, which in the long run is more important than memorizing content.
It is fine and well to define the Six Learning Goals of the Baccalaureate, she said, but for the goals to have a real effect on student learning, faculty members must understand in practical terms how to create assignments that elicit and strengthen the skills embedded in those goals.
Data will be shared
Members of the raters group do not necessarily share assignments, tests or texts — they still work independently within the confines of the course “covenant” — but they do share a willingness to experiment. Richard Law, director of General Education, said he created the group to do research and gather data that will eventually be shared with other faculty who are not part of the group.
To support their effort, the group a year ago received a WSU Teaching and Learning Improvement Grant for a proposal titled, “Breaking the Barriers and Closing the Loop on Assessment in the World Civilizations Program.” Other members of the group include Ken Faunce, Marie Glynn, Susan Kilgore, Robert Staab and Michael Infranco.
The group is on its seventh draft of a critical thinking rubric, Sellen said, because they are constantly tweaking the criteria for what constitutes evidence of critical thinking and looking for language that is equally clear to students.
“We’ve been doing it for two and a half years,” he said, “and it’s tough.”
Methods compared
Not only do the “raters” assess their own work, but they compare student evaluations between their sections and the sections taught by faculty who don’t attend the meetings.
According to data from 2005, as a group, the “raters” scored higher on several measures derived from World Civilizations Program goals and the National Survey of Student Engagement, but about the same or lower on several others measures of the program’s student evaluations.
“Some of the data are disappointing…some of it is very encouraging,” said Law. “We are clearly scoring higher than the larger group on factors related to critical thinking.”
For instance, when asked if they were required to work on projects that integrated ideas and information from different sources, 55 percent of students in “nonrater” sections said yes, compared to 79 percent in “rater” sections. Eighty per cent of students in the raters’ sections agreed that the course gave them opportunities to improve their writing.
Stretching students
The one measure where the “nonraters” earned significantly higher marks from students was on the clarity of grading expectations and assignments.
That is a problem, Law acknowledged, but one to be expected.
“The raters group is experimenting, and that in itself can cause misunderstanding,” he said. “Raters group assignments often take students into unfamiliar or uncomfortable territory.” In effect, they are asking students to “Be prepared to identify a problem, consider multiple perspectives, identify key assumptions, support your argument with credible evidence, put it in context and come up with an insightful conclusion based on the evidence.”
It can be difficult to get students to think for themselves and challenge assumptions — their own and others’ — but the result is often a much richer and interesting learning environment, Jordan said.
