What Will Happen to My Ratings?

At a recent workshop on learner-centered teaching, a participant told us that philosophically she couldn’t agree more with the need to make students more responsible for their own learning. She knew it was right, but she couldn’t go there because her ratings would take a hit. I assumed this meant she was a new faculty member and under scrutiny for tenure. I was amazed when she told me after the session that she was a full professor with years of tenure under her belt. With all that security, the chance of bad ratings was still preventing her from doing what she thought was right. She couldn’t incorporate changes that might improve learning outcomes because she would lose at the ratings game. At the time, I didn’t know how to answer. Now I’ve thought of several responses.

A policy answer—Most institutions overuse summative evaluation. Research on student ratings has shown that for midcareer faculty, ratings—from semester to semester, across a number of courses—are stable. They don’t vary all that much, so why must every class, every semester, be assessed for administrative review? Tenured faculty, through appropriate governance structures, ought to lobby institutions for policies that release a course from summative evaluation when a series of innovations are being implemented in it. Obviously, instructors need formative feedback about the changes, but for the sake of change and refreshment, couldn’t institutions let a faculty member in good standing not evaluate a particular course for a semester or two?

An answer that challenges—Is the assumption that ratings go down when changes are incorporated valid? I can’t find much research that substantiates it. A bit of case study research reports that for major course revisions (not the addition of one, two, or a few techniques), ratings dipped but then recovered. Unless you’ve experienced a drop in ratings when you’ve incorporated a series of changes, I would not assume that change leads automatically to lower ratings. Besides, you can avoid the problem if changes are added gradually, which is probably a more effective way to implement new strategies and techniques.

An answer that suggests an alternative—There is a good deal of research documenting the fact that if you have students do midcourse evaluations and then discuss the results with them, end-of-course ratings increase at statistically significant levels. So, if you anticipate that ratings might decline because of new approaches, test that assumption midcourse. Find out what’s going well and not so well for students. Ask them to suggest ways the new approaches might be refined so that they’ll work better. Discuss the feedback; respond to the suggestions. That doesn’t mean do everything students suggest, but value their input by providing them with a response. If you opt not to do what they suggest, explain the educational rationale behind the approach you’ve chosen.

A straightforward answer—The ratings research gives students more credit than most faculty do when it comes to students understanding what is really going on in a course. Explain up front that you’ve planned some changes for this course. Explain why you need to change—you think they might grasp the material better this new way; you’re tired of the old way; your instructional objectives have changed; whatever. Tell them honestly that you need their feedback, that you’ll solicit it regularly, that you’ll respond to what they suggest and make changes when something is clearly not working well. Invite them to be part of the experiment. Don’t forget, it is ultimately in students’ best interests to have their teachers working on more and better approaches. Give students credit for being able to give you credit for what you’re trying to accomplish.

An answer that suggests another alternative—Stop thinking of global, end-of-course assessments as the only possible feedback mechanism. Students can be asked to provide feedback on any aspect of instruction, including a new assignment, a different text, an alternative approach to grading, group activities, whatever. And feedback can be collected in a variety of ways beyond the machine-scorable short form. Send out an e-mail with a couple of carefully worded queries. Have students meet in groups to briefly discuss an activity and as a group generate feedback. Speak privately to students whose opinions you value. You don’t have to rely solely on the end of course ratings for feedback and to document the nature of that experience for students.

An answer that protects you—For tenured faculty members with lots of security, what’s the problem with a set of low ratings? A smaller merit raise? Most are already so small, would you even notice the difference? I think the problem is more about the personal anguish that results when students negatively respond to a well-intentioned, carefully prepared, thoughtfully designed, and generally well-executed innovation. Protecting yourself begins with recognizing your vulnerability. Don’t ask students whether or not they “liked” the innovation. Opt for questions that explore the impact of that policy, practice, behavior, assignment, or activity on their learning. Keep the focus on the change—don’t ask how it compares with all other changes in the universe. You want to know if and how it did or didn’t help students learn. That’s all that matters, so keep the focus there. Also adopt the golden rule of feedback: Give unto others feedback in the form you’d like to receive it. That doesn’t mean give only positive feedback, but it does mean deliver feedback constructively. Here’s an opportunity for you to help students develop this important skill.

The answer I’d like to give but probably shouldn’t—Grow up! If the chance of a dip in ratings prevents you from doing what you think will positively impact student learning, you are ascribing far more importance to ratings than they deserve. If lower ratings have that much influence at your institution, work with like-minded faculty (I’m sure there are plenty) to change these counterproductive policies. Summative rating data from students offers one view of your teaching. To imagine that it is the whole view, the best view, the correct view, and the only view is to trivialize the complicated way teaching affects learning.

The above article first appeared in Teaching Professor, a monthly newsletter that helps both new and experienced faculty stay on top of the trends in teaching and learning that will help them achieve academic excellence. Click here for more information and to subscribe.