The Value of Student Feedback (Regardless of Response Rates)
By Ken Ryalls, Ph.D.
A recent blog in Faculty Focus by Maryellen Weimer, so aptly titled Course Evaluations: How Can Should We Improve Response Rates?, called for a conversation around how to get students to complete ratings for the right reasons.
At IDEA, we believe that creating value for student feedback is the most essential factor to elicit good response rates. Research and best practice consistently show the single greatest influence on increasing participation in student ratings surveys is for faculty to express and demonstrate how the results are important and used in making meaningful change. The next most influential factor is to set aside time in class to complete the surveys, regardless of delivery modality.
So while we agree that improving response rates should continue to be a goal for campuses whether they use online or paper delivery, we would like to suggest that instead of continuously discussing response rates, we should put greater focus on getting a clear picture over time, regardless of response rate.
Here is why:
We know that ratings based on lower response rates cannot be assumed to represent the overall class perceptions as well as higher response rates. But, representativeness is a different issue than reliability. The former is tied to the percent of students in the class that respond—the greater the response rate, the more representative are the scores derived from the course rating. Reliability, on the other hand, is related to sample size, or the number of student raters. If 50 students out of a class of 100 responded to a survey, their ratings would be more statistically reliable than if 19 students out of a class of 20 responded even though the 19 responders would be more representative. So even though the reliability of any measure does increase as the number of observations increase, it does not follow that a low number of observations means those observations are not reliable; even classes with low response rates can provide useful information for a teacher provided that the data are utilized as part of a holistic analysis of feedback over time.
The ultimate goal should be to gather multiple sources of ongoing teaching and learning feedback so that the faculty has information helpful in ensuring continued growth of program quality over time. As Dr. Weimer states, “They (students) should be doing these end-of-course evaluations because they believe the quality of their experiences in courses matters to the institution.”