Culture and Commitment: The Why and the How of High SRI Response Rates at Champlain College
Guest Blog by: Ellen Zeman, Learning Assessment Director, Champlain College
At Champlain College, a small private professionally oriented institution in Burlington, Vermont, we pride ourselves on what we call a “radically pragmatic” approach to education. Our hands-on pedagogy is about thinking and doing, and teaching is our core mission. Because of the centrality of teaching on our campus, we take seriously our commitment to evaluating the quality and effectiveness of student learning in our classrooms, be they face-to-face or online. Student feedback is an essential part of that evaluation.
History and Culture
Being a teaching-centered institution is one important element in understanding why Champlain College students consistently participate in the IDEA Student Ratings of Instruction at rates above 80%. But there is certainly more to this story, which began more than ten years ago. In 2005, by which time it had become glaringly obvious that the home-designed “Student Description of Teachers” instrument—printed in Courier font, with manual data collection and in use since the early 1980s—was flawed and losing relevance. In 2006, a task force was established to create a comprehensive faculty evaluation system, one part of which would be the instrument used for student feedback. This group collaborated with the Office of Academic Affairs to pilot two new instruments, one of which was the IDEA SRI, alongside our homegrown survey. Among these three, faculty found the IDEA SRI, with its emphasis on learning outcomes and teaching methods, to be best aligned with our mission and pedagogy. Our ten-year, ongoing partnership with IDEA began with faculty choice.
Two important factors, history (we’ve always done course surveys) and culture (we’re a teaching institution), contribute to our high participation rate. Admittedly, these circumstances might be hard to replicate on many campuses. Two other contributors may prove more useful: relevance and practice.
To gain any relevance for students, course ratings first must be relevant to faculty. Because a full complement of student ratings results is required evidence supporting promotion decisions for full-time faculty and performance reviews for both full-time and contingent faculty, instructors make sure to administer the survey in every course section. Only in about 2% of all sections in a given term do instructors fail to administer the SRI. It is an expectation that deans and program directors will examine IDEA SRI results to gauge instructional quality and to inform individual professional development. Our ideal is to utilize the formative design of the IDEA reports to guide faculty in directing or improving their own instructional practice.
Another way to increase relevance of the SRI for faculty is to enhance its validity. A “work in progress” for us is an ongoing effort to collaborate with program directors and individual faculty to align IDEA learning objectives with course and program learning outcomes and college-level competencies. We also use workshops and outreach efforts to help instructors add their own custom questions to the SRI as a way to gain more direct and meaningful feedback from students.
When we first implemented the IDEA SRI in 2006, we used paper forms for all of our on-campus courses. It was (and still is) expected that instructors would administer the survey during class time within a designated two-week period. Because we distributed the paper forms in brightly colored envelopes to faculty mailboxes, division operations managers could keep an eye on who had not yet picked up their IDEA packet and send a nudge by phone or email. Over time we moved from paper to online administration of the SRI in on-campus classes, starting first with sections that met in a computer lab, then adding volunteers and then discontinuing the paper option altogether in 2014. Completion reports go out from the Campus Coordinator to operations managers halfway through the survey period so that they can send reminders to instructors who had not yet administered the survey. To support faculty administering the survey online during class time, the Provost’s office purchased several rolling suitcases filled with Chromebooks that instructors could borrow. Over the past ten years, overall response rates for on-campus sections have remained close to 85%.
The same was not always true for our distance-learning sections. At Champlain our adult continuing education and graduate programs are run online. Students access the IDEA SRI through a module on the LMS course shell that remains open during the survey period, and they take the survey asynchronously. In graduate-level sections, response rates have always been high, but for the first several years, we struggled with low response rates from our adult undergraduate students. High participation in graduate classes could be attributed to relevance for instructors. Program directors in our relatively small graduate programs historically have had close collaborative relationships with their contingent faculty. Through this open channel program directors have been able to communicate the importance and utility of the survey to instructors (who then communicated that importance to their students). To increase student response rates in all of our online sections we implemented an extra-credit program in which adult students (graduate and undergraduate) report, based on the honor system, their completion of the IDEA survey through a “quiz” located in the survey module and receive a 1% increase in their overall course score for a “yes” response. Immediately upon offering extra credit, online student response rates jumped from around 45% to over 80%. Spot checks show that these self-reports are generally accurate.
Taking a “user support” approach to administration and a developmental attitude toward interpreting results goes a long way toward gaining faculty trust and buy-in. Outreach from the Campus Coordinator is both bottom up and top down. Meetings with program directors and deans help them learn how to get good information from IDEA results and how those results can be formative for faculty. We offer workshops and a carefully planned stream of communications throughout the term to help faculty understand administration procedures as well as how to interrogate their IDEA reports to help tell their teaching stories. Along with in-class requirements for traditional and extra-credit for adult students, for Champlain College the key to high student participation is communicating with faculty about what and how they can learn from student feedback.
Ellen Zeman is the Learning Assessment Director at Champlain College and has been the IDEA Campus Coordinator since 2006.