IDEA’s Student Ratings of Instruction are designed to provide feedback you can use—formative feedback that gives you suggestions for improvement—as well as summative feedback that can be used as a component of a more comprehensive system of faculty evaluation.
When you partner with IDEA, all of the tools are available to use.
- Diagnostic Feedback - A comprehensive tool that provides both summative and formative feedback about student progress on relevant course objectives, instructor teaching methods, and overall impressions of the instructor and course.
- Learning Essentials - Provides summative feedback about average student progress on relevant learning objectives and overall impressions of the instructor and course.
- Teaching Essentials - Provides formative feedback about teaching methods highly correlated with instructor and course excellence.
- Instant Feedback- A short instrument that gathers immediate student responses after each class or unit; weekly, or whenever desired, so faculty can see the impact of changes they have made in their class.
How do faculty choose which instrument to use?
Some institutions allow faculty to choose the instrument they wish to use for particular courses while others may make this decision for them based on policies for departments or other units. If you are allowed to choose your own instrument, your institution should have a way for you to communicate that choice because faculty cannot make the technical selection of an instrument--only the administrator can do this when setting up administrations. Contact your on-campus coordinator to find out how to make your choice.
See also this advice for selecting an instrument: Choosing an Instrument.
What the ratings will tell you
IDEA’s Student Ratings of Instruction provide feedback you can use to improve instruction with the focus of the instrument on student learning. The student ratings provide evidence of progress on learning objectives and teaching methods used in a course.
One of the most important of the pieces of evidence is student ratings of how well they achieved learning objectives in your course. Students are asked to rate their achievement of learning objectives in both the comprehensive Diagnostic Feedback instrument and Learning Essentials instrument. In your report, you will be able to see their rating and how you compare to other groups.
One of the unique things about the SRI instruments is that they control for course difficulty beyond the influence of the instructor. In the Diagnostic Feedback instrument, the system adjusts your Progress on Relevant Objectives scores, as well as your “excellent teacher” and “excellent course” ratings, through regression equations, to account for the influence of student background preparation, work habits, motivation, course difficulty, and class size. In Teaching Essentials, average ratings on both the “excellent teacher” and “excellent course” items are adjusted for student work habits, motivation, background, as well as class size.
These adjusted scores may raise your scores some to account for these influences. For instance, in a course where a large number of students report low motivation for the course, your scores are adjusted upward to account for the low motivation. The purpose of adjusting scores is to level the playing field when comparisons need to be made between instructors teaching different kinds of classes. So, for instance, scores of instructors teaching large, required general education classes can be compared more authentically to those teaching smaller, high level, in-major courses.
Adjusted Scores at a Glance [PDF]
The default view you are shown in your reports is this adjusted score, but you can chose to view your Raw Averages if you wish by simply toggling the View option.
What students see and how they access the survey
Students access the survey for all their courses from one place—the Campus Labs portal for your campus. Your On Campus Coordinator may have more directions and information for you about how students can access the portal.
- Once students are logged in, they will see a list of current and upcoming course evaluations to be completed.
- Clicking on the course starts their rating of your course. Once they are in the instrument, it is a simple matter of selecting ratings for the items presented.
- Students can access the instrument through desktop computers, tablets or mobile devices.
To learn more, see Accessing Campus Labs Course Evaluations for students.
What are the questions on the instruments?
Sample versions of each instrument can be found by clicking the links below
Diagnostic Feedback Instrument
Learning Essentials Instrument
Teaching Essentials Instrument
With all instruments except Instant Feedback, it is possible to add custom questions if this option is available at your institution.
See Adding Custom Questions to learn more.
Administering your Survey
In many cases, institutions choose to use only the Diagnostic Feedback instrument for all courses, or specific departments may alternate instruments based on decisions of that unit. In these cases, you as a faculty member, do not need to make any decisions about choosing an instrument. If, however, your institution allows you to select which instruments to use in each of your courses, your first task is to communicate with your IDEA Campus Coordinator about which instrument you will use in each course. Your institution should have its own internal method of communicating about this.
If you are using the Diagnostic Feedback or Learning Essentials instruments, you will need to choose the Essential or Important learning objectives for your course before the administration of the survey begins. See the Choosing Learning Objectives video for more about this. For Teaching Essentials, there is nothing you need to do ahead of time.
ABOUT LEARNING OBJECTIVES
In the Diagnostic Feedback and Learning Essentials instruments, the main indicator of teaching effectiveness is student progress on learning objectives. The instruments are designed around 13 learning objectives that encompass all objectives you would have in a course. These learning objectives are broader objectives that precede your more specific learning outcomes but are subsequent to your major course goals.
Here’s an example for a child development class:
Understand characteristics of each stage of child development
IDEA Learning Objective
Gaining a basic understanding of the subject (e.g., factual knowledge, methods, principles, generalizations, theories)
IDEA Learning Objective
Learning to apply course material (to improve thinking, problem solving, and decisions)
1. Students will describe each of the physical, cognitive, and psychological traits of each stage of development
2. Students will differentiate the stages of child development when presented with examples.
1. When presented with cases, students will explain child behavior based on characteristics of the child’s stage of development
2. When presented with common childhood problems, students will suggest solutions based on child development principles.
In this example, the course goal has two IDEA Learning Objectives
(though it could have more), and each learning objective has two learning outcomes (though there could more be more as well). Organizing your course learning outcomes through IDEA Learning Objectives can help you think through your course more thoroughly and ensure you are covering concepts and applications as thoroughly as needed. You might also consider organizing your course syllabus this way so that students see clearly how your learning outcomes fit with the objectives.
Whether or not you organize all your course learning outcomes by IDEA objectives, those objectives in your course that are Essential or Important must be identified before administering the survey. IDEA instruments double-weight Essential items and single-weight Important items while those of minor or no importance are not used in calculating the ratings. This enables you to focus the survey on those learning objectives that are vital in your individual course.
Choosing Learning Objectives
If you are using the Diagnostic Feedback or Learning Essentials
instruments, the Essential
learning objectives of your course must be identified before administering the survey. Your institution may prescribe which learning objectives are to be rated in your course and pre-select them for you, or it may be left to you to choose. If they are pre-selected, you can upgrade an item (for instance, make an “Important” item “Essential), but you cannot downgrade an item. Check with your On-Campus Coordinator if you do not know the options for this at your institution.
If you are to choose your own, there are few things to consider. As a general rule, limit your selection to between 3-5 objectives by prioritizing those objectives that are Important or Essential in this course. Although all the learning objectives may be addressed to some extent in your course, it is unrealistic to expect that students could make significant progress on all or even most of them in one course. So again, try to identify those 3 to 5 that are most important. But be true to your course. If you are teaching a course where you are targeting only one objective (such as a lab experience), then it would be appropriate to select only one learning objective. On the other hand, if you are teaching a capstone class in your major, and you have targeted many learning objectives, it would be appropriate for you to select more than five.
Selection is more obvious in some courses than others. To help, consider these three strategies:
1. Create a chart that aligns goals, IDEA learning objectives, and your learning outcomes. Ask these questions:
- Is it a significant part of the course?
- Do you do something specific to help students accomplish the objective?
- Does the student’s progress on the objective influence his or her grade?
2. Use the Teaching Goals Inventory to help narrow important objectives
3. Use relevant learning theory such as Bloom's Taxonomy to narrow the focus
To choose objectives, login to your dashboard in the Campus Labs system and choose the Objective Selection Form
Adding Custom Questions
Individual faculty are able to add custom questions
to specific courses as needed (provided this is allowed by your institution) on all instruments except Instant Feedback. For example, you may want to collect student feedback about a particular resource used in the class or their feedback about the usefulness of a grad assistant. Custom questions must be added before the administration period of the survey begins. Institutions often do not want to use custom questions so that surveys between classes are the same length and do not over-burden students, so this option may not be available to you.
Take a look at the following guides for suggestions related to Using Additional Questions
and Using Additional Questions for Online Courses.
Increasing Response Rates
While any feedback from students can be useful to the individual instructor for improving a course, higher response rates help make comparing one course to another more meaningful. To get responses from as many students as possible, there are several things you can do.
- Create Value for Student Feedback. This is the single most important factor to elicit good response rates. If instructors encourage student feedback, students will be more inclined to participate. Faculty can help students learn to value the feedback process by using techniques such as the following:
- Use your syllabus. At the beginning of the course, put the IDEA Learning Objectives that you have selected for this course in the course syllabus alongside the specific course learning outcomes. This can demonstrate for students how the course outcomes relate to the general IDEA learning objectives and how your instructional decisions are informed by student feedback.
- Mid-point reminders. At the midterm point of the course, remind students what the goals of the course are and how they can enhance their learning experience.
- At the end of the course. When the survey has been opened for students to respond, encourage them to complete the form by telling them that throughout the course you have been giving them feedback with their best interest in mind.
- Assure students that their responses are confidential. When their responses are accessed by you or administrators, no identifying information is associated with the responses. Students are more likely to complete a survey if they are assured of the confidentiality of their responses.
- Administer Surveys During Class. Emulate the “captive audience” nature of in-class paper ratings by asking students to login and complete the survey during class. Students can use their mobile phones, laptops, or desktop computers in a lab.
Consult with your On-Campus Coordinator about whether or not your particular institution has specific policies about when/how to administer surveys and the use of other resources such as Email Notification Templates
Using your Feedback
Viewing Your Results
Login to the Campus Labs portal for your institution where you will see a list of your courses and options. When the course evaluation period has ended, click “View Reports” to see results for individual courses.
COURSE AND STUDENT CHARACTERISTICS
The three end-of-term instruments (Diagnostic Feedback, Teaching Essentials, and Learning Essentials) include a summary of student characteristics, and Diagnostic Feedback also includes student perceptions of course characteristics.
On the first page of your report, you will see a representation of these characteristics. The Diagnostic Feedback Report provides you with a summary of how students perceived the difficulty and workload of the course as well as indicators of their motivation and effort in the course. These factors are used in calculating the adjusted scores for the other ratings in the instruments including the excellence of the instructor and excellence of the course items. In a course with unmotivated students, for instance, the adjusted scores will be raised slightly. And reviewing this information yourself will help you interpret the rest of the report in light of the characteristics of the students.
and Teaching Essentials
provide a summary of student motivational characteristics. These are used in calculating adjusted scores as described above, and likewise, can be used to interpret the rest of the report.
INTERPRETING THE DIAGNOSTIC FEEDBACK REPORT
The Diagnostic Feedback instrument is a comprehensive tool that provides summary and detailed, formative feedback for your course. The summative information for your course gives a quick overview of all the ratings of your course, and the formative feedback provides detail about average student progress on relevant course objectives, instructor teaching methods, and overall impressions of the instructor and course. The Diagnostic Feedback instrument controls for extraneous factors (e.g., student work habits and motivation) beyond the instructor’s control.
The Summative tab provides you with a quick overview of “how you did” in the course according to the ratings.
Progress on Relevant Learning Objectives (PRO)
One of the best ways to infer teaching effectiveness is to examine student ratings of progress on relevant objectives that have been chosen as Important or Essential. The average of these ratings provides a good indication of how successfully objectives were reached in the course. From the Summative tab, simply click on “Progress on Relevant Objectives” to open the details for each learning objective.
When you enter the Learning Objectives details, you can see how students rated their learning on that objective and make comparisons to other groups.
This displays the course average for the item in the course.
Converted Average Comparison
This score compares the course relative to the overall mean in the comparison group by telling you if your score is similar, lower, or higher. [The comparison score converts the raw mean score for a class to a scale having a mean of 50 and a standard deviation of 10.] The overall comparison group is all users in the IDEA system. The comparison for disciplines includes only classes in the IDEA database where the instructor selected a specific discipline code during the most recent five years, and for the institution includes only classes at the specific institution during the previous five years. In all types of comparisons, comparison scores are only provided if there are at least 400 classes in the respective comparison group. See Technical Reports 12 and 18 for statistical details.
Formative feedback provides you with student observations about how often they observed the Teaching Methods shown to be effective for achieving the learning objectives identified in your course. When you enter the Formative Feedback area, you can see each item from the instrument and whether your rating results in the suggestion that this is a “strength to retain,” or that you should “retain current use or consider increasing,” or that you should “consider increasing use.”
INTERPRETING THE TEACHING ESSENTIALS INSTRUMENT
Teaching Essentials is a short form (seven items plus a few overall items) that provides formative feedback about teaching methods highly correlated with instructor and course excellence. No information is collected about progress on course objectives as in the Diagnostic Feedback or Learning Essentials instruments. Like other IDEA instruments, the report controls for extraneous factors beyond the instructor’s control and provides converted and comparative scores on excellence of the instructor and the course.
Students rate their observations of the seven teaching methods that are highly correlated with ”course and instructor excellence.” Instructors who are consistently observed by students as not practicing these methods should consider adding them to their instructional practice. For each method, there is a teaching “Note,” which will help you better understand the teaching method and how to use it (see all the Teaching Notes on Instruction).
The seven Teaching Essentials teaching method ratings
- Displayed a personal interest in students and their learning
- Found ways to help students answer their own questions
- Demonstrated the importance and significance of the subject matter
- Made it clear how each topic fit into the course
- Explained course material clearly and concisely
- Introduced stimulating ideas about the subject
- Inspired students to set and achieve goals which really challenged them
INTERPRETING THE LEARNING ESSENTIALS INSTRUMENT
The IDEA survey’s chief indicator of teaching effectiveness is how well students rate their progress on the types of learning that faculty target. These ratings are part of the Diagnostic Feedback instrument and are the primary component of the Learning Essentials instrument.
Before administering Learning Essentials, you or someone on your campus, selected the Essential or Important learning objectives for your particular course. So presumably, these learning objectives were targeted in your course by your course design and instructional practices. Students, then, rate how well they believe they achieved these objectives. When your collective rating on these items is low, it indicates that students believe they did not achieve those objectives very well, and that perhaps, you should consider how you have designed learning around those objectives.
For each objective identified as Important or Essential, you get the following report:
In addition, Learning Essentials includes the “overall” or summative items included with all IDEA instruments: “Overall, I rate this instructor an excellent teacher, and “Overall, I rate this course as excellent.” The ratings on these items may be used to monitor overall student impressions of a course and instructor.
INTERPRETING THE UNIT SUMMARY REPORT
The USR uses data from the Diagnostic Feedback and Learning Essentials instruments
The Interpretive Guide will walk through through the various components.
CREATING A PLAN FOR IMPROVING INSTRUCTION
Each of the IDEA Student Ratings of Instruction instruments provide a number of opportunities for you to work toward improving instruction. Whether you simply use the results to improve your teaching on your own, or as part of a collaborative effort with a colleague, or more formally, as part of an annual professional development plan, the results from each instrument can help guide you toward where you might focus your efforts.
For instance, if students do not rate you very highly on the method, “Made it clear how each topic fit into the course,” (as indicated in the results of your Diagnostic Feedback or Teaching Essentials instruments), you might target this as an area for learning more about and increasing its use in your courses. Document this targeted area for improvement in your annual review report, and develop a plan for it. This may include workshops, conferences, working with your campus Center for Teaching Learning, and as well as the resources provided within your report.
Both the Diagnostic Feedback and Teaching Essentials instruments include links to the Teaching Methods Notes pages for each teaching method. Clicking on that link will take you to a summary of each method, suggestions for implementing it, and references. These notes are a good place to start as you work to increase your use of any particular teaching method.
In the Diagnostic Feedback and Learning Essentials instruments, you get feedback on how well students think they achieved the learning objectives identified as Essential or Important in your course. Research has demonstrated that low ratings on a learning objective is correlated with poor performance on that objective in the course. So consistent low ratings on an item likely means there is room for improvement in how that objective is taught.
When you have identified objectives with low ratings, consider the factors involved. If you have aligned the objectives with your specific course learning outcomes (See About Learning Objectives), look at other evidence you have for student performance on those learning outcomes. Are students indeed scoring poorly on those outcomes? If so, the combined evidence suggests that students are not mastering that objective as well as they should and instructional changes are warranted. Be sure to read the Notes on Learning for each of the learning objectives to learn more about methods of teaching that are most effective for each.
Of course, you have to interpret the reports in light of your course. For instance, just because a particular teaching method is not observed much, it does not necessarily mean you should increase your use of it. Some methods are more appropriate for some disciplines and courses than others. But do not dismiss a low rating without seriously considering it. Work with your instructional lead (department chair, dean, etc.), or contact IDEA, to identify the findings that are more relevant for you and work toward improvement.
Using Instant Feedback
Instant Feedback is a tool you can use at any time during a course to gather formative feedback. That is, feedback that can help you make adjustments to the course during the course rather than after the fact when it is too late for the students in the course.
So you might, for example, ask students to complete the feedback at the end of a class that was particularly challenging, or after using a new instructional method, or teaching a topic for the first time. Using six of the items from Teaching Essentials instrument and the global question, “You understood the material covered today—yes or no” the feedback will help you decide if you need to backup and revisit some parts of the unit or can continue.
You can administer Instant Feedback as often as you wish. The results are only seen by you, not by administrators or anyone else. You get results emailed to you almost immediately after the survey ends.
The quick Instant Feedback Instrument:
How to use Instant Feedback
The Instant Feedback tool must be enabled by your On-Campus Coordinator. Once enabled, you can use it as many times as you wish during the semester. You can set the survey to end automatically two hours after it begins, or you can manually end it after a class. Either way, your results are emailed to you almost immediately. Keep in mind that student enrollments must be setup in your campus system before students can access the survey. Be sure to communicate with your On-Campus Coordinator about this so that you know when enrollments are complete and students are able to access the survey.
You send students to the survey just like you do the end-of-term surveys. You direct them to your campus portal for surveys.
From their course list, they will see the Instant Feedback link for your course. The survey can be completed in less than a minute.
It is recommended that when you use Instant Feedback in face-to-face courses, you ask students to complete it before leaving the classroom so as to capture the feedback immediately for your use. In online courses, students could complete Instant Feedback at any time after a unit or lesson.