Diagnostic Feedback Overview
The Diagnostic Feedback instrument has been updated to reflect the changes in higher education. This 40-item instrument (reduced from 47) includes 19 Teaching Methods, 13 Learning Outcomes, six student and course characteristics and two summative items. A brief summary of changes to the instrument follows:
Newly added Teaching Methods include:
- Diverse perspectives, Helped students to interpret subject matter from diverse perspectives (e.g., different cultures, religions, genders, political views);
- Self-reflection, Encouraged students to reflect on and evaluate what they have learned;
- Service learning, Created opportunities for students to apply course content outside the classroom.
In addition, “Provided timely and frequent feedback on tests, reports, projects, etc. to help students improve” and “Explained the reasons for criticisms of students’ academic performance” were synthesized into a new item “Provided meaningful feedback on students’ academic performance.”
The following Teaching Methods were removed because they were not highly correlated with student progress on any of the learning objectives:
- Displayed a personal interest in students and their learning
- Scheduled course work (class activities, tests, projects) in ways that encouraged students to stay up- to date in their work
- Gave tests, projects, etc. that covered the most important points of the course
Four new learning objectives were added to the Diagnostic Feedback instrument, which include:
- Diversity, Developing knowledge and understanding of diverse perspectives, global awareness, or other cultures;
- Civic engagement, Learning to apply knowledge and skills to benefit others or serve the public good;
- Quantitative literacy, Learning appropriate methods for collecting, analyzing, and interpreting numerical information.
- Ethical reasoning, Developing ethical reasoning and/or ethical decision making replaced Developing a clearer understanding of, and commitment to, personal values.
In addition, minor modifications were made to Information literacy by combining information literacy and lifelong learning items into Learning how to find, evaluate, and use resources to explore a topic in depth. Lastly, “Gaining factual knowledge (terminology, classifications, methods, trends)” and “Learning fundamental principles, generalizations, or theories” were synthesized into new item, Gaining a basic understanding of the subject (e.g., factual knowledge, methods, principles, generalizations, theories).
The following Learning objectives were removed. The first two were integrated into “Learning how to find, evaluate, and use resources to explore a topic in depth”; the third was replaced by “Developing ethical reasoning and/or decision making.”
- Learning how to find and use resources for answering questions or solving problems
- Acquiring an interest in learning more by asking my own questions and seeking answers
- Developing a clearer understanding of, and commitment to, personal values
Student and Course Characteristics
Two items, “Amount of reading” and “Amount of work in other (non-reading) assignments” were synthesized into one, Amount of coursework. Further, background preparation (My background prepared me well for this course’s requirements), which is included on the Learning Essentials instrument, has also been added to Diagnostic Feedback; and a new item related to self-efficacy was developed, When this course began I believed I could master its content.
The following items were removed from the instrument, because they were less important than other items in either predicting student progress on relevant learning objectives or computing adjusted scores:
- I had a strong desire to take this course
- I worked harder on this course than on most courses I have taken
- I really wanted to take a course from this instructor
- As a result of taking this course, I have more positive feelings toward this field of study
- The instructor used a variety of methods--not only tests--to evaluate student progress on course objectives
- The instructor expected students to take their share of responsibility for learning
- The instructor had high achievement standards in this class
- The instructor used educational technology (e.g. Internet, email, computer exercises, multi-media presentations) to promote learning
Variables used in calculating Adjusted Scores
Mean scores on several items pertaining to student and course characteristics are used in computing adjusted scores on student ratings of learning objectives and the two overall summary measures. The adjustments control for extraneous factors that can affect ratings. They are intended to “level the playing field” between instructors who teach highly motivated students with good work habits and those whose students may be lacking in those characteristics. Course Motivation (I really wanted to take this course regardless of who taught it), Work Habits (As a rule, I put forth more effort than other students on academic work), class size and a residual of Course Difficulty (Difficulty of subject matter) remain in calculating the adjusted scores. Background Preparation (My background prepared me well for this course’s requirements) has been added; whereas Student Effort (I worked harder on this course than on most courses I have taken) no longer plays a role in this calculation.