It is not news to anyone teaching in higher education that Student Evaluations of Teaching (SET) are a hotly debated topic. Their validity and reliability are often 
called into question, particularly since they are typically the primary source of evidence used for merit and promotion decisions in regard to one’s teaching effectiveness. Even I have weighed in on this issue along with Philip Stark, Chair of Statistics. Regardless of how they are used, the fact remains that they are used and they can provide valuable formative insights into one’s teaching...if you know how to interpret them. This is a question I’m often asked. So, instead of accepting the praise with nothing more than a proud smile, or dismissing the criticism with a wave of the hand (and turn of the cheek), try some of these tips to garner the most valuable information from SET’s that can inform your teaching now and beyond.
1. The range of responses on quantitative questions tells a richer story than the average alone
Receiving all 5’s on a rating scale item means something different than a scattering of ratings that more closely fall into a grouping of 3’s and 7’s. If student responses fall into this later example of two or more distinct sub-groups, it begs the question, “Why would my class and pedagogy have such a positive impact on some students, yet not equally engage this other group of students?” Having one or two outliers at the extreme are usually not a good call to action. Having larger, distinct groupings of student responses means something is happening that demands your attention. If the qualitative responses do not shed light on this disparity of responses, plan on conducting some mid-semester evaluations the next time you teach to identify this phenomenon earlier and dig deeper with students to understand what can be done to foster a learning experience more will find valuable. Or, you may find that there is nothing you should change other than setting more explicit expectations for student learning in the course.
2. Situate both quantitative and qualitative responses within your unique context along the career continuum, because students rarely do
I often give this advice when working with pre-tenure faculty, especially someone relatively new to teaching. New teachers, or even very experienced teachers teaching a class for the first time, receive critical feedback in the areas of organization, preparedness, etc. Unless you tell the students, they do not know that you are new to teaching, or that this is the first time you are teaching this course, so do not expect them to cut you any slack in this regard. In my experience, ratings and comments in these areas tend to go up as someone gains more experience teaching, or teaches a class the second, third, and fourth time. It is to be expected that your course planning and delivery will be refined each iteration, so “lower” ratings and critical comments in these areas can be expected to a degree the first time around. If that’s not enough for you, be proactive about talking with the students throughout the semester about this being the first time you have taught this course (not necessarily that it’s the first course you have ever taught), and actively solicit their input into it’s organization and design. Simply getting on the same page about what these terms (organization, preparedness) mean can be helpful to your teaching and better inform students’ rating at the end of the semester (Lauer, 2012).
3. Target your highest and lowest rated items to focus on areas for specific improvement, or to help identify your strengths-based pedagogy
Your highest rated items, and the areas where you receive the most student praise, tell you where your teaching strengths may lie. These are areas of teaching that you should harness, deploy more, and craft your course design around. If students LOVE meeting with you in office hours because of the more direct learning and you have a knack for teaching in small groups or one-on-one, this means you should think about ways to create small group work in your course that would allow you to float around and work with each small group to re-create an office hours type experience. On the flip side, your lowest rated items, and the areas you receive the most criticism, tell you what you need to continue to hone as part of your pedagogical craft. Or, if you go for a strengths-based approach, they help identify teaching approaches that you should try to avoid. There are lots of different ways to be an excellent teacher, but not all of them are necessarily right for you, and that’s ok.
4. If particularly critical comments are made, determine if there is some measure of “truth” to them, or if they are an outlier that should be addressed better during the next semester
It is inevitable, you are going to receive a student comment that is going to hurt, sting, pierce, and cut you to the core. It’s the nature of our business. After you give yourself some time to calm, and stop trying to figure out who said it so retribution can be sought, punching a hole in the wall out of anger, or even swear off reading student comments ever again, take a fresh look at the feedback and first determine if there may be a “nugget” of truth hidden inside it somewhere. After some reflection, if you think that the comment/s actually stem from immaturity, frustration, or bitterness about the course (and possibly towards you), heed that as cause to work next semester towards helping bring that outlier back into the fold. A Small Group Instructional Diagnosis (SGID) can be an effective way to help students gain perspective on a course and instructor that they would not gain otherwise. Read about how to conduct an SGID.
5. Review each SET holistically, not just item by item
Assuming the SET asks sound quantitative and qualitative questions, it is a good idea to not only review each item and the student responses, but to also couple them together categorically. Often, qualitative comments offer insight, explanation, and additional detail to the ratings on certain quantitative questions. At times they will match and be highly informative. At times they will not match (e.g., positive comments and a low rating), and that means there’s a discrepancy of some kind in the students mind as they are evaluating the course and your teaching.
These 5 tips should help you to interpret your SET’s and make use of the feedback to inform your teaching practice. With that said, you are not in it alone. It can be a good idea to have a consultation with someone in the Center for Teaching and Learning to seek further assistance in interpreting the evaluations and developing concrete considerations for the next time you teach. Request a consultation by emailing teaching@berkeley.edu with “Consultation” in the subject line.
Tips #1, 3, 4 and 5 have been adapted from Davis’ (2009) Tools for Teaching.
References
Davis, B. G. (2009). Tools for teaching. San Francisco, CA: John Wiley & Sons.
Lauer, C. (2012). A comparison of faculty and student perspectives on course evaluation terminology. To Improve the Academy, 31, 195-211.