Fri 20 Nov 2009
How to “measure” teaching skill?
Posted by Whitney Wilson '90 under Academics, Teaching at 10:44 am
One of the spin-offs from the some of the recent discussions about Bernard Moore was a “debate” about how good or bad he was as a teacher. One student said that he was terrific, several others were quite critical. Without trying to rehash Mr. Moore’s specific talents, I thought the debate raised an interesting question.
Most (all?) of us believe that one of the qualities that makes Williams a great educational institution and distinct from major research universities like Stanford, Cornell, or Yale, is the fact that the professors are expected to and do teach undergraduates. This fact is assumed to attract professors to Williams who are interested in teaching (as opposed to simply being interested in research) and, hopefully, are good at it (whatever that means). During the evaluation process for professors, I understand that teaching ability is an important factor.
But how can we measure or evaluate teaching ability? This, of course, is a problem at all educational levels. At the primary and secondary school level, we can evaluate teachers in part by how much their students learn, typically measured through testing. Good teachers should teach their students more than bad teachers. Is anything similar done at the college level? If so, I am not aware of it.
I think teaching ability is largely measured by student surveys, supplemented by occasional observations. I think its unfortunate, if true, that only the numeric scores from those surveys are shared with the professor’s department, and that student comments are not shared with the department. I think these comments, when viewed as a whole could be very useful. Are the comments only made available to the professor in question? If so, why would that be?
If teaching evaluations are based primarily (almost exclusively) on numerical aggregations of student survey data, I think that is a little troubling, simply because that process is so subjective and subject to the vagaries of sample size, who shows up/bothers to fill out the surveys, etc. I am of the view that great teaching is like obscenity (i.e. you know it when you see it), but is there a good (better?) way of determining who good teachers are?
Input from the many academics here at EphBlog would be appreciated.
Print • Email| « Spoken Up | Freeze Faculty Salaries Again? » |
7 Responses to “How to “measure” teaching skill?”
Leave a Reply
You can follow this conversation by subscribing to the comment feed for this post
If a comment you submitted does not show up, please email us at eph at ephblog dot com. Please note that commenters are required to use a valid email address when submitting comments.
David says:
In as multi-dimensional and transparent a manner as possible.
First, Williams already does a fine job of this. Numeric scores provided by students are useful, as is written feedback. Seniors are often interviewed about all the professors that they have had in the department, allowing them to provide feedback informed by 4 years at Williams. Recent graduates are (still?) sought out for their opinions. (Williams wrote to me (and other philosophy majors, I assume) after graduation when Alas White came up for tenure.) The more information you gather — numeric and written, immediate and measured — the better off you are.
[Whitney, I think you underestimate how much written/interview information from students is gathered, especially for tenure-track faculty. Corrections welcome.]
Second, I would like to see Williams do more in this regard by setting up a series of prizes for great teaching.
Even if you don’t like the cash prize aspect of this, having formal, annual, high profile prizes would aid us in our efforts to identify (and retain!) exceptional teachers.
Third, and much more controversially, I would like to see Williams become much more transparent with regard to teaching evaluation. (Needles to say, I think that transparency would make Williams better in every dimension.) This is a tricky issue and I am happy to leave non-tenured faculty out of the equation.
But why not make the summary SCS form information publicly available (at least on campus) for every tenured member of the Williams faculty? There are some great professors at Williams and some less great ones. The students deserve to have as much information as possible.
kthomas says:
@Whitney: Thanks for spawning this into a separate discussion and removing it from a thread which begins with Prof. Water’s considered remarks on a largely separate topic.
Some comments:
It seems to me that, of course, it is hard. What works for one student, does not necessarily work for another.
Given the sample sizes, I continue to fundamentally distrust the numerical course survey data, especially for new professors who have only a few data points. It is too easy to manipulate the results in one direction or the other.
Finally– on the use of written comments– one particularly influential department chair, while I was at Williams, argued against expanded attention because this would “give students even more influence in the process.” (My gloss).
Aidan says:
Gates Foundation just put up big monies ($335MM) on this
'10 says:
I assume the reason that students’ written comments are not shared with departments is so that students feel free to be blunt in their evaluations. On a couple occasions I’ve taken classes with new profs who I liked a lot, but didn’t have a lot of teaching experience and so definitely had room for improvement teaching-wise. If I knew that my comments would be used to judge those profs, I would have been a lot less honest and frank about (what I saw as) their teaching shortcomings, and that probably would have been much less useful to them.
Also, most (all?) departments do have their senior faculty conduct periodic interviews with students who have taken courses with a junior faculty member, so it’s not as if student opinion isn’t factored into a tenure decision.
hwc says:
Whitney:
This issue (and the related issue of teaching teachers how to teach) are prominent in every college accreditation report I’ve read. Higher education is really struggling with these issues: both evaluation and training.
The short answer is that liberal arts colleges rely on informal mechanisms and there is widespread concern that there needs to be more focused attention paid to the quality of instruction.
One of the reasons that I highlighted something like the writing programs at various colleges is that, by committing full-time faculty resources to writing, this kind of program can be a catalyst for improving how professors teach writing across campus. It’s imperfect, of course, but these are the types of initiatives that the accreditation panels are contemplating when it comes to mechanisms for systematically improving teaching.
The student surveys are viewed as largely counter productive. Every college professor in the country knows that the easiest way to get good student evaluations is to give everyone better grades.
frank uible says:
All them Ben and Jerry’s flavors for all them people.
Derek says:
My thoughts on this issue that all faculty deal with:
1) You can’t measure teaching ability. You can, however, assess or evaluate it. The problem with most assessments or metrics is that they presuppose that everything can be turned into a math problem and that a math problem is somehow better. But this creates precision at the expense of accuracy. Quantitative results on something not inherently quantitative is almost never better than qualitative results. “Dr. Professor is a good teacher in areas x, y, and zed but could afford to work on a and b” is infinitely better than “Dr. Professor scored 3.7 on our 1-5 scale.”
2) Multiple measures is the key. Teaching evals — both quantitative and comments; teaching observations from peers within and outside of the discipline; gathering teaching materials (I think we forget that you can tell a great deal from syllabi, assignments, handouts, and the like –if the inputs are good, the outputs have a better chance at being good); self-assessments. None of these is perfect. But the accumulation can allow for gaps to be covered and for multiple angles on teaching. All of these things can be gamed. But if someone is doing reasonably well on all fronts, they are probably a good teacher. If they are awful on all fronts, there is probably a problem.
3) These things should be driven first and foremost from faculty as expressed by departmental and school priorities and so forth. The further the demand comes from the classroom, the worse the solution is likely to be. At state schools, when the legislature decides to poke their snouts in, a politicized, useless clusterfuck is sure to follow.
dcat