1. Professors that heap on the work and are tough graders typically get lower evaluation scores than those who offer fluff classes and give out As like candy.
2. Even if a professor is "tough", they will still get get okay evaluations if students feel that the class was valuable.
3. Professors that do a consistently poor job will get consistently poor ratings over a period of years. This cannot and should not be ignored.
4. If you make a professor's job contingent on high student evaluation scores only, you may well end up with an education system that is biased toward push-over classes and grade inflation.
So, yes—analyzing student evaluations will weed out the obviously bad professors. However, you may also reward the ones that are bad because they don't really hold students accountable.
The best way to do it would be to more carefully scrutinize those professors with scores that are either too low OR too high. Because chances are that those at either end of the scale may be doing poor jobs. This is not to say that all faculty with high scores are too easy—some may well be truly excellent professors. However, you really do have to ask: why are the scores are so high?
I also think that you need an outside, objective entity to evaluate professors. Peer reviews don't cut it because faculty tend to protect their colleagues—many times out of a sense of self-preservation (e.g., you scratch my back and I'll scratch yours).
I am PhD grad student (two weeks until my defense) who is done quite a bit of teaching and this is spot-on.
I say put cameras in the classroom and base evaluations solely on 1) cogency and clarity of each instructor’s lectures and 2) keeping with high and consistent standards, set by the university, in assessments . Problem solved.
See my POst 76.
I’m wondering how long an “outside, objective entity” would remain either.