Tuesday, May 09, 2006
When Metrics Make No Sense
This article examining student ratings of teachers in universities is a perfect example of when metrics make no sense. Having spent time in universities (Some say too much time) and having been a teacher, I was aware that students don't measure the same way professors do. The measure should be what the student has learned and not whether the student likes the professor. But that's not what ratings do.
Why bring this up to PR practitioners? Because there are PR metrics that make no sense. They look good on paper, but they don't tell one anything. Take, for example, clip counts. Some clients still demand to know how many clips they are getting without reference to the quality of the clip and what the clip says. They forget that one right story in the right place is worth much more than envelopes full of squibs. This is particularly true in an era of media transformation when newspaper circulations continue to drop.
The ultimate PR metric is awareness and perception change among target publics. That requires something more than media metrics but few have ever wanted to pay for finding this information out. So, they resort to other measures and usage windage to make a guess.
I have long felt sorry for professors whom students dislike but are effective teachers nonetheless. I remember one fellow who taught statistics at business school. The students detested him because of his accent and his way of explaining how to look at numbers. I by then had completed four courses in stat and was bored by the same lame way it was taught. This fellow struck me as a professor who understood statistics more than most and was pointing to common-sense ways to think about them. He was the best statistics teacher I ever had. Yet, he sat at the bottom of student rankings.
It is hard to remember that the only good measurement is what counts in the end.
Why bring this up to PR practitioners? Because there are PR metrics that make no sense. They look good on paper, but they don't tell one anything. Take, for example, clip counts. Some clients still demand to know how many clips they are getting without reference to the quality of the clip and what the clip says. They forget that one right story in the right place is worth much more than envelopes full of squibs. This is particularly true in an era of media transformation when newspaper circulations continue to drop.
The ultimate PR metric is awareness and perception change among target publics. That requires something more than media metrics but few have ever wanted to pay for finding this information out. So, they resort to other measures and usage windage to make a guess.
I have long felt sorry for professors whom students dislike but are effective teachers nonetheless. I remember one fellow who taught statistics at business school. The students detested him because of his accent and his way of explaining how to look at numbers. I by then had completed four courses in stat and was bored by the same lame way it was taught. This fellow struck me as a professor who understood statistics more than most and was pointing to common-sense ways to think about them. He was the best statistics teacher I ever had. Yet, he sat at the bottom of student rankings.
It is hard to remember that the only good measurement is what counts in the end.
Comments:
Post a Comment