Wed 24 Oct 2007
Are you thinking about becoming a professor? Even if you aren’t, you probably know a little bit about the tenure process . . . five years of work before a review by the peers of your department, ending either in heartache or a pay bump and a permanent place in your ivory tower. But did you know that formal, quantitative measurements exist to assess a scholar’s contribution to the world? There is even a program to mine the knowledge bases for data and turn them into a “g-index” or “h-index.”
Publish or Perish is a “a citation analysis software program, designed to help individual academics to present their case for research impact to its best advantage.” Using Google Scholar, the program tabulates all citations of the queried paper and can turn the data into the academic-worth statistic of your choice (apparently there are many, and none older than 2005).
The site also has a set of great FAQs providing a good, and clearly knowledgeable, background on associated topics.
I know it sounds like I’m promoting the site or something, but I’m just remarking on a practice to which I was naive, one that seems at once reasonable, strange, and slightly appalling. Is it commonly known that this sort of assessment is used? Is this new only to me? Are there other such tools out there?
I am left with a few thoughts
- I feel a lot less bad about helping promote Factrak, the site where Williams students rated Williams faculty online. (actually, I never really felt bad about it)
- David’s repeated insistence that just about anything can be quantitatively measured is seeming a lot more reasonable.
- Making the focus of your research the devising of an index for the quantitative measurement of tenure-worthiness seems a little too self-parasitic for my taste.
October 24th, 2007 at 8:27 am
Interesting stuff. Could someone please post the results for various Williams professors? (Feel free to include me as well since I am a wanna-be academic.) It would be special fun to look at some of the people who have made news at EphBlog: Morty, Cheryl Shanks, Aida Laleian, Mark Taylor, and so on.
October 24th, 2007 at 10:08 am
The more common database to reference is ISI’s Web of Science. Web of Science has a data base of every reference made in every article in their defined set of journals. This allows them to create citation counts for professors and journal rankings.
Citation counts are a good measure of the degree to which the field has taken note of your work. The counts themselves are objective, so supporters of a professor’s candidacy will marshal the counts in defense.
The weaknesses of the citation counts are:
a) They don’t differentiate between positive and negative citations (i.e., attacks are the same as praise).
b) They don’t count references in books or syllabuses.
c) An article or book might be very high quality, but ignored.
d) ISI references a broad array of journals (and nearly all of the top ones), but it does not include all of them by any stretch of the imagination.
Publish or Perish seems to be an alternative for people who do not score highly in Web of Science. The idea is roughly the same, it just uses Google Scholar rather than an index of thousands of journals. I guess the strength is that you would pick up on working papers referencing work. The downside is that you will double count a citations (a single paper presented at three or four seminars would count as 4 citations). I would be surprised if deans will pay attention to these numbers when judging tenure cases.
It should also be noted that citation counts are relatively less important for tenure. Tenure at most schools is six years. It takes a while for articles to be published, read, incorporated into working papers, which then need to get published themselves. There is a definitely a lag and deans know this. Citation counts are far more important in promotion from associate to full professor or when making senior hires.