Mon 22 Aug 2005
We all know there is a certain arbitrariness to the US News and World Report College Rankings. The selection of criteria and the weighting of the criteria are idiosyncratic. Alternative ranking exist. For instance, some clever economists ranked colleges by the head-to-head choices made by high school seniors). Well, the liberal Washington Monthly weighs in with its own rankings emphasizing public service:
From this starting point, we came up with three central criteria: Universities should be engines of social mobility, they should produce the academic minds and scientific research that advance knowledge and drive economic growth; and they should inculcate and encourage an ethic of service.
The authors are forthcoming with the limitations of the methodology and data (part of their purpose is to encourage universities to release data — something David Kane can fully support). The rankings of universities is radically different from US News and World Report. Here are the top ten universities: 1) MIT; 2) UCLA; 3) UC-Berkeley; 4) Cornell; 5) Stanford; 6) Penn State; 7) Texas A&M; 8) UC-San Diego; 9) U Penn; 10) University of Michigan.
The rankings for liberal arts colleges are not radically different: 1) Wellesley; 2) Wesleyan; 3) Bryn Mawr; 4) Harvey Mudd; 5) Fisk; 6) Amherst; 7) Haverford; 8) Wofford; 9) Colby; 10) Spelman.
Williams comes in at #14.
Williams, which U.S. News ranks as the top liberal arts school in the country, wound up at #14 on our list, one slot below Presbyterian, largely because of its weak service numbers.
One methodological irony is how Washington Monthly measured service. Numbers on teachers and government employees are not readily available. However, ROTC numbers for each college are easy to find. So schools with active ROTC programs are ranked more highly than they might be otherwise. Again, I think David Kane might fully support an expansion of the Williams ROTC program.
This might be the only source of agreement between David and Washington Monthly.
August 22nd, 2005 at 10:49 am
Note that Professor Marc Lynch also has commentary on this.
I do not support the “expansion” of the Williams ROTC program. Unless I am seriously misinformed, there is no Williams ROTC program, nor has there been one for 25 years. I do support the efforts of others to bring ROTC back to elite campuses. But Williams is just not big enough to support a program. I am eager to see more Ephs consider serving in the military, both because it would be good for the country, but also because it would be good for them.
I wonder what Williams rank would be if the Washington Monthly had accurate data on military service.
August 22nd, 2005 at 11:05 am
Although I hold no brief for Washington Monthly and, if I knew something about it, might not approve of it at all, a little more humility concerning college rankings might be salutary for most of us, your correspondent excepted of course.
August 22nd, 2005 at 11:50 am
It has been said that Harvard is #1 in Asia, and that Williams is #1 on Wall Street. Turning out investment bankers does a good deal more for America and the world than promoting hippie-dippie public service as WesTech apparently does.
I think being low on this list is a good thing.
August 22nd, 2005 at 12:13 pm
WM used the number of PhD.s granted in science and engineering as a measure of quality. Most liberal arts colleges don’t grant any PhDs. I wonder if #3 Bryn Mawr (my wife’s alma mater), which does grant PhDs in a limited number of fields through its Graduate School of Arts and Sciences, got a big boost for that reason. Do any other “liberal arts colleges” listed there grant PhDs?
The point about not including teachers is a good one, too. Williams seems to turn out more than our fair share of teachers, though I doubt Washington Monthly sees the education of new generations of prep school students as a public service.
August 22nd, 2005 at 12:34 pm
Who says that Harvard is #1 in Asia and Williams is #1 on Wall Street? Not that I entirely disagree with the sentiment, but I would find it highly surprising if that coinage has any purchase outside the Purple Valley.
Stats geeks might take interest in how Washington Monthly calculated the percentage of pell grantees who graduate. They used “regression” (I’m guessing OLS) to predict the expected graduation rate for the school based partly on the percentage of students receiving Pell Grants. They then used the difference between the predicted graduation rate and the actual graduation rate (i.e., the residual) to estimate the percentage of Pell grantees who graduate.
I used predicted propensity scores in one of my papers. Determining precisely what you are measuring can be a tricky matter. In this instance, I can think of a couple of potential problems:
1) Suppose pell grants are not the best predictor of a school’s overall graduation rate and that another variable is causing low graduation rates (say, high school GPA). If a school is good at getting these students to graduate, then it might appear to be graduating Pell Grantees when it is not.
2) Suppose Pell Grants are one of the best predictors of a school’s overall graduation rate (a scenario I think likely). Some schools may be very good at admitting poor students who are exceptional students and likely to graduate. So the unexpectedly high graduation rate has less to do with a school’s ability to offer instruction, and more to with the ability to cherry pick good students.
3) The precision of the modeling equation needs to be high, otherwise pathologies immediately appear. For instance, suppose a school looks for all the world like a school whose students are unlikely to graduate. However, the administration only can take the Pell grant money as long as the student is enrolled (and I’m not sure how long a student is eligible for Pell grants), so they maintain very low standards to pass classes and graduate. Using this methodology, the graduation rate is taken as a signal of quality, when really the opposite is true.
There are measurement errors with the other two criteria, but they don’t strike me as so problematic. The other criteria simply capture one facet of the issue (e.g., Peace Corp service) rather than another (e.g., teaching or volunteer work or being a decent neighbor).
August 23rd, 2005 at 1:44 pm
I think the lack of civil-service data is especially damning for Ephs…many of my friends here in DC work for the government, in some capacity. Even more work for non-profit organizations.
The particular methodology used seems to be saying that campuses with ROTC chapters are more “service-oriented” than colleges who send all of their graduates into low-paid civil servant positions, or, god-forbid, the non-profit sector. There’s something a little skewed about that. People who work for the State Department are just as service-oriented as those who work for the Peace Corps. In fact, they don’t get as much recognition for their service!
The solution, however, isn’t for the government to somehow track all of its incoming hires so that they can be searched by college. Instead, colleges themselves should release their alumni data concerning this.