the insanity of college rankings

The very best book to read on the subject is Paul Boyer's College Rankings Exposed. It should be available right next to copies of U.S. News and World Reports annual College Rankings issue (what those at USNWR refer to as their swimsuit issue.) In a recent AP piece (Colleges Struggle to Quit the Rankings Habit) Brian Kelly, editor of USNWR, is quoted as saying that its annual rankings of colleges and universities simply provide "consumers" -hungry for hard data- with the same sort of rankings offered for cars, refrigerators, and the like. This is very telling. Are the families who enroll their students in college really consumers? There is a great deal of data on marriage and the sort of characteristics that contribute to a man becoming a good marriage partner. Weddings are, like college, a major expense. USNWR could rank cities across the U.S. according to the quality of its me. Part of these rankings could be based on what thsoe performing marriage ceremonies in other cities have heard about men in a particular city. In the case of some cities they might have direct knowledge -current or not-while in others they may have little or no information at all. We could then place a great deal of weight on these scores regardless. Families with these rankings in hand could make a much more informed decision about what city is the best one to go to if you wish to locate a husband. Einstein is quoted as saying that "everything that can be counted does not necessarily count; everything that counts cannot necessarily be counted". And this is at root the point. Just as there is not one universally valid criteria for selecting someone with whom you will share your life with there is not a universally valid criteria for selecting a college or university. While we can all agree on a fairly limited list of factors that make one big screen TV better than another or, similarly, one set of tires better than another how on Earth can it be argued that there is a relatively finite list of factors that determine waht constututes a good school? Consider also that when it comes to rankings that the most elite schools are so because they serve, as they always have, the elite. Socioeconomic status is a powerful deteminant of academic achievement. Highly ranked schools are largely populated by students from "highly ranked families"; the well-to-do and the powerful. Wealthy alumni can afford to make large contributions. Large contributions together with tuition revenues mean that such schools are able to recruit and retain faculty at relatively high salaries. It also allows them to ask faculty to carry light enough teaching loads that permits success in both teaching and research. Finally having already completed their primary and secondary education at exceedingly good schools, students come far readier for the rigors of higher education. To have completed one's undergraduate career at a highly ranked college or university is more often than not symptomatic of an exceedingly advantaged background rather than having achieved some thing spectacular through hard work, perseverence or exceptional intellectual gifts. For "lesser" institutions to climb the summit means drawing students from such backgrounds. The likelihood of their being able to attract a critical mass of students from such advantaged backgrounds to materially change their fortunes is small at best. Especially for small liberal arts colleges -modelled after top liberal arts colleges from the east coast- founded over a hundred years ago by immigrant communities throughout the U.S. the chances are small. The alternative is to encourage their faculties to develop a more national profile (and, via slick marketing to make these acheivements known to a wider national audience.) in the hopes of raising the heavily weighted peer review in thge USNWR rankings. The fact is that the top ranked schools, thriving in a very different environment, are unlikely to pay much attention. (Check out this story from NPR's Morning Edition September 15 2006, "Marketing Higher Education Gets Sophisticated" ) And as the following illustrates, deans and presidents of competing institutions are unlikely to be swayed by suchinformation no matter how impressive it might actually be. Say that you have two Deans of two colleges, A and B, who are asked by USNWR to rank one another. A can given B a rank of one (a low ranking), a five (a high ranking) or give B a rank of three. B has the same set of choices as far as A is concerned. The deans of colleges A and B may dismiss out of hand the notion of giving their competitor a rank of one. Both Dean reconginize that truly miraculous things are happening at the other college and so briefly consider giving the other college a ranking of 5. What stops either from doing so is the following. Consider college A. If its Dean gives B the ranking it deserves and B does not reciprocate but instead gives A a ranking of 3 (the rank it received in last year's USNWR) then A will drop relative to B in the rankings. Of course if B gives A a rank of 5 then, relative to B, A's position is unchanged. A family whose student is considering both colleges A and B will note that A and B are both highly ranked if the latter happens and that A has a much lower rank if the former happens. Regardless of what B does then, the worst thing that can happen to A if its Dean gives B a ranking of 3 is that its relative ranking will remain unchanged. However , if A's Dean gives college B the ranking that she feels it truly deserves and B does not return the favor the worst thing that can happen is for college A's relative position to fall 2 points. Now, a student who might have considered college A or B will see a ranking of 3 for A and of 5 for B and choose B. It follows from this that both Deans will choose to award a rank of 3 to the other college. In fact, if you look at peer rankings for liberal arts colleges in last year's USNWR -outside the top fifty schools- peer rankings do indeed cluster around three. The end result is a tremendous amount of effort and time are expended to change a situation that look at the problem strategically is unlikely to change. And the peer ranking is important. It is given a great deal of weight in USNWR's overall ranking. In frustration, colleges respond by putting more resources into encouraging students and faculty to perform specatacular feats as well as trying to tweak other numbers that would also help push the overall rankings higher. The most infamous such strategy has been the practice of sending rejection letters to students who appear to have included a given school as a back-up school. (See "Glass Floor: Colleges Reject Top Applicants,Accepting Only the Students Likely to Enroll" by Daniel Golden, WALL STREET JOURNAL, May 29, 2001.) When students either failed to make the cut at their preferred school but were also turned down by the "lesser" school or worse still, they were accepted at their preferred school but turned down by the "lesser" school it became readily apparent that admissions policy was in part being driven by the desire to lower that college's acceptance rate, and, to a degree, bolster its USNWR rankings. Passing over students with perhaps greater potential but with lower ACT/SAT scores (fn1), putting money into campaigns to boost alumni contributions and failing to accept students probably heading elsewhere anyway all have been done in an effort to boost rankings. Silly as these things are, it is quite impossible for a college to unilaterally opt out. To do so would be suicidal. That is what is so heartening about the agreement reached earlier this summer by the Annapolis group. As with all such agreements the risk of defection is high. However, the only way out of such a bad situation is through such a cooperative agreement. While I am sympathetic to the notion that families and their students need help in making the decision about where they are going to go to school. It is not an easy choice. What Vasaar College's President Catherine Bond Hill suggest as an alternative to USNWR's rankings is an online database where students and families would have access to data and could construct school rankings whose value would be based on weights that they choose (see "A Better Way to Rank Colleges" .) One of the key things one should understand to their maerrow after graduating from any institution of higher learning that is worth its salt is that there is seldom one universally valid answer to important questions. Perhaps that lesson should start with the college selection process itself. fn1: See "Paving a Path for Poor Students' College Dreams" where reporter Wendy Kaufman notes that tuition has continued to rise and financial aid and scholarship money has not even more schools seeeling to boost rankings "...are emphasizing schoalrships based on high grades and test scores, not financial need")