EHRENBERG / U.S. News & World Report Rankings 145 The Review of Higher Education Winter 2002, Volume 26, No. 2, pp. 145–162 Copyright © 2002 Association for the Study of Higher Education All Rights Reserved (ISSN 0162-5748) Reaching for the Brass Ring: The U.S. News & World Report Rankings and Competition Ronald G. Ehrenberg The American system of higher education is the envy of the rest of the world. A mixed system of over 3,600 public and private institutions, it is noted for its competitiveness. An institution’s geographical location, selec- tivity, size, whether it is church related, the degrees that it offers, and the range of its curriculum, determine the specific institutions that are its com- petitors. Against these competitors, institutions vie in a variety of ways— for undergraduate, graduate, and professional students, for faculty members, for research dollars, for state and federal appropriations, for private philan- RONALD G. EHRENBERG is the Irving M. Ives Professor of Industrial and Labor Relations and Economics at Cornell University and Director of the Cornell Higher Education Re- search Institute (CHERI). He presented an earlier version of this paper at the Macalester Forum on Higher Education on “Comparative Advantage and Common Purpose in Higher Education, Macalester College, June 2001. He expresses appreciation to the Andrew W. Mellon Foundation and the Atlantic Philanthropies (USA) Inc. for their support of CHERI, to Michael McPherson, and to two reviewers for their comments. His most recent book is Tuition Ris- ing: Why College Costs So Much (Cambridge, MA: Harvard University Press, 2002). His cur- rent research interests focus on public higher education, higher education governance, the growing cost and importance of science to universities, and the implications of the growing dispersion of resources across academic institutions within both the public and private higher educational sectors as well as between the two sectors. Address queries to him at Cornell University, 256 Ives Hall, Ithaca, NY 14853-3901; telephone (607) 255-3026; fax: (607) 255- 4496; e-mail: rge2@cornell.edu. 146 THE REVIEW OF HIGHER EDUCATION WINTER 2002 thropy, and for other sources of revenues such as those generated by dis- tance-learning activities and the commercialization of faculty members’ re- search. Institutions may have different competitors along different dimensions. For example, Cornell University competes directly with a much smaller number of institutions for students than it does for faculty. At the same time they are competing, academic institutions also under- stand that their resources are limited and that there is much to be gained by collaborating with these peers. For example the five academic institutions in the Pioneer Valley region of Massachusetts (Amherst College, Hamp- shire College, Mt. Holyoke College, Smith College, and the University of Massachusetts) have long permitted students from one institution to take courses at any of the other four, thereby increasing the range of courses that students from each institution can access. The behavior of academic institutions, including the extent to which they collaborate on academic and nonacademic matters, is shaped by many fac- tors. This paper focuses on one of these factors, the U.S. News & World Report (USNWR) annual ranking of the nation’s colleges and universities as undergraduate institutions, exploring how this ranking exacerbates the competitiveness among American higher education institutions. After pre- senting some evidence on the importance of the USNWR rankings to both public and private institutions at all levels along the selectivity spectrum, I describe how the rankings actually are calculated, then discuss how aca- demic institutions alter their behavior to try to influence the rankings. While some of the actions an institution may take to improve its rankings may also make sense educationally, others may not and, more importantly, may not be in the best interest of the American higher educational system as a whole. In the final section of the paper, I ask whether the methodology that USNWR uses to calculate its rankings prevents institutions from collabo- rating in ways that make sense both educationally and financially. My an- swer is, by and large, no, although I indicate that USNWR could encourage even more such collaborations by fine-tuning its rankings system. In short, although USNWR rankings cause institutions to worry more about the peers with which they compete than would otherwise be the case, the rankings should not prevent institutions from working productively towards com- mon goals. Put another way, USNWR is not the “evil empire” and academic institutions should not blame USNWR for their failure to collaborate more. IMPORTANCE OF THE USNWR RANKINGS In a relatively short period of time the USNWR annual ranking of the nation’s colleges and universities as undergraduate institutions has become the “gold standard” of the ranking business. Perhaps this occurred because EHRENBERG / U.S. News & World Report Rankings 147 the USNWR ranking has the appearance of scientific objectivity (institu- tions are ranked along various dimensions with explicit weights being as- signed to each dimension). Perhaps this occurred because institutions at the top of each category—for example, the top 50 national universities— are ranked numerically within their categories; and the American public wants to know which institution is number one. Because of year-to-year changes in their rankings on the various dimen- sions that USNWR considers or year-to-year changes in the weight that each dimension is given in computing the overall ranking, an institution’s ranking within its group may change from year to year. It is the change in the numerical rankings of institutions near the top of each institutional category, as well as the changes in the quartile rankings of some lower-ranked institutions from year to year that sells lots of copies of magazines. After all, if the ranking of institutions did not vary over time, there would be no need for families to have the most recent year’s issues. College and university presidents repeatedly publicly pronounce that the USNWR rankings are not a measure of the quality of their institutions, that an institution’s quality cannot be measured by a single number, that changes in an institution’s rank are often due to USNWR’s periodically changing the way the rankings are calculated, and that their university does not pay any attention to the rankings. Increasingly sensitive to the criticism that its rankings have received, USNWR has appointed a college advisory board that annually suggests changes in its methodology. While such changes in- variably lead to changes in the rankings of institutions and to the sales of more magazines, in one of its recent rankings USNWR also explicitly ad- vised readers: “Since we may change our methodology from year to year, we do not invite readers to track colleges annual moves in the rankings” (Morse & Flanagan, 2000, p. 28). However despite the pronouncements of the presidents and the recent advice of USNWR, the rankings of institutions do matter and the institu- tions are very concerned about what the rankings show. And well they should be! A recent study by James Monks and me (Monk & Ehrenberg, 1999) that focused on top national private universities and liberal arts colleges found, other factors held constant, that when an institution improves in the rankings the next year it receives more applicants, can accept fewer of them, sees a greater proportion of its accepted applicants enroll, shows an improvement in the average SAT scores of its enrolled admitted applicants, and can re- duce the amount of institutional grant aid that it spends to attract its class. For example, with other factors held constant, improving 5 positions in the ranking was associated with a drop in the institution’s admit rate of about 2.0 percentage points and a yield (percentage of those accepted who actu- ally enroll) that increased by almost 1 percentage point. In contrast, when its rankings worsen, just the opposite occurs. Other factors held constant, 148 THE REVIEW OF HIGHER EDUCATION WINTER 2002 the next year the institution receives a smaller number of applicants, must admit a greater fraction of them, has a smaller fraction of its admitted ap- plicants enroll, sees a drop in the average test scores of its enrolled admitted applicants, and must increase the amount of institutional grant aid that it spends to attract its class. In short, changes in an institution’s USNWR rankings affect measures both of its academic quality and its financial aid bill. A change in a selective private institution’s ranking greatly complicates the life of its administrators in charge of enrollment management. The Monk/Ehrenberg study dealt with only selective private colleges and universities. Even if critics accept our findings at face value, they might ar- gue that it is only the applicants to the most selective of our nation’s private colleges and universities that pay attention to the USNWR rankings. Inas- much as the vast majority of American college students are educated at public institutions or at less selective private institutions, critics might con- tinue that it is only a small minority of academic institutions that pay at- tention to the USNWR rankings. Moreover, they might add that USNWR publishes precise numerical rankings for only the top 50 or so institutions in a category; other institutions are grouped into broad tiers. So they might ask how the USNWR rankings could be relevant for any but the most selec- tive private institutions. Anecdotal evidence from several private institutions that immediately follows suggests that an institution’s USNWR ranking is important to pri- vate institutions regardless of which selectivity tier they occupy. Subsequent sections will provide similar evidence for public institutions in several se- lectivity tiers. This evidence suggests that private and public institutions at many levels of student selectivity do worry about their USNWR rankings. For example, Hobart and William Smith Colleges had long been ranked in the second tier (quartile) of national liberal arts colleges. When the 2001 USNWR rankings were published in the fall of 2000, the institution found itself ranked in the third tier (quartile), because a senior administrator at the institution failed to report current year data to USNWR. USNWR was forced to compute the institution’s ranking using year-old data, which un- derstated the institution’s current performance on a number of dimensions. This fall in the rankings occurred even though Hobart and William Smith Colleges was ranked higher on academic reputation by academic leaders than 18 of the 29 national liberal arts colleges placed in the second tier in the 2001 rankings. Needless to say, academic reputation is a critical compo- nent of the ranking methodology (Brownstein, 2000). The administrator was fired and Hobart and William Smith’s recently appointed president took vigorous steps to repair the damage. After several discussions with the institution, USNWR sent Hobart and William Smith a letter saying that, if the correct data had been reported, Hobart and Will- iam Smith likely would have been ranked in Tier II for that year. The college EHRENBERG / U.S. News & World Report Rankings 149 promptly distributed this letter and the reason for the institution’s errone- ous fall in the rankings to guidance counselors at high schools that were regular feeder schools to the institution. Concerned that the morale of cur- rent students (which might affect retention), of faculty, and of alumni might also fall, the president and the trustees also vigorously conveyed this mes- sage to all of the college’s constituents. A new president clearly felt that his institution’s USNWR ranking was critical to the institution and that he had to spend a good part of his first months in office on damage control.1 A second example comes from a small liberal arts college located in a Middle Atlantic state. (By agreement, its name cannot be disclosed.) Ranked in Tier 4 (the bottom quartile) of national liberal arts colleges, the college actively explored ways to improve its USNWR rankings as part of its strate- gic planning process. Because an institution’s average faculty salary has sig- nificant weight in the ranking formula, an early draft of a strategic planning document called for an increase in average faculty salaries to improve the institution’s ranking. The fact that an academic institution considered its USNWR ranking in its strategic planning process confirms the importance of the rankings in the institution’s perception of its external image. However, the document did not discuss whether raising faculty salaries would be expected to im- prove the quality of faculty members that the institution could attract, im- prove retention of existing faculty members, or improve faculty members’ performance. Similarly, the document did not discuss whether the funds that would be spent on increasing faculty salaries could be better used in other ways to actually improve the educational experience of its students. The college was planning to spend more with a single goal in mind: an improved USNWR ranking. I mentioned how strange this seemed to me to a member of the college’s board and am happy to report that the final ver- sion of the college’s strategic plan made no mention of spending more to improve rankings. It is clear, however, that the trustees of the institution believed that enhancing the institution’s USNWR ranking would allow the institution to attract more and better applicants and to improve its finan- cial position. THE 2001 USNWR METHODOLOGY USNWR divides academic institutions into categories based on the 1994 Carnegie classification of colleges and universities (Carnegie, 1994). Carnegie 1Conversation with President Mark Gearan of Hobart and William Smith Colleges on October 20, 2000. Hobart and William Smith returned to the second tier of national liberal arts colleges in USNWR’s 2002 rankings. 150 THE REVIEW OF HIGHER EDUCATION WINTER 2002 Research I, Research II, Doctoral I and Doctoral II institutions are all in- cluded in the USNWR National University category. Carnegie Liberal Arts I institutions are included in the USNWR National Liberal Arts College cat- egory. All Carnegie Comprehensive I and Comprehensive II institutions are reported by quartile rankings in the USNWR Regional University cat- egory, with the country divided into four regions for reporting purposes. Finally, the Carnegie Liberal Arts II institutions are all classified as USNWR regional colleges, with quartile rankings again reported for the four regions. The USNWR methodology considers seven broad categories of measures that relate to college quality: academic reputation, student selectivity, fac- ulty resources, graduation and retention rates, financial resources, alumni giving, and graduation rate performance. The weights assigned to each of these broad areas vary across USNWR categories, as does the number of factors included in each category and the weight assigned to each factor. The formulae for national universities and national colleges are identical but differ from the formulae for regional universities and regional colleges. For ease of exposition, I will focus on the national college and university formula below. Table 1 summarizes the ranking criteria and weights used in America’s Best Colleges: 2001. Academic Reputation The first criterion that USNWR uses is the institution’s academic reputa- tion; this factor gets a weight of 25%. USNWR gives this criterion such a weighting because it recognizes the role of an institution’s academic repu- tation in gaining high-paying jobs and admission to top graduate and pro- fessional schools for its graduates (Brewer, Eide, & Ehrenberg, 1999; Eide, Brewer, & Ehrenberg, 1998). USNWR surveys presidents, provosts, and deans of admission of schools in each category for information on academic repu- tation. They are asked to rank each institution in their category on a scale of 1 (marginal) to 5 (distinguished), or to indicate that they are unfamiliar with the institution. USNWR reported a response rate of 67% for its 2002 survey. Many presidents and provosts at major institution refuse to fill out this survey because they do not like participating in a “beauty contest.” Others may realize that it is in their best interest to rank institutions close to them in selectivity very poorly, although this clearly would not be an ethical way for senior academic administrators to respond. Student Selectivity The second criterion that USNWR uses is student selectivity, weighted at 15%. Student selectivity is assumed to have four components. The smaller the fraction of freshman applicants that a school accepts, the more selective it is assumed to be. Similarly, the higher the yield rate, the more selective it EHRENBERG / U.S. News & World Report Rankings 151 TABLE 1 CRITERIA AND WEIGHTS USED IN USNWR 2001 RANKING OF UNDERGRADUATE INSTITUTIONS Ranking Category Category Subfactor Subfactor Weight Weight Academic reputation 25% Academic reputation 100% survey Student selectivity 15% Acceptance rate 15% Fall ’99 entering class Yield 10% High school class 35% standing top 10% High school class 0% standing top 25% SAT/ACT scores 40% Faculty resources (’99) 20% Faculty compensation 35% Percent faculty with top 15% terminal degree Percent full-time faculty 5% Student/faculty ratio 5% Class size, 1-19 students 30% Class size, 50+ students 10% Graduation and retention rate 20% Average graduation rate 80% Average freshman 20% retention rate Financial resources 10% Average educational 100% expenditures per student Alumni giving 5% Average alumni giving rate 100% Graduation rate performance 5% Graduation rate performance 100% Source: America’s Best Colleges, 2001 (Washington, DC: U.S. News & World Report, 2000). is assumed to be. Student selectivity also is assumed to be positively related to the proportion of its freshman class that were ranked in the top 10% of their high school classes and the average SAT (or ACT) scores of its enter- ing students. There are numerous ways that an institution can influence its acceptance rate and yield rate. The president of one midwestern flagship public univer- sity which consistently ranks in the top tier of national universities, told me that his institution currently discourages applicants who have little chance of being accepted at the institution from applying. However, repeatedly ex- pressed concerns from trustees about the institution’s not being ranked high 152 THE REVIEW OF HIGHER EDUCATION WINTER 2002 enough within the top 50 group of USNWR national universities may force his admissions office to actively solicit more applications from lesser-quali- fied students. The result will be added expenses for the admission process and more unhappy rejected applicants throughout the state. A private college president told me about an applicant to his institution who was the son of an influential alumnus. The student had also applied to a large northeastern public university that is currently ranked among the second tier of national universities by USNWR and is aggressively trying to improve its USNWR ranking. Although the applicant’s credentials (test scores, rank in class, grade point average) were far above the typical applicant’s credentials at the public institution, it rejected his application. However, the alumnus reported to the private college president that his son also received a letter from an admissions officer at the public institution indicating that, if the applicant would declare his intention to attend if ad- mitted, the public university would admit him. Clearly this behavior is de- signed to lower the institution’s acceptance rate and improve its yield, but one must wonder how ethical it really is. Lest one think that it is only the publics that would stoop to such behav- ior, a recent Chicago Sun Times article reported that Franklin and Marshall College rejected 140 of its top applicants for the class of 2005, relegating them to its waiting list, because it believed that they really wanted to attend more selective Ivy League institutions (Golden, 2001b). In the process, it and a number of other private institutions that are behaving in a similar manner have increased high school students’ uncertainty, as the concept of a “safety school” has become much less certain.2 A simple way to improve an institution’s yield and lower its acceptance rate, which many selective institutions have aggressively pursued, is to in- crease the proportion of an institution’s class that is admitted through an early-decision process. In this process, an applicant applies to one institu- tion by early November of the applicant’s senior year in high school. At most institutions, the applicant is required to sign a statement to the effect that he or she will enroll if admitted and will withdraw any other applica- tions that have been submitted. The institution typically gives the applicant a decision by mid-December, which takes the form of an acceptance, a re- jection, or a deferral of the applicant to the regular decision process. If ad- mitted, the applicant has a brief period of time within which to notify the institution that he or she has accepted the offer of admission and with- drawn applications pending at other institutions. 2According to Golden (2001a), Franklin and Marshall is only one of a number of private institutions that include, as part of their admissions criteria, their estimates of the probabil- ity that a student will actually enroll if admitted. EHRENBERG / U.S. News & World Report Rankings 153 Expanding the fraction of an institution’s class admitted by the early- decision route lowers the institution’s acceptance rate and increases its yield because early-decision applicants accept offers of admission with a prob- ability of close to one. Thus, to attain any given size class, the institutions can admit fewer students overall. Consequently, expanding early admis- sions relative to one’s competitors will improve one’s USNWR rankings.3 USNWR penalizes institutions that pursue alternative policies, such as early action (early notification of students of their acceptance but no require- ment that the student commit to the institution early) or rolling admis- sions. Enrolling students through an early-decision process benefits both insti- tutions and applicants. Institutions most frequently identify as a benefit having its freshman class composed of students for whom the institution was their first choice. Some students who enroll through the regular admis- sion process may have been rejected from their first-choice schools and ar- rive at their second choice with less enthusiasm.4 This factor may affect their overall attitude towards the institution and also their likelihood of persisting at the institution until graduation. Even if they do graduate, they are less likely to be attached alumni who devote time and money to helping the institution to prosper. Put simply, an institution benefits greatly from enrolling students who really want to attend it. Students who are sure about which school is their first choice also clearly benefit from the early-decision process. Being admitted by the early-decision route allows them to avoid the tensions associated with waiting until their senior spring to learn whether they will be admitted to their first-choice school. 3Suppose that an institution initially has 10,000 applicants, admits 4,000, and enrolls 2,000. Its acceptance rate is 40% (4,000/10,000) and its yield is 50% (2,000/4,000). Now suppose that it offers an early-decision option, 2,000 of its applicants choose this route and it accepts 1,000 of them, all of whom enroll. If the yield rate on its regular applicants re- mains at 50% to attain a class of 2,000, it only needs to admit 2,000 students from its pool of 8,000 regular applicants and the early decision applicants that it has deferred. Hence, its acceptance rate will fall to 30% (3,000/10,000) and its yield will rise to 67% (2,000/3,000). 4An example from my own university illustrates this point. Cornell’s Arts and Science and Engineering colleges’ entering freshmen have the highest average test scores in the uni- versity. They also have among its lowest yield rates because there are many high quality engineering and arts and science colleges in the nation. For many of the students enrolled in these colleges, Cornell was not their first choice. In contrast, the yield rates at Cornell’s School of Industrial and Labor Relations and its Hotel School are among the highest in the univer- sity because these schools are the preeminent places in the nation to study their subject matters, and most of the students enrolled in them are attending their first-choice institu- tion. On average, students from the latter colleges rate their Cornell experience better than students from the former colleges. Granted, the student body is also larger in the former two colleges, which decreases student satisfaction. 154 THE REVIEW OF HIGHER EDUCATION WINTER 2002 A second reason that institutions like early-decision applicants, although this is rarely mentioned publicly, is that early-decision applicants are more likely to come from upper or middle-income families and thus require less institutional grant aid than typical applicants.5 The intuitive explanation is that applicants who are very concerned about financial aid will apply to several institutions to see which one gives them the best offer. An early-decision applicant eliminates this possibility for himself or herself. Hence, increasing the proportion of the class enrolled by early decision is a strategy that insti- tutions can use to dampen the growth rate of their financial aid budgets. Of course, to the extent that financial need is correlated with race and ethnicity, early admission applicants and acceptances are likely to be more heavily white and Asian American than the institution’s total applicant pool (Avery, Fairbanks, & Zeckhauser, 2001). Being admitted for upper- and middle-income White and Asian American applicants will probably be higher in the early-deci- sion pool than in the regular decision pool. Increased awareness of this ten- dency among high school students, parents, and guidance counselors will put additional pressure on students to opt for early-decision applications. Another factor intensifying this trend is that some institutions now explic- itly tell applicants that they give preference to early-decision candidates. Consequently, to the extent that an institution values socioeconomic and racial/ethnic diversity, it will have to give extra attention in its regular ad- mission process to underrepresented minority and lower-income candi- dates. In fact, when the University of North Carolina did not enroll a first-year class for 2002–2003 which was sufficiently representative of North Carolina’s lower-income and minority populations, it announced that it was eliminating its early-decision program, making it the first major uni- versity to do so (Flores, 2002). Public universities face much severer politi- cal pressures to enroll a representative classes than private universities. Thus, while Richard Levin, president of Yale, proposed in 2001 that selective pri- vate colleges and universities as a group agree to abandon or reduce their dependence on early-decision applicants, to date no private institution has (Hoover, 2000). USNWR is thus contributing to the pressures that institutions and stu- dents face to expand the early-decision application process and it is not clear that this is in either the institutions’ or the applicants’ interest. (See Roth and Xiaolin, 1994, for this tendency of markets to “unravel,” with trans- actions occurring earlier and earlier.) More high school students will be making decisions on where to apply earlier in their high school career, and these decisions may not be informed by as much information on institu- tional characteristics as the students should have. They will be forced to 5Avery, Fairbanks, and Zeckhauser (2000) provide supporting evidence from 14 selective private colleges and universities during 1991–1997. EHRENBERG / U.S. News & World Report Rankings 155 worry about higher education options earlier in their high school career before many of their interests are fully formed, thus putting unnecessary extra pressure on them. All students contemplating early-decision application to a selective pri- vate college or university will face very difficult decisions. Should one make early-admission application to the institution that one really desires or to a slightly less selective institution? It will be more difficult to be admitted to the slightly less selective institution during the regular admissions process than during the early-decision process, a fact that may discourage some students from applying to the highly selective institution that they want. If these students enroll at the slightly less selective institutions, they will be less satisfied. Even more dissatisfied will be the students who stretched and made early-decision applications to the highly selective institutions, were turned down, and then found that they could not be admitted to the slightly less selective institutions during the regular admissions process. They will be forced to enroll at even less selective institutions. As a result, it is reason- able to hypothesize that, as the proportion of early-decision admissions expands, more and more freshmen may transfer after their first or second semesters. The third criterion included in student selectivity ratings is the propor- tion of entering students who were ranked in their high school’s top 10%. Using class rank as an admission criterion is currently fashionable at public universities in a number of states, including Florida, Texas, and California. This criterion, however, penalizes students who attend tough schools with highly competitive student bodies. In addition, many high schools no longer rank their students. For example, in a recent year, 80% of the freshman students whose high school class rank was reported to Cornell were ranked in the top 10% of their classes but 35% of Cornell’s entering freshman had no rank reported. Perhaps more disturbing, the class rank variable used by USNWR is only for new freshmen who enroll in the fall of the year. Neither freshman stu- dents who first enroll in the spring or transfer students’ class rank are in- cluded. Thus institutions that are interested in expanding enrollments have an incentive to admit more January freshmen or transfers rather than ex- pand their fall freshman enrollments. Such a strategy may make financial sense for institutions with limited on-campus housing or constraints on the number of sections of widely required courses they can offer. However, it makes less sense to the admitted January freshman and transfer students who would benefit from being integrated into the institution’s community at an earlier stage of their college careers. A similar argument can be made about the use of SAT or ACT average test scores as the last criterion in the student selectivity category. A number of selective private institutions—including Sarah Lawrence, Mount Holyoke, 156 THE REVIEW OF HIGHER EDUCATION WINTER 2002 Bowdoin, Dickinson, Franklin and Marshall, and Connecticut College— have made SAT scores reports optional for their applicants. Several institu- tions made this change in the early 1990s before the USNWR rankings had achieved the prominence that they now have. However, several made the change much more recently at times when they were known to be unhappy about their USNWR rankings. In each case, the institution explained the change as prompted by concern about the possible unfairness of the test to low-income and underrepresented minority applicants, wanting to judge applicants on a much wider dimension of their accomplishments, and en- couraging high schools students to spend time on other activities than pre- paring for tests. Obviously, if the test is optional, only students who score well will report it, causing average test scores for admitted applicants that are reported to USNWR to rise. Inevitably, these institutions will see their USNWR rankings improve. In addition, because SAT scores are no longer required, students with weaker SAT scores (and, arguably, weaker academic preparation) who previously would not have applied to these institutions may do so. An in- crease in applicants will permit the institution to reduce its acceptance rate, which in turn will further improve its USNWR rankings. The magnitude of the rise in the rankings experienced by many of these institutions in the years after they made SAT score reporting optional has already been docu- mented, and these increases in rankings are not restricted to institutions that started off in the top selectivity tier (Yablon, 2001). This is not to say that it was wrong to make the test score reports op- tional. However, it is worth noting that a recent meta-evaluation conducted by researchers at the University of Minnesota concluded that SAT test scores are highly correlated with a wide range of educational outcomes at many institutions (grade point average, rank in class, persistence to graduation) so one wonders why institutions would not want to have the information on all applicants’ test scores and at least be able to consider them in their admissions process. Because the College Board financed this study, some observers have challenged its findings (Jacobson, 2001). A recent proposal by Richard Atkinson, president of the University of California system, to eliminate SAT scores in admissions decisions is not subject to the same criti- cism because Atkinson proposes replacing the SAT by a set of criteria, in- cluding SAT2 or similar tests that measure subject matter achievement (Selingo & Brainard, 2001). How USNWR would alter its rankings meth- odology if large highly selective institutions, such as Berkeley, did not con- sider SAT scores in their admissions processes is not known. Faculty Resources The third criterion that USNWR uses in its methodology is faculty re- sources, giving this criterion a weight of 20%. The factors included in it are EHRENBERG / U.S. News & World Report Rankings 157 average faculty salaries (deflated by a regional cost-of-living index), the per- centage of faculty with terminal degrees, the percentage of full-time faculty, the student-faculty ratio, and the percentage of classes with fewer than 20 students (good) and more than 50 students (bad). While no professor in his right mind, including me, will argue against having more full-time faculty, more small classes, lower student-faculty ra- tios, more faculty with terminal degrees, and higher paid faculty, the heavy weight that faculty salaries get in this criterion (35%) provides an incentive for institutions to continue increasing faculty salaries even if market condi- tion do not warrant such increases, as long as they have the resources.6 Indeed, this factor may provide part of the reason that faculty salaries in private research and doctoral universities have risen so substantially rela- tive to those in public research universities during the last 20 years (Ehrenberg, 2000, chap. 2). While salary increases for faculty at public re- search and doctoral universities have been constrained by limitations on state funding, private institutions have vigorously increased their spending to enhance their activities and their reputations. Graduation and Retention Rates USNWR’s fourth criterion is the institution’s six-year graduation rate and freshman retention rate. This criterion receives a weight of 20%. All academic leaders should be in favor of improving their students’ retention and graduation performance. However, such success can be achieved, not only by improving students’ educational experiences and financial support but also by watering down requirements for academic progress. USNWR focuses on these outcomes, not on the methods by which they are achieved. Financial Resources Per Student USNWR’s fifth criterion is financial resources per student, measured by the average educational expenditures per student and weighted at 10%. USNWR has paid considerable attention to improving this variable’s mea- surement in recent years, including giving less weight to expenditures on research and adjusting for the fraction of an institution’s students enrolled in graduate programs.7 Use of this measure places academic institutions in a very difficult posi- tion. On the one hand, they would like to hold their expenditures down to 6USNWR’s use of average faculty salaries also ignores the fact that average salaries may differ across institutions because of differences in their faculty’ field and age distributions. For example, if two institutions offer the same average salaries for faculty members in any given field, but the first hires a much larger fraction of its faculty in high-paying fields, such as law or business, the first will have a higher average salary overall than the second. 7Changes in measuring the expenditure per student variable resulted in much of the variation in the rankings that my own institution experienced in the late 1990s (Ehrenberg, 2000, chap. 4). 158 THE REVIEW OF HIGHER EDUCATION WINTER 2002 reduce increases in undergraduate tuition. On the other hand, if any insti- tution unilaterally reduces its rate of growth of expenditures per student, let alone cuts the level, its USNWR ranking would fall because of the weight placed on expenditures per student in the ranking methodology. You can imagine the reaction of the Cornell Board of Trustees’ Finance Committee, who were berating the Cornell administration to behave more like a busi- ness and hold the university’s costs down, when I explained to them one year that the university could not unilaterally do so because of what it would do to our USNWR ranking and, in turn, to our ability to attract high-qual- ity students. Put simply, USNWR encourages institutions to spend more, not to spend less. Alumni Contribution and Graduation Rate Performance The final two criteria—the fraction of alumni who make annual contri- butions to the university and an estimate of graduation rate performance that controls for the quality of an institution’s entering class and the gener- osity of its financial aid policies—are minor, receiving a combined value of only 10% in the ranking scheme. I have discussed elsewhere the actions that Cornell took to try to improve its performance on the alumni giving mea- sure (Ehrenberg 2000, chap. 4). Some of these actions were “improvements” in the quality of the underlying data that made the institution look better. Virtually every academic institution engages each year in a process of examining all of the data it is planning to submit to USNWR to see if it can make legitimate adjustments to the data that will improve its ranking. Of course it is a rare institution that carefully examines whether it uninten- tionally erroneously reported something that overstates its position. One may well wonder if the resources that each institution devotes to preparing, checking, and adjusting its data could more productively be either saved or used to educate students. EVALUATING USNWR RATINGS There is a tendency among some academic leaders to blame USNWR for many of the ills that their institutions suffer. However, the USNWR rankings are probably more symptomatic of the increasingly competitive environment in which academic institutions find themselves than its underlying cause. While I have indicated that the USNWR ranking methodology provides incentives for institutions to take actions that are not always socially desirable, the methodology does not penalize institutions for cooperating in ways that improve the education they are providing for their students or for increasing the efficiency of their operations. Some concrete examples illustrate this point. A consortium of 15 institutions associated with the Associated Colleges of the South has created a virtual department of classics (Morrell, 2001). EHRENBERG / U.S. News & World Report Rankings 159 Prior to this development, the department at each member institution had at most three or four faculty members—not enough to provide a rich range of undergraduate classics’ courses to each institution. Most classics faculty members at the institutions devoted the majority of their time to teaching a few basic courses, and enrollments were seldom large enough to justify of- fering a large number of elective courses. Now a faculty member can offer a course simultaneously to students at his or her own institution and also to students at one or more other consortium institutions through distance- learning technologies. As a result, other faculty members’ teaching time is freed up, enabling them to offer a wider range of electives in the same man- ner. The classics courses had small enrollments to start with, and combined enrollments across institutions in any one course are still probably less than 20. So unless USNWR gets in the business of counting courses taught by faculty from other institutions by distance-learning technologies as courses taught by part-time faculty, no consortium member’s USNWR ranking will be hurt by this change. The education of students at all of the institutions will be substantially improved. Professor Sarah Turner of the University of Virginia and I each teach a course on the economics of higher education. During the fall of 2001, we taught a number of classes simultaneously to Cornell and Virginia students using a two-way compressed video system transmitted over the Internet. Students enrolled in the course listed at their home institution with the faculty member from that institution as the instructor. Class sizes at each institution did not change, and no revenue passed between the institutions. However, students from each institution benefited from interacting with two faculty members who had somewhat different perspectives on a number of issues and from hearing the views of students from a different institution. Two or more institutions located close to each other can, and do often, collaborate in making courses taught at one institution available to stu- dents from other institutions—as the institutions in Massachusetts’ Pio- neer Valley already do. Class-size limits may be necessary not to spoil an institution’s USNWR ranking on the proportion of classes it teaches that enroll 20 or fewer students, or USNWR may be encouraged to change its ranking system so that institutions will not be penalized if their reported class sizes increase when such collaborations occur. Still, such student ex- changes clearly benefit students and allow institutions to diversify their cur- riculum by drawing on neighboring institutions. As distance-learning technologies improve, such exchanges will increas- ingly transcend the limitations of geography. For example, a Cornell law professor regularly teaches a course on a very specialized legal topic to law students enrolled at Cornell and three other law schools. Interest in the subject is not sufficient at any one institution to warrant regularly offering the course and the other three law schools do not have a professor capable 160 THE REVIEW OF HIGHER EDUCATION WINTER 2002 of teaching it. By sharing the cost of providing the course at Cornell, all four institutions benefit from it. This course is taught by a combination of real-time two-way compressed video classes and Web-based lectures, read- ings, and discussion groups. Each student receives course credit at his or her own institution. To take another example, in the fall of 2001, ReCAP (the Research Col- lections and Preservation Consortium of Columbia, Princeton, and the New York Public Library) built a single off-campus storage facility to house rarely used books from the three libraries. During the initial five years of opera- tions, this consortium plans to build a joint collection of over 6 million items with delivery of a needed book guaranteed to users at each consor- tium institution within 24 hours. (See http://www.columbia.edu/cu/lweb/ indiv/recap/index.html.) By pooling their collections in one place, the three institutions can deacquisition duplicate copies and thereby achieve consid- erable savings in preservation and storage. The savings can be redirected towards other educational uses at each institution, and deacquisitioned books can be donated to other libraries. In addition, the joint collection effectively becomes the “property” of each library, so each can claim to have more books in its collection than it did before the project began.8 While library collection size does not enter into the USNWR rankings, libraries use it as a standard in their national comparisons. Academic institutions can share resources in many other ways. Adminis- trative savings could result from combined “back office” operations. For example, if multiple institutions develop a common purchasing department, they can probably achieve economies of scale and negotiate better prices for suppliers. The savings that they achieve can be redirected towards im- proving the education that they provide to their students. The bottom line is that the USNWR ranking methodology does not dis- courage academic institutions from collaborating with their competitors to improve the educations they are offering undergraduate and graduate stu- dents. Currently, however, USNWR does not reward institutions for doing so. USNWR would perform a real public service if it changed its ranking methodology in a way that encouraged institutions to collaborate on aca- demic matters. Nor should the USNWR rankings methodology be seen as discouraging institutions from collaborating on nonacademic operations to achieve fi- nancial savings. While its methodology rewards increased spending, it does not penalize institutions for reallocating financial savings from nonacademic 8For example, if each of the three institutions had 2,000 rare books and 1,500 of these were unique (held only by a single library) while 500 were triplicates (held by all three li- braries), the collection held in the combined storage facility would be 5,000 books, while 1,000 books (two copies of each of duplicated 500 books) could be deacquisitioned and donated to other libraries. EHRENBERG / U.S. News & World Report Rankings 161 operations to academic operations. Again USNWR would provide a valu- able public service if it modified its ranking methodology to explicitly give colleges and universities credit for devoting a greater share of their resources to their academic enterprises. REFERENCES Avery, C., Fairbanks, A., & Zeckhauser, R. (2001, October). Joining the elite: The early admissions game. Cambridge, MA: John F. Kennedy School of Public Policy. Avery, C., Fairbanks, A., & Zeckhauser, R. (2000, August). What worms for the early bird: Early admissions at elite colleges. Cambridge, MA: John F. Kennedy School of Public Policy. Brewer, D. J., Eide, E. R., & Ehrenberg, R. G. (1999, Winter). Does it pay to attend an elite private college? Cross-cohort effects of college type on earnings. Journal of Human Resources, 34, 104–123. Brownstein, A. (2000, September 21). Administrator ousted after failing to reply to ‘U.S. News’ questionnaire. In Chronicle of Higher Education. Retrieved July 16, 2002, from http://chronicle.com. Carnegie Foundation for the Advancement of Teaching. (1994). A Classification of Institutions of Higher Education. Princeton, NJ: Author. Ehrenberg, R. G. (2000). Tuition rising: Why college costs so much. Cambridge, MA: Harvard University Press. Eide, E. R., Brewer, D. J., & Ehrenberg, R. G. (1998, Winter). Does it pay to attend an elite private college? Evidence on the effects of undergraduate college quality on graduate school attendance. Economics of Education Review, 17, 371–376. Flores, C. (2002, May 3). U. of North Carolina at Chapel Hill drops early-decision admissions. Chronicle of Higher Education, p. A38. Golden, D. (May 29, 2001a). Glass floor colleges reject top applicants, accepting only the students likely to enroll. Wall Street Journal, p. A1. Golden, D. (2001b, October 29). Some schools shun top grads; It’s done to boost their ‘yield rates’ weeding out those least likely to enroll. Chicago Sun Times, p. 5. Hoover, E. (2001, December 14). Yale’s president proposes abandoning early-deci- sion admissions. Chronicle of Higher Education. Retrieved July 16, 2002, from http://chronicle.com. Jacobson, J. (2001, April 27). A study financed by the College Board, bolsters the reliability of the SAT. In Chronicle of Higher Education. Retrieved July 16, 2002, from http://chronicle.com. Monks, J. & Ehrenberg, R. G. (1999, November/December). U.S. News and World Report rankings: Why they do matter. Change, 43–51. Morrell, K. (2001). Now that students have wings. In M. E. Devlin (Ed.), Forums futures 2001: Exploring the future of higher education (pp. 39–42). Cambridge, MA: Forum for the Future of Higher Education. Morse, R. J., & Flanigan, S. (2000). How we rank colleges. In America’s Best Colleges 2001 (pp. 26–29). Washington, DC: U.S. News & World Report. 162 THE REVIEW OF HIGHER EDUCATION WINTER 2002 Roth, A. E., & Xiaolin, X. (1994, September). Jumping the gun: Imperfections and institutions relating to the timing of market transactions. American Economic Review, 84, 992–1044. Selingo, J., & Brainard, J. (2001, March 2). Call to eliminate SAT requirements may reshape the debate on affirmative action. Chronicle of Higher Education, p. A21. Yablon, M. (2001, October 30). Test flight: The scam behind SAT bashing. New Republic, pp. 24–25.