The Big Test

Dan Seligman

In The Big Test, Lemann makes this tale immensely readable.
7#151; Wall Street Journal

Hardy Green

One of the most astute and disquieting efforts of the season. What begins as a chronicle of the evolution of standardized testing ends as a penetrating commentary on today's frantic skirmish for a place in the professional upper middle class. Business Week

Chester E. Finn, Jr.

Until recently, the operations of this Princeton, New Jersey-based institution [ETS] have been shrouded in an almost CIA-like secrecy. But some of [its] secrets have now been laid bare in a long-awaited book by the journalist Nicholas Lemann. [But when] Lemann turns from institutional history to a broader discussion of testing itself, his book sheds whatever character it has as a fact-finding excursion and increasingly takes on the appearance of an ideological tract. Lemann turns out to dislike test based meritocracy altogether.
— Commentary Magazine

Andrew Sullivan

The story he tells -- and often beautifully -- seems such an obviously progressive one that framing it as a case study in failure seems willfully perverse...[H]e does an excellent job of evoking characters in thumbnail sketches and a few memorable scenes.
— NY Times Book Review

Caleb Crain

There was little outcry in 1966 when Harvard Law School relaxed its test-score standards in order to admit more black students. The year before, Lyndon B. Johnson had signed an executive order quietly launching affirmative action, and no one had objected to that, either. The moves were so politically palatable that newspapers and voters scarcely even noticed.

The public would not be so indifferent today. In 1996, in a bitter, headline-grabbing fight, California voters approved Proposition 209, which ended affirmative action in the state's public colleges and universities. Washington State voters and a Texas federal appeals court soon followed the California example. Why?

In The Big Test: The Secret History of the American Meritocracy, journalist Nicholas Lemann offers a provocative answer. Affirmative action galls, Lemann suggests, because it depends on a premise that is not only unappealing but deeply un-American: At the top of the United States today sits a meritocracy, an elite who believe that their intellectual achievement earned them their high status. Meritocrats think of themselves as progressive and antiracist, but they are certified into the elite by the SAT, an IQ-like test on which whites and Asians consistently outscore blacks and Hispanics. Thus, as a compensation, faith in affirmative action has become one of the group's shibboleths. In The Big Test, Lemann explains how the Educational Testing Service (ETS), the nonprofit corporation that administers the SAT, arrived at the center of America's educational culture. He suggests that the recent setbacks to affirmative action have thrown the meritocrats' assumptions about themselves into disarray. And then he argues that the meritocracy -- and ETS -- should be dislodged.

This is quixotic, but it makes for a good read. Most writers on affirmative action have given numbers and argumentation; Lemann gives personalities and plot. There is Henry Chauncey, president of ETS from 1945 to 1970, an affable Episcopal scion of one of the strictest Puritan families of 17th century New England. A former assistant dean at Harvard, Chauncey excelled at ETS thanks to his prowess at bureaucratic judo and his fervent, unskeptical love of all mental tests. There is James Bryant Conant, ETS's godfather. When the dour chemistry professor became Harvard's president in 1933, he undertook to turn the college's idle-rich-boy culture on its head, forcing the gentleman's C to defer to the scholar's A. There is Princeton psychology professor Carl Brigham, at first an ardent eugenicist, then the author of the first SAT, and finally a skeptic who recanted his earlier faith in intelligence as an inborn trait, writing instead that "test scores very definitely are a composite including schooling, family background, familiarity with English, and everything else, relevant and irrelevant." Allan Nairn -- in the news recently as a journalist arrested and released in East Timor -- has a cameo as a young apprentice to Ralph Nader in the 1970s.

Focusing on changes in the admissions policies and academic cultures of Harvard, Yale and the University of California system, Lemann documents a historic shift that is hard to appreciate today, so completely do we inhabit the current dispensation. "The machinery that Conant and Chauncey and their allies created is today so familiar and all-encompassing," Lemann writes, "that it seems almost like a natural phenomenon, or at least an organism that evolved spontaneously...It's not. It's man-made." This "machinery," Lemann emphasizes, is not even necessarily what its creators had in mind. It appalled Conant, for example, when the Selective Service hired ETS during the Korean War to identify the most intelligent American college students and reward them with a draft deferment. Even Chauncey had his moments of dissatisfaction, dreamily wishing for a test that measured something broader and more humane than the IQ-like characteristic captured by the SAT.

It's refreshing to read about the flip-flopping opinions and open-minded debates of ETS's founders, because today the discussions have become harshly polarized -- Dr. Pangloss meets Ebenezer Scrooge. Take, for example, the storm that broke out last month, when the research department of ETS revealed that it was experimenting with a new way to report SAT scores to colleges. The new formula would have identified "Strivers" -- students who scored 200 points higher on the SAT than their socioeconomic background would predict. One version of the Strivers formula would have accounted for race, and upon editorial-page writers, this had the effect of blood poured into a shark tank. Within days, even the president of the College Board, the association that oversees ETS, was disparaging the innovation.

The Strivers were burked, but not quickly enough to prevent a hail of statistics and canards from pelting the media. "When you look at a Striver who gets a 1000," ETS vice president Anthony Carnevale told the Wall Street Journal, "you're looking at someone who really performs at a 1200." Nonsense, countered Harvard professor Abigail Thernstrom in the New Republic. In fact, Thernstrom wrote, it was "one of the buried but depressing facts" in William Bowen and Derek Bok's pro-affirmative-action book, The Shape of the River, that black students already "earn substantially lower grades in college than their SATs would lead us to predict."

Thernstrom was right about the academic underperformance of blacks, and she was right to call it depressing. But she was wrong to say that Bok and Bowen "buried" this fact. Actually, the 1998 book is so forthcoming with facts it could bliss out even the most hardened empiricist. "Black students with the same SAT scores as whites tend to earn lower grades," Bok and Bowen point out, and they accompany this observation with an 18-page analysis and an easy-to-read chart.

Bok and Bowen's book does reward close reading, however; as former university presidents, the two are men of tact, and so there are facts they do bury. My own favorite is in Table 5.1, which compares the average 1995 earned incomes of a group who entered college in 1976. The data are sorted by race, gender, SAT scores and the selectivity of the college attended. You would expect whites to earn more than blacks, and they do. You would expect men to earn more than women, and they do. You would expect graduates of more prestigious colleges to earn more than those of less prestigious colleges, and they do. You would expect people who score high on the SAT to earn more than those who score low.

They don't. As it turns out, the highest wages belong to the white men with the lowest SAT scores at the most prestigious colleges. This is a remarkable statistic. Opponents of affirmative action like to talk about academic merit, and overall, black and Hispanic applicants to selective colleges benefit from an admissions handicap of about 0.67 GPA points and 400 SAT points. But Table 5.1 suggests that on the margins, where the white would-have-been-admitteds confront the black wouldn't-have-been-admitteds, the disparity in academic standards is dwarfed by the disparity in future earned income.

Originally, "SAT" stood for "Scholastic Aptitude Test," later changed to "Scholastic Assessment Test." According to Lemann, today the letters "literally don't stand for anything." Whatever the test measures, SAT scores in and of themselves do not have all that much to do with earning potential. They have a great deal to do with what college you go to, however. And the next link in the chain is where the correlation pays off: Where you go to college has everything to do with your future income. Conservatives from time to time float the specter of the minority student whose life was ruined because affirmative action threw him into a rigorous academic environment he couldn't handle. No such animal, statistically speaking: Attending a selective college substantially improves a minority student's chances of graduating. It also adds a bonus to the minority student's future income. The trouble is that, as Bok and Bowen's Table 5.1 demonstrates, it also adds a bonus for whites, in even greater dollar amounts.

Nonetheless, elite colleges unexpectedly and rather doggedly prefer a black alumnus who earns $86,700 a year over a white alumnus who earns $132,700. Their motive, according to Bowen and Bok, is enlightened self-interest. Elite institutions believe that the nation as a whole will prosper politically if its minorities have educated leaders, and that it will prosper economically, too, because black and Hispanic executives will be better able to sell to minority consumers and motivate minority workers. Unfortunately, this high-capitalist rationale doesn't sit all that well with the whites shut out of elite colleges. In fact, as California's Proposition 209 demonstrates, its condescension is fiercely resented.

For good or ill, race was the last thing on anyone's mind while ETS was building its machinery, according to Lemann. But the institutional politicking was nonetheless fierce, and it is in accounts of euphemism, logrolling, double-dealing and gamesmanship that Lemann shines as a reporter. The complexity of his chronicle makes it easy to see how the original principles (and doubts) of the machinery's inventors were mislaid along the way. As a private, nonprofit corporation, ETS has never been under government supervision, and it has always had to make money to survive. Thus its evolution has been insulated from politics but buffeted by economics, and as Lemann's book advances, the reader can't help but notice how many cruxes in the testing machinery's development were resolved on economic grounds. (It is cheaper to test for IQ than for knowledge of a set curriculum; it is cheaper to pluck out a few promising students than to improve public education generally.) The story of ETS begins to sound like the story of capitalism begetting upon itself the means of selecting its ideal executives. It does not, after a while, seem at all wonderful that this efficient, bland entity could have shuffled the membership of the American upper class more thoroughly than anything since industrialization.

Lemann admires the rise of ETS, the way one admires a crocodile that had to eat a number of its siblings. But Lemann worries, too, because no one ever voted for the crocodile, and yet it seems to be in charge. The SAT was first administered in 1926, and today it is so deeply embedded in the self-image of the American elite that any change would be resisted with the ingenuity and tenacity of a legislator under threat of redistricting. Nonetheless, as Lemann observes, meritocracy is just a neologism for aristocracy, and America shouldn't have one. "How may an aristocracy of intellect justify itself to all men?" the University of California's Clark Kerr once wondered. "Good question," Lemann comments drily. Much has been given to the new meritocrats, Lemann notes. "And what [have] they done, really, to earn such privilege or authority, other than to get high test scores and good grades?"

In Lemann's opinion, the meritocrats deserve a comeuppance. To some extent, he believes, they find it whenever they venture outside their narrow fields of specialization. At large in America, meritocrats discover they are not the only kind of elite. In fact, meritocrats are the least popular sort. The general public has much more respect for the elites that Lemann refers to as Lifers (who earned their eminence by long and loyal service, such as Colin Powell) and Talents (who won their places with unusual achievements, such as Steve Jobs). But a comeuppance is supposed to induce self-scrutiny, and skirmishes with the Lifers and Talents haven't forced the meritocrats to question their assumptions.

Here Lemann's narrative takes a sudden, hazardous turn. The single greatest comeuppance of the meritocrats, he believes, has been at the hands of the masses, in a slap delivered just when the meritocrats thought they were being most altruistic. The recent setbacks to affirmative action, Lemann suggests, have been salutary for a complacent would-be ruling class.

The big test of The Big Test turns out to be not the SAT, but the meritocracy's 1996 ordeal in California. The last third of the book charts the rise and fall of race-sensitive college admissions. Here, too, Lemann excels as a reporter, revealing unlikely alliances, unexpected motives, clandestine political bargains and yet more evidence of the Clintons' slipperiness. But there is a strange hollowness to the struggle, at least as Lemann describes it, because his heroines -- the two women leading the fight to save affirmative action in California -- lack any abiding faith in their cause. Even before its demise, attorney Connie Rice has "concluded that, as a solution to the problems of black America, affirmative action was a joke." And soon after the liberals lose, Molly Munger decides that affirmative action was "the wrong fight to be in." Lemann agrees. "The right fight," he writes, in an approving echo of her thoughts, "was the fight to make sure that everybody got a good education and a chance to live a life of decency and honor."

This is a logical error, of the form "Sex is better than cookies; therefore, cookies are bad." Cookies are not bad, even if sex is better. Of course a campaign to improve the country's elementary and secondary schools would in the long run be far superior to affirmative action. But that does not mean that affirmative action is not worthwhile in the interim.

"The project of picking the members of [the] elite properly does not confer an aura of justice on the whole society," Lemann writes. This is true, and worth saying, if only to prick the meritocrats' moral smugness. It's grandiose of the meritocrats to justify affirmative action as if it were charity, when the expense is only partially theirs. But affirmative action is more than the meritocracy putting on airs; all the numbers indicate that the power of an elite college education is real. The more familiar temptation in America is to downplay the hand you were dealt. If a group of people have admitted to a privilege so great they feel obliged to offset it with a responsibility, it probably isn't wise to refuse them just because we know they could do better.
— Salon

Publishers Weekly

In a country obsessed with educational opportunity, the principal institution for overseeing the distribution of access to higher education, the Educational Testing Service, was founded in "an atmosphere of intrigue, corruption, competition, and disorder." So contends Lemann (The Promised Land) in this enthralling, detailed story of how the Scholastic Aptitude Test (SAT) became enshrined in U.S. culture. Although the idealistic, patrician pioneers of testing may have wished to displace the entitlements of birth and wealth for what they saw as the more democratic entitlements of scholastic aptitude, at the end of the 20th century "their creation looks very much like what it was intended to replace." This story is compelling in itself, but Lemann's exploration of how the politics of American meritocracy turn on the issue of race makes his history absolutely indispensable to current affirmative action and education debates. Lemann's treatment of the 1996 battle over California's anti-affirmative action Proposition 209 convincingly shows how what is nominally a democratic process actually works. The current crises in American education have deep roots: "America had channeled all the opportunity through the educational system and then had failed to create schools and colleges that would work for everybody, because that was very expensive and voters didn't want to pay for it." The real costs of this situation are now clear; anyone concerned about it should heed this book. Copyright 1999 Cahners Business Information.

Library Journal

This book treats two distinctive but distinctly interrelated themes in which Lemann (The Great Migration) has evinced sustained interest: educational opportunity in America as it determines socioeconomic success and (in)equity as it reflects educational opportunity. Lemann does not altogether succeed in integrating these two stories. For lengthy stretches, this book is about the ostensible development of an objective elite through standardized testing and the establishment of Educational Testing Service (ETS) and its major product, the Scholastic Aptitude Test (SAT), as an American state religion. Lemann shows how a handful of eccentric, Depression-reared members of the WASP elite went about reforming access to Ivy League education by pushing their confidence in the quantitative social sciences to the point where SAT scores, not family origin, became what mattered most to young adults' chances. As Lemann relates that history, he interjects the personal stories of a later generation of eccentrics at Yale and Harvard Law in the 1960s who fought to make access to higher education yet more inclusive. Finally, Lemann makes clear that the SAT and civil rights come out of egalitarian impulses that might each resist the other. Lemann's work in the archives of ETS is commendable, and overall this is an important contribution to American sociology by a lay journalist. Recommended for academic, public, and high school libraries. [Previewed in Prepub Alert, LJ 6/15/99.]--Scott H. Silverman, Bryn Mawr Coll. Lib., PA Copyright 1999 Cahners Business Information.

Booknews

Lemann, a longtime journalist with , , and other magazines, describes the attempt to set up a new social order in the US during the second half of the 20th century based on standardized testing. He traces the origins of the idea and how it was put into effect, as well as broader issues relating to the value of testing and the meritocracy it created and sustained. Annotation c. Book News, Inc., Portland, OR (booknews.com)

David Karen

In The Big Test, Nicholas Lemann has forced us to recognize that the SAT is not simply a "natural" means of selection; it has a history that could have been different. He has produced a book of broad scope and much original research. He reminds us that any plans for redistributing opportunity in society must be willing to address issues of race and class.

The Nation

Erin Kelly

[R]ead Nicholas Lehmann's facinating and thoughtful new book, The Big Test: The Secret History of the American Meritocracy. It documents how the SAT has changed our lives and our country...His new book is written with his usual grace and is told movingly through the lives of the people involved; The Big Test makes you look at your world, your successes, and you high school boyfriend's mother a bit differently.
— Fortune

Kirkus Reviews

Enlightenment is available here for anyone who has struggled to understand why their future depended upon filling in little ovals with a #2 pencil in response to odd questions about vocabulary. Lemann (The Promised Land: The Great Black Migration and How It Changed America, 1991) describes a project whose purpose was nothing less than the replacement of family background with intellectual capacity as the basis for social and economic hierarchy in America. The SAT's originators saw it as the scientific path to the optimal utilization of society's human resources, matching abilities with educational options and guiding individuals into suitable employment and social positions. Such social utopian convictions today seem naive and possibly dangerous, but in the post-WWII environment the leadership of the Educational Testing Service and its meritocratic allies were able to put the machinery of this system in place, and testing became "the all-powerful bringer of individual destiny." However, while testing advocates championed equal opportunity for all against the patrician social status quo, the tests they administered reapportioned opportunity rather than expanding it; the criterion for social discrimination changed, not the fact of its existence. Most of the antagonisms engendered by this new system of sorting winners and losers could be deflected by claims of scientific objectivity, but testing advocates didn't foresee the race factor. Blacks were particularly ill-positioned to do well on tests measuring capacities, such as vocabulary skills, dependent upon education and environment; consequently, testing reinforced racial inequity. Lemann follows the story through the ups anddowns of affirmative action and concludes with an unusually cogent analysis of what an educational system should be in a democracy and what a genuine American meritocracy would look like. Lemann has produced a suitably big book, sprawling across most of a century and multiple major issues, told through the lives of numerous fascinating figures, and ultimately providing an original, perceptive, and powerful analysis of institutions that are too often taken for granted.

From the Publisher

“I use The Big Test in my Perspectives on Secondary Education class. It makes students come face-to-face with issues of race, class, and inequality in secondary education, and since everyone has taken ‘the big test,’ it really hits home.” —Susan Semel, Associate Professor, Hofstra University

“An engaging, enlightening historical analysis of the idea of the SAT, dramatized by the stories of the people who designed it, the students who benefited from it, and recent battles over standardized testing and affirmative action.”—Wendy Kaminer, The Boston Globe

“Engrossing . . . . The narrative of The Big Test is carried along by a string of life stories [that] once more display Nicholas Lemann's talent for distilling social analysis out of personal history.”—Alan Ryan, The New York Review of Books