April 9, 2006
Class-Action Lawsuit to Be Filed Over SAT Scoring Errors
By KAREN W. ARENSON, NY TIMES

LINK

Three Minnesota law firms have begun a class-action lawsuit against the College Board and one of its contractors over scoring errors for thousands of students who took the SAT last October.

Papers were served Friday for a suit in state court in Hennepin County, Minn. Officials say it is the first legal action in the matter; a handful of other firms have expressed interest in the case.

The College Board disclosed last month that 4,411 students out of about 500,000 who took the SAT reasoning test in October received incorrect scores. The errors were partly due to moisture that interfered with the scanning of the students' answer sheets by Pearson Educational Measurement, the company that handled that part of the scoring for the College Board. Pearson's parent company, NCS Pearson Inc., is based in Minnesota.

"The College Board contracted with Pearson despite the fact that Pearson is no stranger to botching test scores," the 48-page complaint said.

The board said most of the students received scores that were too low, some by as much as 450 points of a maximum possible 2,400, and those scores were being corrected. It also said that about 600 students received scores that were too high, by as much as 50 points, but that under board policy, those scores were not corrected.

Some college admissions officers have criticized that policy, which the board has said it is reviewing.

Chiarra Coletti, a spokeswoman for the College Board, and David R. Hakensen, a spokesman for Pearson, said yesterday that they could not comment on the suit.

T. Joseph Snodgrass, a partner at Larson King in St. Paul, one of the three firms bringing the suit, said the lead plaintiff was a high school senior from Dix Hills, N.Y., on Long Island, who received an incorrect score when he took the exam. The student was not identified in the suit.

Mr. Snodgrass said yesterday that the firms planned to seek an injunction requiring the College Board to correct the inflated scores as well as those that were too low.

"It is unfair that regular students have to compete against those students with inflated scores for admission, scholarships and financial aid," Mr. Snodgrass said.

The suit defined two classes of students: those who received mistaken scores "and everybody else who has to compete against the students with inflated scores," he said. That may amount to millions of students who applied to college this year, he said.

Robert A. Schaeffer, public education director for FairTest, a group that says standardized tests are relied on too heavily, said more than half a dozen other law firms were working on similar suits.

"Our hope is that this lawsuit will finally get to the bottom of what really happened," Mr. Schaeffer said, "and why it took so long for the College Board to make the problem public."

The board has said it was alerted to the problem by two students who questioned the scores they received in December; after finding that their tests were misscored, the board investigated and learned the problem was widespread.

The three law firms  Larson King, and Zimmerman Reed and McSweeney & Fay, both of Minneapolis  joined in another suit against Pearson over a misscored state test in 2000 that kept some students from graduating with their class. In 2002, Pearson settled it for $12 million, with about $7 million for students and $4.5 million in lawyers' fees.

SAT errors may jeopardize some scholarships
Financial aid, not college admissions, is most at risk from College Board scoring mistakes.
Karen W. Arenson, New York Times, March 30, 2006

LINK

At many colleges, the biggest impact of the mistakes made by the College Board in scoring the October SAT will be on eligibility for scholarships, not on admissions decisions, officials say.

"With admissions, the colleges say they are practicing holistic review," said Donald E. Heller, an associate professor of education at Pennsylvania State University and an expert in student financial aid. "But with scholarships, some use flat cutoff points with the SAT score. They say if you score above 1,200 or 1,800 on the SAT, you are eligible for a scholarship. If you don't get that score, you don't get that scholarship."

Jennifer Topiel, a spokeswoman for the College Board, said that the board has recommended scores not be used that way.

But the reality is that they are used in numerous college and statewide scholarship programs. Heller said he found in a recent study that seven out of 14 states that offered broad-based merit scholarship programs used the SAT scores as a primary award criteria, usually along with students' grade point averages. And, he said, many colleges that offered their own merit scholarships did the same.

Christine A. Halloran, an assistant director of admissions at the College of New Jersey, called the scoring revisions a "nonevent" in terms of admissions because much of the decision-making "is based on the strength of the academic transcript."

But she said that under the state's merit scholarship program, which is tied closely to how students perform on the SAT, about five students would receive better scholarships because the board had raised their October scores.

Franklin and Marshall College, in Lancaster, Pa., which does not have a set cutoff for scholarship eligibility but takes the SAT scores into account, had one applicant whose score correction of more than 300 points meant the difference between a $5,000 scholarship and one worth $12,500.

"I know it is really hard for the public to understand why 50 points can make a difference," said Dennis Trotter, a vice president and dean of admissions. "But when it comes down to it, we might be looking at 200 students who might qualify for these scholarships, and they go head to head."

NATIONAL DESK
SAT Problems Even Larger Than Reported
By KAREN W. ARENSON, NY TIMES, March 23, 2006

LINK

The College Board disclosed yesterday that the problems resulting from the misscoring of its October SAT examination were larger than it had previously reported.
In a statement, the organization said it discovered last weekend that 27,000 of the 495,000 October tests had not been rechecked for errors. It said that after checking those exams and one other overlooked set, it had found that 400 more students than previously reported had received scores that were too low.

A board official added that the maximum error was 450 points, not 400.

This is the third time in two weeks that the board, which administers the exam, has acknowledged that its earlier assessment of the problems was wrong. In its statement, the board also outlined steps it planned to avoid mistakes.

The disclosures prompted fresh criticism that the board had not been as forthcoming as it should have been in disclosing the problems promptly and in detail.

'Everybody appears to be telling half-truths, and that erodes confidence in the College Board,' said Bruce J. Poch, vice president and dean of admissions at Pomona College in Claremont, Calif. 'It looks like they hired the people who used to do the books for Enron. My next question is what other surprise we're going to hear about next.'

The board said two weeks ago that it had found scoring problems on the October SAT after two students requested in December that their tests be re-scored by hand. In the review, the board became aware of a more widespread problem.

It asked Pearson Educational Measurement, the large testing company that scores the exam, to rescore the October exams. As a result, the board found that 4,000 students had received understated scores and that 600 had overstated scores. The policy of the board is to change just scores that are too low. Pearson has said the errors resulted in part from too much moisture when it scanned the answer sheets to be graded by machine.

Last week, the board said 1,600 exams, separated for special processing because of security and other questions, had not been rescored. The board asked Pearson to rescore those tests. While awaiting that rescoring, the board asked Pearson to confirm again that all the October tests had been scored a second time. It turned out that they had not been.

Last weekend, the board said, Pearson informed board officials that 27,000 tests had not been 'fully evaluated.' Neither the board nor Pearson explained how or why those tests had been overlooked.

In rescoring the 27,000 tests this week, 375 were found to have scores lower than they should have been. The incidence of problems -- 1.4 percent of the 27,000 -- was significantly higher than in the first batch of problems, in which eight-tenths of 1 percent of the tests were misscored. An additional 18 misscored tests were found among the 1,600 separated from the rest of the October exams for special processing.

According to the board statement yesterday, the total number of students who received scores too low was 10 percent larger than it had reported before, approximately 4,400 rather than 4,000. The board said yesterday that 613 others had received scores higher than those they had earned on the three-part exam, which has a possible 2,400 points.

The vice president for public affairs at the board, Chiarra Coletti, said it would notify college admissions officers and high school guidance counselors last night through an 'e-mail alert,' and inform affected students today.

Pearson, one of the biggest players in the testing industry, has experienced other scoring problems. It started scoring the SAT last year.

In its statement yesterday, the board said Pearson would ensure that all answer sheets were 'acclimatized before scanning' and would scan each answer sheet twice. Pearson will also improve its software to detect whether answer sheets have expanded because of humidity.

In addition, the board said Booz Allen Hamilton, the consultants, would conduct a 'comprehensive review, with emphasis on the scanning process,' over the next 90 days, and would recommend improvements.

Ms. Coletti said that she did not know how much the new procedures would cost, but that the test fee for the rest of this year would 'certainly remain the same.'

The board statement quoted Douglas Kubach, chief executive of Pearson Educational Measurement, as saying that the company regretted 'the uncertainty and disruption these issues caused' and was 'determined to take every possible necessary step to restore confidence in this process.'

'Electronic scanning of answers is essential to giving the large number of students who take the SAT the speed and accuracy they require in this important test,' he added.

A spokesman for Pearson, David Hakensen, said he could not provide more information on whether the new steps would mean higher prices.

Robert A. Schaeffer, public education director for FairTest, a group that criticizes heavy reliance on testing, said the new announcement reinforced 'the need for an outside independent investigation to find out how many more problems have not been reported.'

'The College Board and Pearson are clearly not competent to police themselves,' Mr. Schaeffer said.

NATIONAL DESK
Testing Errors Prompt Calls For Oversight
By KAREN W. ARENSON, NY TIMES
Published: March 18, 2006

LINK

As the College Board races to score the final 1,600 exams from its problem-ridden October SAT test, a string of recent testing errors around the country has college and high school officials, testing experts, students and parents asking with rising urgency, Who is watching the testing industry?
Spurred largely by the No Child Left Behind law, testing has exploded in recent years. Educators are now trying to measure factors like whether toddlers in Head Start know their letters and whether elementary and high school students are making progress in reading and math.

The states alone are administering about 45 million tests this school year.

And for students headed to college, there are the Advanced Placement exams, the SAT and the ACT.

The volume is stretching the $2 billion-a-year education testing industry, educators say, taxing its ability to draw up enough tests and score them quickly and reliably. The states are struggling to find experienced officials to provide quality control.

The resulting flurry of errors has educators and lawmakers calling for better disclosure and oversight. Some are even proposing a national agency like the Food and Drug Administration to regulate testing.

'We need accountability,' said George F. Madaus, a research professor at the Center for the Study of Testing, Evaluation and Educational Policy at Boston College. 'I certainly wouldn't get rid of testing, but we need to be much more aware than we are now about the shortcomings, the limitations and the fallibility of the technology.'

The past few weeks have shown the range of the problems. The College Board's disclosure that at least 4,600 students out of 495,000 who took its October SAT test had scoring errors was followed this week by an $11 million settlement by the Educational Testing Service of a case involving scoring errors for 27,000 people on tests that more than half the states use for teacher certification.

The Illinois superintendent of education, Randy J. Dunn, threatened last week to recommend ending the state's $44.5 million five-year contract with Harcourt Assessment because state tests were delivered late with misprints and collating errors. (He later backed away.) And New York State said this week that its seventh- and eighth-grade math tests had several questions that had been in test preparation materials.

'All of these tests have errors,' said John Katzman, chairman and founder of the Princeton Review, a test-coaching company that has benefited from the testing boom. 'The questions might be flawed in some way. The scoring might be flawed. The administration is often flawed.'

A large part of the problem, policy makers and educators say, is that the demand for tests is outstripping the abilities of the industry, and testing and scoring quality have deteriorated. Adding to the pressure, each state has its own specifications.

Robert L. Linn, a professor emeritus at the University of Colorado at Boulder who serves on a College Board advisory committee, said that problems like those with the October SAT were not uncommon in testing.

'These things happen now and then in a lot of testing programs,' Dr. Linn said. 'But there are too many now; the industry is stretched pretty thin.' He added, 'It's pretty clear, I guess, that the quality-control issues need to be looked at again.'

In a recent study for EducationSector, a new education research organization, Thomas Toch, the group's co-director, found that the high stakes of the No Child Left Behind law, which sanctions schools that do not improve, had states trying to administer tests as late as possible so children had the most time to prepare. But they still want scoring completed in time for summer school placement, giving companies little turnaround time.

Mr. Toch said states were having trouble recruiting and retaining experts, leaving testing companies largely responsible for their own quality control. Companies, too, are struggling with these demands.

'They get so little money for these contracts that they are hard pressed to hire all the people they need to do this immense amount of work without making mistakes,' he said.

Joyce E. Karon, a member of the Illinois State Board of Education, said that although she believed Harcourt was 'a reliable company,' like other test providers, it was 'working at the brink, and when you work at the brink, things happen.'

Mrs. Karon said that before the board chose Harcourt, it asked other states whether they had experienced problems. 'Almost universally,' she said, 'they had all had some problems.'

Rick Blake, a spokesman for Harcourt Assessment, said the company regretted the hardship that the problems had created for Illinois and was working to ensure that the rest of the testing went 'smoothly and without further delay.'

Still, as the pressure for more tests intensifies, testing experts and policy makers are beginning to weigh whether more oversight is needed.

One suggestion would require that testing problems be reported publicly. W. James Popham, an emeritus professor at the University of California at Los Angeles, said that more exposure might help. 'Frankly, because it is in the best public-relations interest of both the scoring service and the state officials who hire them, many scoring muck-ups are masked,' Dr. Popham said.

One step toward disclosure could come from the inspector general of the federal Education Department, who is planning to study what states are doing about errors and whether there is need for federal oversight.

Another approach being suggested by Mr. Toch, Dr. Madaus and others is to have some kind of auditor or oversight board, an independent entity or a federal body. 'There are all kinds of things in society that get monitored,' Dr. Madaus said. 'Nobody is seriously looking over the shoulder of those testing programs.'

But some in the industry say regulation is unnecessary and will raise the price of testing. 'When something like this occurs, you want to make sure you don't create regulations or ways of doing things that increase costs and don't improve the quality of service,' said Gaston Caperton, the College Board president.

Kenneth P. LaValle, the New York State senator responsible for the state's 1979 Truth in Testing law mandating disclosure, said he believed more disclosure of test questions and answers might be needed.

'We are now living in a testing culture,' said Mr. LaValle, Republican of Long Island. 'We need accuracy and security and all these things.'

Some counselors are urging college-bound students to pay extra to obtain answers after a test is administered to check their own performance, and for hand scoring.

Eugene Falik, a computer specialist in Far Rockaway, N.Y., said that after his daughter signed up recently for an Advanced Placement exam, he spotted the hand-scoring option. 'The clear implication is that these marked sense sheets, or optical mark readers, are not reliable,' Mr. Falik said.

More colleges have stopped requiring the SAT. Joanne V. Creighton, president of Mount Holyoke College in Massachusetts, said she felt her college's decision five years ago to make the tests optional was reinforced when she learned of the SAT scoring errors. She said the college had found this made 'no measurable difference' in quality.

But others say testing is a crucial tool. Richard P. Mills, education commissioner in New York State, said recent problems had not changed his mind about the usefulness of testing. 'It can be done right,' Mr. Mills said. 'Does that mean it's flawless? No. Errors crop up. No error is acceptable. But testing is indispensable.'

NATIONAL DESK
1,600 SAT Tests Escaped Check for Scoring Errors
By KAREN W. ARENSON, NY TIMES
Published: March 14, 2006

LINK

The College Board disclosed a new problem yesterday in its efforts to assess and correct mistakes in the scoring of its October SAT test: an overlooked batch of 1,600 exams that have not been checked for errors.
The admission that there were still unchecked tests came a week after the board began notifying colleges that it was raising the SAT scores of 4,000 students whose tests had been graded incorrectly because of processing problems at a Texas scanning facility.

The revelation meant that colleges were likely to face a second scramble to reassess additional applicants just as the admissions season was drawing to a close.

Chiara Coletti, the College Board's vice president for public affairs, said the 1,600 exams had been separated for 'special processing' for a variety of reasons, including security. Ms. Coletti said she could not say how many would show scoring errors, though she acknowledged some might.

Tests are sometimes pulled for special processing when students' scores are so different from those they earned on previous exams that they raise questions about whether two different people may have taken the tests, pretending to be the same person.

The 1,600 tests were in the custody of the Educational Testing Service, which once handled most test scoring for the College Board but now handles more limited functions, including test development and security.

Raymond Nicosia, executive director of the testing service's office of testing integrity, said last night that employees of the College Board were aware that the 1,600 tests had not been rescored. Mr. Nicosia said they would be sent for re-examination Tuesday.

Hours before Ms. Coletti reported the new problem, Gaston Caperton, the College Board's president, said in his first interview since the SAT errors were disclosed that he regretted what had occurred but that the College Board needed no outside audits and that Pearson Educational Measurement, the contractor that scores the tests, was making needed changes.

'We are very sorry it happened,' Mr. Caperton said. 'We've looked back over all of this and we would not have done anything different. We moved as fast as we could and as professionally as we could.'

Ms. Coletti said last night that Mr. Caperton had not learned about the 1,600 additional exams until the afternoon, and that he stood by the comments he made in the morning interview.

But with colleges scrambling and angry students saying that their erroneous scores steered them away from applying to certain colleges, critics said an outside investigation of the episode was needed.

'The more we learn, the worse the problems are,' said Robert A. Schaeffer, public education director of FairTest, which says there is overreliance on standardized testing. 'There needs to be an outside investigation to identify everything that went wrong, why, and whether it has been fixed.'

Kenneth P. LaValle, a New York state senator from Long Island who is chairman of the Senate's higher-education committee, said he wanted to study whether tighter regulation of testing was warranted and planned to call in officials of the College Board for questioning.

Mr. LaValle was the architect of New York's 1979 truth-in-testing legislation, which helped require agencies like the College Board to release test questions, test answers and the students' own answers for a fee.

The complaints of two students who took advantage of that option were what led to the College Board's discovery of wider problems with the October test.

In an hourlong interview in his orderly fourth-floor office near Lincoln Center, Mr. Caperton and a senior official for operations, Laurence E. Bunin, elaborated on how they then became aware of, and tried to resolve, the broader problems, which in the case of some students resulted in underscoring by as many as 400 points.

Mr. Caperton said that although the requests by two students for hand-scoring of their exams arrived in late December, it was not until Jan. 31 that the hand-scoring took place and the board realized that errors needed to be investigated.

Mr. Bunin said the board had worked as fast as it could, including on nights and weekends, to assess and correct the errors. He said that in early February the board notified Pearson Educational Measurement that there were problems. Pearson has said that damp test papers led to some of the problems.

'We reported them as promptly as humanly possible,' Mr. Bunin said.

He said that while big swings in scores would raise red flags about possible cheating, there had not been an unusual number of problems to catch anyone's attention. He said that the number of misscored tests was still small compared with the total number, 495,000.

'When you're talking about a dozen things among half a million, that's not very much,' Mr. Bunin said.

Mr. Caperton and Mr. Bunin said yesterday that besides the 4,000 students whose scores had been understated, 600 students had scores that had been overstated, but they could not say by how much. The board's policy is not to reduce scores, an approach that has riled some colleges. The board is reviewing the policy.

Mr. Caperton's assurances that the board was doing what it could were small consolation to students like Amanda Hecker, one of the 4,000 who learned that their initial scores were lower than they should have been.

A senior at Freehold Township High School in New Jersey, Ms. Hecker took the SAT exams last March. She did well, but hoped to do even better, so she took the exam again in October.

Although most students get higher scores when they retest, she did not: the scores on all three parts of her test fell, and her total plunged 180 points, to 1,890 out of a possible 2,400. The normally upbeat student slashed Harvard and Yale off her list of college applications.

'The College Board's analysis shows that it is a one-time anomaly and blames it on the rain,' she said. 'Should America now be quick to trust this organization that has such a clutch on the college admissions process and college-bound students' emotions?'

In an era when families are paying thousands of dollars to raise their children's scores by 50 or 100 points, some students said they had lost the chance of a lifetime when they made their decisions last fall on where to apply to school.

'I didn't apply to certain schools and I almost didn't apply to others,' said Amanda M. Hellerman, a senior at Yorktown High School. Her scores rose more than 300 points once they were corrected; she had chosen not to apply to Brown because of the mistakes.

'I tried to not let it get to me,' Ms. Hellerman said, 'but to an extent, I did wonder why my performance had declined.'

Even now that she knows her scores should have been 320 points higher -- with 740's in both reading and writing and a 610 on math -- she said she did not think 'too fondly' about testing, and wondered 'how accurately it gauges your intelligence.'

But she had one other pleasant surprise last week: Franklin & Marshall, a liberal arts college in Pennsylvania, said that her higher scores meant that she qualified for a $12,500 scholarship. Ms. Hellerman said she was thinking seriously about the college.