Rule changes aided school progress
By Dan Hardy, The Philadelphia Inquirer Online, October 28, 2004

LINK

School accountability gains that Pennsylvania education officials lauded resulted from lower standards, not improved performance, according to an Inquirer analysis.

More than twice as many schools would not have made what the state considers "adequate yearly progress" toward goals set under the federal No Child Left Behind Act if the rules had not been changed.

The changes allowed schools with lower graduation rates, lower standardized test scores, or lower attendance than in previous years to win passing marks. For example, a 2003 standard requiring high schools to have graduation rates of 95 percent or to show improvement was reduced to 80 percent or improvement for 2004.

Pennsylvania was among dozens of states allowed by the U.S. Department of Education to change the standards. New Jersey, too, was among those states, but it made relatively few changes.

The changes in Pennsylvania were significant.

In 2004, 81 percent of the state's schools met the act's so-called adequate yearly progress benchmarks using the new standards.

But the Inquirer analysis found that if the same rules used in 2003 had been used in 2004, the number of schools falling short of the yearly benchmark would have grown from 566 to 1,164. Instead of 81 percent meeting the benchmark, just 61 percent would have succeeded. Last year, 63 percent of schools made the benchmark.

More than 100 schools in Philadelphia and its suburbs that met the benchmark this year would not have done so if the old standards had been applied.

Schools not making the benchmark have to adopt a series of corrective measures; those that don't improve are eventually subject to sanctions, including possible conversion into charter schools.

In a presidential election year in which the No Child Left Behind law has drawn fire, some critics contend that the rule changes were part of a U.S. Department of Education effort to downplay controversy about the law by softening its impact.

"It is curious to me that when it was not the season it is now, they were very resistant in Washington to making changes, and now, all of a sudden, they are very flexible," said Carolyn C. Dumaresq, executive director of the Pennsylvania State Education Association, a teachers' union. "I think it is politically motivated, but I also think the changes were reasonable."

Jo Ann Webb, a spokeswoman for the U.S. Department of Education, responded that "Pennsylvania, like many states, made changes to its accountability plan to best reflect the educational needs of the students in that state... . It refined its system to focus efforts on schools most in need of improvement."

Brian Christopher, a spokesman for the Pennsylvania Department of Education, said that the changes were designed to "make the system more valid and reliable... . We do not want a system that might overidentify and wrongly classify schools and districts."

Pennsylvania's rule changes lowered the threshold for acceptable attendance and graduation rates. They also allowed schools within a margin of error of 5 percent of achieving a math or reading standard to pass anyway and allowed "Safe Harbor" schools that showed substantial improvement to make less progress but still meet the mark.

The state changed the rules for children with limited English proficiency to exclude some newly arrived immigrant students and to include others with higher levels of English proficiency. And it changed the method of determining the participation rate of test-takers, another benchmark standard.

When Pennsylvania's Education Department submitted its proposed changes to the federal government in March, it said that hundreds of schools that would not have met the 2003 standards would likely pass muster because of the changes.

But in late August, when the Education Department announced that only 566 of 3,009 public schools failed to meet the standard, there was no mention of the role the rule changes played in the "significant gains" made. "We are extremely excited... because 20 percent more schools are making [yearly progress] due to the hard work of Pennsylvania's teachers and the state's overall emphasis on accountability," acting state Education Secretary Francis V. Barnes said.

Had the state not changed the 2003 rules, at least 598 more schools would not have met the mark.

The state also changed procedures for counting "feeder schools" - those that did not have any tested grades. Last year's progress count included only feeder schools that sent students to schools that did not meet the standards. This year, it added 251 feeder schools that sent students to schools meeting the mark. Had those 251 schools not been counted, the percentage of schools making progress this year would have fallen to only 58 percent, going by the 2003 rules.

Christopher defended the state Education Department's characterization of the progress results. "There was improvement in the PSSA scores of students," he wrote. "It is by no means inappropriate to celebrate improvement in the performance of our students."

In a recent defense of the No Child Left Behind law, Rod Paige, the U.S. secretary of education, wrote that "in just one year, Pennsylvania has gone from 62 percent of schools meeting their annual goals to 81 percent." And the U.S. Education Department's Webb said that "perhaps the most important reason that more schools in Pennsylvania... are making [progress] is because kids are learning and test scores are up."

But several educational testing experts said that the improved showings were often due to changes in the rules.

"I think that the message ought to go out that there is no apples-to-apples comparison between this year and last year, because states made the changes they did," said Brian Gong, associate director of the National Center for the Improvement of Educational Assessment, a nonprofit group working with states to improve their accountability systems.

He said that when the 2003 failure rates were released, "the Department of Education and members of Congress were concerned. I think they were looking for rates of 20 percent and that was not what they got... . In my opinion, a lot of the changes were intended to cut the number of schools that were identified" as not making [progress].

The number of schools making progress did in fact go up in many states this year. In the 35 states reporting complete results, the percentage of schools meeting the standard rose from 63.3 percent in 2003 to 70.7 percent, said Scott Young, a senior policy specialist with the National Conference of State Legislatures.

An analysis by the Center on Education Policy found that 46 states were granted some changes to their accountability systems. A dozen received modifications like those made in Pennsylvania that allowed schools to make progress if their scores were within a statistical margin of error, or "confidence interval," of the passing score.

"The accountability-plan changes... are likely to make it easier for schools to demonstrate [progress], at least in the short term," the analysis said.

New Jersey changed how students with limited English proficiency were counted, changed calculations of test participation rates, and increased the number of special education students who had to be in tested grades for their scores to count in annual calculations.

Of New Jersey's 2,398 public schools, 1,777 met the federal requirements this year, an increase of 228 schools over the 1,549 schools that met the requirements in 2003.

Richard Vespucci a spokesman for the state Department of Education, said that a data analysis had not yet been completed but that so far the department believed that the rule changes were "a factor, but not a major factor," in the improved performance.


Contact staff writer Dan Hardy at 610-701-7638 or dhardy@phillynews.com.