Culture, Candor, Cost: Is America Ready for School Inspections?
by GUEST BLOGGER on JANUARY 27, 2012
in ACCOUNTABILITY
Guest blog post written by Craig Jerald of Break the Curve Consulting.
LINK

Over the past two weeks, Education Sector has hosted a series of deeply thoughtful guest posts on the possibility of adopting school inspections in the United States, prompted by my report On Her Majesty’s School Inspection Service. Clearly, states would need to address many challenges to ensure that inspections work as well here as they do in England, from the financial to the cultural. But I believe that at least on the cultural front, we’ve already begun to lay the necessary groundwork.

In his guest post, Charles Barone argues that delivering candid feedback to other adults can be very uncomfortable, which can undermine rigor when rating the performance of schools and other institutions. It’s certainly valid to wonder whether American inspectors could deliver the same kinds of tough judgments Kate McGovern recalls from her experience in a London school: “The inspection process correctly identified that in spite of boosting its test scores, Willow Tree still wasn’t serving its children as well as it should.” And Barone correctly points out that the necessary change isn’t technical but cultural, making it that much more difficult to tackle.

But American education has already begun to face up to the very same challenge on another front: teacher evaluation. Although TNTP argued in its 2009 Widget Effect report that decades of inflated evaluation ratings have created “stiff cultural resistance” to candid evaluations, recent efforts to reform evaluations have shown it is possible to overcome the problem. In November, researchers reported that classroom observations in Chicago’s teacher evaluation pilot identified the same low-performing teachers as seemingly more “objective” measures like value-added scores. Similarly, data released by the National Institute for Excellence in Teaching (NIET) have shown that observation-based judgments of teaching practice in TAP System schools are not inflated and are correlated with value-added scores.

NIET says that one of its biggest “lessons learned” over the past decade is that education leaders need to confront the cultural challenge directly and concretely. Leaders in TAP schools use a range of strategies (this is where the “technical” meets the “cultural”) to promote a culture of candor. State officials would have to do the same to promote a culture of candor for school inspections. Based on my study of England’s inspection system, some of the more important strategies would include the following:

Independence and professionalism: One way England has ensured that standards remain rigorous is by creating an independent inspectorate external to its Department for Education and staffed by highly trained and qualified inspectors. Asking already overstretched state department of education employees to conduct inspections would save money but undermine confidence in inspections and accuracy of inspection judgments. Relying on volunteers from other schools would be cheaper, too, but such an approach also would undermine accuracy and candor, as the experience with regional accreditation has shown. As guest blogger David Plank put it last week, “The main strengths of the English inspectorate are its independence and professionalism. It’s not yet clear what [California] Governor Brown means by ‘site visitation panels,’ but it’s hard to imagine that they would share those virtues with Ofsted, and it’s all too easy to imagine who would volunteer to serve on the panels.”

Transparency: A state would need to publish a detailed set of expectations and guidelines that include clear descriptors of performance for each possible inspection rating, in language that school personnel can understand. That would prevent inspections from being seen by educators as an unfair “gotcha” exercise to be resisted (or gamed), and help them calibrate their own expectations with the official ones. The English inspectorate Ofsted publishes three documents that offer excellent examples.

Relevance: The best way to promote a culture of candor is to ensure that inspections are useful for those being inspected. To that end, states would need to carefully design inspection systems to provide diagnostic feedback that can help educators improve. When they find feedback to be fair and useful, educators value critical candor. As guest blogger Louis Freedburg pointed out on Friday, section 1117 of NCLB required states to constitute “school support teams” to provide diagnostic feedback for school improvement, but states have been negligent in complying. Freedburg argues that the concept should be buttressed in the next version of the ESEA. But a state could build such diagnostic feedback into its accountability measures themselves, through an inspection system, rather than tacking it on as an afterthought.

Monitoring and quality assurance. England employs a variety of strategies to assure quality of inspections, including a close read of each draft inspection report by one of Her Majesty’s Inspectors (HMIs) who did not serve on the inspection team. In addition, a state also could continue to publish “objective” data on student outcomes (test results, value-added scores, graduation rates) separately so that members of the public, the media, and watchdog groups could compare those data with the published inspection judgments for schools.

Admittedly, some of these strategies would have significant cost implications. As Patrick Riccards and David Plank noted last week, that presents a major challenge for cash-strapped states. One thing is certain: Doing inspections on the cheap will not promote the “culture of candor” American education so desperately needs.

Lessons From England on School-Level Accountability
by GUEST BLOGGER on JANUARY 10, 2012
in ACCOUNTABILITY
LINK

Guest Post written by Craig D. Jerald of Break the Curve Consulting.

On Friday the Bill & Melinda Gates Foundation released the latest findings from its $45 million Measure of Effective Teaching (MET) Project. MET researchers concluded that teachers should be evaluated using “multiple measures” of teaching effectiveness, including student surveys, value-added estimates based on student test scores, and direct observations of classroom practice. Indeed, the MET Project recommended that evaluation systems should include observations even though they are expensive to conduct reliably and add little to the predictive power of value-added scores alone. Despite facing very tight budgets, not one single state or district will ignore that recommendation and evaluate teachers without directly observing them in action.

The reason for that costly consensus is that observations contribute something that test-score based measures simply cannot. As the MET Project put it, “the real potential of classroom observations is their usefulness for diagnosis and development of instructional practice.” In other words, everyone agrees that teacher evaluations should be about more than simply measuring performance; evaluations also should be about helping educators improve their performance. And it’s worth spending more on a set of evaluation measures that can accomplish both.

Yet many education advocates who wholeheartedly support the use of multiple measures for teacher evaluation still resist such measures when it comes to evaluating schools, including the school-level analogue of classroom observations: on-site “inspections.” In a paper published by Education Sector today, On Her Majesty’s School Inspection Service, I use examples from England to show how inspections can contribute the kind of diagnostic feedback and support for improvement nearly completely lacking under No Child Left Behind.

England is just one of dozens of countries around the world where inspectors visit schools. Just as most U.S. policymakers would never consider evaluating teachers without observing them, those countries would never consider holding schools accountable without taking time to inspect them.

Of course, U.S. advocates who worry that inspections might “water down” school accountability systems do have a point. Frankly, I shared similar concerns when Education Sector invited me to write a report on this topic. But after spending months studying England’s approach, I became convinced that there are ways to incorporate inspections into school accountability systems in ways that do not sacrifice “rigor.”

In fact, I concluded that England’s current inspection system offers clues to solving many challenges related to holding schools accountable in the post-NCLB era, including the following:

Judging schools on a broader range of evidence without losing sight of the fundamental importance of student achievement, including standardized test scores;
Leveraging expert judgment rather than relying solely on quantitative formulas, yet still ensuring sufficient safeguards against inconsistent or inflated ratings; and perhaps most importantly
Achieving a better balance between rigorous evaluative ratings and better diagnostic feedback to help schools improve.

For all of those reasons, state and federal policymakers should give careful consideration to the potential for using inspections to improve school-level accountability systems. It’s time to think about “observing” schools, too.

Tagged as: Inspecting the Inspectorate Series, Inspectorate Model, MET Project, School Accountability, School Inspections, School observation

COMMENTS
Craig Jerald January 12, 2012 at 2:48 pm
Gideon, you’re quite right that many U.S. charter school authorizers inspect schools. Ultimately, I chose to focus on England’s model because charter inspections tend to be for a narrower purpose, charter renewal, and I thought England’s approach more relevant to the broader accountability debate. As I mention on page 3 of the report, “Some states require charter schools to undergo reviews by visiting teams before their charters can be renewed. Massachusetts consulted with experts familiar with Ofsted’s approach in order to model its charter reviews on English inspections, and the state publishes each school’s ‘renewal inspection report’ online. However, so far the state only inspects charter schools, and the inspections play no role in regular accountability determinations.” But for states considering inspections, it would be worthwhile to speak with some of those charter authorizers.

Gideon January 11, 2012 at 2:07 pm
No one seems to have noticed that some high quality charter school authorizers do use regular inspection visits as part of their high stakes accountability systems. They even post their findings for the public to read. The key is to have clear standards for school performance and focus on quality programs rather than compliance monitoring. They often use external experts to augment their inspections teams, providing targeted expertise and objectivity.