Does your district have what it takes to significantly improve?

Do you count on EdSource’s education coverage? If so, please make your donation today to keep us going without a paywall or ads.

The question struck Arun Ramanathan while he endured a nine-hour meeting of little consequence in Hayward Unified in the East Bay: Is there an inverse (or is it perverse) relationship between the length of a school board meeting and the effectiveness of a district?

As it turned out, probably not.

But the late-night musing of the CEO of Oakland-based Pivot Learning, a national nonprofit that works with school districts on improving classroom instruction, inspired a deep exploration into factors behind a successful school district. 

And that led to the release last week of the District Readiness Index. An interactive website based on publicly available data, it lists the organizational strengths that are vital to the creation of enduring school improvement. It then measures how well 420 California school districts — those with at least 2,500 students — meet them.  

Too often, districts focus on new programs and curriculums but don’t consider underlying conditions. Then they wonder why the reforms don’t take place or stick, Ramanathan said.

“Despite billions in investment and decades of effort, new reform efforts rarely have intended impacts at the district, school and classroom level, and sustained improvement is rare,” he said. “There are rarely insights into why. The District Readiness Index resulted from that questioning.”

Now, with a “shared urgency for improving student outcomes,” there is a need to understand the elements behind a district’s capacity for change, he said, and to learn from other districts that excel in areas that your district doesn’t.

“Before you start adopting and implementing a bunch of new programs, you really want to make sure that you have these foundations ready to go,” said Sean Arseo, Pivot Learning’s research manager who guided the project. “Because if you don’t have the staffing you need, the leadership to see it through, the community relations that help you implement it, you’re not going to see the return on investment that you’d like.”

Three years and multiple iterations in the making, the index consists of five foundational areas of operation that would affect whether an initiative succeeds: family and community engagement, financial management, leadership and governance, school personnel and work environment. For each foundation, the index uses a point system to measure five to eight indicators of readiness for improvement.

For example, two measures of school personnel are whether the principal retention rate is greater or equal to the national average and whether inexperienced teachers are concentrated in high-needs schools. Measures of work environment include whether a district offers bonuses for special assignments or certifications, whether teachers and the district signed a distance learning memorandum within 30 days during the pandemic in 2020-21 and whether midpoint salary for teachers is greater than or equal to similar districts’ statewide average.

Based on research and interviews, some indicators are given double weight, such as whether there were attempted or successful recalls of board members in the previous five years — a measure of cohesion in a district. Bonus points are given for adopting an equity policy and establishing a labor-management initiative.  

There’s a point total for each of the five foundational areas and then an overall district rating based on the scores of the five areas. Blue signifies strong foundations with mostly high scores positioning them to “initiate and sustain innovative efforts to improve student outcomes.” Yellow designates partial foundations with a mixture of high and low scores. Orange designates few foundations, with at least two of five areas with low scores.

About half of districts are yellow; a little more than 40% are blue, and two dozen districts — about 5% — are orange.

In line with perceptions

The index quantifies widespread perceptions of some of the most effective and the most troubled districts. San Francisco Unified, in the headlines over the past year for budget troubles, a recall election of three school board members and a malfunctioning payroll system, is an orange district, as are Oakland Unified and Los Angeles Unified, with low scores in finance and leadership. Ravenswood City Elementary, a low-income district abutting Palo Alto, is also orange, with low scores in family engagement and personnel, but strong in leadership.

It’s also not surprising that Garden Grove Unified in Orange County, a largely low-income district with stable leadership and many state-recognized “distinguished schools,” is among 70 all-blue districts with high scores in every domain. Others include Corona-Norco Unified and Jurupa Unified in Riverside County, Simi Valley Unified in Ventura County, Santa Clara Unified in Santa Clara County and ABC Unified in Los Angeles County.

And yet there are low-scoring indicators, signaling room for improvement, in every district as well as areas of strength in nearly all. The index was designed as a tool for district improvement, not for accountability. It’s set up to provide comparisons among similar and neighboring districts.

Matt Navo, executive director for the California Collaborative for Educational Excellence, a state agency that oversees district improvement efforts, said the index corresponds to data that the agency uses for its technical assistance work. Pivot Learning “has done a good job for the purpose of generating conversations on why districts are rated and what they can do differently,” he said. “It could be a way to create public dialogue, or for districts to use internally.”

But if used to “weaponize and punish districts,” districts will dismiss and ignore it, he said.  

That is not the intent, said Aliza Husain, Pivot Learning’s vice president of research and evaluation. “We don’t want it to be a tool that others can use to say, ‘This is a bad district. We don’t want our kids to go to school here,’” she said.

A Pivot Learning analysis found that urban districts encompass the plurality of districts with few of the five foundational areas and that a district with few foundations on average enrolls twice the proportion of Black students and about 40% fewer white students as a randomly selected district.

Black students are especially concentrated in districts with lower scores in financial management and school personnel while English learners are also underrepresented in districts with high scores in school personnel, Pivot found.

School personnel and financial management, along with community and parent engagement, also are the areas that were most predictive of a district’s scores on the latest Smarter Balanced assessments, Pivot Learning said. With some exceptions, greater levels of readiness are generally associated with higher math and English language arts scores, it said.

Through time and effort, Pivot Learning was able to compile new, useful indicators from multiple sources. These include rates of teacher turnover, the number of impasses in negotiations with teachers, and the extent to which students attend schools where teachers look like them. This is the first time that all the measures can be compared in one place.

Imperfect proxies

Ramanathan and Arseo acknowledge that some of the indicators are imperfect proxies — trade-offs when creating a project at scale that relies on quantitative, not qualitative measures. An attempted recall of board members by a disgruntled minority may not be an accurate indictment of leadership, for example. Similarly, the number of labor impasses may not accurately capture a toxic work environment. “In the actual day-to-day interactions, we’re not really getting at how do people work with one another? How do they collaborate?” Arseo said.

“Nuances matter,” Navo said. “The index can give a sense of what is happening, but it’s a starting point, not an end.”

Three of the five measures of community and parent engagement are based on districts’ self-evaluations for their annual budget document, the Local Control and Accountability Plan. An administrator’s candor can make the difference between yellow and orange.

Still, it’s useful data for the public to see. Victor Valley Union High School District in San Bernardino County, which rated orange for low marks in personnel management and community engagement, acknowledged it had made little progress in systematically working with and building trusting relationships with families.

There is clear evidence of the need to do better. The nonprofit Public Advocates filed a series of complaints that Victor Valley wasn’t straight about how it spent money for low-income students; in August, the U.S. Department of Education found that the district overdisciplined Black students, violating their civil rights.

Pivot Learning sees the initial index as a base, with future iterations adding new measures and modifying others. But Pivot, which will soon merge with UnboundEd, a national nonprofit that does complementary school improvement work, is hoping to hand off the index to someone else, perhaps a university with research capacity.

The index’s purpose and benefit shouldn’t change, Husain said.

“If this tool works out the way we want, I think it will be the background to work that any district does,” she said.

Do you count on EdSource’s reporting daily? Make your donation today to our year end fundraising campaign by Dec. 31st to keep us going without a paywall or ads.