Feed on
Posts
Comments
How can authorizers track progress in the absence of commonly accepted measures?

Photographed by Jeff Sheldon

In Washington DC, there are 66 early childhood public charter school programs that serve more than 15,000 students from age 3 to second grade. Collectively, they use more than 30 different assessments to measure reading and math skills and evaluate their programs’ academic performance… and that’s just the tip of the iceberg. Standardized testing kicks in at the 3rd grade, but accountability is just as important for schools that serve children at a younger age.

 

How can authorizers track progress in the absence of commonly accepted measures?

The Public Charter School Board (PCSB) in Washington, DC measures school quality using its Performance Management Framework (PMF) for grades 3 -12 and  adult education programs. In thinking about how to evaluate early childhood programs, we first noticed that some of the reading assessments measured several different literacy skills while others only evaluate one, such as vocabulary identification. This makes comparisons and identification of quality programs problematic for our board, as well as for parents and other stakeholders. Another challenge we have found was that the PCSB’s enabling statute states that it cannot mandate all EC schools to adopt the same assessment. Charter schools have the freedom to choose assessments that align with their mission, while DC’s public schools use the same assessments to measure the quality of all of their early childhood programs. An additional hurdle is determining how quality indicators are typically defined for EC programs. Early experts tend to look at inputs as indicative of quality EC program, such as the ratio between the number of students to teachers or the quantity of teacher certifications – but, again, that very Washington, DC law does not allow authorizers to hold schools accountable for these inputs. We determined that in order to assess school quality, and thus the academic performance of its programs, we would need to measure student outcomes.

 

Developing an Early Childhood Performance Management Framework

Beginning in 2013 we met monthly with the PMF Task Force, comprised of representatives from charter schools and charter advocacy groups. This group was tasked with creating a common accountability framework for early childhood programs and considering the pros and cons of implementing different approaches. Task force members focused especially on student progress and/or achievement measures. To fully understand what the measures implied we investigated the assessments schools had already been using, and researched the grade level expectations determined by each publisher. We also identified a cohort of students whose second-grade achievement on the assessment and third-grade proficiency rate on the state assessment could be reviewed. This step helped us determine if the grade level expectations were reasonable yet rigorous enough to help students succeed by third grade and beyond.

After several task force meetings we determined that the early childhood PMF would give each school an overall numerical score and locate it within a tier, the same general approach we have used in our other PMF, based on:

  • Student progress for those in pre-kindergarten (ages 3 and 4)
  • Student achievement or progress for grades K-2
  • CLASS, spell out and identify, scores for pre-kindergarten classrooms
  • Attendance
  • Parent satisfaction as indicated by the number of families re-enrolling their child

Schools are rated tier 1, 2 or 3, with 1 being the best. Tier 1 schools are high performing scoring between 100-65; tier 2 or mid performing schools score 64.9-35; and tier 3 or low performing schools score 34.9 to 0.

 

Strengthening the Framework

As with any plan, report or assessment tool, our work had just begun. Last year PCSB released the first Early Childhood Performance Management Framework (EC PMF) for the 2013-14 school year. Because this new framework was drastically different from previous early childhood accountability plans the task force asked us, PCSB, not to publish the overall score and tier. As we plan to release the second EC PMF for the 2014-15 school year we realize that the framework needs to be strengthened in order to show the difference between high and low quality programs. We are focusing more on ensuring that this PMF highlights schools which are truly preparing students to achieve proficiency by the third grade. This means extending the dialogue with the PMF Task Force and schools that continue to use multiple assessments. Our number-one priority is to provide a PMF that measures real rigor and positive outcomes for students.

We know that designing a framework to measure early childhood program quality is a work in progress. As we work to strengthen the early childhood performance framework, our door remains open for comments, suggestions and ideas. If your organization is also struggling to develop an early childhood performance management framework, please sign up here or drop a comment below.

 

 

Erin Kupferberg is a manager in the school quality and accountability department at the DC Public Charter School Board. The board is responsible for academic achievement for the 112 public charter schools in Washington, DC.

I offer a tip of the hat to the State University of New York (SUNY) Charter Schools Institute for their oversight of a struggling charter school that posed exceedingly tricky politics.

This whole thing was political from the start. The New York City teachers union, the UFT, created the charter school intending to demonstrate that running a school under the city’s collective bargaining agreement was not an impediment to success. It didn’t turn out as planned. The school has struggled. With the exception of the high school from this K-12 endeavor, performance has been unacceptable. Last week the school’s board decided to close the elementary and middle-grade portions of the school.

That is an appropriate action that is in the best interests of the students. It makes sense to close a chronically low-performing charter school. But it is easy to imagine this process devolving into the ugly politics of a proxy fight. That didn’t happen. Credit for the relatively smooth course of events should go to the SUNY authorizing shop–in addition to the board from the school.

One of the strategies SUNY employed, which we have discussed on this site previously, was a short-term renewal with a set of mandatory benchmarks. When the school came up for latest renewal in 2013, it was granted a two-year renewal. The renewal contract listed outcomes that the school had to meet within two years to be renewed. By agreeing to the pre-established standards, both sides agreed the school would close if it did not meet the benchmarks.  A similar strategy has also been used in recent years by Denver Public Schools.

At NACSA we generally recommend against short-term renewals of failing schools. Ideally, the original charter contract is based on similar benchmarks. A performance framework, based on multiple measures, articulates metrics that inform the contract. That contract clearly outlines what is expected of every school if it is going to be renewed. SUNY’s leadership and other stakeholders in New York have argued that the first five years is not enough time to have measures of growth. So, the authorizers in the state frequently employ a short-term renewal strategy.

Continue Reading »

Buckeye Blues Again

Back in 2006, NACSA, the National Alliance for Public Charter Schools, and the Thomas B. Fordham Institute issued ‘Turning the Corner to Quality,” a tough report on Ohio’s charter sector whose message was summed up in its first major recommendation: “Clean House.”  There were too many failing charters, oversight had gone from bad to worse after the legislature removed chartering authority from the state education department, and the state’s charter cap was effectively shutting out strong operators.

In the intervening 8 years, a lot of good things have happened, including successful charter ventures like Cleveland’s Breakthrough Schools;  a default-closure law that has eliminated 24 low-performing charters; and most recently, a concerted effort by the state agency’s Quality School Choice office, led by former NACSA staffer David Hansen, to bring accountability to the state’s multitudinous authorizers.

Yet the muck persists. Last week, CREDO at Stanford reported that on the whole, students in Ohio’s charters are getting 14 fewer days of learning in reading and 36 fewer days in math than their counterparts in district-run schools.  There are some bright spots. Cleveland charters outperform the district; performance is better in charters for black students and those in poverty; middle schools do comparatively well; and there seems to be a trend toward improvement among urban charters. But overall performance hasn’t improved since CREDO’s 2009 Ohio study, and is particularly weighed down by woefully deficient results in the roughly half of Ohio charters operated by charter-management organizations.

There’s been some giddy blogging about one comment made by the report’s author Macke Raymond, in a Columbus speech introducing the findings: “[Education] is the only industry/sector where the market mechanism just doesn’t work.” Expect to hear that repeated ad nauseam as an indictment of choice itself,  which it assuredly is not. Raymond’s early work was on regulation in the telecommunications industry and she has always linked good outcomes to good oversight.  And in Ohio, you simply can’t discuss the weak results without asking what role authorizers are playing — as noted in the rest of her comment: “We need to have a greater degree of oversight of charter schools. But I also think we have to have some oversight of the overseers.”

So it was good to see that issue taken up forcefully in another report issued yesterday by Bellwether Education Partners in conjunction with the Fordham Institute (which, let’s remember, is not only a DC think tank but also a successful Ohio authorizer).  My former Alliance colleague Andy Smarick, who’s emerged as a formidable education thinker and activist, is co-author of the report, which bears the fire n’ brimstone title “The Road to Redemption.”

Continue Reading »

OntheRodLinkedIn recently reminded me of my five-year anniversary at NACSA.  Like most unsolicited emails from an App, it was an unwelcome reminder that an algorithm in a server farm is better at tracking my life than I am. It also led me to think about the last decade of charter school policy development. That is timely because this week NACSA released a report on policies that support strong authorizing. On the Road to Better Accountability: An Analysis of State Charter School Policy, presents eight policies we recommend that support accountability and excellence in the charter sector and details the extent to which each state has adopted similar policies.

We hope this report will inform and stimulate discussions about each state’s charter authorizing environment.  In the coming weeks, I’ll write about specific policies and lessons learned. In this post, , I’d like to share a note on where these ideas came from.

All of the ideas in this report – bar none – came from the inspiring work of authorizers, charter educators, policymakers, and leaders in our field.

My colleagues at NACSA have been busy for the past 15 years.  They have reviewed more than a hundred charter applications for various authorizers; they have evaluated or provided technical assistance to authorizers responsible for overseeing about 40 percent of the charter schools in the country. Our staff have been on the ground, evaluating authorizers, helping them to design and implement tools for performance management; and they have helped emerging professionals engage in the work of authorizing.

The experiences of these authorizers informs all our work at NACSA. In my five years, one of the tasks has been to bridge between that knowledge and a parallel set of knowledge we have gained by working with charter advocates and authorizers in the field involved in state policy discussions.  We are regularly invited into state-level policy discussions by authorizers, charter school advocates, legislators, governors, and other education leaders and asked to respond to policy proposals.

Leaders in the charter school sector often advance what opponents of charters consider “ironic” proposals. That includes proposed laws from charter support organizations seeking to mandate the closure of failing schools in Ohio and Florida; or schemes to sanction the worst authorizers, promoted by pro-charter chiefs in Indiana and charter advocates in Minnesota.

The heads of state-wide authorizers, in places like Nevada and South Carolina, asked NACSA to help them establish policies that gave them the authority and responsibility to use performance frameworks, performance contracts, and differentiated strategies to replicate high-quality schools.

After policies were underway, we were asked to help refine policies.  In Indiana, the State’s Charter School Board and major authorizers sought help in closing the worst schools. Soon they needed to figure out how to stop authorizer hopping by failing schools that were attempting to switch from a rigorous authorizer, intent on closing their school, to one willing to let them stay open.

In Colorado, I had the pleasure of personally chairing a state task force composed of school district leaders and charter school and authorizer representatives. That effort was prompted by the Colorado League of Charter Schools. We were charged with reviewing and recommending standards for charter school authorizing that were based on NACSA’s own Principles & Standards and eventually adopted by the State Board in state rules.

Continue Reading »

Open for Comment

Whenever a federal announcement capitalizes 19 out of 32 words (including prepositions and conjunctions) you can bet it is important. It can also require translation. Last week, the U.S. Department of Education announced the publication of its “Notice of Proposed Priorities, Requirements, Definitions, and Selection Criteria (NPP) for the Charter Schools Program (CSP) Grants to State educational agencies (SEA).”  The CSP grant for SEAs is the primary source of funding for States that award start-up grants to new charter schools.

Here’s what this means:

  • The feds released draft materials that outline how they plan to evaluate applications and award the grants they give to states under the CSP.
  • After a comment period (going on NOW), the feds will finalize priorities, requirements, and definitions and release a final Notice Inviting Applications for New Awards—this will likely incorporate some of the material in this current notice.
  • Many states will decide to compete for this program. States that have programs and policies that best match the new priorities will have a slight advantage in the upcoming competition in 2015. State applications will also be scored in some manner based on the plan the State puts forward to carry-out certain activities reflected in the proposed selection criteria.
  • If a state receives a grant, the new provisions may place a few new requirements on these states regarding charter school oversight, as well as on what states, authorizers, and schools do to ensure that students with disabilities’ have access to charters and are well served when they enroll.

NACSA’s preliminary analysis indicates that several of the new provisions will support and incentivize state action to strengthen accountability in the charter school sector, but, of course, there are also details that should be changed.

This announcement is important to our members for two main reasons:

  1. Your state may be interested in applying for a CSP grant, and this is a great time to start planning. Once the feds receive input on the draft materials, they will release final materials and run a grant competition in late winter or early spring of 2015.
  2. You can have input on the priorities, requirements, definitions, and selection criteria for the CSP grants to SEAs. The public has until January 5th, 2015 to submit written comments. NACSA will provide input on the draft materials, and we are reaching out for input from the field.  Please contact Whitney Spalding Spencer (whitneys[@]qualitycharters.org) if you have comments or just want to stay in the loop on the process.

If you’re interested in the CSP SEA grant competition, be on the lookout for more info from NACSA or reach out to Whitney Spalding Spencer to share your thoughts and get more info.

 

NACSA President and CEO Greg Richmond comments on Sarah Carr’s piece  in The Atlantic on discipline in New Orleans’ charter schools over at Education Post:

…Carr’s article does an excellent job of describing how school leaders, parents, students and communities are re-thinking how the implementation of high behavioral expectations happen on a day-to-day basis.

Many of them come to appreciate the intense structure, but only if they also come to trust the mostly young educators who enforce it. As school leaders in New Orleans are discovering, forging that trust is far harder than teaching someone to say thank you and toe an orange line.

And importantly, it appears that part of that re-thinking is reducing the number of kids who are suspended and expelled—demonstrating that it’s possible to embrace “no excuses” and “strict discipline” without removing a lot of students from school as many effective schools have done for decades…

Read the whole thing here.

 

First Page PCSBA new case study, funded by the Michael and Susan Dell Foundation (MSDF), and conducted by the FSG consultants, was released today.  The study, Transforming Education in the Nation’s Capital, examines the practices of D.C.’s charter school authorizer, the District of Columbia Public Charter School Board’s (DCPCSB). This is a welcome analysis of a top-flight authorizer in the country. We need more detailed examinations like this one that highlight the practices used by authorizers to promote quality charter schools.

The study is important for several reasons.  First, it is empirically focused on a city where there is evidence of success.  As the study outlines, charters have a strong record of performance in the District. The results charters are getting there are among the strongest in the nation’s charter sectors.  They beg the question: what are people doing in DC to support this kind of sector-wide performance?  What role could DCPCSB be playing? This report offers some answers.

Second, it presents a broad range of actionable strategies that authorizers in other settings can bring to their own work. The case study isn’t describing a silver-bullet. Instead, it describes a body of work, and a set of practices that fit together in a comprehensive approach that addresses student performance, equity, and oversight in the public interest. No single piece of the DC strategy likely makes “the” difference. But other authorizers and policy makers can look at this set of practices, which align closely with NACSA’s Principles and Standards, and figure out what they can do to bring about similar action in their community.

Finally, it is provides a substantive and positive counter-foil to so much of what we see passing for policy debate.  Hopefully, additional case studies of other strong leaders in our sector will follow.  And more and more of the stories about what leaders, like Scott Pearson recently, and Josephine Baker before him, have done to give us the chance to acknowledge, and even celebrate, when things work.

We learned a lot about what is working this week at the NACSA Annual Leadership Conference in Miami. Close to 500 authorizers and others interested in advancing quality in the charter school sector shared effective practices, discussing common challenges, and learning from each other. We asked each other hard questions. We focused on difficult issues. We even got worked up sometimes about things that matter, not just to authorizers and others who care about expanding quality educational opportunities but also to the millions of children who are depending on our success. We also made sure to acknowledge the hard work, brave leadership, and successes that authorizers are achieving every day, all over the country.  We still have a lot of work to do, but we are making great progress. Those working in the charter sector see this progress all the time. It is nice to have somebody focus on it and describe it in detail.

 

Older Posts »