Virtual School Meanderings

August 15, 2018

What Effect Did The New Orleans School Reforms Have On Student Achievement, High School Graduation, and College Outcomes?: A Q&A With NEPC Fellow Douglas Harris

An item from the National Education Policy Center.  Readers should note that the same “solution” to public education that was imposed on New Orleans following Katrina is now being imposed on Puerto Rico following Irma.

What Effect Did the New Orleans School Reforms Have on Student Achievement, High School Graduation, and College Outcomes?: A Q&A With NEPC Fellow Douglas Harris

Tuesday, August 14, 2018


What Effect Did the New Orleans School Reforms Have on Student Achievement, High School Graduation, and College Outcomes?: A Q&A With NEPC Fellow Douglas Harris


TwitterEmail Address

In 2005, Hurricane Katrina forever changed the city of New Orleans and its schools. In the wake of the deadly storm, the educational system underwent a series of dramatic and controversial transformations. Nearly all the schools were converted from traditional public institutions to publicly-funded, independently-operated charters. Families selected schools rather than being assigned to buildings close to their homes. The district dismissed all of its employees, and many never returned. Per-pupil funding increased by 10 to 15 percent.

How did all of this change impact key educational outcomes?

In July, National Education Policy Fellow Douglas N. Harris and his co-author Matthew F. Larsen addressed this question in What Effect Did the New Orleans School Reforms Have on Student Achievement, High School Graduation, and College Outcomes?, a policy brief and accompanying technical report published by the Education Research Alliance for New Orleans at Tulane University.

The study found that student achievement, high school graduation rates and college outcomes all improved in the wake of the reforms. In the weeks since its publication, New York Times columnist David Leonhardt has cited the study in support of charter schools, while charter skeptics have raised objections.

In the Q&A below, Harris discusses the study and its results. 

QNEPC has done a lot of work focused on closing opportunity gaps, based on the foundational premise that students learn more when they have greater opportunities to learn – and that achievement gaps are the direct result of opportunity gaps. Those opportunity gaps arise from many different sources, inside of schools and outside of schools. Have you and your colleagues found that the post-Katrina schools in New Orleans are increasing opportunities to learn (OTL) in concrete ways? If so, what are some examples?

Harris: OTL usually includes things like instructional quality, curriculum, class sizes, and school climate. In New Orleans, the student-teacher ratio is essentially unchanged. Teacher experience is lower. On other measures, perhaps the best evidence we have comes from a survey of teachers who taught in New Orleans schools both pre- and post-Katrina. The results present a mixed picture. School climate and support for teachers improved, but teachers are less satisfied with their jobs and with the evaluation process.

We don’t want to stop there, however, because OTL is hard to measure and because, while OTL measures are positively related to student outcomes, their effects on student outcomes are generally small. Therefore, it’s also worth thinking about outcomes and achievement gaps—whether opportunity to learn turns into measurable learning. We found that all racial/ethnic and income groups saw increased outcomes on every measure in New Orleans—test scores, high school graduation, college entry, college persistence, and college graduation. Also, achievement gaps within the city declined or remaining unchanged on almost all of these measures.

QCharter school research from next door in Texas, conducted by Will Dobbie and Roland Fryer Jr., found that later earnings outcomes are actually worse for students in charters, even when test-score outcomes improve. Outcome measures for charters are better in places like Boston. More generally, research measuring differences in outcomes between charter schools and public schools suggests that such differences are non-existent or very small. Do you have a hypothesis or a hunch as to why NOLA’s results would be better? Are the examples that you cite, things that NOLA is doing differently than, for example, Texas charters?

Harris:The differences in outcomes between charter and traditional public schools are indeed small. In the early years, charters looked slightly worse on average, and now they look slightly better. An important question going forward is whether that upward trend will continue.

One clear pattern in the research is that “no excuses” schools seem to have more positive effects on typical student outcome measures than other kinds of charter schools. This is true in Boston as well as in the Dobbie and Fryer study. (Actually, the pattern with no excuses also aligns with the old effective schools literature.) New Orleans, too, has had a large share of schools that might be described as no excuses.

No excuses schools also tend to spend more money, and we do see higher spending in New Orleans. It may be the combination of schooling model and spending.

Q: The technical report discusses six possible threats to validity in the study, one of which concerns the question of how much of the measured outcomes might be due to increased resources. The school-choice reforms were accompanied by an increase of almost $1,400 annually per student. You’ve noted that this increase in spending likely contributed some to the overall outcome improvement that you found. But you also noted that the increased funding would not have been forthcoming if the district didn’t adopt the school-choice reforms. So you said that it’s hard to interpret the funding increase as a cause that’s separate or alternative from the school-choice reforms. This leads to several questions: How do we know how much (if any) of the outcome improvements was due to resources and how much (if any) is due to school choice? Given the research from Kirabo Jackson and his colleagues, finding that a 20 percent increase in school spending led to 25 percent higher earnings and a 20 percentage-point reduction in the incidence of adult poverty, can you estimate how much of the outcome improvements you’re seeing in NOLA might be due to, e.g., school-choice reforms and how much due to the increased spending? I.e., what percentage of the variability is explained by school funding increases versus other factors?

Harris: First, it’s important to recognize the important contributions of the studies by Jackson, Johnson, and Persico and Lafortune, and Lafortune, Rothstein, and Schanzenbach in identifying causal effects of school spending. Money matters, and that’s why we carried out the study showing the increase in funding that you’re referring to.

It is difficult to estimate the role of funding or really any specific factor since this was a system-level change, involving several interconnected factors. Certainly, getting something as precise as “the percentage of variability” is impossible. Even if we could, there is the other issue you raised—that the increased spending was partly caused by the reforms.

QWhat lessons do you think the NOLA reform offers for places that don’t include the hefty funding increase?

Harris:The effects would almost certainly be smaller, but it’s hard to tell by how much. A related question is, what would have happened if we had kept the old system, but still increased funding? The answer is probably “not much.” Even today’s critics of the charter-based reforms say that the district was in need of an overhaul. Pouring money into a failing district isn’t the answer, nor is it politically plausible in the long run.

QA columnist for the New York Times has been using your study as part of his advocacy for charter schools. This has drawn some responsesas you know. To be fair to the columnist, an op-ed isn’t amenable to a discussion of the potential limitations mentioned in the technical report. But to what extent, if any, is it appropriate to draw upon your study to advocate for charter schools in settings outside of New Orleans? To what extent, if any, have the New York Times columns extrapolated lessons that are beyond the scope of your findings? 

Harris: In some sense, any policy advocacy based on research requires some degree of extrapolation. Certainly, that’s true here as well. As we emphasized in the summary of our briefing paper, New Orleans was uniquely situated for these reforms to work. The district was extremely low-performing and pretty much everyone agreed that some type of major change was in order. It’s easier to improve from such a low starting point. Also, the national interest in rebuilding the city and being part of the reform effort made it easier to attract educators, especially in the early years. Cities tend to have advantages over suburbs and rural areas as well. In short, I don’t think we can extrapolate New Orleans to most of the country. It’s more like a best-case scenario.

QFrom outside of New Orleans, it seems like a great deal has been changing not just in the city’s schools, but also in its communities—which have had to be rebuilt as well. NEPC’s opportunity-gap work has been strongly influenced by the body of research showing that outside-of-school difference in resources and opportunities are substantially more important for kids’ educational outcomes than are inside-of-school differences. What sorts of changes, if any, have you and your colleagues seen in New Orleans since the reforms that might be important? Are those differences ones that researchers can account for?

Harris: Researchers usually break these “outside-of-school” factors into two categories: home and community. The family income and parent education of individual students, in particular, are strong predictors of education outcomes. I think you’re coming to a question about population change later.

On community factors, New Orleans, unfortunately, has a long history of violence, mass incarceration, racism, and deeply impoverished neighborhoods. We are currently studying the possibility of school reform effects on crime. All we know at this point, however, is that crime and incarceration rates are somewhat lower in New Orleans compared with pre-Katrina, but this is also true statewide. So, while we see no obvious indications of relative improvement on crime and incarceration, as indicators of progress in the community, we can’t draw any conclusions about them at this point.

Q:How, if at all, did your study account for any changes in the rate of concentrated poverty at the school level pre- versus post-Katrina?

Harris: The New Orleans reforms affected the whole system and, as we’ve discussed, poverty and demographics in the public schools were essentially unchanged. In other words, it’s really the overall public school demographics in the city that could affect our results. Since those demographics changed only very slightly, it’s hard to see how this could influence the results.

But the concentration of poverty by school is important for other reasons and we’ve studied that, too. In an earlier study, with Lindsay Bell Weixler, Nathan Barrett, and Jennifer Jennings, we focused on cross-school segregation across a wide range of student characteristics—race, income, special education, English Language Learners, and achievement. We found, first, that the schools were very segregated prior to the reforms. After the reforms, we found no systematic changes in segregation in these measures at the elementary level. At the high school level, we see increased segregation on poverty and race, but decreased segregation on achievement and special education.

Q:  You and your colleagues clearly attempted to control for pre- and post-Katrina changes in population. To what extent do you think that the demographic variables available to you as a researcher can account for the change in the composition of the student population that occurred pre- versus post-Katrina? To what extent is it the case that the pre-Katrina student population was substantially more disadvantaged than the post-Katrina population?

Harris: The New Orleans population was disadvantaged before and that stayed about the same. We’re confident about that because we come to the same conclusion from three entirely different types of analysis. First, we looked at the share of students eligible for free or reduced prices lunches. Second, we commissioned the U.S. Census Bureau to provide detailed family data on public school students, before and after the reforms. Third, we looked at the pre-Katrina scores of students who returned to the city and compared those with students who did not return. All three analyses suggest the demographics changed very little, so little that they can’t have more than a negligible effect on student outcomes. Some of the analyses even suggest that students became more disadvantaged after the reforms, indicating that we might be slightly understating the reform effects.

Just to clarify, the population of the city has definitely changed, but the demographics of the city don’t mirror the population of the public schools. Many families don’t have children, and New Orleans sends about 25 percent of school-age children to private schools (that hasn’t changed much either). The overall city demographics therefore aren’t very informative.

QCollege enrollment/persistence/completion rates are probably not impacted by gaming or accountability in K-12. But to what extent might the improvements in test scores and high school graduation rates post-Katrina New Orleans be attributable to gaming incentivized by high-stakes accountability?

Harris: Gaming the system on tests, such as drilling test questions, is the hardest to gauge. There’s no question that New Orleans schools are data-driven and under tremendous pressure to get scores up, so there is probably some of that going on. On the other hand, Louisiana has aggressive accountability statewide, so it could be that other districts, which comprise our comparison groups, may also be doing this, so some of it washes out when we estimate the effects. Unfortunately, we can’t use the NAEP to help us out here.

With high school graduation, the potential for gaming is there, but we were able to test for it directly—and we don’t see any evidence of it. And, as you say, this isn’t an issue with college graduation.

Q: I wonder if you could tell us how we should think about the internal validity of the study (i.e., how should a reader define and bound the intervention, and how sure are you that you’re measuring that intervention?). Similarly, how should readers think about the external validity of the study? To what extent should someone in Memphis or Minneapolis or Modesto assume that they would get similar results if they followed the New Orleans playbook, and what are the key elements of that playbook?

Harris: Overall, I would say that internal validity is stronger than external validity. With internal validity, the main complicating factor is the role of funding. One option is to just think of the increased spending as part of the reform “package.” The other is to try and isolate the two, but as we discussed earlier, that’s complicated by the fact that the reforms partly caused the funding increase.

On external validity, we have good reasons to expect that market-based reforms work better in urban areas and that any reform will tend to generate more improvement when the existing system is failing—in the sense of generating little outcome growth or value-added. So, this type of reform, and others, are more likely to work in the very lowest performing urban districts in the country. Also, having a lot of universities and college-educated workers will make it easier to attract educators who fit the educational models that charter schools want to create.

It’s important to note that these conditions aren’t very common nationally. Again, New Orleans is probably a best-case scenario in terms of student outcomes.

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at:

Copyright 2018 National Education Policy Center. All rights reserved.

August 13, 2018

Are Private Schools Really Better?

An item from the NEPC that should up in my inbox late last week.  I have to be honest and say I’m not familiar with any research that has been done on this topic in relation to private online schools – but there are a number of them out there.

Are Private Schools Really Better?

Thursday, August 9, 2018


Are Private Schools Really Better?


TwitterEmail Address

It is conventional wisdom: Private schools are “better” than public schools. But is it really true? A growing body of evidence suggests the answer is no. A recent installment of the Washington Post’s Answer Sheet column reported on the results of a study conducted by Robert Pianta and Arya Ansari of the University of Virginia. They tracked more than 1,000 public and private school students from birth to age 15. At first glance, the private school students appeared to be performing better on multiple measures. But the differences were deceptive. As soon as the researchers accounted for family income and parental education levels, the advantage evaporated. Private school students did not outperform their public school peers on academic tests, assessments of social adjustment, attitudes and motivation, or on behavioral metrics.

These most recent findings echo those of a 2013 book, The Public School Advantage, co-authored by the husband-wife team of Sarah Theule Lubienski and NEPC Fellow Christopher Lubienski. They too found that, compared to their private school counterparts, public school students performed the same if not better on achievement tests once demographics were taken into account.

Other studies have found that children do worse when they use vouchers to move to private schools—studies that NEPC Fellows Kevin Welner and Preston Green discuss in a 2018 working paper for the UCLA Civil Rights Project.

Combined, this research casts serious doubt on the idea that private schools are somehow better. The simple truth is that schools are “better” when they provide richer opportunities to learn. Some private schools have the resources to make this happen; others do not.

This research also raises obvious questions about the merits and wisdom of voucher programs that funnel public funds into private education via cash payments or tax credits. Such programs are popular with current U.S. Department of Education Secretary Betsy DeVos. Earlier this year, for example, DeVos proposed using public funds to help military families’ children attend private schools—until the plan met with opposition from military groups that did not want their public schools undermined.

Additional Resources on Public versus Private Schools:

Dynarski, M., Rui, N., Webber, A., & Gutmann, B. (2017). Evaluation of the DC Opportunity Scholarship Program: Impacts after one year (NCEE 2017-4022). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education.

Figlio, D., & Karbownik, K. (2016, July 7). Evaluation of Ohio’s EdChoice scholarship program: selection, competition, and performance effects. Washington, DC: Fordham Institute.

Lee, J. (2007). NEPC Review: Are private high schools better academically than public high schools? and Monopoly versus markets: The empirical evidence on private schools and school choice. Boulder, CO: National Education Policy Center.

Lubienski, C. & Lubienski, S. (2006). NEPC Review: On the public-private school achievement debate. Boulder, CO: National Education Policy Center.

Lubienski, C & Lubienski, S. (2013). The public school advantage. Chicago, IL: University of Chicago Press.


Mills, J.N., & Wolf, P.J. (2017, February 17). Vouchers in the bayou: The effects of the Louisiana scholarship program on student achievement after two years. Education Evaluation and Policy Analysis, 39(3), 464-484.


Pianta, R. & Ansari, A. (2018, July 9). Does attendance in private schools predict student outcomes at age 15? Evidence from a longitudinal study. Educational Researcher

Welner, K. & Green, P. (2018). Private school vouchers: Legal challenges and civil rights protections. Los Angeles, CA: The Civil Rights Project.



The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at:

Copyright 2018 National Education Policy Center. All rights reserved.

August 10, 2018

NEPC – Diane Ravitch’s Blog: National Education Policy Center: Virtual Charter Schools are a Sham and Waste Taxpayers’ Dollars

This showed up in my inbox or RSS reader earlier this week.

Diane Ravitch’s Blog: National Education Policy Center: Virtual Charter Schools are a Sham and Waste Taxpayers’ Dollars

The National Education Policy Center recently issued a bulletin about the negative results of virtual charter schools. To see all the links embedded, open the NEPC report. Betsy DeVos wants more of these fraudulent “schools” to open.

It is no secret. The news media is full of reports about problems with cyber schools. Some recent examples include:

In January 2018, the nation’s largest virtual school, Electronic Classroom of Tomorrow (ECOT), closed. There was a subsequent failure to determine what happened to 2,300 of 11,400 students. The school shut down after the state of Ohio found that ECOT had overstated its enrollment by more than 9,000 students, resulting in a $60 million overpayment.

The Akron Digital Academy quietly closed last month because it could not repay the state the $2.8 million it owed for failing to correctly track enrollment. Akron Public Schools dropped its sponsorship of the school in 2013 due to problems such as poor student performance.

The state of New Mexico is in the process of shutting down the state’s largest virtual school, also for poor academic performance.

To continue reading, visit

August 9, 2018

As School Segregation Intensifies, What Can We Do About It?

Note this item from the National Education Policy Center.

As School Segregation Intensifies, What Can We Do About It?

Tuesday, August 7, 2018


As School Segregation Intensifies, What Can We Do About It?


TwitterEmail Address

In a recent piece in The Atlantic, Will Stancil concludes that school segregation is on the rise in America:

According to my analysis of data from the National Center on Education Statistics, the number of segregated schools (defined in this analysis as those schools where less than 40 percent of students are white), has approximately doubled between 1996 and 2016. In that same span, the percentage of children of color attending such a school rose from 59 to 66 percent. For black students, the percentage in segregated schools rose even faster, from 59 to 71 percent.

Stancil points to multiple causes, including the termination of hundreds of court-ordered desegregation plans, the secession of white neighborhoods and cities from large, diverse Southern school districts, and the expansion of highly segregated charter schools that are typically exempt from desegregation efforts.

In her NEPC policy brief, Professor Amy Stuart Wells describes these and other factors impacting segregation as “ostensibly ‘colorblind’” education policies. Such policies are not necessarily focused on race, but they intensify racial segregation and race-based achievement gaps, nonetheless.

Wells concludes by recommending that the nation create and sustain more racially and ethnically diverse schools. She suggests accomplishing this objective by:

  • Supporting and sustaining diverse districts and communities in conjunction with local zoning boards, developers, real estate agents and the U.S. Department of Housing and Urban Development’s (HUD) effort to “Build Integrated and Sustainable Communities.”
  • Fostering cross-district cooperation and collaboration with magnet programs and other special offerings.
  • Encouraging inter-district transfers to promote diversity.

Wells also stresses the necessity of supporting curriculum, teaching, and assessment that emphasize the educational benefits of diversity. Means of working toward this goal include:

  • Expanding legal challenges based on the educational benefits of diversity. Stancil notes that these benefits include stronger academic achievement, reduced exposure to the criminal justice system, superior professional and educational outcomes, better connections that lead to jobs or college, reductions in racial prejudice, and higher levels of preparation for living and working in diverse communities.
  • Tapping into the progressive potential of the Common Core State Standards to focus more on deeper learning and critical citizenship.
  • Placing less emphasis on standardized tests that often lead to poor outcomes for students of color.

Wells concludes:

Much can be done in the policy arena to support and further the educational benefits of diversity. Such efforts will only progress, however, in a context in which people in power admit that they and their constituents can indeed see color. In fact, we know that seeing is believing in the potential of the most racially and ethnically diverse democracy in the world.

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at:

Copyright 2018 National Education Policy Center. All rights reserved.

August 3, 2018

The Demise of Channel One News

An item from the National Education Policy Center.

The Demise of Channel One News

Thursday, August 2, 2018


The Demise of Channel One News


TwitterEmail Address

It happened not with a bang but a whimper: Houghton Mifflin Harcourt announced that its in-classroom TV service, Channel One News, had aired the final broadcast of its 29-year history.

The announcement attracted scant attention, especially for a project described breathlessly by a 1995 New York Times article as “the highest-profile—one might even say glamorous—controversy in American education.”

Founded in 1989 by entrepreneurs Christopher Whittle and Ed Winter, Channel One sparked debate by requiring participating schools to show students 12 minutes of commercial-filled television daily in classrooms. In exchange, the schools received equipment such as satellite dishes and televisions. Critics, including the Campaign for a Commercial-Free Childhood, objected to the insertion of corporate advertising into schools. Research co-authored by Alex Molnar, co-director of the NEPC’s Commercialism in Education Research Unit (CERU), found that the broadcast cost taxpayers an average of $229 per pupil in 1994 dollars (which would equal $389 in 2018 dollars) in lost school time.

As co-director of CERU, NEPC research associate Faith Boninger is an expert on commercialism in schools. In this Q&A, she reflects upon the controversial project.

This interview has been edited for brevity and clarity.

Q:What is the significance of Channel One’s demise? In other words, does it signify anything about broader trends in education or commercialism in education?

A: It’s good news—there are now a lot of kids who will no longer be manipulated by Channel One and its clients. But unfortunately, there’s still a lot of marketing to children in schools. Many schools accept “free” digital resources, for instance, that put children at risk for information about them to be collected and used for marketing purposes. One significant example is Google products. Many schools use Google Classroom, which is free, and/or provide students Google Chromebooks, which are relatively low-cost, compared to other computers. This gets children to use, become familiar with, and like Google products. Also, YouTube is the second most accessed site from school Chromebooks. YouTube (owned by Google), of course, advertises to and collects information from its student users. This is a real problem.

Q: What does research have to say about the impact of Channel One on the students who were exposed to it?

A: Channel One was, first and foremost, a commercial enterprise designed to deliver a captive audience of children to advertisers. You can see the true goal of the project in its marketing materials to advertisers, in which it promised to deliver a teen audience that was forcibly prevented from engaging in any distraction. You can always tell a company’s true colors by how it portrays itself to advertisers.

Research on Channel One supported what other marketing research shows: that advertising is effective. For example, a study that compared students who had to watch Channel One in school to students who didn’t found that those who watched Channel One evaluated products advertised there more favorably. They also expressed stronger consumption-oriented values than did the students who didn’t watch—agreeing more that designer labels were an important feature of their clothing and that they usually wanted what was advertised on television.

Another study, co-authored by Alex Molnar, examined the true costs of Channel One to schools. The “sell” was that the school would get “free” equipment in exchange for making the kids watch the program every day. But it turns out that there are hidden costs to accepting “free” equipment. The study calculated the dollar value of the time schools are required to allocate daily to Channel One and compared it to the cost of the equipment provided. Even when they conservatively calculated the value of time lost from other educational experiences (counting only the daily two minutes of commercials rather than the full 12-minute broadcast time), they found that the cost of instructional time lost was substantially greater than the value of the equipment provided.

Q: What are some common critiques of Channel One? What are some common benefits perceived by the districts or schools that adopted it?

A: Critiques involved the abusive advertising to a captive audience (and its effects on children) and hidden costs. There was quite a bit of lost educational time. It takes more than 12 minutes per day (the length of the broadcast) if you include the transition time, also. The perceived benefit was the seemingly free equipment. Schools are strapped for money, and they are susceptible to offers of free equipment.

Q: What, if anything, currently exists that resembles the Channel One model? 

A: The same argument about there being “no free lunch” that applied to Channel One applies to ed-tech. Google is a major example, but only one; there are literally hundreds of companies looking to provide ed-tech to schools, and many of their products are “free.” Data collection is a hidden cost.

Q: Why do you think Channel One was discontinued? 

A: The business struggled in the face of continued criticism of how it took advantage of and manipulated children and schools. It changed hands. In recent years its business model had to change to advertise relatively more subtly. Instead of straight-out commercials, it would promote things like movies, or stars, in the guise of news segments on them. Houghton Mifflin Harcourt finally pulled the plug.

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at:

Copyright 2018 National Education Policy Center. All rights reserved.

Next Page »

Blog at