Virtual School Meanderings

June 11, 2019

Lessons From Long Ago… Evaluating Virtual Schools: Lessons For Evaluators And Policymakers

At some point over the past week, someone shared this document with me again.  It was a presentation given to the National Legislative Program Evaluation Society (NLPES) Fall 2008 Conference by Jeff Shinn of the Office of Performance Evaluations for the Idaho Legislature.  The presentation was entitled Evaluating Virtual Schools: Lessons for Evaluators and Policymakers, and you can access it at:

http://www.ncsl.org/print/nlpes/shinn0908.pdf

What I was struck by, as I reviewed these slides was the consistency that exists 11 years later.  For example, the third slide is titled “Issues for Policymakers and Evaluators” and reads:

Virtual schools often differ from “brick and mortar”) schools (traditional and charter)

  • Operation and oversight
  • Curriculum development
  • Instruction methods

Under the “Operations and Oversight” slide, it reads:

  • Definition
    • Virtual school vs. virtual program
    • Virtual school vs. charter school
  • Requirements of Approval process
    • Laws and rules specific to charter schools
    • General education laws and rules
    • Reporting and accountability

Later it reads:

  • Approval process and oversight are not tailored to virtual school operations
  • Current statutory definition is vague; no clear framework for virtual schools to operate within
  • Wide variations exist in school operations
    • Curriculum development
    • Delivery of instruction
    • Required student-teacher contact

The recommendations made during the presentation included:

  1. Clarify definition of virtual school to address areas of school operations and make a distinction between virtual schools and programs
  2. Seek clarification about what rules apply to charter schools
  3. Update the petition review checklists to reflect requirements specified in rule
  4. Require petitioners to address the findings in the State Department of Education sufficiency review
  5. Analyze the relationship between variations in operations and student outcomes
  6. Require all existing virtual schools to be approved by commission
  7. Consider adding annual reporting requirements for virtual schools
  8. Address whether any public school operating a virtual program be subject to oversight similar to virtual schools

These recommendations were followed by these final thoughts:

Virtual schools are emerging as an innovative flexible approach, providing opportunities for students who may not fit in a traditional classroom setting

Variations in school operations may include differences in attendance requirements, required levels of student-teacher interaction, activities that count as course work, and student-to-teacher ratios

Statutes should:

  • Define virtual schools in clear, comprehensive language
  • Outline operating requirements
  • Provide mechanism for school approval and ongoing oversight
  • Provide framework for virtual schools to maximize flexibility in operations

I guess what strikes me the most is that while this presentation was given in the Fall of 2008, is there anything I have quote above that couldn’t have been included in a presentation given last week?  I guess we’ll file this under “the more things change, the more they stay the same!”

June 10, 2019

Press Release: CREDO At Stanford University Releases First In-Depth Examination Of Charter School Impacts In Idaho

This is the third of three entries that was referenced about sixty minutes ago (see here).  If you haven’t looked at that first entry, I would STRONGLY encourage that you do for a brief primer on some of the methodological issues.

CREDO at Stanford University Releases First In-Depth Examination of Charter School Impacts in Idaho

STANFORD, Calif. – Stanford University’s Center for Research on Education Outcomes (CREDO), found that on average, students in Idaho charter schools experience similar learning gains in math and stronger learning gains in reading compared to their traditional public school student (TPS) peer. The report studies Idaho state charter students’ performance over three years, beginning with the 2014-2015 school year and ending with the 2016- 2017 school year.

“We are always excited to work with a new state, and our first in-depth look in Idaho provided many unique insights. Idaho is distinctive for numerous reasons — different geography, different student populations — but the findings show that the policy framework of charter schools can be successful in Idaho as much as elsewhere. Idaho provides a unique proof point to the nation, and we look forward to following the story,” said Margaret Raymond, Director of CREDO at Stanford University.

Key Findings

  • This analysis spanned two growth periods and used a total of 14,915 and 14,814 charter school student records from 55 and 56 charter schools in reading and math, respectively.
  • In Idaho, there are both online and brick-and-mortar charters. Our investigation revealed remarkably weaker growth in both reading and math among online charter students relative to the average TPS students or brick-and-mortar charter students. In fact, it is the poor performance of online charter schools that drags down the overall charter impact on student academic growth.
  • Students in rural charter schools have stronger gains in both reading and math compared to their TPS counterparts.
  • At the school level, around 40 percent of Idaho charter schools outpace their local TPS peers in learning in reading and math. Still, 17 percent of charter schools have results that are significantly worse than TPS peers for reading and 20 percent of charter schools are underperforming in math relative to their local TPS peers.

To download a copy of the full report, visit: http://credo.stanford.edu

About CREDO at Stanford University CREDO at Stanford University produces rigorous, non-partisan research and evaluation to enhance the body of empirical evidence, driving education policy decisions toward improved education outcomes for all students.

Note the portions I have highlighted in red that would be of interest to readers of this space.

The direct link to the report is available at:

http://credo.stanford.edu/pdfs/Idaho_report_Final.pdf

Again, another result that confirms findings in a report released in the past two weeks by the National Education Policy Center (of which I am a co-author).

Press Release: CREDO At Stanford University Finds Limited Improvement In Charter School Impact In Ohio

This is the second of three entries that was referenced about thirty minutes ago (see here).  If you haven’t looked at that first entry, I would STRONGLY encourage that you do for a brief primer on some of the methodological issues.

CREDO at Stanford University Finds Limited Improvement In Charter School Impact In Ohio

CREDO releases a new report examining the impact of Ohio Charter Schools from 2013-2017

STANFORD, Calif. – Stanford University’s Center for Research on Education Outcomes (CREDO), found that the typical charter school student in Ohio makes similar progress in reading and weaker growth in math compared to their traditional public school peer (TPS).

“The performance in Ohio charter schools has been consistent since our initial investigation in 2009. We intend to continue to study the impact of the bipartisan legislation HB2 and other policies,” said Margaret Raymond, Director of CREDO at Stanford University. “We continue to be grateful for our long-term partnership with the Ohio Department of Education to provide impartial analysis.”

Key Findings

  • This report provides evidence for charter students’ performance in Ohio over four years, beginning with the 2013-2014 school year and ending in 2016-2017.
  • In Ohio, there are both online and brick-and-mortar charters. Our investigation revealed remarkably weaker growth in both reading and math among online charter students relative to the average TPS students or brick-and-mortar charter students. In fact, as CREDO has found in other states, it is the poor performance of online charter schools that drags down the overall charter impact on student academic growth.
  • Greater academic progress is found for charter black students, including black students in poverty for reading, but not among other subgroups.
  • At the school level, around 34 percent of Ohio charter schools outpace their local TPS peers in learning in reading and 29 percent in math. Still, 14 percent of charter schools have results that are significantly worse than TPS peers for reading and 32 percent of charter schools are underperforming in math relative to their local TPS peers.

To download a copy of the full report, visit: http://credo.stanford.edu

About CREDO at Stanford University CREDO at Stanford University produces rigorous, non-partisan research and evaluation to enhance the body of empirical evidence, driving education policy decisions toward improved education outcomes for all students.

Note the portions I have highlighted in red that would be of interest to readers of this space.

The direct link to the report is available at:

http://credo.stanford.edu/pdfs/OH_state_report_2019.pdf

Again, another result that confirms findings in a report released in the past two weeks by the National Education Policy Center (of which I am a co-author).

Press Release: CREDO At Stanford University Finds Little To No Progress In Charter School Impact In Pennsylvania Since Release of CREDO’s 2013 National Charter School Report

Over the next 90 minutes or so I’m going to be posting three press releases from Stanford University’s Center for Research on Education Outcomes (CREDO), a research body that utilizes their own virtual twin methodology that CREDO themselves developed.  As I have written in the past, there are a number of problems with this kind of comparison.

The first issue is the inability to control for the reasons why [virtual school] students enrolled in their virtual course. Many of these students chose virtual education because there were circumstances preventing them from being successful in that particular course in their brick-and-mortar school. Research has indicated a range of reasons for this decision, such as the course not being offered, a conflict in the student’s timetable, a conflict between the student and the face-to-face teacher, the student being bullied in school, specific learning disabilities or preferences, a lack of success in the past, or numerous other reasons or even a combination of several of those reasons.37 The issue arises when the report claims that improved educational outcomes are the result of the student being enrolled in a virtual environment, when they may simply be due to a lessening of the circumstances that caused the student to leave the traditional setting in the first place. For example, if a student being bullied in a brick-and-mortar school transfers to a cyber school, any improved performance may be completely divorced from the technology or delivery method, but rather could be attributable to the fact the student is no longer being bullied. While that is a benefit of virtual education, it wasn’t what the authors argued or were even researching.

Essentially, the control group and the treatment group are two non-randomly constituted groups, one of which is set to regress upward (i.e., from their poor motivation or attention or performance or whatever circumstance may have led them to select the virtual environment), and students in the control group, who are not on the same trajectory. In the language of experimental design, this source of internal invalidity is labeled a Regression-by-Selection Invalidity.38

37 Barbour, M. K., & Reeves, T. C. (2009). The reality of virtual schools: A review of the literature. Computers and Education, 52(2), 402–416.

Cavanaugh, C. (2013). Student achievement in elementary and high school. In M. G. Moore (Eds.), Handbook of distance education (3rd ed.) (pp. 170-184). New York: Routledge.

38 Campbell, D., & Stanley, J. (1963). Experimental and quasi-experimental designs for research. Chicago, IL: Rand-McNally.

————

Taken from: Barbour, M. K. (2014). Review of “Virtual schooling and student learning.” Boulder, CO: National Education Policy Center. Retrieved from https://nepc.colorado.edu/sites/default/files/ttr-fla-virtual-pepg.pdf

I mention the methodology up front because it is a procedure that CREDO developed on their own, and in the vast majority of cases by using their own methodology they have been able to find in favour of almost all forms of neo-liberal-style educational reform initiatives (e.g., charter schools, vouchers, etc.).  However, importantly, they still have consistently found that online charter schools perform poorly – a result that confirms findings in a report released in the past two weeks by the National Education Policy Center (of which I am a co-author).

Anyway, the press release that was posted read:

CREDO at Stanford University Finds Little To No Progress In Charter School Impact In Pennsylvania Since Release of CREDO’s 2013 National Charter School Report

CREDO releases a new report examining the impact of Pennsylvania Charter Schools from 2013-2017

STANFORD, California, June 4, 2019–STANFORD, Calif. – Stanford University’s Center for Research on Education Outcomes (CREDO) found over four years of study that the typical charter school student in Pennsylvania makes similar progress in reading and weaker growth in math compared to their traditional public school peer (TPS).

“Our recent analysis mirrors our last investigation in 2013, which yielded similar findings. With nearly one-quarter of [charter] schools posting student results that lag in reading and one third doing so in math, the collective impact on students’ academic careers and later life outcomes remains of deep concern. We will continue to study the impacts of this sector as we know that the Pennsylvania Department of Education has focused on strengthening the accountability of the online sector over the past year,” said Margaret Raymond, Director of CREDO at Stanford University. “We continue to be grateful for our long-term partnership with the Pennsylvania Department of Education in our shared commitment to student success.”

KEY FINDINGS

  • This report provides evidence for charter students’ performance in Pennsylvania over four years, beginning with the 2013-2014 school year and ending in 2016-2017. The study examines the progress students make from one year to the next, comparing charter school students to matched peers in traditional public schools in the same communities.
  • In Pennsylvania, there are both online and brick-and-mortar charters. Our investigation revealed remarkably weaker growth in both reading and math among online charter students relative to the average TPS students or brick-and-mortar charter students. In fact, as CREDO has found in other states, it is the poor performance of online charter schools that drags down the overall charter impact on student academic growth.
  • Greater academic progress in reading is found for charter students attending urban brick and mortar schools. Additionally, greater academic progress for Hispanic students attending brick and mortar schools in reading was found, but not among other subgroups.
  • Thirteen percent of charter schools included in the study exhibited high achievement and high academic growth in English language arts; fewer than 10 percent of charters met this standard in math.

To download a copy of the full report, visit http://credo.stanford.edu

About CREDO at Stanford University – CREDO at Stanford University was established to improve empirical evidence about education reform and student performance at the primary and secondary levels. CREDO at Stanford University supports education organizations and policymakers in using reliable research and program evaluation to assess the performance of education initiatives. CREDO’s valuable insight helps educators and policymakers strengthen their focus on the results from innovative programs, curricula, policies or accountability practices.

Note the portions I have highlighted in red that would be of interest to readers of this space.

The direct link to the report is available at:

http://credo.stanford.edu/pdfs/2019_PA_State_Report_FINAL_06052019.pdf

May 19, 2019

Media Release: Reports Shed New Light On California Charter Schools’ Fiscal Impact

Note that the only reference to cyber or virtual charter schools in this series is in the second report, where it indicates that they analysis excludes the 25,000 students who enrolled in virtual charter schools from their analysis.

I should also note that one of the difficulties with this type of costing analysis is that it assumed that the loss is uniform.  For example, if a district loses ~30 students and the associated funding that they can simply cut a class of students (as ~30 students is roughly one class).  The problem is that in these ~30 students there may be three grade 1 students, five grade two students, one grade three student and so on.  This means that the school would not be able to cut a single class of students, but make do with the lower level of funding while trying to accommodate the same number of grades and classes.

If you’re having trouble viewing this email, you may see it online.
Share this:    
crpe.org
Media Release
Contact:
Travis Pillow
407.376.3105
Reports shed new light on California charter schools’ fiscal impact

Seattle, Wash. May 17, 2019 – The Center on Reinventing Public Education (CRPE) is releasing three briefs that challenge some commonly held beliefs about the impact California charter schools have on school districts.

The reports come as a state task force reviews potential charter school policy changes, and state lawmakers prepare to debate a series of bills affecting charter schools. They conclude:

Charter school growth does not account for all district enrollment declines. In the most recent school year (2018–2019), charter school enrollment growth can account for little or none of the enrollment loss experienced by Los Angeles and San Diego Unified school districts. In Oakland, the district gained more students than did charter schools. District enrollment losses stem from a combination of demographic shifts and students attending schools in nearby districts, private schools, homeschooling, or charter schools. Read our full brief on district enrollment losses.

There is no evidence that charter schools are to blame for fiscal distress in California school districts. Based on state data, we find no relationship between the share of students enrolled in charter schools and the likelihood school districts will enter fiscal distress. Between 1998 and 2015, an average of just 1.5 percent of school districts where charter schools enroll more than 10 percent of students entered fiscal distress. Read our full brief on district fiscal distress.

Charter schools have, according to available evidence, important benefits for California communities and limited costs to the state. Like all policies, charter schools impose costs and generate benefits for California students. For example, in Southern California and the Bay Area, they have been shown to lift student achievement in reading and math—and particularly for students who are black, Hispanic, and living in poverty. These benefits should be considered alongside any costs. Read our full brief on costs and benefits.

“Charter school policy is too important for California to get wrong,” said Robin Lake, CRPE’s director. “It is important for policymakers to carefully analyze the impact of charter schools for all California students, and the full range of challenges facing public education in the state.”

The reports and an accompanying blog post offer recommendations for policymakers:

1.  The state should work with school districts to develop more accurate budget and enrollment projections. This will help restore trust with taxpayers and teachers negotiating salaries, and enable an honest conversation with taxpayers and state policymakers about public schools’ funding needs.

2.  Policymakers should consider transition aid for districts where students leave for charter schools, as states like Massachusetts have done. This could help districts deal with costs they can’t easily shed, and compete more effectively to attract students.

3.  In districts struggling with the effects of persistent enrollment declines, charter schools and the state could pay into debt reduction funds that a district could tap if it made strides reducing costs and increasing financial transparency. Charter schools, in return, could receive improved access to district facilities.

For more information, or to arrange an interview with CRPE expert, please contact Travis Pillow at 407.376.3105 or travis.pillow@gmail.com.

About CRPE

Part of the University of Washington Bothell, the Center on Reinventing Public Education is one of the nation’s leading sources for transformative, evidence-based ideas to improve education. To ensure all students are prepared for a rapidly changing future, CRPE puts forward rigorous research and policy analysis to help educators, policymakers, civic and community leaders, parents, and students themselves reimagine education systems and structures. Learn more about CRPE’s current research here.
Center on Reinventing Public Education
Improving education through transformative,
evidence-based ideas about/research/etc
600 1st Avenue, Suite 206 | Seattle, WA 98104 United States

Got this as a forward? Sign up to receive our future emails.
Next Page »

Blog at WordPress.com.