Virtual School Meanderings

April 7, 2014

AERA 2014 – Examining Variation in Achievement Impacts Across California’s Full-Time Virtual Schools

This is the twenty-fourth session – and final one for Monday (and the conference) – that I am blogging from the 2014 annual meeting of the American Education Research Association (AERA) in Philadelphia.  This session was a part of a symposium that was described as:

Virtual Schools in the United States 2014: Politics, Performance, Policy, and Research Evidence

In the past decade, virtual education has moved quickly to the top of the K-12 public education reform agenda. Though little is known about the efficacy of online education generally or about individual approaches specifically, states are moving quickly to expand taxpayer-funded virtual education programs. The main purpose of this session is to understand the specificities of today’s virtual school movement as it moves from novelty to mainstream. Drawing from a rich array of theoretical perspectives and content disciplines, we will examine the performance of full-time, publicly funded K-12 virtual schools, describe the policy issues raised by the available evidence, assess the research evidence that bears on K-12 virtual teaching and learning, and offer research-based recommendations to help guide policymaking.

The actual session is described in the online program as:

Examining Variation in Achievement Impacts Across California’s Full-Time Virtual Schools
Charisse Atibagos Gulosino, University of Memphis; Jonah Liebert, Teachers College, Columbia University

Perhaps the most significant current trend in education reform is the growth of virtual (online) schools (Watson et al., 2011, 2012). While these schools offer the potential to radically restructure the way that teaching and learning happens, they also present challenges for researchers and policymakers who want to know whether they work. Specifically, the extent to which virtual schools depart from traditional brick-and-mortar schools creates difficulties with respect to assessing what these schools are doing in terms of teaching and learning and how well they are doing it.

This study uses longitudinal student-level data covering all full-time virtual schools (thirty-two total) in California from 2010-2012 to study the effect of virtual schools on student performance. Based on our web-based research, all full-time virtual schools in California are contracted to run as charter schools. Full-time virtual schools are defined as those schools in which instruction is delivered entirely or primarily through online methods (Watson et al., 2012). However, students self-select into virtual schools, making it difficult to estimate the effects of these schools on achievement. This study addresses this challenge using propensity score matching (PSM). Following the counterfactual framework (Rosenbaum & Rubin 1983; Rubin 1974), PSM matches virtual school students (“treatment” group) to those who are non-virtual school students but similar in all other preexisting observed characteristics (“control” group), based on a propensity to attend a full-time virtual school. In addition, this study addresses selection bias due to both observed and unobserved covariates. Previous studies employing the PSM approach have focused mainly on selection bias due to observed covariates (Chevalier & Viitanen, 2003). Using the Rosenbaum bounds method (Rosenbaum 2002), we evaluate the extent to which selection bias on unobserved covariates would nullify propensity score matching estimates of the effects of virtual schools.

The data for this study come from the California Department of Education (CDOE). The Department maintains longitudinal records on all public school teachers and students, including test scores (CALPADS), demographic data, enrollment and attendance information. This study supports our larger project’s focus on heterogeneity in the achievement effects of charter school attendance across demographic groups in California. We focus on the difference in impact between virtual school students and non-virtual school students in our sample, primarily because of the finding of large positive impacts in urban charters and non-significant or negative impacts in non-urban charters has been noted in our prior analysis.

Although virtual schooling is gaining ground in the K-12 classroom (Molnar et al., 2013), its impact on academic performance remains largely unexplored. Considered one of the largest markets of virtual school programs in the United States, California offers a fertile context for the study of virtual school impacts and thus serves as the focus of our study. Ultimately, our study focuses on discovering which policy-amenable aspects of virtual schools—their characteristics and conditions— are related to their ability to maximize student learning and close the achievement gap.

While a part of our symposium, this portion was not part of the National Education Policy Center’s report Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence.

This was a difficult one for me to follow, as it was highly statistical.  The other difficulty was that the presenter was not able to present the results.  Basically, the California Department of Education has indicated that they are unable to verify the data that Charisse and her team have collected – even though the data has come from the California DoE’s own website – they have forbidden her from presenting the data (something I was led to believe came about after she submitted her AERA proposal).  She – and others in the room – were hopeful that this might be worked out at some stage.

Some notes that I was able to capture.  Her data was from the 2010-11 school year to the 2012-13 school year.  It used a PMS feeder school model to address selection bias (a technique commonly used by CREDO in their studies of charter schools and cyber charters).  She used a multivariate analysis strategies, that had three matching procedures, as a way to try and compare apples to apples.  That’s about all I was able to get from the procedure – and, as I mentioned, she was unable to present the actual data.

A couple of resources that she did mention that helped inform her work included:

Cassandra Guarino, Ron Zimmer, Cathy Krop, and Derrick Chau. Nonclassroom-based Charter Schools in California and the Impact of SB 740. RAND: MG-323-EDU, February 2005.

Ron Zimmer , Richard Buddin, Derrick Chau, Brian Gill, Cassie Guarino, Laura Hamilton, Cathy Krop, Dan McCaffrey, Melinda Sandler, and Dominic Brewer. Charter School Operation and Performance: Evidence from California. RAND: MR-1700, July 2003.

So I wanted to share those as well.

This is me officially signing off from AERA 2014…

AERA 2014 – What Do We Actually Know? Examining the Research Into Virtual Schools for Useful Models

This is the twenty-third session that I am blogging from the 2014 annual meeting of the American Education Research Association (AERA) in Philadelphia.  This session was a part of a symposium that was described as:

Virtual Schools in the United States 2014: Politics, Performance, Policy, and Research Evidence

In the past decade, virtual education has moved quickly to the top of the K-12 public education reform agenda. Though little is known about the efficacy of online education generally or about individual approaches specifically, states are moving quickly to expand taxpayer-funded virtual education programs. The main purpose of this session is to understand the specificities of today’s virtual school movement as it moves from novelty to mainstream. Drawing from a rich array of theoretical perspectives and content disciplines, we will examine the performance of full-time, publicly funded K-12 virtual schools, describe the policy issues raised by the available evidence, assess the research evidence that bears on K-12 virtual teaching and learning, and offer research-based recommendations to help guide policymaking.

The actual session is described in the online program as:

What Do We Actually Know? Examining the Research Into Virtual Schools for Useful Models
Michael Kristopher Barbour, Sacred Heart University

The purpose of this systematic review of the literature is to identify trends in the research regarding K-12 online learning related to the delivery of virtual schooling.

While the use of K-12 online learning at the K-12 level has been practiced for approximately two decades, the availability of published research to inform that practice has not kept pace. Barbour (2011) reviewed 262 articles from the main distance education from 2005 to 2009, only 24 articles related to K-12 distance education. Further, Cavanaugh et al. (2009) stated that the literature was “based upon the personal experiences of those involved in the practice of virtual schooling” (p. 5). Finally, Rice (2006) lamented that “a paucity of research exists when examining high school students enrolled in virtual schools, and the research base is smaller still when the population of students is further narrowed to the elementary grades” (p. 430).

There is general agreement about the themes that have been dominant in the limited amount of research conducted on K-12 online learning to date. Rice (2006) described the research into K-12 online learning as either being comparisons of student performance between those enrolled in online and face-to-face environments or examinations of the qualities and characteristics of the online learning experience. An examination of the findings related to comparison of student performance in K-12 online learning environments and the traditional classroom has been mixed (Cavanaugh et al., 2004; Means et al., 2009). Further, Cavanaugh et al. (2005) speculated virtual school students took their assessment were more academically motivated and naturally higher achieving students.

There is no shortage of issues within the realm of K-12 online learning that need to be examined. For example, Cavanaugh et al. (2009) recommended researchers establish best practices for online teaching, improve the identification and remediation of factors related to success for online learners, investigate the nature of support provided to online learners by how school-based teachers. However, Barbour and Reeves (2009) went even further on how future research should be conducted, recommending a design-based research approach.

To date, the only exception to this pattern is the Virtual High School (VHS). SRI International, based upon seven goals identified in conjunction with their VHS partners, conducted several annual evaluations, content-specific investigations in focus areas where VHS was not meeting their initial goals, and a final evaluation. VHS were full participants in this research process: assisting in the identification of issues to be examined, the design of the research, the implementation of the recommendations, and then beginning the process again to ensure that recommendations actually addressed the original problem. This cyclical research process that the VHS and SRI International engaged in was able to resolve many of the initial issues in the implementation of what was then a new model of educational delivery. The findings from this design research approach should form the starting point for additional research into similar K-12 online learning settings.

For those that aren’t aware, this session was based on the National Education Policy Center’s report Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence, and my section was Section II.

Since this was a session that I am involved in, I’ll just post the slides below.

AERA 2014 – Key Policy Issues in Virtual Schools: Finance, Instructional Quality, and Teacher Quality

This is the twenty-second session that I am blogging from the 2014 annual meeting of the American Education Research Association (AERA) in Philadelphia.  This session was a part of a symposium that was described as:

Virtual Schools in the United States 2014: Politics, Performance, Policy, and Research Evidence

In the past decade, virtual education has moved quickly to the top of the K-12 public education reform agenda. Though little is known about the efficacy of online education generally or about individual approaches specifically, states are moving quickly to expand taxpayer-funded virtual education programs. The main purpose of this session is to understand the specificities of today’s virtual school movement as it moves from novelty to mainstream. Drawing from a rich array of theoretical perspectives and content disciplines, we will examine the performance of full-time, publicly funded K-12 virtual schools, describe the policy issues raised by the available evidence, assess the research evidence that bears on K-12 virtual teaching and learning, and offer research-based recommendations to help guide policymaking.

The actual session is described in the online program as:

Key Policy Issues in Virtual Schools: Finance, Instructional Quality, and Teacher Quality
Luis Alberto Huerta, Teachers College, Columbia University

Scaling up virtual school reform presents significant implementation and accountability challenges, as several recent research and technical reports on virtual schools have illustrated. Although there have been some recent legislative efforts to clarify expectations in such areas as accountability and standards, states are struggling to establish accountability mechanisms appropriate for both guiding and auditing virtual schools—even as they allow them to expand. In 2011, for example, Wisconsin, Oregon, Louisiana and Michigan either increased or eliminated enrollment caps for full-time virtual schools; however, none of those states passed legislation strengthening accountability and oversight mechanisms. A continuing challenge for states will be to reconcile traditional funding mechanisms, governance structures, and accountability demands with the unique organiza¬tional models and instructional methods found in virtual schools.
Drawing on recent reports and our own research on virtual charter schools, we consider relevant policy issues in the following critical areas:

• Finance : Much of the debate over virtual schools focuses on appropriate funding levels compared to traditional brick-and-mortar schools. Funding formulas for virtual schools must be reconsidered and adjusted to account for the actual costs associated with the new instructional delivery model. In addition, given the potential of virtual schools to expand access beyond the traditional geographic boundaries associated with brick-and-mortar schools, governance systems must be structured to address the challenges associated with extended attendance boundaries.
• Instructional program quality: Accountability mechanisms for virtual schools must address not only their unique organizational models but also their instructional methodologies. Quality of content, quality and quantity of instruction, and quality of student achievement are all important aspects of program quality that have yet to be addressed by accountability models linked to unique delivery models of virtual schooling.
• High quality teachers : The common assumption that effective teachers will wholeheartedly embrace digital tools and be motivated to teach in a one-dimensional virtual environment must be carefully examined. Factors that support teachers and promote effective teaching include strong leadership, peers, professional development, books, materials and a myriad of other resources. Policymakers must ensure that such support, or other types of support necessary in a digital environment, is available to professionals teaching online. Effective recruitment, professional development, assessment, and retention of high quality teachers are all critical components of a strong virtual environment in which both teachers and students thrive.

This section will identify relevant common assumptions; and, related but unanswered key empirical questions linked to the policy issues outlined above. And lastly, we advance a set of policy recommendations intended to help policymakers and practitioners address the challenges identified.

For those that aren’t aware, this session was based on the National Education Policy Center’s report Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence, and Luis’ section was Section I.

Unfortunately Luis had to go back to New York unexpectedly, so he was not available to do his portion.

AERA 2014 – Full-Time Virtual Schools: Enrollment, Student Characteristics, and Performance

This is the twenty-first session – and first one for Monday – that I am blogging from the 2014 annual meeting of the American Education Research Association (AERA) in Philadelphia.  This session was a part of a symposium that was described as:

Virtual Schools in the United States 2014: Politics, Performance, Policy, and Research Evidence

In the past decade, virtual education has moved quickly to the top of the K-12 public education reform agenda. Though little is known about the efficacy of online education generally or about individual approaches specifically, states are moving quickly to expand taxpayer-funded virtual education programs. The main purpose of this session is to understand the specificities of today’s virtual school movement as it moves from novelty to mainstream. Drawing from a rich array of theoretical perspectives and content disciplines, we will examine the performance of full-time, publicly funded K-12 virtual schools, describe the policy issues raised by the available evidence, assess the research evidence that bears on K-12 virtual teaching and learning, and offer research-based recommendations to help guide policymaking.

The actual session is described in the online program as:

Full-Time Virtual Schools: Enrollment, Student Characteristics, and Performance
Gary J. Miron, Western Michigan University; Brian Horvitz, Western Michigan University; Charisse Atibagos Gulosino, University of Memphis

Full-time virtual K-12 schools, also known as cyber schools or online schools, are schools that deliver all curriculum and instruction via the Internet and electronic communication, usually with students at home and teachers at a remote location, and usually with everyone participating at different times (Means, B. e al., 2010). Although increasing numbers of parents and students are choosing this option, little is known about virtual schooling in general, and very little about full-time virtual schools in particular (Watson et al., 2011, 2012). For example, information has not been available on such basic questions as the number of virtual elementary and secondary schools operating, the number of students enrolled in them, and the rate at which they have expanded. Moreover, despite a dearth of research evidence useful in shaping policy, many states have adopted legislation permitting full-time virtual schools or removing the caps that once limited their growth (SREB, 2013).

The primary goal of our study is to describe the education landscape of all full-time virtual schools in the U.S. for which data are available for the 2012-13 academic year and to provide an estimate of their growth. We raise three research questions. They include: (1) How many full-time virtual schools operate in the U.S.? How many students do they enroll? (2) What are the demographic characteristics of students enrolled in full-time virtual schools? Within individual states, how do demographic data differ for students enrolled in virtual schools and those enrolled in brick-and-mortar schools? (3) How do full-time virtual schools perform in terms of student achievement relative to other public schools?

Our study is based on publicly available data, collected, audited, and warehoused by public authorities. The scope of the study is limited to full-time, public elementary and secondary virtual public schools serving U.S. students. This includes virtual schools operated by for-profit Education Management Organizations (EMOs) as well as virtual schools operated by states or districts. Private virtual schools are excluded. Also excluded are schools offering a combination of full-time virtual programs and blended programs, unless it is possible to separate data for the full-time virtual school component. The primary sources for total enrollment and school performance data are state-level datasets, Common Core of Data (CCD), and school report cards for the 2012-13 school year. Aggregated data reflect weighted averages based on enrollment. That is, averages have been calculated so that the influence of any given school on the aggregated average is proportional to its enrollment. Comparisons are made to norms for all public schools in the United States.

The results of our comparisons using AYP, state ratings and on-time graduation rates suggest that virtual schools are not performing as well as brick-and-mortar schools. The two themes that run through our findings are: (1) the inadequacies of state data and accountability systems on virtual schools; (2) the insufficiency of state policies and authorizing practices to design new outcome measures appropriate to the unique characteristics of full-time virtual schools.

For those that aren’t aware, this session was based on the National Education Policy Center’s report Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence, and Gary’s section was Section III.

Gary began by providing some background into the study, the fact that it is in its second year, and some of the methodology for his specific section.  Based on their data collection, they were able to identify 338 full-time virtual schools that were serving 243,000.  61% of these were charter schools, and they accounted for 85% of the enrollment in full-time virtual schooling.  44% of the full-time virtual schools were run by for-profit EMOs, and this represented 80% of the full-time virtual school students.  For-profit EMOs had an average of 1,230 students, while those operated by non-profit EMOs or no EMO enrolled an average of 470 and 362 students respectively.  Gary stressed the fact that the data is poor in many respects.

Gary also underscored the fact that based on their data have more White students, fewer minority students, fewer free and reduced lunch students, fewer special education students, and fewer English language learners as compared to the national averages.  This was consistent with the findings that Gary had in 2013, and in the examination of solely K12, Inc. data that he completed in the past.  Gary did not that the K12, Inc. proportion of special education students (~10%) was higher than the full-time virtual school average (~7%).

In terms of looking at student performance, Gary and his team looked at the state’s performance rating, which factors in many issues.  Interestingly, only 33.76% of full-time virtual schools had an acceptable rating by their state.  Independent full-time virtual schools (i.e., district-based programs) were much more likely to achieve the state’s acceptable rating than their for-profit counterparts.

Gary concluded with highlighting the recommendations from the NEPC report.

 

April 6, 2014

AERA 2014 – Legal Responsibility for Special Education in Cyber Charter Schools

This is the twentieth session – and last for Sunday -  that I am blogging from the 2014 annual meeting of the American Education Research Association (AERA) in Philadelphia. This session was a part of a roundtable session entitled “Charter School Management: Cyber Schools and Performance.”  The actual session is described in the online program as:

Legal Responsibility for Special Education in Cyber Charter Schools
Regina R. Umpstead, Central Michigan University, Robert Andersen, Cooley Law School, Bruce Wells Umpstead, BrightBytes

[No abstract provided in the online system]

In this study, Regina and Bruce (at least they were the two presenters present) studied cyber charter school legal responsibilities with regards to special education in California, Illinois, Michigan, Ohio, Pennsylvania, and Texas.  The project was something that they were asked to investigate based on a request from the Michigan Department of Education – and Regina mentioned some involvement of the Center for Online Learning and Students with Disabilities  at the University of Kansas.

Bruce began his portion by describing the cyber charter landscape in each of the states that was studied – actually, he spoke to both the cyber school (i.e., full-time) and virtual school (i.e., supplemental) landscape.  For the purpose of the study, cyber charter schools were full-time online schools that operated through primarily online means.

Regina then shifted the focus back to students with disabilities, and began by highlighting the different laws (see my notes on AERA 2014 – A Legal Framework for Special Education and Charter Schools for the complete list).  She underscored the notion of services being provided in the least resistant environment, which parents want to argue as the home but many cyber schools argue is the online environment.  This could form many of the next round of legal battles.

Bruce then focused on Michigan and mentioned that Michigan was the first cyber charter school in the United States was in the state in 1999, when Noah Webster created a dial-up system that targeted a home schooling audience.  This program was shut down by the legislature in reaction to a law suit that, according to Bruce, “went no where.”  He then outlined the special education law, where the state was the most interesting in their opinions – because by state law there is a requirement that both the local education authority (LEA) (i.e., the cyber charter school) and the district of the student’s residence are required to provide special education service.  Apparently, this is a bit contentious within the state and the State has refused to publish their legal brief.

In California, cyber charters are also considered LEAs.  But it is the State Department of Education that decides upon funding.

In Illinois, most of the cyber charters are authorized by local school districts.  One of the issues is that the district of residence is expected to provide funding for services.  Regina indicated that cyber charter laws in Illinois enumerate a list of state education laws that they have to comply with, which are only a select few of the education laws that all traditional public schools have to abide by.  Even more interesting was the fact that state special education laws were not included in the list that cyber charter schools have to comply with (although they would still have to comply with federal laws).

In Ohio, the district of residence is responsible for special education.  However, when a student enrolls in a cyber charter school the responsibilities for special education transfers solely to the cyber charter school.

In Pennsylvania, cyber charters are exempted from education law in most areas.  However, they are considered LEAs for the purposes of special education.

Regina indicated that Texas was an interesting model for cyber charters and special education – and one that might be seen as a model for other states.  The use of a statewide network provided a model that removes the notion or conflict of district of residence that occurs in many states.

From my reading (or listening), it appears that the greatest tensions appear to be source of funding and whether it is the district of residence or the cyber charter or both that provide special education services.

« Previous PageNext Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 2,468 other followers