Virtual School Meanderings

July 14, 2014

Reviews/Summaries of Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence

Filed under: Academic Achievement,Alex Molnar,Brian Horvitz,Charisse Gulosino,Columbia University,Connections Academies,Connections Academy,Curriculum,cyber school,Edison Schools,education,Education Management Organization,EMO,Gary Mirion,high school,Insight Schools,Jennifer Rice King,K12,K12 Inc.,Kaplan Virtual Education,learning coach,Leona Group LLC.,Luis Huerta,Michael K. Barbour,Mosaica Inc.,National Education Policy Center,NEPC,Policy/Regulation,Public Schools,report,research,Research Methodology,Roads Education Organization,Sacred Heart University,Sheryl Rankin Shafer,Teachers College,technology,University of Maryland,virtual school,Western Michigan University,Western Washington University,White Hat Management — Michael Barbour @ 1:41 pm
Tags: , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,

Since these came across my desk a little while ago, I wanted to pass them on to my readership…

Basically, someone over at the International Center for Home Education Research Reviews reviewed (as they called it, but it is more of a summary) each of the three sections of the Virtual Schools in the U.S. 2014: Politics, Performance, Policy, and Research Evidence published by the National Education Policy Center that I contributed to (note Part 2 is my section).

Anyway, just wanted to pass this along…

Note I have used the same tags used by the author(s) of these reviews, along with my own that I have used for this report in the past on this blog.

June 27, 2014

News From The NEPC: Tepid Evaluation Is Built On Sand

From yesterday’s inbox…

Research and analysis to inform education policy
and promote democratic deliberation
Is this email not displaying correctly?
View it in your browser.

Tepid Evaluation
Is Built on Sand

NEPC review questions RAND report’s attempt to evaluate the effectiveness of a principal preparation program using
value-added measures
of students’ test scores

 

Contact:

William J. Mathis, (802) 383-0058, wmathis@sover.net

Edward J. Fuller, (814) 865-2233, ejf20@psu.edu

URL for this press release: http://tinyurl.com/memtb3d

(June 26, 2014) – An evaluation of the New Leaders principal preparation program concludes that the program has a slightly positive effect on student test scores, though only for certain grade levels, subject areas, and districts. But a review published today cautions that the evaluation, even with such tepid conclusions, overreaches.

Reviewer Edward J. Fuller of Penn State University notes that the evaluation, conducted by researchers at the RAND Corporation, found effect sizes that were quite small where they existed at all, and that “the study’s results are more mixed than its bottom-line conclusion would suggest.” More importantly, Fuller explains that the research concerning principal effectiveness expressly warns against using value-added estimates—of the sort used by RAND—to attempt to capture such effectiveness in a high-stakes context.

Professor Fuller reviewed Preparing Principals to Raise Student Achievement: Implementation and Effects of the New Leaders Program in Ten Districts for the Think Twice think tank review project. The review is published by the National Education Policy Center, housed at the University of Colorado Boulder School of Education. Fuller is associate professor and executive director of the Penn State Center for Evaluation and Education Policy Analysis whose research examines a variety of areas relating to educator preparation, quality, and career pathways, as well as school improvement, evaluation, and charter schools.

The report was written by a team of researchers for RAND led by Susan Gates. The evaluation was sponsored by the New Leaders principal preparation program, a non-profit organization founded in 2000 and based in New York City, which describes its mission as improving student outcomes by preparing effective leaders and improving working conditions of school principals.

The RAND evaluation attempts to determine the New Leaders program’s impact on student test scores at primary and secondary grades, although the data allow for only weak analyses at the upper grades.

At the lower grades, the effect sizes associated with principals prepared through the New Leaders program are less than two percentile points. Overall, Fuller adds, “the study’s results are more mixed than its bottom-line conclusion would suggest” – with most results finding no statistically significant impact of New Leaders principals on test scores, and nearly as many negative findings as positive ones.

Because of the general weaknesses of the value-added approach, and because of additional causal difficulties of attributing some defined aspect of students’ test score growth to their principals, Fuller points out that the evaluations’ findings have weak validity. This does not mean that New Leaders principals are poorly prepared; they might be extremely professional and effective. But the RAND evaluation is wrong to attempt to make causal claims.

Fuller observes that the study does provide “a rich description of a thoughtful approach” to evaluating the effectiveness of principals and of principal preparation programs. But, the reviewer warns, it is also potentially harmful, if it leads policymakers to conclude that the evaluation approaches employed offer a valid basis for high-stakes accountability systems for principals or preparation programs.

“Current research is very clear about this—the estimates presented in the study do not accurately capture principal effectiveness and should not be used to make high-stakes decisions about individuals or programs,” Fuller concludes.

“Thus, ultimately, this study misses a very important opportunity to discuss these issues and inform policymakers about the problems and prospects of using the strategies it employs.”

Find Edward J. Fuller’s review on the NEPC website at:
http://nepc.colorado.edu/
thinktank/review-
preparing-principals
Find
Preparing Principals to Raise Student Achievement: Implementation and Effects of the New Leaders Program in Ten Districts, by Susan M. Gates et al. and  published by RAND Corp. with sponsorship by New Leaders, on the web at:
http://www.rand.org/
pubs/research_reports/
RR507.html
The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on the NEPC, please visit http://nepc.colorado.edu/.

This review is also found on the GLC website at http://www.greatlakescenter.org/..

If you are not already subscribed to this newsletter and would like to receive it regularly, click
http://nepc.colorado.edu/
and then click the button in the upper right-hand corner that looks like this: 


The National Education Policy Center (NEPC) is housed at the University of Colorado Boulder School of Education. Its mission is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information about the NEPC, please visit http://nepc.colorado.edu/.


 

Copyright © 2014 National Education Policy Center, All rights reserved.
You’re receiving this email because you have opted in at our website or sent a personal request to be included. Thank you.
Our mailing address is:

National Education Policy Center

School of Education, 249 UCB
University of Colorado

Boulder, CO 80309-0249

Add us to your address book

For all other communication with NEPC, write to nepc@colorado.edu.

June 25, 2014

News From The NEPC: Debate Intensifies over Value-Added Research

From yesterday’s inbox…

Research and analysis to inform education policy
and promote democratic deliberation
Is this email not displaying correctly?
View it in your browser.

Debate Intensifies
over Value-Added Research

NEPC reviewer responds to research team’s claim of a connection between teacher value-added scores and student lifetime earnings

Contact: 

William J. Mathis, (802) 383-0058, wmathis@sover.net

Moshe Adler, (917) 453-4921, ma820@columbia.edu

URL for this press release: http://tinyurl.com/n9wnvry

BOULDER, CO (June 24, 2014) – Although policymakers may grab onto easy answers, questions about teacher effectiveness—how we measure it and what we can conclude about a teacher’s long-term impact—are being heatedly debated among scholars.

Today, the National Education Policy Center published a clear and detailed response to some of the most influential research claims about teacher effectiveness.

Those claims were made by researchers Raj Chetty, John Friedman, and Jonah Rockoff, who assert a connection between teachers’ “value-added” scores and what their students will earn over their lifetimes. Those assertions made their way into President Obama’s State of the Union message two years ago; they also surfaced in this month’s ruling by a California judge, in the Vergara case, who found that certain due process protections for teachers violated that state’s constitution.

That research, however, cannot bear the weight of critical scrutiny, according to an expert review by Moshe Adler published by the NEPC in April.

Adler’s review—published by NEPC as part of its Think Twice think tank review project—examined two working papers presenting research by Chetty and his colleagues and published by the National Bureau of Economic Research. Adler is an economist affiliated with both Columbia’s Urban Planning Department as well as the Harry Van Arsdale Jr. Center for Labor Studies at Empire State College, SUNY, and the author the 2010 book, Economics for the Rest of Us: Debunking the Science That Makes Life Dismal.

Chetty, Friedman, and Rockoff wrote a response—which is posted on the NEPC website—taking issue with the criticisms. Accordingly, we asked Adler to continue this important debate, replying to those responses. Adler has now presented an item-by-item explanation of why those responses are inadequate to address the study’s weaknesses.

The influence of the Chetty team’s research includes this statement in President Obama’ 2012 State of the Union address: “We know a good teacher can increase the lifetime income of a classroom by over $250,000.” Similarly, the judge in California’s Vergara litigation cited Chetty’s testimony and the team’s research as evidence that “a single year in a classroom with a grossly ineffective teacher costs students $1.4 million in lifetime earnings per classroom.”

The research claim rests on complex statistical analyses that attempt to attribute a student’s test score growth over a year to one or more of that student’s teachers—and then attempts to link those scores to subsequent student earnings.

According to Adler, however, the judge’s and the president’s conclusions step far beyond what the research can validly demonstrate. Adler explains why the evidence that Chetty and his colleagues cite cannot adequately support either claim: that a teacher’s value-added scores reflect that teacher’s quality, or that the scores predict future student earnings.

“Despite widespread references to this study in policy circles, the shortcomings and shaky extrapolations make this report misleading and unreliable for determining educational policy,” Professor Adler concluded in his April 10 review. That review raised at least nine different concerns, including the improper use of prior research and the failure to report important results when those results contradicted  the authors’ conclusions.

The points Adler makes should give pause to policymakers who have assumed that the evidence is sufficiently solid to be relied upon to frame education reform strategies.

Although the concerns Adler discusses are sometimes technical, they are very clearly explained. Policy makers and researchers considering using the study from Chetty and his colleagues are strongly encouraged to read this robust exchange, which brings forth and clarifies serious and important issues.

Find Moshe Adler’s review ; the Chetty, Friedman, and Rockoff reply; and Adler’s response to their reply, on the NEPC website at:
http://nepc.colorado.edu/
thinktank/review-
measuring-impact-of-teachers
.
Find

Measuring the Impacts of Teachers, by Raj Chetty, John N. Friedman, and Jonah E. Rockoff, on the web at:
http://www.nber.org/papers/w19423 and
http://www.nber.org/papers/w19424.
The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on the NEPC, please visit http://nepc.colorado.edu/.

The initial review is also found on the GLC website at http://www.greatlakescenter.org/.
If you are not already subscribed to this newsletter and would like to receive it regularly, click
http://nepc.colorado.edu/ 
and then click the button in the upper right-hand corner that looks like this: 


The National Education Policy Center (NEPC) is housed at the University of Colorado Boulder School of Education. Its mission is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information about the NEPC, please visit http://nepc.colorado.edu/.


 

Copyright © 2014 National Education Policy Center, All rights reserved.
You’re receiving this email because you have opted in at our website or sent a personal request to be included. Thank you.
Our mailing address is:

National Education Policy Center

School of Education, 249 UCB
University of Colorado

Boulder, CO 80309-0249

Add us to your address book

For all other communication with NEPC, write to nepc@colorado.edu.

June 20, 2014

News From The NEPC: Fordham Puts Lots Of Little Choice Carts Before The Evidence Horse

From yesterday’s inbox…

Research and analysis to inform education policy
and promote democratic deliberation
Is this email not displaying correctly?
View it in your browser.

Fordham Puts Lots of Little Choice Carts before the Evidence Horse

Report describes how to bring school choice to the level of individual classes – but it never considers research regarding whether this would be beneficial

Contact: 

William J. Mathis, (802) 383-0058, wmathis@sover.net

Patricia Burch, (909) 272-5839, pburch@usc.edu

URL for this press release: http://tinyurl.com/lua4872

BOULDER, CO (June 19, 2014) – A recent report provides guidelines for an expansion of school choice policies, urging policymakers to turn the selection of school classes themselves into a form of market competition, with each student choosing and designing a personal menu of classes.

The drawback to that idea, according to a new review, is that the report makes no effort to evaluate the underlying proposal before offering policymakers a “guide” for how to put it into effect.

Patricia Burch of the University of Southern California, along with Jahni Smith of USC and Mary Stewart of Indiana University, reviewed Expanding the Education Universe: A Fifty-State Strategy for Course Choice for the Think Twice think tank review project. The review is published today by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.

Expanding the Education Universe, written by Michael Brickman and published by the Thomas B. Fordham Institute, envisions a future when students design their own selection of online and off-line school classes. As the reviewers note, the report possesses a great deal of confidence that such a change would ease transportation and expand students’ options in many beneficial ways. Course providers for the proposed system could be for-profit as well as not-for-profit providers, including school districts and other public institutions.

But in offering a “guide” to solve practical problems of policy and implementation, the report acts with undue haste. It “assumes, without solid evidence, that course choice, electronic educational provisions, and the like are viable, effective, and proven methods,” the reviewers write.

“No direct research is presented, and relevant related research that might support the efficacy of the method is not included,” Burch and her colleagues point out. “Accordingly, the piece rests entirely on assumptions and assertions.”

Lacking the necessary evidence and detail to demonstrate that the proposal holds any promise, policymakers and the public should lack confidence in the report’s proposal. The report, the reviewers conclude, offers “little basis for assessing the benefits and liabilities of a program that potentially has enormous financial costs and educational quality implications for public education.”

Find the review by Patricia Burch, Jahni Smith, and Mary Stewart the NEPC website at:
http://nepc.colorado.edu/
thinktank/review-expanding-
the-education-universe
.
Find

Expanding the Education Universe, by Michael Brickman, on the web at:
http://edexcellence.net/publications/
expanding-the-education-universe-a-fifty-state-strategy-for-course-choice
.
The Think Twice think tank review project (http://thinktankreview.org) of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound e

.vidence. For more information on the NEPC, please visit http://nepc.colorado.edu/This review is also found on the GLC website at http://www.greatlakescenter.org/.
If you are not already subscribed to this newsletter and would like to receive it regularly, click
http://nepc.colorado.edu/
and then click the button in the upper right-hand corner that looks like this: 


The National Education Policy Center (NEPC) is housed at the University of Colorado Boulder School of Education. Its mission is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information about the NEPC, please visit http://nepc.colorado.edu/.


 

Copyright © 2014 National Education Policy Center, All rights reserved.
You’re receiving this email because you have opted in at our website or sent a personal request to be included. Thank you.
Our mailing address is:

National Education Policy Center

School of Education, 249 UCB
University of Colorado

Boulder, CO 80309-0249

Add us to your address book

 For all other communication with NEPC, write to nepc@colorado.edu.

June 18, 2014

News From The NEPC: Earnest Oversimplification

From yesterday’s inbox…

Research and analysis to inform education policy
and promote democratic deliberation
Is this email not displaying correctly?
View it in your browser.

Earnest
Oversimplification

Plan for states to turn school reform over to nonprofits may be sincere, but it lacks evidence and ignores the real world 

Contact:

William J. Mathis, (802) 383-0058, wmathis@sover.net

Peter Cookson, (202) 403-6203, pcookson@air.org

URL for this press release: http://tinyurl.com/k7fm3d2

BOULDER, CO (June 17, 2014) – A recent report suggesting states should turn education reform over to “an ecosystem of nonprofit organizations” lacks substantiation and ignores the realities of school, policy and politics, according to a new review.

The report presumes that “educational justice is a technical problem that can be resolved by tinkering with governance issues,” says Peter W. Cookson Jr., a professor at Georgetown University who reviewed The State Education Agency: At the Helm, Not the Oar for the Think Twice think tank review project. The review is published today by the National Education Policy Center, housed at the University of Colorado Boulder School of Education.

At the Helm, Not the Oar was written by Andy Smarick and Juliet Squire and published by the Thomas B. Fordham Institute. The report argues that state education agencies should step back from direct involvement in school reform, handing the task over to nonprofit organizations that would be charged with implementing school improvement. The report offers as its rationale the assertion that state education agencies are hobbled by cumbersome, time-consuming procedural requirements, statewide politics, and “institutional sclerosis.”

Cookson’s review, however, finds the report “sincere and well-written but methodologically and politically unsophisticated.” Its findings and recommendations are unsubstantiated, and it “oversimplifies social complexity,” the reviewer writes. “Privatizing educational reform is an idea whose time has not come, and most likely never will,” because it is premised “on a model of American education disconnected from the democratic ethos that animates public education.” Along with market education reform advocates generally, the report’s authors do not acknowledge that “change does not take place in a sociological vacuum; policymakers and educators live in the real world of structural racism, blocked mobility, and opportunity hoarding by the affluent.”

Contrary to the report’s apparent aim, “There is no way to eliminate politics from educational change—nor should there be,” Cookson writes. “Debate and difference are what make democracy strong and are likely to lead to solutions that reflect the public good.”

To the degree that the report forces people to question the unexamined assumptions that underlie their approaches to the issue of education reform, however, the report could prove useful, Cookson concludes. Its real utility is to focus attention squarely on market and quasi-market educational reform arguments “not as ideology, but as a change model,” he writes. And that, he suggests, is something “that needs to be addressed honestly” by supporters and critics alike of the market model.

Find The State Education Agency: At the Helm, Not the Oar, written by Andy Smarick
and Juliet Squire and
published by the Thomas B. Fordham Institute,
on the web at:
http://edexcellence.net/
publications/the-state-education-agency-at-the-
helm-not-the-oar
.
The Think Twice think tank review project (http://thinktankreview.org)
of the National Education Policy Center (NEPC) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. NEPC is housed at the University of Colorado Boulder School of Education. The Think Twice think tank review project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice.

The mission of the National Education Policy Center is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information on the NEPC, please visit http://nepc.colorado.edu/.

This review is also found on the GLC website at http://www.greatlakescenter.org/.

If you are not already subscribed to this newsletter and would like to receive it regularly, click
http://nepc.colorado.edu/
and then click the button in the upper right-hand corner that looks like this: 


The National Education Policy Center (NEPC) is housed at the University of Colorado Boulder School of Education. Its mission is to produce and disseminate high-quality, peer-reviewed research to inform education policy discussions. We are guided by the belief that the democratic governance of public education is strengthened when policies are based on sound evidence.  For more information about the NEPC, please visit http://nepc.colorado.edu/.


 

Copyright © 2014 National Education Policy Center, All rights reserved.
You’re receiving this email because you have opted in at our website or sent a personal request to be included. Thank you.
Our mailing address is:

National Education Policy Center

School of Education, 249 UCB
University of Colorado

Boulder, CO 80309-0249

Add us to your address book

For all other communication with NEPC, write to nepc@colorado.edu.

Next Page »

The Rubric Theme. Blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

Join 2,472 other followers