Virtual School Meanderings

April 26, 2017

News from the NEPC: Report Wrongly Claims to Provide Answers on Wisconsin School Choice Policies

From yesterday’s inbox…  Note the line below that reads: “the report attempts to use a methodology known as “fixed effects” to analyze test score data in districts outside Milwaukee, but such a methodology is not possible with the data described in the report.”

Insufficient data hinders report’s comparison of relative effectiveness across school sectors.
Is this email not displaying correctly?
View in your browser

Report Wrongly Claims to Provide Answers on Wisconsin School Choice Policies

Key Review Takeaway: Insufficient data hinders report’s comparison of relative effectiveness across school sectors.

Contact:

BOULDER, CO (April 25, 2017) – A recent report from the Wisconsin Institute for Law and Liberty attempts to compare student test score performance for the 2015-16 school year across Wisconsin’s public schools, charter schools, and private schools participating in one of the state’s voucher programs. Though it highlights important patterns in student test score performance, the report’s limited analyses fail to provide answers as to the relative effectiveness of school choice policies.

Apples to Apples: The Definitive Look at School Test Scores in Milwaukee and Wisconsin was reviewed by Benjamin Shear of the University of Colorado Boulder.

Comparing a single year’s test scores across school sectors that serve different student populations is inherently problematic. One fundamental problem of isolating variations in scores that might be attributed to school differences is that the analyses must adequately control for dissimilar student characteristics among those enrolled in the different schools. The report uses linear regression models that use school-level characteristics to attempt to adjust for these differences and make what the authors claim are “apples to apples” comparisons. Based on these analyses, the report concludes that choice and charter schools in Wisconsin are more effective than traditional public schools.

Unfortunately, the limited nature of available data undermines any such causal conclusions. The inadequate and small number of school-level variables included in the regression models are not able to control for important confounding variables, most notably prior student achievement. Further, the use of aggregate percent-proficient metrics masks variation in performance across grade levels and makes the results sensitive to the (arbitrary) location of the proficiency cut scores. The report’s description of methods and results also includes some troubling inconsistencies. For example the report attempts to use a methodology known as “fixed effects” to analyze test score data in districts outside Milwaukee, but such a methodology is not possible with the data described in the report.

Thus, concludes Professor Shear, while the report does present important descriptive statistics about test score performance in Wisconsin, it wrongly claims to provide answers for those interested in determining which schools or school choice policies in Wisconsin are most effective.

Find the review by Benjamin Shear at:
http://nepc.colorado.edu/thinktank/review-milwaukee-vouchers

Find Apples to Apples: The Definitive Look at School Test Scores in Milwaukee and Wisconsin, by Will Flanders, published by the Wisconsin Institute for Law and Liberty, at:
http://www.will-law.org/wp-content/uploads/2017/03/apples.pdf

The National Education Policy Center (NEPC) Think Twice Think Tank Review Project (http://thinktankreview.org) provides the public, policymakers, and the press with timely, academically sound reviews of selected publications. The project is made possible in part by support provided by the Great Lakes Center for Education Research and Practice: http://www.greatlakescenter.org

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at: http://nepc.colorado.edu


Copyright © 2017 National Education Policy Center. All rights reserved.

Leave a Comment »

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.

%d bloggers like this: