Virtual School Meanderings

May 24, 2019

Review Identifies Flaws In High-Profile “Gold Standard” Study Used To Market TFA

An item from the National Education Policy Center.

Review Identifies Flaws in High-Profile “Gold Standard” Study Used to Market TFA

Thursday, May 23, 2019


Review Identifies Flaws in High-Profile “Gold Standard” Study Used to Market TFA


TwitterEmail Address

Andrew Brantlinger is a former public school math teacher who is now an associate professor at the University of Maryland’s Department of Teaching and Learning, Policy and Leadership. Earlier in his academic career, he worked with data concerning the New York City Teaching Fellows alternative certification program. So Brantlinger was intrigued when, six years ago, the federal Institute of Education (IES) Sciences published a report entitled, The effectiveness of secondary math teachers from Teach For America and the Teaching Fellows programs, finding that Teach for America corps members significantly out-performed other teachers at their high-poverty schools. This IES-funded high-profile study, which was authored by researchers at Mathematica, a non-partisan, research organization, is prominently featured in TFA promotional material.

TFA selects high-achieving college graduates and places them in these high-poverty schools after several weeks of preparation. Although the TFA corps members start off uncertified, the placement is followed by ongoing, on-the-job support, and many do eventually gain standard certification.

Brantlinger was eventually able to obtain the data used in the IES/Mathematica study and, along with co-author and University of Maryland doctoral candidate Matthew Griffin, he was able to perform a secondary analysis of the study data.

In a Review Worth Sharing published today by the National Education Policy Center, Brantlinger and Griffin explain that the original analysis was flawed in three primary ways:

  • First-year Teach for America teachers were under-represented in the study (while second-year corps members were over-represented). This matters because teachers typically make considerable professional growth in their initial years on the job.
  • Poorly qualified teachers were over-represented in the comparison group. For example, nationwide, 80 percent of 8thgrade math teachers at high-poverty schools are fully certified. Yet just 40 percent of the comparison group were fully certified, while 58 percent of the TFA teachers in the study were fully certified. Keep in mind that alternative-certification programs, by definition, generally place teachers in schools before they are certified—making the situation studied here difficult to generalize. This may limit the study’s applicability to other schools and also bias the results in TFA’s favor.
  • TFA teachers were likely trained to teach to the exams used as study outcomes, since such an approach is part of the program. The study did not account for this likely alignment between the outcome measure and the TFA focus.

Despite assertions to the contrary on TFA’s website and promotional materials and by the authors of the Mathematica report, the effect size identified by the study was small—certainly small enough to be explained by these three flaws in data and methods.

The Mathematica study was designed as an experiment, with students randomly assigned to matched pairs of TFA and comparison teachers. Randomization studies are sometimes described as the “gold standard” for research because they reduce the odds that treatment and control groups are not comparable. However, as Brantlinger and Griffin’s analysis highlights, the on-the-ground reality of experimental studies does not always translate into unbiased comparison groups in real-world schools. And in cases in which the participants may be anomalous (e.g., control group teachers who are less qualified than typical non-TFA teachers), the study results may not be generalizable to other TFA-employing schools.

Brantlinger said he conducted his analysis because, as a former Chicago Public Schools math teacher who has observed TFA math classrooms many times, he was concerned about the potential impact if the study did, indeed, turn out to have important flaws.

“I’m not against alternative certification,” he told us. He continued:

But it bothered me that TFA was parading this around. It bothered me because the organization gets so much federal and private money [and] in part because they say, ‘we’ve got proven evidence that our teachers are better.’ I think that the Mathematica study is not sufficient evidence.

He added, “I’ve been in a lot of these classrooms. Some TFA corps members are well meaning. Some become good math teachers. But a lot of them are really under-prepared, especially in mathematics.”

NEPC occasionally publishes reviews it has not commissioned because we believe they contribute to our goal of helping policymakers, reporters, and others assess the social science merit of reports and judge their value in guiding policy. These reviews have not gone through NEPC’s editorial process. The views, analyses, and conclusions expressed in them belong entirely to the authors. As is the case with NEPC Reviews, NEPC encourages the authors of the reviewed reports to engage with these additional reviews, consistent with our core belief that readers benefit from healthy, substantive exchanges of ideas and contentions.

This newsletter is made possible in part by support provided by the Great Lakes Center for Education Research and Practice:

The National Education Policy Center (NEPC), housed at the University of Colorado Boulder School of Education, produces and disseminates high-quality, peer-reviewed research to inform education policy discussions. Visit us at:

Copyright 2018 National Education Policy Center. All rights reserved.

Congratulations Michael, You Reached A Milestone

And another item from one of my open scholarship networks.

Your publication has a new achievement:
Social Constructivist E-Learning: A Case Study
View achievement
ResearchGate GmbH, Chausseestr. 20, 10115 Berlin, Germany. Imprint.
See our Privacy Policy and Terms of Service.

The milestone was:

Nice work, Michael!
Your article reached 300 reads

Michael, A Recent Article Cited Your Research

Another item from one of my open scholarship networks.

Michael K. Barbour
Michael, we found a recent citation of your research:
Asynchronous and synchronous online teaching: Perspectives of Canadian high school distance educatio…
View citing research
ResearchGate GmbH, Chausseestr. 20, 10115 Berlin, Germany. Imprint.
See our Privacy Policy and Terms of Service.

The citing research was:

Examining Chinese beginning online instructors’ competencies in teaching online based on the Activity theory
  • May 2019
  • DOI:
  • 10.1007/s40692-019-00140-w

Michael, People Are Reading Your Work

An item from one of my open scholarship networks.

Michael K. Barbour
View report
ResearchGate GmbH, Chausseestr. 20, 10115 Berlin, Germany. Imprint.
See our Privacy Policy and Terms of Service.

The actual report read:

Report for week ending
May 18, 2019
  • +2.7
    Research Interest
    View more
  • +2
    View more
  • 0
    View more
  • +231
    View more
People who read your work
Your most read publications
Show only publications
Show all
Your citations
Reads by institution
  • United States
  • Hooksett
  • Spain
  • Móstoles
Reads by country
United States
Your publication achievements
View all achievements
Your article reached 50 reads
Learning Styles: A Focus upon E-Learning Practices and Pedagogy and their Implications for Designing E-Learning for Secondary School Students in …
Your article reached 100 reads
Distributed Learning in British Columbia: A Journey from Correspondence to Online Delivery

May 23, 2019

EDEN 2019 Annual Conference Bruges – Detailed Programme Available

An update from this European-based distance and online learning conference.  Those interested in K-12 distance, online and/or blended learning sessions might find the following of interest:

Monday, 17 June : 15:45-17:15 Afternoon Parallel Sessions (B)

Paper Session B3
The Instructional Design of an Online Learning Environment (RISSC) for Upper Secondary School Students’ Research Skills
Louise Maddens, Fien Depaepe, Annelies Raes, Jan Elen, Katholieke Universiteit Leuven, Belgium

Tuesday, 18 June 14:30-15:45 Afternoon Plenary Sessions (D)

Poster Session D1
Higher Motivation and Sustainability of Knowledge through Social Media for Teens
Natasa Rizman Herga, Andreja Kolar, Primary school Ormoz, Slovenia, Dejan Dinevski, University of Maribor, Slovenia

Wednesday, 19 June: 9:30-11:00 Morning Parallel Sessions (F)

Paper Session F3
Five Years of Kahoot! in the Classrooms – What does Research Tell Us?
Matthias Murawski, Md Tawhid Hasan, Markus Bick, ESCP Europe Business School Berlin, Germany

Anyway, the update and the link to the full program is available below.

EDEN 2019 Annual Conference: #EDEN19
Connecting Through Educational Technology
In search for contemporary learning environments

Bruges, Belgium, 16-19 June 2019


Detailed programme available!

The Conference programme includes plenaries, parallel sessions with paper presentations, workshops, trainings, moderated poster and demo sessions and the synergy strand, ensuring digital interactivity and cooperation on the social web.
Read the detailed programme

Registration is open

The EDEN Secretariat is taking care of the registration for the Annual Conference. Registration is open!

The registration fee includes access to all conference facilities, the electronic Conference Proceedings, the Welcome Cocktail, lunches and refreshments during the conference days. The fee does not include accommodation and the conference dinner.

For young scholars, a special discount will be applied.

Register for the Conference

Conference live stream

Good news for everyone who can’t make it to the conference: as in previous years, all plenary sessions of the Annual Conference will be streamed on EDEN’s YouTube channel – you will be able to follow the conference welcome, all keynote speeches and the closing remarks from the comfort of your chair.

Social Events

Welcome Reception – 16 June, 19:30

The Welcome Reception helps you to set the mood for the Conference. Meet your colleagues and enjoy the welcoming atmosphere. The EDEN Senior Fellow and Fellow Awards will be presented at this occasion.

The EDEN Fellow and Senior Fellow Awards will be presented at the Welcome Reception.

The Reception will take place on the VIVES campus..

Wine & Games Event – 17 June, 17:30

VIVES Bruges holds the VIVES Archive of Games, the largest collection of games in the world, with more than 20.000 titles and many educational (party) games. In the archive, you will find the most recent (educational) games, but also some very old games are kept in the closed section. The oldest game dates back to 1810!

EDEN members and conference delegates are invited to visit the Archives, entertain with exciting games and savour good wines.

Conference Gala Dinner – 18 June, 20:00

The Conference Gala Dinner will be held in the Market Halls of Bruges.

Standing on the market square of Bruges there is one building that immediately attracts your attention, the market halls and its Belfry.

They have been recognized as UNESCO World Heritage sites since 1999. The halls were built very early in the Middle Ages and were mainly used as an indoor marketplace.

The Best Research Paper Award and Young Scholar Award will be presented at the Conference Gala Dinner.

Guide to Bruges

VIVES University students have created a website to help visitors get to know Bruges and its must-see sights, historic landmarks, charming spots and of course the best culinary experiences in town  – take a look here, and plan your visit!

Plan your visit

Quality and Recognition

EDEN Fellow and Senior Fellow Awards – the ceremony will be held at the Welcome Reception

The purpose of the EDEN Fellow scheme is to provide validation and support to professionals in Europe in the field of open, distance and e-learning, and to enhance their mobility within Europe through a respected scheme of recognition.

Best Research Paper Award & Young Scholar Award- the ceremony will be held at the Conference Dinner

The tradition of awarding the EDEN Best Research Paper will be continued. Since 2008, EDEN has bestowed the Best Research Paper Award at its Annual Conferences and the bi-annual Research Workshops. The selection process takes place in collaboration with the Ulrich Bernath Foundation for Research in Open and Distance Learning.

The Award for the Young Scholars’ Best Paper will also be issued at the Conference Dinner.

Open Badges

Authors of accepted submissions receive a Speaker badge and all conference participants will be awarded a Participant badge.

Learn more about Bruges

The conference is hosted by the VIVES University of Applied Sciences.


Conference website:

Inquiries in relation to the conference should be sent to:

The 28th EDEN Annual Conference is the initiative of EDEN supported by the ERASMUS+ Programme of the European Union
EDEN – European Distance and E-Learning Network Secretariat
Tel: +36 1 463 1628, 463 2537 Fax: +36 1 463 1858
EDEN website
EDEN website
EDEN Facebook
EDEN Facebook
EDEN Twitter
EDEN Twitter
EDEN YouTube
EDEN YouTube
Conference Website
Conference Website
Copyright © 2019 EDEN – European Distance and E-Learning Network Secretariat, All rights reserved.
You are receiving this email because you subscribed to our mailing list.Our mailing address is:

EDEN – European Distance and E-Learning Network Secretariat

Budapest University of Technology and Economics
Egry J. u. 1.

Budapest, Budapest H-1111

Next Page »

Blog at