Greatest Benefits Of K-12 Online Learning And Visible Learning

A couple of months ago I posted a poll asking What Do You Believe To Be The Greatest Benefit To K-12 Online Learning? In response, I received the following totals:

Individualized instruction – 23% (6 votes)
Being able to access learning at any time – 31% (8 votes)
Learning catered to student’s learning style – 19% (5 votes)
Access to highly qualified teacher – 15% (4 votes)
Ability to progress at any pace – 8% (2 votes)
Other (please add additional responses to the poll comments) – 4% (1 votes)

The other in this instance was no bus ride and no school lunch.

A short while later, my good friend Darren – over at Teaching and Developing Online – asked his students at the Saskatoon Catholic Cyber School the same question and reported the results at K-12 Online Learning Benefits.  Essentially they broke down like this:

Individualized instruction – 6% (3 votes)
Being able to access learning at any time – 26% (14 votes)
Learning catered to student’s learning style – 8% (4 votes)
Access to highly qualified teacher – 4% (2 votes)
Ability to progress at any pace – 47% (25 votes)
Other (please add additional responses to the poll comments) – 9% (5 votes)

And I don’t know what the “Other” entries were in this poll because most of Darren’s students left comments (again see K-12 Online Learning Benefits).  I mention these two polls because I used this poll specifically because I wanted to come back to it later and discuss some of the items listed here.  The reason I wanted to use them was because these were what I have perceived to be many of the purported benefits of online learning by proponents of K-12 online learning.  If you listen to many speeches or read many press releases from iNACOL, or any of the cyber charter companies (and even some of the supplemental state-wide programs), these are the benefits to K-12 students they state online learning brings.  Shortly after I originally posted this poll, I used the monthly Meandering Out Loud feature to talk about K-12 online learning and a new book I had come across – John Hattie’s Visible Learning: A Synthesis of Over 800 Meta-Analysis Relating to Achievement.  I wanted to pick that conversation up again now that I have spent more time with Hattie’s book.

Anyway, to provide some background first…  A meta-analysis is essentially where a researcher takes a bunch of existing studies that compare the use of X teaching innovation vs. a situation without X, and then combine the results of all of those studies to get a result based on an expanded sample (click the link for a more technical definition).  For example, there may be five studies of varying sample sizes with differing results that look at whether assigning homework has any effect on student performance.  A meta-analysis would take the results of those five studies and combine them in such a way to give an overall effect size.  Most researchers argue that this combined effect size should be higher than 0.6 or 0.8 for the intervention to be considered useful.

What Hattie has done in his work is combine over 800 of these meta-analysis (so he is combining studies that have already combined studies) to come up with what I will call these super results or effect sizes.  In his book (and I suspect in his articles too – see Hattie & Marsh (1996, 2002)), Hattie argues that we should use an effect size of 0.4 as a determinant on whether an intervention should be consider – and he argues this based upon the amount of work involved with different interventions (and uses the example that it may not cost that much to organize instruction based on behavioural objectives  (d=0.20), whereas the cost may be too much to implement reciprocal teaching (d=0.74), and since the cost to implement behavioural objectives is so low it is worth doing – in very much a cost-benefit approach).

So, if you consider the “purported benefits of online learning by proponents of K-12 online learning” I included in my original poll, there are only two that have direct instances in Hattie’s findings:

Individualized instruction – d=0.23
Learning catered to student’s learning style (“matching style of learning” in Hattie’s book) – d=0.40

Based on Hattie’s perspective, the “individualized instruction” isn’t worth even considering because it doesn’t provide a positive different in achievement for a high enough percentage of students and the “matching style of learning” with a d=0.40 would be worth considering based on the cost effectiveness of it (although keep in mind many others believe you need an effect size of 0.6 or 0.8 before you should consider adopting any intervention).  Also note that I’ve discussed the pseudo-science of learning styles before (see Using Learning Styles for one of the more recent entries or my review of Disrupting Class at Theory of Disruption Applied to K-12 Doesn’t Compute).

The other three items in our list of “purported benefits of online learning by proponents of K-12 online learning” don’t have direct correlations, but there are some points of comparison.  In theory, “access to highly qualified teacher” should translate into the “quality of teaching” (d=0.44).  However, the way in which states operationalize “highly qualified teacher” varies from state-to-state.  While the federal definition for “highly qualified teacher” is basically a core academic subject is one who holds at least a bachelor’s degree, is appropriately licensed for the assignment and demonstrates subject matter competence (i.e., content knowledge) in the core academic subject(s) she/he teaches, states have operationalized this from copying the federal definition word-for-word to defining it as any teacher that holds a Master’s degree or is Nationally Board Certified.  With such variation, it is actually worthwhile to look at some of the other teacher-related interventions that Hattie reports.  The ones that were 0.4 or greater include:

Providing formative evaluation (d=0.90)
Micro teaching (d=0.88)
Teacher clarity (d=0.75)
Reciprocal teaching (d=0.74)
Providing feedback (d=0.73)
Teacher-student relationships (d=0.72)
Professional development (d=0.62)
Teaching strategies (d=0.60)
Cooperative vs. individualistic learning (d=0.59)
Study skills (d=0.59)
Direct instruction (d=0.59)
Mastery learning (d=0.58)
Worked examples (d=0.57)
Concept mapping (d=0.57)
Goals (d=0.56)
Peer tutoring (d=0.55)
Cooperative vs. competitive learning (d=0.54)
Questioning (d=0.46)
Expectations (d=0.43)

I should also note that “teacher training” (d=0.11) and “teacher subject matter knowledge” (d=0.09) were very low ranked items.

The closest that I was able to find to “ability to progress at any pace” was “spaced vs. mass practice” (d=0.71) and to “being able to access learning at any time” was “time on task” (d=0.39); although I’d argue that “student control over learning” (d=0.04) applies to both of these purported benefits.

I find it interesting in making these comparison that in many instances the “purported benefits of online learning by proponents of K-12 online learning” don’t measure up to the most systematic, reliable and valid research that we have available to us at this time.  Which leads me to believe that the “purported benefits of online learning by proponents of K-12 online learning” are in many instances simply rhetoric used for political purposes (but I’m generally known as a cynic when it comes to these things and you may see it differently).

Of note for those interested in K-12 online learning might also include:

Home environment (d=0.57)
Parental involvement (d=0.51)
Second and third chance programs (d=0.50)
Small group learning (d=0.49)
Reducing anxiety (d=0.40)
Computer assisted instruction (d=0.37)
Frequent/effects of testing (d=0.34)
Decreasing disruptive behaviour (d=0.34)
Programmed instruction (d=0.24)
Class size (d=0.21)
Charter schools (d=0.20)
Web-based learning (d=0.18)
Home-school programs (d=0.16)
Distance education (d=0.09)

Bibliography:

Hattie, J. (2009). Visible learning: A synthesis of over 800 meta-analysis related to achievement. New York: Routledge.

Hattie, J., & Marsh, H. W. (1996). The relationship between research and teaching: A meta-analysis. Review of Educational Research, 66, 507–542.

Marsh, H. W., & Hattie, J. (2002). The relationship between productivity and teaching effectiveness. Journal of Higher Education, 73(5), 603-641.