Virtual School Meanderings

July 9, 2010

Series: What Constitutes Quality in Virtual Schooling?

I’ve been cleaning out my Bloglines, as I get ready to post a few series and other regular monthly features, and I came across this series of blog entries over at The Quick and the Ed that were kind of like a debate/discussion or series that Bill Tucker posted with one of his colleagues.

Essentially, the first entry saw Bill arguing in favour of using K-12 online learning in a greater capacity to provide parents with school choice and allowing more players into that online learning space.  To ensure the quality of those new (and I suppose the existing) online learning players, we could consider some kind of accreditation process – similar to what is found in higher education.  However, Bill is more in favour of quality being measured as scores on bubble tests (that’s not how he puts it, but the notion of “pay them on a performance basis” and “You only get public funds when a student succeeds and passes the course.” does mean how students score on standardized testing).  Bill is even generous enough to allow for bonuses in his pay for test scores system, to allow for more money if the students are “disadvantaged and special needs students”.  Essentially, it becomes a market driven system – the same kind of system you see in the United States with health care (and we see how well that works for people on the lowest rung of the socio-economic ladder).

His colleague, Rob Manwaring, responded to Bill in the second entry by suggesting that the strict pay for performance model – particularly if it is based solely on test scores or course completion – is a flawed system that is ripe with the possibility of abuse.

To his credit, in the third and final entry linked above, Bill does argue with Rob that a strict pay for performance model does have the potential for abuse.  He also acknowledges that they both see issues with the traditional post-secondary accreditation system (so that isn’t a preferred option to ensure quality), and then kind of leaves it at that…  A discussion between two individuals – one of whom I know has a strong understanding of the issues related to virtual schooling (sorry Rob, but I don’t know your background) – about how we ensure quality in K-12 online learning that ends where it started.

I’ll be honest and say that – like these two – I don’t have an easy answer.  I’m not as down on accreditation as these two individuals seem to be (although we’re still in the middle of a process of trying to become re-certified to offer the State of Michigan educational technology endorsement in my program, so I may have a different opinion by the time we are finished).  One of the things that I would suggest is that the metrics valued by the previous administration (and really the current administration as well, cause they have really shown no concrete signals that they disagree with the focus on standardized testing in English, mathematics and science) are not those valued by or useful in society.  The ability to memorize information and then recall that information in order to shade in the correct bubble on a piece of paper is not going to be useful to most students ten or twenty years from now (unless they are a census taker or make a living completing mail-in surveys).  Personally, I’d argue that one of the reasons that America has continued to slide in most measures of educational competitiveness is because we value (i.e., teach and test) the wrong kind of information processing.

I’ll be the first to admit that designing assessment that adequately measures other forms of information processing (e.g., critical thinking, problems solving, abstract conceptualization, analysis, synthesis, evaluation, etc.) isn’t the easiest thing to do.  But before we can begin to talk about how do we ensure quality, I think the conversation needs to begin with what kind of learning do we value and what constitutes a quality learning experience.  Once we have answers to those questions – and those questions aren’t limited to the K-12 online learning environment – then we can begin to talk about the appropriate metrics and whether or not there is a way to tie funding to those metrics (which in my gut I still think it not the right path to be on, but it does seem to be the path we are heading).


  1. So, I’m the idealist.

    Can’t we at least have some discussion about the ideal. We’ll never get to it if all we do is continue to muddle in the current system models that at least the three of you agree have flaws. While Bill’s model of high stakes testing is where we are, as long as we talk about perpetuating that model the longer we’ll have it, even if everyone who supports it admits it has flaws.

    Of course, technology does enable different forms of assessment, if the institute of evaluation “mafia” would accept it. Or if we remove the power of the “mafia”.

    And I agree with Michael that this isn’t just about online education. Any measurement (assessment) system should work effectively and efficiently with on-ground as well as online education.


    Comment by Ray Rose — July 9, 2010 @ 6:48 pm | Reply

  2. Ray, thanks for the comments. So, in your idealistic vision, how should we ensure quality? As a follow-up, how should we fund schools?

    Comment by mkbnl — July 9, 2010 @ 8:40 pm | Reply

    • I think we start ensuring quality through the establishment of standards, and by making sure that the folks who will be designing and teaching in the online courses understand online education, the standards and the online research.

      One of my frustrations is the NIH mentality in education. Too many of the online programs have said; “developing an online program is easy” and they haven’t looked at what the leaders in the field have been doing and what they’ve learned. Then rather than build on field knowledge, they repeat the early struggles the leaders have gone through and previously solved.

      I believe that’s one of the reasons the field hasn’t advanced the way I believe it could.

      At SITE this year, I asked the 60+ folks in my session how many were teaching online, and had over half respond positively, but when I then asked how many folks were preparing their graduates to teach online — to be teachers in a virtual school the number dropped to a handful.

      I don’t believe the majority of folks in higher education who teach online have any real understanding of effective online pedagogy. I’ve been at a few higher ed conferences listening to the ID folks who are preparing faculty to teach online, and what they do is focus on how to use the technology. they don’t even know about differences between effective on-ground and online pedagogy. I’m also seeing that with too many of the programs to prepare K-12 folks to teach online.

      It’s nice that iNACOL has published standards, but my concern is that the wording on many of them leaves a great deal to the understanding of the reader. My favorite example (of course)is the standard about access. If you don’t understand the issue of access to begin with, they you don’t know what the words mean. How many courses have you looked at that supposedly meet the iNACOL standards, but use video and don’t have it captioned, nor have a transcript? How many don’t use descriptive alt tags for all their graphic images?

      So, back to the quality issue, we need to do more to help the field understand first, what are the essential elements of a online quality program, and in doing that, to insure that those folks understand that online education isn’t second best to on-ground, it’s just different.

      We have evidence (look at the national drop out rate) that the current on-ground education structure doesn’t work for everyone. We know there are learners who, even though successful in on-ground programs prefer online.

      We need more research on the elements of effective, high quality online education. But, we need that done by folks who have a good understanding of online education, not just those grad students looking for a new dissertation topic. These is, at this time, a need for an accreditation process for online programs, but I’m not convinced the current accreditation folks are the right ones for that. I had advocated iNACOL start that 4 years ago, but now I’m not sure if that organization has become too political to take on that and do it effectively.

      That leaves it to us to be the voices advocating for quality and explaining how it can be accomplished. We need to insure that all online programs are doing a good job — not an okay job — and then, I believe we’ll have less trouble with the funding. But as long as there are poorly run programs — programs that are not about insuring all their students have a quality education; programs that garner headlines because of mis-management; programs that don’t care, because they see online as a second best option — then online education will have trouble getting the funding it should have.

      Comment by Ray Rose — July 11, 2010 @ 3:07 am | Reply

      • One of the issues with funding is that some of the virtual education programs are (or feel they are) fighting for the same dollars. That element of competition has reduced the amount of sharing that would help the field move forward.

        I know of one program that was early-on making all of it’s program information available. They put everything up on their website. But the CEO became frustrated because so many times found new programs had appropriated materials and not given credit to the originators. Now, a lot of the information and experience isn’t as readily accessible. That hurts the field. But, programs without a solid financial base, also need to feel they have a product or brand that they need to protect.

        So, back to the question of how to fund online programs… We need support from sources that also require significant dissemination of program information. The VHS project’s final report is an example; the report was “The Virtual High School, Teaching Generation V”. We need more of that attitude. Programs do share today, but with a great deal more caution.

        Comment by Ray Rose — July 11, 2010 @ 4:43 am

      • Ray, thanks for these additional comments. I wonder if Bill Tucker and Rob Manwaring are following this thread and would care to comment – either by leaving a comment here or by writing something over at The Quick and the Ed blog.

        For myself, I agree that the iNACOL standards are problematic – in my opinion, largely due to the fact that there is no open way to determine the process they undertook (with possibly the exception of the program administration ones). Standards can be developed in several ways, although the most common is to use a combination of the research literature and expert review to create a set of draft standards, that are then applied in a variety of settings by a variety of people so that you can determine things like the validity of the standards, the reliability of the instrument, inter-rater consistency, etc.. With the exception of the program administration standards (which I’m led to believe did the expert review portion), the is nothing in print to indicate that iNACOL used the research literature or open, systematic expert review to come up with the “National Standards” they have published – and I know for a fact that there has been nothing published that would valid any of the three sets of standards.

        As I mentioned in my initial entry, like you I think, I’m not necessarily against some kind of accreditation process. I do agree with you that iNACOL is a political or lobbying body – and not suitable to fulfill an accrediting kind of role. But I do think that some kind of systematic, open process would help with the quality issue when it comes to many of the K-12 online learning programs out there.

        On the issue of the sharing of resources… I have always found the question, “How many online versions of Algebra I do we really need?” a compelling one. Every state requires at least one full year of mathematics and it has long been rumoured that Algebra I and Algebra II are the two most enrolled supplemental courses in the United States. Makes you wonder if we won’t start to see more models like the Open High School in Utah.

        Finally, I’ll be honest and say that I don’t have a good answer on the funding question. The best model that I have seen to date is the one in the Canadian province or British Columbia. And if I mention his name, hopefully Tim Winkelmans will get a Google alert and be able to come here and describe that funding system for you much better than I ever could.

        Comment by mkbnl — July 11, 2010 @ 5:59 pm

  3. Series: What Constitutes Quality in Virtual Schooling?…

    Essentially, the first entry saw Bill arguing in favour of using K-12 online learning in a greater capacity to provide parents with school choice and allowing more players into that online learning space. To ensure the quality of those new (and I suppo…

    Trackback by Teaching And Developing Online — July 10, 2010 @ 10:10 am | Reply

  4. What K-12 Can Learn from For-Profit Higher Ed…

    Good Education Week overview on the growth of for-profit virtual learning companies in K-12 education. Predictably, the article quotes a detractor focused on the tax status of those providing the courses: “I haven’t seen anything in this industry that …

    Trackback by The Quick and The Ed — July 15, 2010 @ 9:32 am | Reply

  5. Thanks for continuing this important discussion. I’ve posted a reply at and hope we can continue to work on this challenge together. Would love to learn more about the model in British Columbia.

    On the multiple versions of Algebra, agree that increasingly the content will be open and sharable. But the teaching and execution, including how well students are supported, can take many, many different forms and I’d love to see more opportunities for creativity and excellence.

    Comment by Bill Tucker — July 15, 2010 @ 9:51 am | Reply

    • Bill, thanks for the comment (and the entry at The Quick and The Ed). I’ve sent notice of this thread to a colleague of mine in British Columbia, hoping he’ll come in and explain the BC model for us.

      Comment by mkbnl — July 15, 2010 @ 10:13 am | Reply

  6. […] Series: What Constitutes Quality in Virtual Schooling? […]

    Pingback by Day 5 – 7 Days To A Better EduBlog « Virtual School Meanderings — July 23, 2010 @ 4:50 pm | Reply

  7. Apologies for my delay in responding to this thread and shedding a little light on the British Columbia BC situation.

    I’ll begin with some stats, so you get a sense of the scale and context. Note that we use the term distributed learning in BC to cover off virtual schooling and other distance delivery modes. Of our 60 public school boards, 47 operate one or more distributed learning schools. This past year, individual student headcount in these programs was 71,000, or over 10% of student population. For our Gr 10-12 students, participation reached over 25%. Most of the enrolments are students in neighbourhood schools seeking additional courses. Students in Gr 10-12 do not need permission of local principal to take a course. Ministry funds are provided to schools based on # of courses delivered – split funding follows the student and there is no direct equivalent between a student and an FTE.

    The thread focus is quality. We believe we created multiple ways to support quality. Most of the key documents described below are found at
    1. Distributed Learning Agreement. A school board must sign one of these to operate a distributed learning school. The schools must meet all general requirements of schools in legislation and policy, modified by Agreement stipulations. These include following standards, hiring teachers based on DL experience and professional training, providing the Ministry with special data, participating in quality review and improvement activities, and other stipulations. The Agreement links to specific policies on the BC website. The ministry can amend the agreement at any time, usually through special provisions triggered by other quality mechanisms.
    2. Distributed Learning Standards. These were created in a participatory manner with government and distributed learning personnel, drawing from research and BC’s long experience with distance education. There are standards for practice and for content development.
    3. Compliance Audits. BC has specific participation rules for funding eligibility (Active Policy, found at These rules, some of the Agreement requirements, and common expectations about teacher certification and learning resources are monitored through the audit program. The criteria and instructions are available online at
    4. Achievement Data Collection. BC has a central student information data warehouse. The ministry recently began collecting course achievement results for each DL student in each DL course in each DL school. As a new reporting requirement, it had to spend some time defining requirements and working with schools to ensure the collection and reporting elements are robust. Very soon, ministry staff should be able to identify effective schools to determine and share their practices. They will also be able to talk to specific schools about results that appear weak.
    5. Quality Review. The compliance audits mentioned above have little to do with pedagogical practice. The quality review is based on participatory qualitative evaluation approaches. It assumes DL schools will use the DL standards as part of school internal review (planning) activities. An external team visits the school to discuss, verify, and comment upon the internal review, as well as to observe and identify effective practices that can be shared with other schools. The team will also provide suggestions to a school re areas for improvement or further exploration. The DL school’s administrator is involved in creating the final report.

    The ministry is (currently) not responsible for creating content or providing professional development, these are Board of Education roles.

    These mechanisms do not guarantee quality, but they set expectations and provide tools that allow the province to respond to emergent issues.

    Comment by Tim — August 6, 2010 @ 5:55 pm | Reply

    • Tim, thanks for this contribution. I’ll make sure that I pass along this to Bill Tucker and hopefully he’ll re-join the conversation.

      Comment by mkbnl — August 6, 2010 @ 8:41 pm | Reply

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at

%d bloggers like this: