Virtual School Meanderings

March 17, 2009

K-12 Online Learning And Open Access Literature

Note: Make sure to read the comments for additional, relevant information about the content of this entry.

A week or so ago, I made an announcement that an article that Cathy Cavanaugh, Tom Clark and I had published in International Review of Research in Open and Distance Learning (see K-12 Online Learning Research: Article Publication).  During that time, there have been a number of items that have been blogged about this article.  For example…

However, there has been one that has found a way to get under my skin… 

For those of you who haven’t visited this entry, it reads:

A literature review limited to OA literature

Cathy S. Cavanaugh, Michael K. Barbour, and Tom Clark, Research and Practice in K-12 Online Learning: A Review of Open Access Literature, The International Review of Research in Open and Distance Learning, February 2009.

Abstract:   The literature related to online learning programs for K-12 students dates to the mid-1990s and builds upon a century of research and practice from K-12 distance education. While K-12 online learning programs have evolved and grown over the past decade, the amount of published research on virtual schooling practice and policy is limited. The current literature includes practitioner reports and experimental and quasi-experimental studies, both published and unpublished. This paper reviews open access literature in K-12 online learning and reports on a structured content analysis of the documents. Themes in the literature include steady growth and a focus on the benefits, challenges, and broad effectiveness of K-12 online learning. In addition, newly developed standards for K-12 online learning are emerging in descriptions of effective practices.

Comment. This is the first literature review I’ve seen which deliberately limits itself to OA literature.  Don’t jump to conclusions about why the authors did it this way.  They do not believe “if it’s not OA, then it’s not worth reading”. They did not decide to review what was ready to hand because they lacked access to much of the TA literature.  (Some of the co-authors have published previous literature reviews focusing on the TA literature.)  They did not assume that OA literature and TA literature differ in the topics they cover or conclusions they draw, which one could only know by reviewing of the TA literature as well.  From the body of the paper:

The decision to use only open access documents was made for two reasons. The initial search of literature revealed that individuals outside of the academy authored the majority of documents; thus, the authors may not have regular or free access to subscription-based publications. Also, because the authors were interested in presenting this paper to the practitioner community, we wanted to ensure that this audience was able to access the documents on which our metasynthesis was based….

One day soon we’ll see another kind of literature review limited to OA literature:  one based on sophisticated text mining.  The authors will explain that only OA literature is technically and legally amenable to that kind of analysis.

I guess the reason that has gotten under my skin a bit is threefold.  The first is the attribution that while we have focused on only open access literature because we did it for reasons the author questions, that somehow the end product is not of the same quality.  I’m wondering how the author of this blog thinks we were able to come up with the 226 publications we reviewed, if it wasn’t from a systematic search of the open access literature using a variety of terms to mine the open databases?!?

The second reason is the claim that “they [meaning us] lacked access to much of the TA literature”.  Please note that we had access and could have included the over 500 items that we discovered in our initial mining of both the open access and closed databases. If you read the section this blog author quoted, it states:

The initial search of literature revealed that individuals outside of the academy authored the majority of documents; thus, the authors may not have regular or free access to subscription-based publications.”

Allow me to translate…  The authors of the literature we were finding were for the most part not from the academic community and would not have access to many of the pieces included in our metasynthesis – which would prevent others from replicating or building upon it.  You see, the vast majority of people who have written about k-12 online learning have been practitioners and, as such, they do not have access to the same kind of closed databases that those of us in the academic community have (usually through subscriptions maintained by our institutional libraries).

The third, and probably the most important, reason is the close-mindedness of the author of this blog entry.

Don’t jump to conclusions about why the authors did it this way.  They do not believe ‘if it’s not OA, then it’s not worth reading’.”

One day soon we’ll see another kind of literature review limited to OA literature:  one based on sophisticated text mining.  The authors will explain that only OA literature is technically and legally amenable to that kind of analysis.”

I mean, come on…  If it isn’t open access than it isn’t worth reading?  How close-minded can you be?  There is a lot of good literature, based on strong research methodologies, that is being published in both open access and closed database sources.  To say that if it isn’t open access it is not worth reading is kind of like saying I’m going to surf the Internet, but I’m only going to visit websites that were created using Dreamweaver because I personally believe that Dreamweaver is a superior product to anything else on the market.  It is exactly the same thing (and yes, just as dumb!).

There are many reasons why an author will choose a particular publication as an outlet for their work.  For example, we choose the International Review of Research in Open and Distance Learning because we were looking for a publication that focused on distance education/online learning, that was open access, that had a wide and international readership, was peer reviewed and that was known for being of high quality.  We made these decisions because the content of our article was about online learning, the limiting of our methodology to open access literature, the fact the literature we reviewed came from multiple countries, and because two of the authors are in academic environments and had promotion and tenure considerations.

The final item, promotion and tenure considerations, is an important one for those of us in the academy.  Not all of us are at the top of the academic ladder (like the author of the blog entry in question), and are able to essentially do whatever we please – academically speaking – and not worry about the consequences of those actions.  For most of us, we can’t be an ideologue and limit ourselves from quality publications simply because they aren’t open access.  We make decisions based on where is the most appropriate place for a particular manuscript – and that is the way it should be.

Now would I like to see a world where all scholarship is open access?  Definitely!!!  But until then to claim that what gets published in closed journals is not worth reading is nothing more than a lie!!!  And until we reach a stage where everything is open access, when I submit my manuscripts I will continue to consider the audience I want to reach, the content of the manuscript, and the highest impact that I believe I can have based on the quality of the manuscript.  And I won’t apologise when I submit something to a closed journal – and when I get tenure I won’t look down my nose at those who do the same thing I did before I got tenure.  I would hope more of my colleagues, even those who have become ideologues, would do the same!

8 Comments »

  1. Hi Michael: You reversed my point entirely. I didn’t say that you believed that “if it’s not online, it’s not worth reading.” I said that you did *not* believe that. And so on for several other possibilities. Please read it again. I was trying to warn my readers *away* from the interpretations that got under your skin. When I quoted your reasons for limiting your review to OA literature, I did it with approval.

    Comment by Peter Suber — March 17, 2009 @ 11:10 pm | Reply

  2. Peter, if that was your intent it didn’t come across that way – and I do apologise. I even read it one day, and did what my wife always encourages, left it alone and slept on it before coming back the following day and read it again and left with the same interpretation.

    Even reading it again now, I’m still a little puzzled. Since I have you hear, I want to make sure I understand your original entry then… The lines:

    “One day soon we’ll see another kind of literature review limited to OA literature: one based on sophisticated text mining. The authors will explain that only OA literature is technically and legally amenable to that kind of analysis.”

    Are the ones that throw me the most (and since you end the entry this way it appears that this is the last thing you want the reader to take away). These two sentences seem to indicate that an analysis based on sophisticated text mining can only be done using open access – “ONLY open access literature is technically and legally amenable to that kind of analysis” (i.e., an analysis “based on sophisticated text mining”). Can you explain what you meant by this, cause the is where the lasting negative impression that I leave your entry with comes from?

    Comment by mkbnl — March 17, 2009 @ 11:17 pm | Reply

  3. Michael: Apologies if my original post was unclear.

    My final paragraph (on text mining) was not about your article or about your reasons for limiting review to OA literature. After quoting your reasons (and contrasting them with some bad reasons that did not pertain to you), I wanted to offer another possible good reason to stand alongside yours.

    I was trying to address readers who couldn’t imagine any good reasons to limit a review to OA literature. First there were your reasons. Then, beyond them, there was this additional reason based on text mining.

    Text mining requires machine access to the corpus of texts you want to mine. For non-OA literature, that requires payments or permissions or both. Hence, it’s expensive or time-consuming or both. Without the permissions, it can be illegal, since it requires making full-text copies for processing. OA literature removes all these problems. Hence one good reason to limit review to OA literature is to use some kind of sophisticated text mining in the review.

    I expect that the growth of OA will nurture the growth of text mining, and I expect that most text mining projects will limit themselves to OA literature.

    Comment by Peter Suber — March 18, 2009 @ 9:46 am | Reply

  4. So text mining and data mining are two different things then? Which may be where my confusion came in. As I understand it, data mining is simply running keywords through databases that you have access to (open or closed) and the actual process behind the scenes doesn’t require the copying of the manuscripts that you describe above.

    While I’ve heard of the process that you refer to, as there was a program in the mid-1990s developed in the UK for political purposes (at least that was the circles I was in at the time and your political parties (cause you’re on that side of the pond if I read your website correctly) would scan in all of the newspapers in an OCR fashion and the software would mine what the media were writing about – and you could search it based upon certain keywords (e.g., you could see what came up about your own political party, leader, opposition, etc.).

    Is the text mining you’re talking about kind of like a completely electronic version of that? If so, than it was the term that threw me. Is the term text mining common in open access circles? Or is it more common and something I should have known as an academic? I ask cause if it is the latter, I have some reading to do.

    Comment by mkbnl — March 18, 2009 @ 9:56 am | Reply

  5. Text mining and data mining are similar projects in different domains. Yes, text mining is very much like the process you describe. It’s a way to digest more literature than you could actually read.

    The term (“text mining”) is fairly common, though not only in OA circles. Here’s the Wikipedia article about it.

    Comment by Peter Suber — March 18, 2009 @ 1:21 pm | Reply

  6. Thanks Peter… Both for the link with the additional information and for engaging in this dialogue to help clarify your original blog entry. See that I have posted a quick note at the top of my original entry to get folks to read your additional comments below for more information about the content of this entry.

    Comment by mkbnl — March 18, 2009 @ 1:25 pm | Reply

  7. Thanks, Michael. I’ve also updated my post to add a link to this one.

    Comment by Peter Suber — March 18, 2009 @ 8:07 pm | Reply

  8. Cool… And thanks again Peter for this extended dialogue. It has been informative and appreciated.

    Comment by mkbnl — March 18, 2009 @ 8:27 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Blog at WordPress.com.