Virtual School Meanderings

April 9, 2016

AERA 2016 – Content and Rigor of Algebra Credit-Recovery Courses

As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days.  That means that I will be blogging many of the sessions throughout the week.  The sixth session that I am blogging is:

Content and Rigor of Algebra Credit-Recovery Courses

  • In Event: Getting At-Risk Students Back on Track: Results From a Randomized Trial of Algebra Credit Recovery

4:05 to 6:05pm, Marriott Marquis, Level Two, Marquis Salon 2

Abstract

Purpose
This paper describes the content, organization and rigor of the f2f and online summer algebra courses that were delivered in summers 2011 and 2012. Examining the content of both types of courses is important because research suggests that algebra courses with certain features may be better than others in promoting success for struggling students. Examining course rigor is important because it further describes the context through which students recovered course credit, the study’s primary short-term outcome.

Theoretical Framework
The study’s theory of action assumes that students who fail traditional f2f algebra may benefit from taking an online course because it is a different mode of delivery that is potentially more individualized, interactive and personalized. Analyzing the nature of the content further describes how these features played out in both types of courses.

Methods and Data Sources
Math experts systematically analyzed the content of the online course and the f2f courses (the f2f teachers determined the content of their courses, per district and school policy). The content analyses examined (1) coherence: the extent to which topics were logically sequenced within and across units; (2) difficulty: the extent to which topics were pre-algebra, first semester algebra or second semester algebra; and (3) knowledge emphasis: the extent to which lessons and exercises emphasized procedural fluency, conceptual understanding and applied problem solving. The experts used established frameworks and existing curricular materials to guide both sets of analyses.

Data sources include archival data generated from the online course, course materials (syllabi, annotated tables of contents) from the f2f course, and teacher surveys on which teachers indicated the criteria they used to determine grades in their credit recovery courses.

Results
Results suggest that the online course (in both summers), in comparison to the f2f courses, was more coherently sequenced but more difficult in terms of topics covered and more demanding in terms of grading criteria. The online course content was considered typical of second semester algebra and included a fixed set of topics that were organized sequentially. Conversely, the topics and sequencing of the f2f courses varied widely among the f2f teachers. The topics included not only second semester algebra content, but also first-semester algebra and pre-algebra content, and topics did not typically follow the sequential structure of the online course. Neither the online or f2f lesson materials focused strongly on developing conceptual understanding or applied problem solving, but emphasized procedures and skills instead.

Scholarly Significance
Information from this paper will be used to help frame the short- and long-term study results and to contribute to the relatively thin research base on what it means for students to be proficient in algebra. Although the content of first and second semester algebra is fairly uniform and well specified, the proportion of topics that students should master is not. This issue is increasingly salient in light of the Common Core State Standards for Mathematics, which place a stronger emphasis on conceptual understanding and problem solving than more traditional algebra courses.

Authors

  • Kirk Walters, American Institutes for Research
  • Nicholas Andrew Sorensen, American Institutes for Research

 

Kirk’s portion focused on the content, specifically:

  1. did the course cover the was content with similar sequencing?
  2. how much of the content did students complete and with what level of mastery?
  3. how did grading policies compare?
  4. how did students perform on the end of course algebra assessment?

The online course used in the study was developed by Aventa Learning (i.e., K12, Inc.).  It was self-paced and the standard teacherless online course: do pre-assessment, get multimedia content on what you don’t know, then do post-assessment.

The F2F teachers were certified math teachers, selected by the school according to the typical summer school hiring process.  There was a wide variety of textbooks, supplemental materials, and instructional practices.

The findings included:

  • only half of the content of the average F2F class was Algebra 1B (the rest was Algebra 1A or pre-algebra), whereas the online courses were exclusively Algebra 1B content
  • in the online course the sequencing had a coherent organization, whereas almost a third of the F2F sequencing was coded as incoherent (i.e., the order of content just didn’t make sense in terms of building upon prior knowledge and moving from foundational to more difficult concepts)
  • the online students completed about 2/3 of the course content and only passed an average of 1.7 out of 5 end-of-unit exams (70% was considered passing)
  • the grading policies were more rigorous in the online than in the F2F classes (e.g., tests and quizzes accounted for 60% of the students’ grade in the online cohort, compared to an average of 52% in the F2F)
  • the F2F students performed better on pre-algebra, Algebra 1A, and Algebra 1B items on the posttest than the online students

The main takeaways, at least in the short-term, the online students didn’t do as well, they thought that the class was harder, so maybe it just isn’t appropriate?  An alternate theory is that maybe the school-based mentors needed to be more active in their role in supporting students.

Other entries from this session:

AERA 2016 – Impact of Online Versus F2F Algebra Credit Recovery

As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days.  That means that I will be blogging many of the sessions throughout the week.  The fifth session that I am blogging is:

Impact of Online Versus F2F Algebra Credit Recovery

  • In Event: Getting At-Risk Students Back on Track: Results From a Randomized Trial of Algebra Credit Recovery

4:05 to 6:05pm, Marriott Marquis, Level Two, Marquis Salon 2

Abstract

Purpose
This paper presents results from a randomized control trial designed to assess the impact of online versus f2f credit recovery in Algebra I. The study, through a grant from the U.S. Department of Education, provided resources to Chicago high schools to implement credit recovery courses during the summers of 2011 and 2012 to ninth graders who failed Algebra I in their first year of high school. Students were randomly assigned to take Algebra I as either an online course or as a f2f course.

Theoretical Framework
The theory behind the study is that students fail algebra because they have low incoming skills, are poorly engaged, and exert little effort. Students learn little and subsequently fail. Without understanding algebra, they struggle in subsequent math courses. Students then have insufficient credits to graduate, and eventually drop out. Online credit recovery is hypothesized to interrupt this process by reengaging students via a more individualized, interactive experience with personal support from both an online teacher and an on-site mentor, thus leading to success in algebra and later success in subsequent courses.

Methods and Data Sources
The study employs an experimental design with student-level random assignment to either online or f2f Algebra I credit recovery courses in CPS high schools. The study took place in 17 CPS high schools with relatively high Algebra I failure rates.

A total of 1,224 first-time freshmen participated in summer 2011 or 2012. They were randomly assigned by the study team to take either the online course or a f2f course on the first day of summer school.

The online course was developed by Aventa Learning. Students took the course in computer labs at their local high schools, with a trained on-site mentor. They also had an online algebra teacher, provided by Aventa. The control condition was the typical f2f Algebra IB course offered in participating schools, taught by a licensed CPS teacher.

The study uses achievement, course-taking, and graduation data available via district administrative records. Short-term outcomes were obtained via primary data collection during the summer sessions, including student survey measures and an end-of-course algebra posttest.

Results
Results show that most study students (71%) in both the online and f2f conditions successfully recovered algebra credit; there were some short-term benefits for f2f over online credit recovery; and no differences on longer-term outcomes. The online course was perceived by students as more difficult and less clear than the f2f algebra credit recovery course, and students in the f2f course had higher grades, higher pass rates, and scored significantly higher on an algebra posttest than students in the online course. However there were no significant differences by condition on any measured outcomes during the second or the third year of high school. Graduation outcomes will be in-hand for all study students by November 2015 and will be included in the presentation.

Scholarly Significance
This paper will present findings through high school graduation of the only rigorous study of credit recovery conducted to date.

Authors

  • Jessica Heppen, American Institutes for Research
  • Nicholas Andrew Sorensen, American Institutes for Research

This is basically part of a two hour all AIR session.  It is based on “The Back on Track Study: Using Online Courses for Credit Recovery” study (see http://www.air.org/project/back-track-study-using-online-courses-credit-recovery) – and the press release for this project is available at http://www.air.org/news/press-release/students-using-online-credit-recovery-make-freshman-algebra-fare-less-well-peers.  They began with some background, including the fact that success in Algebra I is often a predictor of success in high school (i.e., graduation), and online credit recovery is the fastest growing sector of the K-12 online learning environment (at least in the public school sector).  Jessica than began to give us all of the proponent used rationale for why online learning, at least online credit recovery, might be a better alternative to face-to-face credit recovery.  The main goals for these sessions (i.e., this and the next four blog entries) was to test the efficacy of online vs face-to-face algebra credit recovery.

Some of the main findings for the overall session include:

  • most students who show up for summary school do recover Algebra I credit
  • F2F students had higher grades, pass rates, and score on an algebra post=test, but no differences in subsequent outcomes including graduation
  • the online course content was more rigorous and coherently sequenced than the F2F course content
  • in online course sections with instructionally supportive in-class mentors, students’ credit recovery rate were on par with F2F students’
  • ninth graders who fail Algebra I are often far off-track, and taking Algebra I online is difficult for at-risk students
  • increasing early credit recovery rates does not appear to affect performance throughout high school or graduation outcomes.

Nicholas then transitioned to the study timeline and key outcomes.  Basically, at the end of the first year of high school if the student failed Algebra 1B, they were entered into a summer school credit recovery program (and randomly assigned to a F2F or an online version).  At the end of the summer the outcomes they looked at were attitudes, grades/credits, and algebra posttest.  Over the next three years they looked at a variety of measures from PSAT, on-time graduation, total math credits, etc..  There were 1,224 students in the student – 613 in the online cohort and 611 in the F2F cohort.

Nicholas then previewed some of the general results, which included:

  • 71% were successful in recovering credit
    • 76% in the F2F
    • 66% in the online
  • 12% of F2F got an A, while 2% of online got a A
  • 23% of F2F got a D, while 36% of online got a D
  • while F2F students did better than online students in the algebra posttest, the difference didn’t carry over to the PLAN test at the end of the second year (and there continued to be no difference in ACT scores in year three or math credits earned and on time graduation rates in year four)

Overall, in the short-term it did appear that the F2F cohort did better.  But these gains do not persist beyond that immediate benefit.  There was also the perception that the online credit recovery course was more difficult.

AERA 2016 – The Launch of the Philadelphia Education Research Consortium: Lessons From the First Year of Implementation

As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days.  That means that I will be blogging many of the sessions throughout the week.  The fourth session that I am blogging is:

The Launch of the Philadelphia Education Research Consortium: Lessons From the First Year of Implementation

  • In Event: Roundtable Session 24
    In Roundtable Session: Advancing Reform: The Role of Relationships, Networks, and Partners in District Improvement

2:15 to 3:45pm, Convention Center, Level Two, Exhibit Hall D Section B

Abstract

The Philadelphia Education Research Consortium was launched in July 2014 as an innovative place-based consortium of educational research partners from multiple sectors. Its primary objective is to provide research and analyses on some of the city’s most pressing education issues. This paper’s guiding research question is: How does an urban education research consortium fil gaps in district/charter capacity for conducting applied research by establishing working relationships with practitioners and drawing on the deep pool of research talent that exists in the area? To address this question, the paper details the process by which PERC selected and executed its first research project: an in-depth analysis of how best to apply the emerging concept of Blended Learning to Philadelphia school contexts.

Authors

  • Kathleen M. Shaw, Research for Action
  • Tonya E. Wolford, School District of Philadelphia
  • Liza M. Rodriguez, Research for Action
  • Brittan Hallar, Research for Action
  • Jessica K. Beaver, Research for Action

The session began with the presenters handing out the following item (I tried to capture it using the camera in my phone).

IMG_5937 IMG_5938

Click on any image to see a larger version

IMG_5939 IMG_5940

Click on any image to see a larger version

IMG_5941 IMG_5942

During the first year, their major project focused on a blended learning approach that was being adopted by schools (both within the district and in the charter sector) throughout the Philadelphia area.  The researchers began – even before actually operationalizing the project itself – by looking at address this question:

What are the conditions that you need to established to ensure that the lessons from the research are going to be used by practitioners?

  1. establish a shared purpose
  2. create structures that foster purposeful collaboration and mutual exchange
  3. prioritize and nurture relationships and trust among partners
  4. contextualize research findings through a real understanding of the complexity of district and school decision making
  5. communicate efficiently and strategically (emphasis by presenters)

The actual research project resulted in two briefs: an initial brief to establish a common vocabulary and a second brief that outlined best practices for successful implementation.

Much of the actual presentation at the roundtable focused on process – what they did and then what they plan to do from a research project management perspective.  That is beyond the two briefs that I have linked above, which were just referenced during the presentation (as opposed to what the research found).

The organizational website for this project is available at:

http://www.phillyeducationresearch.org/projects/blended-learning/

AERA 2016 – Evaluation and Approval Constructs for K–12 Online and Blended Courses and Providers

As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days.  That means that I will be blogging many of the sessions throughout the week.  The third session (and first for today) that I am blogging is:

Evaluation and Approval Constructs for K–12 Online and Blended Courses and Providers

  • In Event: Online Teaching and Learning SIG Paper Session 5

10:35am to 12:05pm, Marriott Marquis, Level Two, Marquis Salon 14

Abstract

Public Act 60 (2013) of the Michigan Legislature tasked Michigan Virtual University, through its Michigan Virtual Learning Research Institute, to “research, develop, and recommend annually to the department criteria by which cyber schools and online course providers should be monitored and evaluated to ensure a quality education for their pupils.” This study provides an overview of existing models of cyber and online evaluation, both of which serve to inform the forthcoming recommendations. In this study, the authors review existing literature related to the evaluation of cyber schools, online, and blended providers, and identify five constructs for course and provider evaluation and approval.

Authors

  • Michael Kristopher Barbour, Sacred Heart University
  • Thomas A. Clark, TA Consulting
  • Kathryn M. Kennedy, Michigan Virtual University
  • Kristen DeBruler, Michigan Virtual University

As this is my own session, I have provided the slides below.

April 8, 2016

AERA 2016 – Culturally Inclusive Referral for Cyber School Students: From Prereferral to Appropriate Eligibility for Diverse Students

As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days.  That means that I will be blogging many of the sessions throughout the week.  The first session that I am blogging is:

Culturally Inclusive Referral for Cyber School Students: From Prereferral to Appropriate Eligibility for Diverse Students

  • In Event: Special Education Research SIG: Accessibility and Accommodations

4:05 to 5:35pm, Convention Center, Level One, Room 160

Abstract

The paper presentation will consists of three primary parts. First, the authors will provide a preliminary overview of the Individual Disabilities Improvement Education Act (“IDIEA’” 2004), the ever expanding special education law as it relates to assessing students with a learning disability who also present with diverse educational and other needs. Conversely, the paper will discuss the evolution of cyber charter schools using one state’s laws as a point of reference. Next, the study will identify barriers and strengths of a culturally inclusive and responsive pre-assessment and formal assessment process for culturally and linguistically diverse cyber school students suspected of having a learning disability or Other Health Impairment (“OHI”) under the IDIEA (2004).

Authors

  • York Williams, West Chester University of Pennsylvania
  • Nicole Snyder, Latsha Davis & McKenna

The presenters began with what I would describe as a marketing gimmick for what charter schools could provide (e.g., quality education, opportunity for student to succeed, drive high student expectations and academic achievement – all those things proponents claim, but the research has consistently shown isn’t true).  Charter schools, including cyber charter schools, must live up to various federal laws (such as IDEA, section 504 of the Rehabilitation Act, etc.).

Navigating the Intersection of Complex Laws and Regulations

“Educating students with disabilities in virtual schools entails not only molding state charter school laws to fit a specialized type of charter school, but also adopting federal and state special education guidelines aimed at providing special education in traditional brick and mortar buildings.” (Demystifying Special Education n Virtual Charter Schools by Lauren Morando Rhim & Julie Kowal, Public Impact, 2008)

The study utilized a case study methodology, that focused on questionnaires, virtual observations, document analysis, interviews and virtual focus groups with participants from Pennsylvania’s largest cyber charter school during 2013-14 to address this research question:

  1. What are the factors that impact the effective special education pre-referral and referral process overall for identified special education students in a virtual learning framework?

Twenty-seven of 45 potential teachers participated (with at least two years experience at the cyber school), along with three school leaders (with at least three years experience at the cyber school).

In this particular cyber school there were 11,541 students, and 1,436 of those students were special education students.

  • virtual trust and reciprocity – if the home is the classroom (i.e., least restrictive environment), teachers needed to observe the setting to determine the students needs
  • corrective or corrupted assessment – if you correct the students with the parent looking over your shoulder and correcting the teacher
  • learning coach as monitor – tension between parent and teacher, particularly when it came to assessing for supports

Overall, the two presenters provided a great deal of background, and then weren’t able to spend a lot of time on the findings of the study (although they did allocate about a third of their time to discussing the methodology).  And of course they went over time by two or three minutes.

« Previous PageNext Page »

Blog at WordPress.com.