As I mentioned in the entry entitled AERA 2016 and K-12 Online Learning, the 2016 annual meeting of the American Education Research Association is occurring in Washington, DC over the next few days. That means that I will be blogging many of the sessions throughout the week. The sixth session that I am blogging is:
Content and Rigor of Algebra Credit-Recovery Courses
- In Event: Getting At-Risk Students Back on Track: Results From a Randomized Trial of Algebra Credit Recovery
4:05 to 6:05pm, Marriott Marquis, Level Two, Marquis Salon 2
This paper describes the content, organization and rigor of the f2f and online summer algebra courses that were delivered in summers 2011 and 2012. Examining the content of both types of courses is important because research suggests that algebra courses with certain features may be better than others in promoting success for struggling students. Examining course rigor is important because it further describes the context through which students recovered course credit, the study’s primary short-term outcome.
The study’s theory of action assumes that students who fail traditional f2f algebra may benefit from taking an online course because it is a different mode of delivery that is potentially more individualized, interactive and personalized. Analyzing the nature of the content further describes how these features played out in both types of courses.
Methods and Data Sources
Math experts systematically analyzed the content of the online course and the f2f courses (the f2f teachers determined the content of their courses, per district and school policy). The content analyses examined (1) coherence: the extent to which topics were logically sequenced within and across units; (2) difficulty: the extent to which topics were pre-algebra, first semester algebra or second semester algebra; and (3) knowledge emphasis: the extent to which lessons and exercises emphasized procedural fluency, conceptual understanding and applied problem solving. The experts used established frameworks and existing curricular materials to guide both sets of analyses.
Data sources include archival data generated from the online course, course materials (syllabi, annotated tables of contents) from the f2f course, and teacher surveys on which teachers indicated the criteria they used to determine grades in their credit recovery courses.
Results suggest that the online course (in both summers), in comparison to the f2f courses, was more coherently sequenced but more difficult in terms of topics covered and more demanding in terms of grading criteria. The online course content was considered typical of second semester algebra and included a fixed set of topics that were organized sequentially. Conversely, the topics and sequencing of the f2f courses varied widely among the f2f teachers. The topics included not only second semester algebra content, but also first-semester algebra and pre-algebra content, and topics did not typically follow the sequential structure of the online course. Neither the online or f2f lesson materials focused strongly on developing conceptual understanding or applied problem solving, but emphasized procedures and skills instead.
Information from this paper will be used to help frame the short- and long-term study results and to contribute to the relatively thin research base on what it means for students to be proficient in algebra. Although the content of first and second semester algebra is fairly uniform and well specified, the proportion of topics that students should master is not. This issue is increasingly salient in light of the Common Core State Standards for Mathematics, which place a stronger emphasis on conceptual understanding and problem solving than more traditional algebra courses.
- Kirk Walters, American Institutes for Research
- Nicholas Andrew Sorensen, American Institutes for Research
Kirk’s portion focused on the content, specifically:
- did the course cover the was content with similar sequencing?
- how much of the content did students complete and with what level of mastery?
- how did grading policies compare?
- how did students perform on the end of course algebra assessment?
The online course used in the study was developed by Aventa Learning (i.e., K12, Inc.). It was self-paced and the standard teacherless online course: do pre-assessment, get multimedia content on what you don’t know, then do post-assessment.
The F2F teachers were certified math teachers, selected by the school according to the typical summer school hiring process. There was a wide variety of textbooks, supplemental materials, and instructional practices.
The findings included:
- only half of the content of the average F2F class was Algebra 1B (the rest was Algebra 1A or pre-algebra), whereas the online courses were exclusively Algebra 1B content
- in the online course the sequencing had a coherent organization, whereas almost a third of the F2F sequencing was coded as incoherent (i.e., the order of content just didn’t make sense in terms of building upon prior knowledge and moving from foundational to more difficult concepts)
- the online students completed about 2/3 of the course content and only passed an average of 1.7 out of 5 end-of-unit exams (70% was considered passing)
- the grading policies were more rigorous in the online than in the F2F classes (e.g., tests and quizzes accounted for 60% of the students’ grade in the online cohort, compared to an average of 52% in the F2F)
- the F2F students performed better on pre-algebra, Algebra 1A, and Algebra 1B items on the posttest than the online students
The main takeaways, at least in the short-term, the online students didn’t do as well, they thought that the class was harder, so maybe it just isn’t appropriate? An alternate theory is that maybe the school-based mentors needed to be more active in their role in supporting students.
Other entries from this session: