Mastery – Page 2 – Learning Aloud

Class Disrupted

This is a recommendation for the Class Disrupted podcast. This podcast is now in its 4th season and is hosted by Dianne Tavvener and Michael Horn. I follow several podcasts that include a focus on mastery learning in k12 settings and this wide-ranging podcast does frequently discuss mastery learning topics.

Aside from mastery learning, I am fascinated with this series because there is a lived historical element there for consideration. The podcast started in response to the COVID disruption. Rather than first listening to the most recent episodes or to episodes that have an interesting description, I would recommend listening from the first episode on. If you have an interest in technology and the now related topic of the reaction of many educators to the forced use of technology during the “learn from home” years or are intrigued by what has to have been one of the most disruptive periods in our lives and the long term impact of this disruption, there is plenty to consider by working your way through the episodes from the beginning. What were our expectations? What were our plans? How are things working out?

Addressing middle school math

Educators are likely familiar with the learning challenges students experienced during the COVID years when face-to-face instruction became impractical. The concern for student achievement during this period of time has been documented in declining performance on the NAEP scores. It appears that math achievement was particularly hard hit and the middle school years which set students up for the study of algebra represents a unique problem area.

I decided to focus on a resource that called attention to this problem mostly because it proposes one productive response would be to make more frequent use of mastery instructional strategies in middle school math classes. Readers who follow this blog will understand that mastery learning is one of the topics I spent time writing about. [other more detailed posts about mastery instruction can be found by selecting the category “mastery” that can be found in the left-hand column of this blog]

Mastery instruction individualizes learner experiences meeting students at the level of their understanding and advancing them as individuals when understanding has been achieved. It is an approach concerned that learning goals be met even when group-based instruction would likely move ahead leaving some students missing skills that are prerequisite to new material.

Among the other recommendations to address the middle school math challenge is to double up on math class frequency. This would be a second way to provide additional time to assure the mastery of essential skills, but it is more of a group-based approach than the individual learner emphasis on mastery learning.

Mastery methods in applied settings – reality

Great ideas don’t always meet their argued potential when implemented. Reality has a way of adding complexity that reduces potential. Here are some examples of this effect I am aware of that apply to mastery learning.

Variability in time to learn

Individual learners vary in their speed of learning. Any teacher knows this. The issue in any classroom, whether implementing a mastery strategy or not, is how this variability is handled.

Some have studied variability and what happens when student variability meets a mastery approach. Arlin (1984, 1984b) conducted several experiments and referenced other work to indicate that a group-based mastery approach does not eliminate differences in time to learn. Time to learn appears to remain constant unit after unit.

Arlin (1984, 1984b) challenges what he claims is Bloom’s group-based mastery promise that over time the Bloom approach will eliminate individual differences in the rate of learning. Arlin offers multiple studies that indicate with group-based mastery variability between individuals remains relatively constant. I must admit I was surprised that Bloom argued individual differences would be eliminated, but references to Bloom’s writings appear to indicate I was wrong. Bloom appears to shy away from the existing knowledge and aptitude distinction I make. Arlin does find that given extra time most students will learn what is taught.

Arlin references other scholars with notions of a “wait around” or “Robin Hood” effect for more capable students. This concern argues it is possible more capable students can be held back by certain implementations of a group-based approach. However, I would suggest a) group-based approaches could provide supplemental learning activities not focused on the learning goals of a given unit and I would think most educators would understand this, and b) the Arlin position fails to acknowledge that traditional instruction must teach to a point at which the rate of learning is not optimized for more capable learners.

Conclusion: Most students can learn what is taught if given sufficient time and appropriate instruction and b) student differences in what is sufficient time will not be eliminated. How much time is required – I remember (no reference I can point to) that a 2:1 ratio will be sufficient for 80% of students to reach goals. Recognize that this means twice as much time to learn the same thing.

Procrastination

Studies of college students engaged in Keller-type individualized mastery learning demonstrate a high drop-out rate. What appears to happen when students are given a great amount of independence is that other requirements are prioritized (usually implied to be other courses, but I am guessing other personal priorities should be included here), and study within the mastery course and evaluation test completion lag. Students get significantly behind an acceptable pace and when they try to re-engage find that catching up is not as easy as they had hoped. They drop the course unable to see themselves finishing.

A remedy sometimes described as “the doomsday” contingency (early Keller advocates tended to be behaviorists) set a standard for completing the first several units (e.g., finish two units in the first two weeks) or students faced being dropped. This approach improved completion rates giving students a taste of the effort required. Purists might argue this type of approach was inappropriate.

Hoping for minimal effort

I conducted several studies of what I came to call effort errors (Grabe, 1982, 1994). Several of these studies involved a one-retake option for all course exams. This is not a pure mastery system, but it turned out to be a way to demonstrate the extent to which students bought into a mastery approach.

For example, if a control group and a retake available group are provided, a mastery advocate would predict that groups would be similar on the original exam and the retake group would improve the performance on the second opportunity if students chose to take it. Not so. The one-take group (traditional instruction) performed significantly better on the same initial exam. Clearly, the students who knew they had a second opportunity were not giving their best effort.

I took to identifying different types of what I would call “effort errors” – skipping the initial exam; taking a second exam, but scoring below the score on the first exam (I used a several point differences before this type of decline counted as an error); and skipping the second exam opportunity with a score of C or lower on the initial exam. More effort errors predicted lower course grades and were more common among students most in need of additional opportunities. Lower final cumulative exam scores related to more effort errors likely indicated a general lack of motivation.

Conclusion – motivation to spend additional effort cannot be assumed.

References

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103-120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65-86.

Grabe, M. (1982). Effort strategies in a mastery instructional system: The quantification of effort and the impact of effort on achievement. Contemporary Educational Psychology, 7(4), 327-333.

Grabe, M. (1994). Motivational deficiencies when multiple examinations are allowed. Contemporary Educational Psychology, 19(1), 45-52.

Mastery approaches

I became familiar with the idea of learning personalization for mastery in the 1970s. Even though there are recent applications of mastery concepts making use of technology, I continue to point to the early mastery work because of the research base associated with that time period. The examples of more recent mastery approaches I will link to these early efforts do not come with a rich collection of peer reviewed studies I look for when advocating for what amounts to a serious departure from day to day classroom practice.

To me there were two very different approaches labelled as mastery methods – Fred Keller’s Personalized System of Instruction (PSI) and Bloom’s Learning for mastery.

The Keller Plan

Keller (1968) advocated a truly individualized approach to instruction based primarily on written material (not be confused with the programmed instruction of that time which was often paper-based as well). Keller did make use of teacher presentations, but these were not used for the core approach. Keller liked written materials because individual students could work on written content on their own and could read at whatever rate was productive for them. Instructional text was associated with study guides for guidance. When students felt prepared, they would ask a tutor for an examination over that material. The tutor presented the assessments, evaluated the assessments, and helped learners with challenges they seemed to have encountered. Movement to the next unit depended on a satisfactory score on the unit exams and failure to meet this standard directed the learner to restudy the same material.

Bloom’s Learning for Mastery

Bloom’s (1968) approach to mastery learning was group based. A group of learners would focused on content (e.g., chapter) to be learned for approximately a week and would then be administered a formative evaluation. Those who passed this evaluation would continue to supplemental learning activities and those who did not pass would receive remediation appropriate to their needs. At the end of this second period of time (at about the two-week mark), students would receive the summative examination to determine their grade.

There are many variations and details of these approaches not explained here. My intent was to establish the more individualized and the more group-based approaches. I see the Kahn Academy as similar to the Keller Plan and Modern Classroom Project as similar to Bloom’s approach. My guess is more educators are aware of the Kahn Academy and understand that students can work on this technology-delivered content demonstrating mastery of specific skills at different rates. Many use this content for supplemental learning, but it can also be used as the basis for comprehensive approach. The Modern Classroom does not individualize progress to the same degree and is not necessarily as dependent on technology administered mastery checks. I encourage exploration of the links provided here for those unfamiliar and interested in the present, more technologically based mastery approaches receiving a lot of attention at present.

The idea of mastery and what teaching for mastery means in practice varies to some degree to how essential it is to master specific skills or concepts. I would think that all knowledge/skill deficits are not equally damaging. It might be useful to differentiate general and prerequisite deficits. A prerequisite deficit would describe a skill or concept necessary in the short term to understand/master a more advanced skill/unit of understanding that builds on the deficit skill. A general deficit would identify a skill or a unit of understanding that is missing, but not necessary for the mastery of other units soon to be taught. Original approaches to mastery (Bloom, Keller) focused on an acceptable level of general skill. Kahn approach is more focused on the identification and remediation of specific deficits. I would think technology would offer a much more practical approach to the linking skills and for tracking individual student mastery of prerequisite.

Note that both Keller and Bloom are not absolutists. Technology allows a much more specific approach with Kahn’s complex identification of prerequisites and specific mastery checks in the Modern Classroom approach. Being specific about the identification of unmastered skills does not stop progress as learners can continue to work on other skills with technology allowing the more careful identification of problem areas in contrast to the mastery approaches of the 1960s.

References

Benjamin, S., Dhew, E., & Bloom, B. (1968). Learning for mastery. Eval. Comment, 1, 1-12.

Keller, F. S. (1968). “Good-bye teacher”. Journal of Applied Behavior Analysis, 1, 79–89

Components of mastery instructional methods

I explained what I considered the essential components of a mastery system to my educational psychology and educational technology students for years. I used the following list based on my reading of multiple sources and explained using terminology I hoped would be easy to understand. I see different mastery approaches as implementation of these components to varying degrees so I will try to describe the components as ideal and then it becomes useful to explore the degree to which different implementations approach each ideal.

Components of a mastery system:

Objectives / Goals
Small Units of Instruction
Multiple, nonpunitive assessments
Mastery before progress
Keyed remediation

Objectives / Goals – mastery requires the identification of what is to be learned in a way that is more explicit than is the case in a traditional approach. You cannot teach a “bunch of stuff”, but because of the requirements of some of the other components you must specific what is to be learned and what is based on or follows what.

Small units of instruction – mastery approaches attempt to reduce individual differences in existing knowledge. When the pace of instruction exceeds the time required for an individual to learn, a student may be expected to advance lacking prerequisite knowledge. Small units of instruction combined with other components (mastery before progress) to reduce the gaps in knowledge important in learning related material. Smaller units reduced the probability gaps will be missed.

Multiple, nonpunitive assessments – to use assessment to guide instruction, it may be necessary to reteach and retest. Multiple assessments determine when the expected level of understanding/mastery has been achieved. Nonpunitive implies that the number of assessments necessary to demonstrate understanding will not be used in evaluating/grading learners. When content is mastered, it is mastered.

Mastery before progress – a standard for mastery is set and meeting that standard determines when a student moves from one unit to the next.

Keyed remediation – assessment results should be used in targeting instruction when an assessment determines that content has not been mastered. It is also possible that a previous method of instruction will be changed as that method may be partly responsible for a student failing to achieve mastery

It is interesting to note that early approaches were couched in what I would label as a behavioral tradition. For example, the specification of “objectives”. This can be seen in the general focus journals hosting some of the earliest descriptions of mastery approaches. Historically, from my experience, acceptance of some of these ideas may be related to resistance to mastery concepts. I see this as unfortunate as core ideas about what learning is can easily be described within a cognitive tradition. For example, building from what is known fits with Piaget’s notions of assimilation and accommodation or with concepts such as conceptual change theory describing the interaction of existing models with new information.

Identification of these components serve multiple purposes. I ask students to use these components to identify existing practices that may meet individual components. I use these components to contrast different instructional tactics labelled as a mastery or competency-based approach to describe if and how well the different components are addressed. Researchers have studied many studies of mastery tactics in attempts to determine which components are most essential.

Buskist and colleagues (1991) provide one example of the type of component analysis that has been conducted. The components used differ a bit from my list. This is in part results from their analysis of a single mastery approach, The researchers conclude that unit mastery, multiple short quizzes, quick performance feedback, and review units were concluded to generate achievement advantages. Self-pacing (students having freedom work work when they wanted) and time spent with proctors were not.

My final post in this series will use the components I describe here to differentiate a couple of the original mastery strategies and then link these original strategies to my interest in technology-enhanced mastery approaches.

Reference:

Buskist, W., Cush, D., & DeGrandpre, R. J. (1991). The life and times of PSI. Journal of Behavioral Education, 1(2), 215-234.

Research on mastery instruction is extensive

I have decided to write several posts focused on mastery learning. My motivation for doing this comes from a recent Chalkbeat article claiming that mastery approaches are receiving greater attention since the COVID pandemic, but “evidence remains thin”. Having followed instructional approaches labeled mastery learning since the 1970s, I am troubled by the message this title offers. There have been hundreds of studies evaluating mastery strategies using achievement data. The Chalkbeat article is focused on technology-enabled approaches and this happens to be my own interest. Perhaps less is known about mastery approaches that put technology in a central role, but the underlying concepts of mastery instruction are well researched (see citations at conclusion of this post).

So, my effort here will be to offer a broader background on what mastery learning is and to describe some of the original models similar to more recent technology-enabled approaches.

Mastery learning has always intrigued me because the underlying assumptions make so much sense. I would describe the most basic argument of mastery learning to be that learners master content and skills at different rates. I understand these differences in learning rate as the consequence of a combination of aptitude and existing knowledge. These factors are interrelated in practice.

I would describe aptitude as similar to what others might think of as intelligence. As an individual difference, I don’t think it matters much if intelligence is a biologically based variable. This is a different topic. I think it is obvious that the rate at which learners can learn differs from individual to individual. Label this difference as you see fit. I call this capacity to learn “aptitude”.

Aside from aptitude, existing knowledge plays an important role in learning rate. Differences in existing knowledge have been demonstrated in some circumstances to play a more important role than aptitude (e.g., studies of reading comprehension based on reading skill and topic-related knowledge). These two variables are related because education does not assure that learning has occurred when the system of instruction moves on. This means that learners move ahead differing in aptitude and also differences in the existing knowledge and skills that may be necessary for new learning. Variability increases over time in a system that does not adjust to the needs of the individual increasing both the difficulty of learning new skills or knowledge AND the motivation to deal with the personal circumstances that new learning involves.

Individualization does not make the rate of learning equal, it makes the differences more determined by aptitude than the combination of aptitude and existing knowledge. For many content areas, most individuals would learn what is being taught if the learning environment allows them sufficient time. Addressing the “IF” is the key.

Some major reviews of mastery versus traditional instruction:

Kulik, C., Kulik, J. & Bangert-Drowns, R.L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60, 265-299.

Kulik, C., Kulik, J. & Bangert-Drowns, R.L. (1990). Is there better evidence on mastery learning? A response to Slavin. Review of Educational Research, 60, 303-307.

Kulik, J. A., Kulik, C. L. C., & Cohen, P. A. (1979). A meta-analysis of outcome studies of Keller’s personalized system of instruction. American psychologist, 34(4), 307- 318

Slavin, R. (1987). Mastery learning reconsidered. Review of Educational Research, 57(2), 175-213.

Mastery Learning

I have been writing about and conducting research on mastery learning since the late 1970s. I always thought some of the core concepts (learners develop knowledge and skills at different rates, ignoring such differences leads to failed understanding and learner frustration) were recognized by educators, but largely accepted as the way things have to be because of practical demands of group instruction.

Since I began working with technology in the late 80s and even more so when the Internet allowed wide access to designated resources, I thought technology might overcome some of the practical challenges. There have been hints that others have recently come to the same conclusion. Kahn of Kahn Academy fame described one potential application of the tools he and colleagues were creating as offering a mastery approach (One World Schoolhouse).

Even more recently, a group labeling itself the Modern Classroom Project has proposed an instructional approach that sounded very much like mastery learning to me. You have to work at finding references to mastery learning in the description of their approach, but I was pleased to read this post from the Cult of Pedagogy blog which is based on an interview with Kareem Farah of the Modern Classroom group entitled “How to setup mastery grading in your classroom”. I don’t like the emphasis on “grading” in the description, but the content provided is quite useful.

If the idea of individualizing for mastery interests you, I offer a broader perspective on the basics of mastery learning on my site.