Tag: mastery

The concept of aptitude and how differences in aptitude influencing learning could be reduced through mastery strategies have interested me throughout my academic career. I understood aptitude as something I thought of as intelligence. Intelligence is an abstraction that researchers attempt to measure with intelligence tests and investigate in practice through correlations with academic progress. Intelligence tests are not a direct measure of aptitude, but really an estimate based on differences in what individuals have learned and can do. Even the simple representation of intelligence as IQ (intelligence quotient) imagines intelligence as how much has been learned (mental age or MA) divided by age (chronological age).

Intelligence tests have come under a great deal of criticism based on potential racial/SES biases. These criticisms are certainly fair, but the tests do predict academic achievement and I was never convinced to support the abandonment of the development and use of such tests. The correlations measure something, and whatever this is does not disappear when tests are not given. If both tests and educational practice are biased, why not recognize that this is the case?

The theoretical basis for mastery learning (see Arlin and Bloom references) proposes that educators consider the rate of learning and accept that the rate of learning differs greatly among individuals. To me, this sounded very much like intelligence, and the concept of IQ is obviously related to learning rate (how much was learned per unit of time). However, what these researchers and educational theorists proposed was that other factors were involved in traditional educational practice and these other factors had a significant impact on achievement. While time required for learning was determined by aptitude, it was also influenced by whether the method of instruction met individual needs and by differences in existing knowledge. Think of it this way. If aptitude-based differences in learning create a range of learning speeds and a class of students moves through learning experiences faster than some students can master some important skills and concepts, in the future some students will be burdened not only by learning at a slower rate, but also by missing knowledge prerequisite to new skills and concepts they are trying to learn. Over time, these missing elements (Sal Kahn calls this Swiss cheese learning) will accumulate increasing failure and frustration in some learners. Mastery learning strategies focus on limiting the accumulation of knowledge prerequisites by individualizing the rate of learning to the rate of mastery. Some students in completely individualized approaches do move more slowly (and some faster), but the theory proposes that the rate of actual mastery would be faster than without mastery for all learners because deficits would not accumulate in learners needing more time and more capable students could move more quickly. The work of Arlin attempted to demonstrate what these changes in the rate of learning might be. When ratios such as 5:1 or 7:1 are proposed, it is easy to see why some students would fall hopelessly behind.

Individualization is challenging. Tutoring has always been a personal interest, but not economically feasible. With access to personal computers in the 1990s I saw the first method that might be available to provide individualization and this continues as an interest. Many attack present attempts to make use of technology in direct instruction as boring and depersonalizing. I think these folks have the wrong idea, but this is a topic I address elsewhere. Here, I want to recognize recent research that claims individualized instruction with technology (Koedinger, et al) may not only deal with individual differences in background knowledge, but also challenge the notion there are meaningful differences in the rate of learning.

How variable is the impact of aptitude?

Koedinger and colleagues studied the work of thousands of students from all grade levels working on different types of content using the type of technology-enabled methods I described above. Their focus was different in being based on the mastery of very specific capabilities rather than courses or even weeks of work. The learning experiences consisted of initial exposure to information (video or written) followed by a sequence of worked activities. I suppose a worksheet would be an example of a worked activity, but the variety and type of activities included a many different activities. The goal was to reach 80% mastery on a worked activity. The authors found that in the first attempt following the acquisition phase, the top half of students scored 75% and the bottom half scored 50%. The top half then required 3.7 practice trials to reach mastery (80%) and the bottom half 13 trials. What startled the researchers was that the gain per practice trial was very similar leading the researchers to conclude learning rate was very similar once existing knowledge was addressed. Aptitude (if I can be allowed to switch terms here) accounts for little difference in speed.

I am not convinced I would interpret these results in the same way given the method, but I do like the demonstration that allowing additional learning trials allows students the same level of achievement. I encourage interested parties to review the study themselves and see if they agree with my assessment. The statistical method is quite sophisticated and I wonder what interpretations the method allows. I would be more convinced had the researchers carried their research over an extended period of time and actually determined what happens when individual differences in existing knowledge are eliminated. The difference in understanding after the individual phase of exposure to new content was substantial and while likely a partial product of existing differences in background it does not seem to me that the difference would not partially also be due to aptitude differences. Since learners with existing background knowledge are not involved, it seems to me there is no demonstration that aptitude does not play a role in determining the number of practice trials that are required.

I am pleased to see that this type of research continues and assume this study will generate replications and hopefully extensions.

Additional comments on mastery learning and learning speed

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103–120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65–86.

Bloom, B. S. (1974). Time and learning. American Psychologist, 29(9), 682–688.

Koedinger, K. R., Carvalho, P. F., Liu, R., & McLaughlin, E. A. (2023). An astonishing regularity in student learning rate. Proceedings of the National Academy of Sciences, 120(13), e2221311120.

Khan, S. (2012) The One World Schoolhouse?—?Education Reimagined. Hodder and Stoughton, London, 2012 and Twelwe, Boston & New York.

Educators are likely familiar with the learning challenges students experienced during the COVID years when face-to-face instruction became impractical. The concern for student achievement during this period of time has been documented in declining performance on the NAEP scores. It appears that math achievement was particularly hard hit and the middle school years which set students up for the study of algebra represents a unique problem area.

I decided to focus on a resource that called attention to this problem mostly because it proposes one productive response would be to make more frequent use of mastery instructional strategies in middle school math classes. Readers who follow this blog will understand that mastery learning is one of the topics I spent time writing about. [other more detailed posts about mastery instruction can be found by selecting the category “mastery” that can be found in the left-hand column of this blog]

Mastery instruction individualizes learner experiences meeting students at the level of their understanding and advancing them as individuals when understanding has been achieved. It is an approach concerned that learning goals be met even when group-based instruction would likely move ahead leaving some students missing skills that are prerequisite to new material.

Among the other recommendations to address the middle school math challenge is to double up on math class frequency. This would be a second way to provide additional time to assure the mastery of essential skills, but it is more of a group-based approach than the individual learner emphasis on mastery learning.

Great ideas don’t always meet their argued potential when implemented. Reality has a way of adding complexity that reduces potential. Here are some examples of this effect I am aware of that apply to mastery learning.

Variability in time to learn

Individual learners vary in their speed of learning. Any teacher knows this. The issue in any classroom, whether implementing a mastery strategy or not, is how this variability is handled.

Some have studied variability and what happens when student variability meets a mastery approach. Arlin (1984, 1984b) conducted several experiments and referenced other work to indicate that a group-based mastery approach does not eliminate differences in time to learn. Time to learn appears to remain constant unit after unit. 

Arlin (1984, 1984b) challenges what he claims is Bloom’s group-based mastery promise that over time the Bloom approach will eliminate individual differences in the rate of learning. Arlin offers multiple studies that indicate with group-based mastery variability between individuals remains relatively constant. I must admit I was surprised that Bloom argued individual differences would be eliminated, but references to Bloom’s writings appear to indicate I was wrong. Bloom appears to shy away from the existing knowledge and aptitude distinction I make. Arlin does find that given extra time most students will learn what is taught. 

Arlin references other scholars with notions of a “wait around” or “Robin Hood” effect for more capable students. This concern argues it is possible more capable students can be held back by certain implementations of a group-based approach. However, I would suggest a) group-based approaches could provide supplemental learning activities not focused on the learning goals of a given unit and I would think most educators would understand this, and b) the Arlin position fails to acknowledge that traditional instruction must teach to a point at which the rate of learning is not optimized for more capable learners. 

Conclusion: Most students can learn what is taught if given sufficient time and appropriate instruction and b) student differences in what is sufficient time will not be eliminated. How much time is required – I remember (no reference I can point to) that a 2:1 ratio will be sufficient for 80% of students to reach goals. Recognize that this means twice as much time to learn the same thing.

Procrastination

Studies of college students engaged in Keller-type individualized mastery learning demonstrate a high drop-out rate. What appears to happen when students are given a great amount of independence is that other requirements are prioritized (usually implied to be other courses, but I am guessing other personal priorities should be included here), and study within the mastery course and evaluation test completion lag. Students get significantly behind an acceptable pace and when they try to re-engage find that catching up is not as easy as they had hoped. They drop the course unable to see themselves finishing.

A remedy sometimes described as “the doomsday” contingency (early Keller advocates tended to be behaviorists) set a standard for completing the first several units (e.g., finish two units in the first two weeks) or students faced being dropped. This approach improved completion rates giving students a taste of the effort required. Purists might argue this type of approach was inappropriate.

Hoping for minimal effort

I conducted several studies of what I came to call effort errors (Grabe, 1982, 1994). Several of these studies involved a one-retake option for all course exams. This is not a pure mastery system, but it turned out to be a way to demonstrate the extent to which students bought into a mastery approach.

For example, if a control group and a retake available group are provided, a mastery advocate would predict that groups would be similar on the original exam and the retake group would improve the performance on the second opportunity if students chose to take it. Not so. The one-take group (traditional instruction) performed significantly better on the same initial exam. Clearly, the students who knew they had a second opportunity were not giving their best effort.

I took to identifying different types of what I would call “effort errors” – skipping the initial exam; taking a second exam, but scoring below the score on the first exam (I used a several point differences before this type of decline counted as an error); and skipping the second exam opportunity with a score of C or lower on the initial exam. More effort errors predicted lower course grades and were more common among students most in need of additional opportunities. Lower final cumulative exam scores related to more effort errors likely indicated a general lack of motivation.

Conclusion – motivation to spend additional effort cannot be assumed. 

References

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103-120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65-86.

Grabe, M. (1982). Effort strategies in a mastery instructional system: The quantification of effort and the impact of effort on achievement. Contemporary Educational Psychology, 7(4), 327-333.

Grabe, M. (1994). Motivational deficiencies when multiple examinations are allowed. Contemporary Educational Psychology, 19(1), 45-52.

I have decided to write several posts focused on mastery learning. My motivation for doing this comes from a recent Chalkbeat article claiming that mastery approaches are receiving greater attention since the COVID pandemic, but “evidence remains thin”. Having followed instructional approaches labeled mastery learning since the 1970s, I am troubled by the message this title offers. There have been hundreds of studies evaluating mastery strategies using achievement data. The Chalkbeat article is focused on technology-enabled approaches and this happens to be my own interest. Perhaps less is known about mastery approaches that put technology in a central role, but the underlying concepts of mastery instruction are well researched (see citations at conclusion of this post). 

So, my effort here will be to offer a broader background on what mastery learning is and to describe some of the original models similar to more recent technology-enabled approaches.

Mastery learning has always intrigued me because the underlying assumptions make so much sense. I would describe the most basic argument of mastery learning to be that learners master content and skills at different rates. I understand these differences in learning rate as the consequence of a combination of aptitude and existing knowledge. These factors are interrelated in practice.

I would describe aptitude as similar to what others might think of as intelligence. As an individual difference, I don’t think it matters much if intelligence is a biologically based variable. This is a different topic. I think it is obvious that the rate at which learners can learn differs from individual to individual. Label this difference as you see fit. I call this capacity to learn “aptitude”.

Aside from aptitude, existing knowledge plays an important role in learning rate. Differences in existing knowledge have been demonstrated in some circumstances to play a more important role than aptitude (e.g., studies of reading comprehension based on reading skill and topic-related knowledge). These two variables are related because education does not assure that learning has occurred when the system of instruction moves on. This means that learners move ahead differing in aptitude and also differences in the existing knowledge and skills that may be necessary for new learning. Variability increases over time in a system that does not adjust to the needs of the individual increasing both the difficulty of learning new skills or knowledge AND the motivation to deal with the personal circumstances that new learning involves. 

Individualization does not make the rate of learning equal, it makes the differences more determined by aptitude than the combination of aptitude and existing knowledge. For many content areas, most individuals would learn what is being taught if the learning environment allows them sufficient time. Addressing the “IF” is the key.

Some major reviews of mastery versus traditional instruction:

Kulik, C., Kulik, J. & Bangert-Drowns, R.L. (1990). Effectiveness of mastery learning programs: A meta-analysis. Review of Educational Research, 60, 265-299.

Kulik, C., Kulik, J. & Bangert-Drowns, R.L. (1990). Is there better evidence on mastery learning? A response to Slavin. Review of Educational Research, 60, 303-307.

Kulik, J. A., Kulik, C. L. C., & Cohen, P. A. (1979). A meta-analysis of outcome studies of Keller’s personalized system of instruction. American psychologist, 34(4), 307- 318

Slavin, R. (1987). Mastery learning reconsidered. Review of Educational Research, 57(2), 175-213.

I have been writing about and conducting research on mastery learning since the late 1970s. I always thought some of the core concepts (learners develop knowledge and skills at different rates, ignoring such differences leads to failed understanding and learner frustration) were recognized by educators, but largely accepted as the way things have to be because of practical demands of group instruction.

Since I began working with technology in the late 80s and even more so when the Internet allowed wide access to designated resources, I thought technology might overcome some of the practical challenges. There have been hints that others have recently come to the same conclusion. Kahn of Kahn Academy fame described one potential application of the tools he and colleagues were creating as offering a mastery approach (One World Schoolhouse).

Even more recently, a group labeling itself the Modern Classroom Project has proposed an instructional approach that sounded very much like mastery learning to me. You have to work at finding references to mastery learning in the description of their approach, but I was pleased to read this post from the Cult of Pedagogy blog which is based on an interview with Kareem Farah of the Modern Classroom group entitled “How to setup mastery grading in your classroom”. I don’t like the emphasis on “grading” in the description, but the content provided is quite useful.

If the idea of individualizing for mastery interests you, I offer a broader perspective on the basics of mastery learning on my site.

My past couple of posts have returned to a topic I have addressed several times over the years – technology can made practical the benefits of personalized mastery learning. To conclude this focus and offer an example to justify my claim of practicality, I want to bring attention to legislation recently passed in the state of North Dakota. I spent a good part of my working life as an academic in North Dakota. I don’t regard North Dakota as a progressive state for educational innovation, but perhaps that is my point. Innovation can be a function of the efforts of individual innovators as administrators or educators with administrative support and such individuals can be quite influential. 

North Dakota just passed legislation makes it easier for districts wanting to institute personalized approaches to pace and the demonstration of competence in meeting graduation requirements. In addition, the state will facilitate the development of a learning continuum as a resource for those schools wanting to offer students an alternative to the more traditional time-based definitions of courses and progress.

Shall facilitate the development and implementation of a North Dakota learning continuum in collaboration with the department of career and technical education, upon the recommendation of the kindergarten through grade twelve education coordination council.

Here are a couple of resources explaining the intent of the legislation.

Statement from the Office of the Governor.

Wording of registrations

My point in this post is to offer the suggestion that the concepts and procedures identified in the two previous posts about standards for hybrid learning and plans for personalization instruction through what I refer to as mastery methods are being implemented in locations and this activity may not be familiar to many educators. 

Mastery learning is described in some detail in previous posts over the years. Here are a couple of important topics for this post. First students vary significantly in what might be called “learning rate”. The word “aptitude” would be my personal preference, but this term is less descriptive. While the origins of differences in learning rate are perhaps disputed (e.g., is it what was traditionally called intelligence), I don’t think any practitioner would dispute that students acquire knowledge and skills at different speeds. An important issue is whether the instruction that students receive accommodates these differences. If instruction moves on to new material before a student has learned the present focus of instruction, students at best would have a knowledge or skill gap and at worse would be missing the background to learn something new that is based on or assumes other learning has been accomplished. Over time, these gaps accumulate, increasing the variability of learning rate. The frustration students experience in dealing with demands they are ill-prepared to master only magnifies the problem.

There have been attempts to deal with the variability in the rate of learning through practices such as ability grouping (tracking) and tutoring. Ability grouping comes with multiple difficulties such as labeling and equity issues and tutoring while very successful is prohibitively expensive. 

Mastery instructional techniques have been investigated for years. My personal study of this topic begins with publications in 1968 – Bloom’s group based mastery and Keller’s personalized system of instruction (PSI). I provide greater detail on these approaches elsewhere. Interest in such approaches seemed to wane likely because of practical issues and because the systems required alterations in traditional educational practices rather than because of lack of effectiveness. The potential of technology for taking on some of the components of mastery instruction increasing efficiencies and changing attitudes of educational practice may encourage reconsideration of mastery approaches possibly with different terminology and different variations on the components of a mastery approach.

I consider the components of PSI and Bloom’s group-based mastery strategies to be – a) clearly stated and integrated objectives/goals, b) small units of instruction, c) mastery before progress, d) multiple, nonpunitive assessments, and e) remediation keyed to individual needs. When I work with graduate students on this topic, I typically ask them to explain how these components appear in PSI and Bloom’s group based mastery, but also in new instructional approaches that mention a commitment to typical mastery goals.

Briefly, here are a couple of examples of more recent instructional approaches with mastery characteristics. The Kahn Academy is a recent example of a technology-enabled approach that Kahn began to label as mastery (e.g., see One World Schoolhouse). My focus is not on the Kahn Academy here, but it seems to me to be similar to PSI when implemented as a full-blown system (using the short videos, competence checks, and hierarchical linking of learning units). 

A second example I want to highlight is called the Modern Classroom Project. The modern classroom model is based on three components – blended instruction, self-paced learning, and mastery evaluation. Blended instruction proposes that educators replace face-to-face lectures to a group with short videos – think flipping the classroom. The idea is lecture involves limited interaction so why waste valuable face-to-face instructional time when videos can be viewed whenever a learner wants. Keller had the same idea in 1968 which he captured in the title of his paper – Goodbye teacher …  The title might be misinterpreted if the paper is not actually read. What Keller noted was that lecturing to a group was not productive when students could read (back in the day when reading the textbook was assumed preparation for class). Reading could be completed whenever and to give Keller credit for an important insight reading was under the control of the learner – content could be reread if necessary and at the pace required by the learner. 

Self-paced learning in the Modern Classroom to me is similar to Bloom’s group-based method of instruction. Bloom did not employ a pure mastery system absolutely requiring mastery before progress, but divided content into units often of two weeks. At a point during this time, students completed a formative assessment (you may have heard this term – this is where it comes from) on essential content. Those who passed this check were often moved on to self-guided supplemental activities while the teacher worked on the most essential skills with those not meeting the expectations of the assessment. A summative unit concluded the unit and all moved on. The Modern Classroom includes content identified as must do, should do, aspire to do. Students have great flexibility when they attempt to demonstrate competence of the related skills with the instructor keeping an eye on things and working with the students most needing assistance. 

Mastery assessment is pretty much what it sounds like. Performing at the expected level of the knowledge/skill checks. While this is the only use of the work mastery in descriptions of the Modern Classroom, I hope that you can see the system applies other of the core mastery components I have identified under other labels. Technology plays a second role in the Modern Classroom in tracking goal accomplishments and helping the teacher identify those most in need of help. 

Advocates of the Modern Classroom urge creativity in educators applying the core concepts they have identified. So, examples of implementation come in many different shapes and sizes. 

The Modern Classroom Project is an organization offering resources and mentorship to interested schools. A free tutorial on the core concepts is available.