Mastery – Learning Aloud

AI Tutoring Update

This is an erratum in case my previous posts have misled anyone. I looked up the word erratum just to make certain I was using the word in the correct way. I have written several posts about AI tutoring and in these posts, I made reference to the effectiveness of human tutoring. I tend to provide citations when research articles are the basis for what I say and I know I have cited several sources for comments I made about the potential of AI tutors. I have not claimed that AI tutoring is the equal of human tutoring, but suggested that it was better than no tutoring at all, and in so doing I have claimed that human tutoring was of great value, but just too expensive for wide application. My concern is that I have proposed that the effectiveness of human tutoring was greater than it has been actually shown to be.

The reason I am bothering to write this post is that I have recently read several posts proposing that the public (i.e., pretty much anyone who does not follow the ongoing research on tutoring) has an inflated understanding of the impact of human tutoring (Education Next, Hippel). These authors propose that too many remember Bloom’s premise of a two-sigma challenge and fail to adjust Bloom’s proposal that tutoring has this high level of impact on student learning to what the empirical studies actually demonstrate. Of greater concern according to these writers is that nonreseachers including educational practitioners, but also those donating heavily to new efforts in education continue to proclaim tutoring has this potential. Included in this collection of wealthy investors and influencers would be folks like Sal Kahn and Bill Gates. I assume they might also include me in this group while I obviously have little impact in compared to those with big names. To be clear, the interest of Kahn, Gates, and me is really in AI rather than human tutoring, but we have made reference to Bloom’s optimistic comments. We have not claimed that AI tutoring was as good as human tutors, but by referencing Bloom’s claims we may have led to false expectations.

When I encountered these concerns, I turned to my own notes from the research studies I had read to determine if I was aware that Bloom’s claims were likely overly optimistic. It turns out that I had read clear indications identifying what the recent posters were concerned about. For example, I highlighted the following in a review by Kulik and Fletcher (2016).

“Bloom’s two sigma claim is that adding undergraduate tutors to a mastery program can raise test scores an additional 0.8 standard deviations, yielding a total improvement of 2.0 standard deviations.”

My exposure to Bloom’s comments on tutoring originally had nothing to do with technology or AI tutoring. I was interested in mastery learning as a way to adjust for differences in the rate of student learning. The connection with tutoring at the time Bloom offered his two-sigma challenge was that mastery methods offered a way to approach the benefits of the one-to-one attention and personalization provided by a human tutor. Some of my comments on mastery instruction and the potential of technology for making such tactics practical are among my earlier posts to this site. Part of Bloom’s claim being misapplied is based on his combination of personalized instruction via mastery tactics with tutoring. He was also focused on college-aged students in the data he cited. My perspective reading the original paper many years ago was not “see how great tutoring is”. It was more tutoring on top of classroom instruction is about is good as it is going to get and mastery learning offers a practical tactic that is a reasonable alternative.

As a rejoinder to update what I may have claimed, here are some additional findings from the Kulik and Fletcher meta-analysis (intelligent software tutoring).

The studies reviewed by these authors show lower benefits for tutoring when outcomes are measured on standardized rather than local tests, sample size is large, participants are at lower grade levels, the subject taught is math, a multiple-choice test is used to measure outcomes, and Cognitive Tutor is the ITS used in the evaluation.

However, on a more optimistic note, the meta-analysis conducted by these scholars found that in 50 evaluations intelligent tutoring systems led to an improvement in test scores of 0.66 standard deviations over conventional levels.

The two sources urging a less optimistic perspective point to a National Board of Educators Research study (Nickow and Colleagues, 2020) indicating that human tutoring for K-12 learners was approximately .35 sigma. This is valuable, but not close to the 2.0 level.

Summary

I have offered this update to clarify what might be interpreted based on my previous posts, but also to provide some other citations for those who now feel the need to read more original literature. I have no idea whether Kahn, Gates, etc. have read the research that would likely indicate their interest in AI tutoring and mastery learning was overly ambitious. Just to be clear I had originally interpreted the interest of what the tech-types were promoting as mastery learning (personalization) which was later morphed into a combination with AI tutoring. This combination was what Bloom was actually evaluating. The impact of a two-sigma claim when translated into what such an improvement would actually mean in terms of rate of learning or change in a metric such as assigned grade seems improbable. Two standard deviations would move an average student (50 percentile) to the 98th percentile. This only happens in Garrison Keeler’s Lake Wobegon.

References:

Bloom, B. S. (1984). The 2 sigma problem: The search for methods of group instruction as effective as one-to-one tutoring. Educational researcher, 13(6), 4-16.

Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: a meta-analytic review. Review of educational research, 86(1), 42-78.

Nickow, A., Oreopoulos, P., & Quan, V. (2020). The impressive effects of tutoring on prek-12 learning: A systematic review and meta-analysis of the experimental evidence. https://www.nber.org/papers/w27476

Is there such a thing as aptitude?

The concept of aptitude and how differences in aptitude influencing learning could be reduced through mastery strategies have interested me throughout my academic career. I understood aptitude as something I thought of as intelligence. Intelligence is an abstraction that researchers attempt to measure with intelligence tests and investigate in practice through correlations with academic progress. Intelligence tests are not a direct measure of aptitude, but really an estimate based on differences in what individuals have learned and can do. Even the simple representation of intelligence as IQ (intelligence quotient) imagines intelligence as how much has been learned (mental age or MA) divided by age (chronological age).

Intelligence tests have come under a great deal of criticism based on potential racial/SES biases. These criticisms are certainly fair, but the tests do predict academic achievement and I was never convinced to support the abandonment of the development and use of such tests. The correlations measure something, and whatever this is does not disappear when tests are not given. If both tests and educational practice are biased, why not recognize that this is the case?

The theoretical basis for mastery learning (see Arlin and Bloom references) proposes that educators consider the rate of learning and accept that the rate of learning differs greatly among individuals. To me, this sounded very much like intelligence, and the concept of IQ is obviously related to learning rate (how much was learned per unit of time). However, what these researchers and educational theorists proposed was that other factors were involved in traditional educational practice and these other factors had a significant impact on achievement. While time required for learning was determined by aptitude, it was also influenced by whether the method of instruction met individual needs and by differences in existing knowledge. Think of it this way. If aptitude-based differences in learning create a range of learning speeds and a class of students moves through learning experiences faster than some students can master some important skills and concepts, in the future some students will be burdened not only by learning at a slower rate, but also by missing knowledge prerequisite to new skills and concepts they are trying to learn. Over time, these missing elements (Sal Kahn calls this Swiss cheese learning) will accumulate increasing failure and frustration in some learners. Mastery learning strategies focus on limiting the accumulation of knowledge prerequisites by individualizing the rate of learning to the rate of mastery. Some students in completely individualized approaches do move more slowly (and some faster), but the theory proposes that the rate of actual mastery would be faster than without mastery for all learners because deficits would not accumulate in learners needing more time and more capable students could move more quickly. The work of Arlin attempted to demonstrate what these changes in the rate of learning might be. When ratios such as 5:1 or 7:1 are proposed, it is easy to see why some students would fall hopelessly behind.

Individualization is challenging. Tutoring has always been a personal interest, but not economically feasible. With access to personal computers in the 1990s I saw the first method that might be available to provide individualization and this continues as an interest. Many attack present attempts to make use of technology in direct instruction as boring and depersonalizing. I think these folks have the wrong idea, but this is a topic I address elsewhere. Here, I want to recognize recent research that claims individualized instruction with technology (Koedinger, et al) may not only deal with individual differences in background knowledge, but also challenge the notion there are meaningful differences in the rate of learning.

How variable is the impact of aptitude?

Koedinger and colleagues studied the work of thousands of students from all grade levels working on different types of content using the type of technology-enabled methods I described above. Their focus was different in being based on the mastery of very specific capabilities rather than courses or even weeks of work. The learning experiences consisted of initial exposure to information (video or written) followed by a sequence of worked activities. I suppose a worksheet would be an example of a worked activity, but the variety and type of activities included a many different activities. The goal was to reach 80% mastery on a worked activity. The authors found that in the first attempt following the acquisition phase, the top half of students scored 75% and the bottom half scored 50%. The top half then required 3.7 practice trials to reach mastery (80%) and the bottom half 13 trials. What startled the researchers was that the gain per practice trial was very similar leading the researchers to conclude learning rate was very similar once existing knowledge was addressed. Aptitude (if I can be allowed to switch terms here) accounts for little difference in speed.

I am not convinced I would interpret these results in the same way given the method, but I do like the demonstration that allowing additional learning trials allows students the same level of achievement. I encourage interested parties to review the study themselves and see if they agree with my assessment. The statistical method is quite sophisticated and I wonder what interpretations the method allows. I would be more convinced had the researchers carried their research over an extended period of time and actually determined what happens when individual differences in existing knowledge are eliminated. The difference in understanding after the individual phase of exposure to new content was substantial and while likely a partial product of existing differences in background it does not seem to me that the difference would not partially also be due to aptitude differences. Since learners with existing background knowledge are not involved, it seems to me there is no demonstration that aptitude does not play a role in determining the number of practice trials that are required.

I am pleased to see that this type of research continues and assume this study will generate replications and hopefully extensions.

Additional comments on mastery learning and learning speed

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103–120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65–86.

Bloom, B. S. (1974). Time and learning. American Psychologist, 29(9), 682–688.

Koedinger, K. R., Carvalho, P. F., Liu, R., & McLaughlin, E. A. (2023). An astonishing regularity in student learning rate. Proceedings of the National Academy of Sciences, 120(13), e2221311120.

Khan, S. (2012) The One World Schoolhouse?—?Education Reimagined. Hodder and Stoughton, London, 2012 and Twelwe, Boston & New York.

Mastery of what?

I want to make what follows as practical as I can. I understand that standards, curriculum, and instructional approaches (e,g., mastery) can drift into the realm of what some regard in a negative way as theoretical, but I think I can frame what I have to say here in terms of some questions teachers must answer all of the time.

Let me begin with the question of what you regard as the essential knowledge and skills your students should acquire. I would suggest that you in one way or another address this question by answering a series of questions.

What content and experiences will I use with the students in my class?

What will I do to evaluate student mastery of these experiences and content?

What will I do with the students who perform poorly on the methods of evaluation I have applied?

My proposal is that this sequence of questions provides a way to look at the label of essential? Some experiences and content were essential enough to provide. Some experiences and content were essential enough to evaluate. It is this third question I am most interested in because all educators face this challenge. It really gets at the core of the designation of “essential”? What happens when skills or knowledge are not developed? Is the answer “I move ahead to new content and new skills”? I refer some students for outside help? I take students or small groups aside and work with them in an effort at remediation.

Old school mastery proponents (e.g., Bloom, Keller) addressed what must be mastered in a fuzzy way. Rather than identify specific things that must be known, they hedged. Bloom proposed a group-based mastery system. Imagine a textbook chapter and related classroom contributions to be mastered over a two-week period of time. Bloom proposed that teachers first focus on essential skills and knowledge (not really clear to me how this material was identified). At the end of maybe a week, students completed an evaluation related to these materials that Bloom labeled a formative evaluation. Those learners who “passed” this evaluation went on to supplemental goals and those who failed to achieve mastery received further help with the essential goals. At the end of the time set aside for the unit, students completed a summative evaluation over the essential goals and everyone moved on essential goals met or not.

Keller’s PSI (personalized system of instruction) focused heavily on written content as a way to allow personalized progress – think textbook again. Reading is an individual way to confront new information. When students felt they were ready to be evaluated on their mastery of a unit, the asked a tutor to provide an assessment. Pass/not pass was based on an overall score so what was mastered was not really determined at the level of specific elements of understanding. Those who passed went on to the next unit and those who did not pass continued to study the chapter yet to be mastered with some assistance from a tutor.

Modern mastery (Kahn Academy, Modern Classroom Project) advocates confront the question of essential more directly. Before I try to address how, I will try to answer my original question – “Mastery of what?” I would suggest essential means a) knowledge or skill is necessary for learning some other essential knowledge or skill or b) knowledge or skill the system has a responsibility to develop and this development is expected of the course or grade level I teach.

For example, double digit subtraction is essential to being able to master long division. The “North Dakota Studies” course is likely the one time you would learn why the Red River Valley has some of the richest farmland in the world. Okay, maybe this is not essential, but it matters to those who live in this area and depend on agriculture. Essential is a squishy thing and one could argue that a cellphone would allow anyone to perform long division and explain the soil quality of the Red River Valley without knowing how to subtract or basic geological facts. However, I assume there are essential things we teach that are a subset of all things we teach.

The Kahn Academy uses a complex model of the content with multiple strands identifying which skills/knowledge are prerequisites to what other skills. Students make progress across strands and must show mastery of prerequisites when identified within a given strand. Kahn complained about “Swiss cheese knowledge” that can be generated when students advance without prerequisite knowledge leaving gaps in skills and understanding that make future learning more difficult.

The Modern Classroom Project suggests educators identify differences in the importance of specific knowledge or skills using a triage of sorts – must do, should do, and aspire to do or need to know, good to know, and aim to know. This approach allows classroom educators to differentiate objectives in a way that allows more uniform progress within a group and still requires an extended focus on some prerequisites.

My long-time interest in mastery learning more recently combined with my interest in the classroom benefits of technology allow what I consider improvements in both the value and the practicality of mastery approaches. The value concerns a way to address the difficulty of new learning when past learning does not provide important existing knowledge. The efficiency associated with technology comes from the tracking of what has been learned and what should be learned next on a far more specific and individual student level. As I hope my analysis has made clear, the specificity of what should be learned next sometimes matters and sometimes does not. “Just in-time learning” is always possible, but this concept still requires a method of identification and application that group based approaches to teaching/learning do not make practical. Using teacher skills in a different way (tutor, coach) in combination with the value of technology in tracking individuals and delivering learning experiences seems a productive alternative to group-based approaches.

As a final comment, I wonder if big data will provide a way to address the issue of necessary prerequisites in a more specific way. Would there be a computational way of creating the strands of knowledge/skill units Kahn has identified based on intuition?

References

Bloom, B. S. (1968). Learning for Mastery. Instruction and Curriculum. Regional Education Laboratory for the Carolinas and Virginia, Topical Papers and Reprints, Number 1. Evaluation comment, 1(2), n2.

Khan, S. (2012). The one world schoolhouse: Education reimagined. Twelve.

Keller, F. S. (1968). “Good-bye teacher”. Journal of Applied Behavior Analysis, 1, 79–89

Modern Classroom Project – https://intercom.help/modern-classrooms/en/articles/5261634-must-do-should-do-and-aspire-to-do

Are we ignoring differences in rate of learning?

I can identify a half dozen or fewer themes that have captivated my professional imagination over the 40+ years of my academic career. So many of these themes often were at the core of specific research interests and my applied work. Sometimes a theme was something I found interesting at the time it was first encountered, but I saw no practical way the idea could be implemened. Sometimes this situation has changed. The best example of this “opportunity discovered” comes from my original interest in individual differences in the rate of learning and my later interest in technology and how the affordances of technology could make responding to differences practical.

The concept of aptitude is a topic educational psychologists teach. We may talk about issues associated with aptitude tests and perhaps biases in these tests as measures of aptitude or perhaps problems in the way test results were applied. Intelligence tests make perhaps the best example of an attempt to estimate general aptitude. Aptitude tests are about prediction and intelligence scores are predictive of achievement. Past achievement may be a better predictor of future achievement, but sometimes there is value in breaking down the components that contribute to achievement differences. Aptitude as an estimate of potential does not guarantee that potential will be realized and this difference, if real, is worth investigating.

As I said originally, I am interested in individual differences in the rate of learning and the practical consequences of these differences in rate under different classroom circumstances. I can trace my personal interest back to the theoretical work of Carrol (1963, 1989) which proposed what I interpreted as an optimistic model of learning. The model proposed that most individuals could learn most things if provided enough time. Carroll then differentiated the time required the time provided and then broke time required down according to variables that were influential. Aptitude proposed that aptitude was a way of understanding the time required under ideal conditions of optimal instruction and the presence of relevant existing knowledge.

I saw a connection to the notion of IQ which few seemed to make. The classic representation, IQ=MA/CA, is really about time and rate of learning. CA (chronological age) is the time available for learning and MA (mental age) is really how much has been learned estimated as the average knowledge of others of a given age. Hence MA/CA is rate of learning. The amount of general knowledge that has been acquired relative to what is typical is one way to estimate this rate. It is problematic in practice because it assumes equal opportunity which is of course idealistic.

A different way to estimate rate of learning might be to measure it directly and this is possible with various forms of individualized instruction. I remember the time when individualization was called programmed instruction and was accomplished using sequenced paper materials (see Yaeger). For example, I remember a reading comprehension implementation based on a box of cards with short reading passages and related questions that reflected different levels of text complexity. I remember this as an SRA reading product. The box of cards was based on a color scheme representing each level (e.g., brown cards, green cards, orange cards) and there were multiple cards at each level. Students would start at a common level, read a card, and attempt the related questions. If they obtained an established score, they were advanced to the next level. If not, they would take a different card of the same color and try again. Students would progress at different rates and the difference in time required to advance from level a to level could be used as one way to estimate reading aptitude.

There are now multiple technology-supported systems (e.g., Kahn Academy) based on a similar model (I refer to such approaches as mastery learning after the use of this term by Bloom, and Keller in the late 1960s).

Rate of learning could also be impacted by the presence or absence of relevant background knowledge. More recently, Kahn (Kahn Academy) has described this as the problem of Swiss cheese knowledge. Do students have the relevant prerequisites for acquiring a given new skill or concept?

How little variability in the rate of learning would exist given ideal instruction and the mastery of prerequisites has become an interesting question. To me, this seems similar to asking the question if there are really differences in the theoretical notion of intelligence or are the individual differences we observed due to differences in motivation, background knowledge, and instructional quality.

Why does it matter? I think it matters because educators and on a different level our models of education must deal with individual differences. However conceptualized, every teacher must make decisions about the rate of presentation that slows down the rate at which some students could learn and moves too fast for other students. The reality of aptitude as differences in rate of learning is there whether we choose to ignore it or not. Estimates of this variable range from 3:1 to 10:1 (Arlin). I liked to pick 5:1 and proposed to future teachers that some of their students would “get it” during their class on Monday and suggest they would have to work on the same concepts for the rest of the week to get most of the students to the same place. What should they do between Monday and Friday?

I would suggest that techniques have been available to provide a solution since the late 1960s. Mastery learning proposes to create settings that address differences in background knowledge by focusing on assuring students progress when ready and not so much the calendar says it is time to begin the next unit. My way of describing the goal would be to say the goal is to reduce the variability in time required to the bare minimum required by differences in aptitude by addressing differences in background knowledge and moving ahead at a rate individual students can handle reducing their frustration at not being able to succeed at meeting learning goals.

I see two practical ways to accomplish an approach of this type – tutoring and technology. Tutoring is very effective in meeting individual student needs, but expensive. Technology provides a more cost effective approach and offers advantages in content presentation, evaluation of understanding, and record keeping over early implementations of mastery learning. Technology can free teachers from having to take total responsibility for these functions and to provide more time to function as an individual or small group tutor. More on some of these ideas in future posts.

Related references:

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103-120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65-86.

Bloom, B. S. (1974). Time and learning. American psychologist, 29(9), 682-688.

Carroll, J. B. (1963). A model of school learning. Teachers college record, 64(8), 1-9.

Carroll, J. B. (1989). The Carroll model: A 25-year retrospective and prospective view. Educational researcher, 18(1), 26-31.

Keller, F. S. (1968). Goodbye teacher… Journal of Applied Behavior Analysis Vol. 1, pg. 79-89.

Khan, S. (2012). The one world schoolhouse: Education reimagined. Twelve.

Yeager, J. L., & Lindvall, C. M. (1967). An exploratory investigation of selected measures of rate of learning. The Journal of Experimental Education, 36(2), 78-81.

Class Disrupted

This is a recommendation for the Class Disrupted podcast. This podcast is now in its 4th season and is hosted by Dianne Tavvener and Michael Horn. I follow several podcasts that include a focus on mastery learning in k12 settings and this wide-ranging podcast does frequently discuss mastery learning topics.

Aside from mastery learning, I am fascinated with this series because there is a lived historical element there for consideration. The podcast started in response to the COVID disruption. Rather than first listening to the most recent episodes or to episodes that have an interesting description, I would recommend listening from the first episode on. If you have an interest in technology and the now related topic of the reaction of many educators to the forced use of technology during the “learn from home” years or are intrigued by what has to have been one of the most disruptive periods in our lives and the long term impact of this disruption, there is plenty to consider by working your way through the episodes from the beginning. What were our expectations? What were our plans? How are things working out?

Addressing middle school math

Educators are likely familiar with the learning challenges students experienced during the COVID years when face-to-face instruction became impractical. The concern for student achievement during this period of time has been documented in declining performance on the NAEP scores. It appears that math achievement was particularly hard hit and the middle school years which set students up for the study of algebra represents a unique problem area.

I decided to focus on a resource that called attention to this problem mostly because it proposes one productive response would be to make more frequent use of mastery instructional strategies in middle school math classes. Readers who follow this blog will understand that mastery learning is one of the topics I spent time writing about. [other more detailed posts about mastery instruction can be found by selecting the category “mastery” that can be found in the left-hand column of this blog]

Mastery instruction individualizes learner experiences meeting students at the level of their understanding and advancing them as individuals when understanding has been achieved. It is an approach concerned that learning goals be met even when group-based instruction would likely move ahead leaving some students missing skills that are prerequisite to new material.

Among the other recommendations to address the middle school math challenge is to double up on math class frequency. This would be a second way to provide additional time to assure the mastery of essential skills, but it is more of a group-based approach than the individual learner emphasis on mastery learning.

Mastery methods in applied settings – reality

Great ideas don’t always meet their argued potential when implemented. Reality has a way of adding complexity that reduces potential. Here are some examples of this effect I am aware of that apply to mastery learning.

Variability in time to learn

Individual learners vary in their speed of learning. Any teacher knows this. The issue in any classroom, whether implementing a mastery strategy or not, is how this variability is handled.

Some have studied variability and what happens when student variability meets a mastery approach. Arlin (1984, 1984b) conducted several experiments and referenced other work to indicate that a group-based mastery approach does not eliminate differences in time to learn. Time to learn appears to remain constant unit after unit.

Arlin (1984, 1984b) challenges what he claims is Bloom’s group-based mastery promise that over time the Bloom approach will eliminate individual differences in the rate of learning. Arlin offers multiple studies that indicate with group-based mastery variability between individuals remains relatively constant. I must admit I was surprised that Bloom argued individual differences would be eliminated, but references to Bloom’s writings appear to indicate I was wrong. Bloom appears to shy away from the existing knowledge and aptitude distinction I make. Arlin does find that given extra time most students will learn what is taught.

Arlin references other scholars with notions of a “wait around” or “Robin Hood” effect for more capable students. This concern argues it is possible more capable students can be held back by certain implementations of a group-based approach. However, I would suggest a) group-based approaches could provide supplemental learning activities not focused on the learning goals of a given unit and I would think most educators would understand this, and b) the Arlin position fails to acknowledge that traditional instruction must teach to a point at which the rate of learning is not optimized for more capable learners.

Conclusion: Most students can learn what is taught if given sufficient time and appropriate instruction and b) student differences in what is sufficient time will not be eliminated. How much time is required – I remember (no reference I can point to) that a 2:1 ratio will be sufficient for 80% of students to reach goals. Recognize that this means twice as much time to learn the same thing.

Procrastination

Studies of college students engaged in Keller-type individualized mastery learning demonstrate a high drop-out rate. What appears to happen when students are given a great amount of independence is that other requirements are prioritized (usually implied to be other courses, but I am guessing other personal priorities should be included here), and study within the mastery course and evaluation test completion lag. Students get significantly behind an acceptable pace and when they try to re-engage find that catching up is not as easy as they had hoped. They drop the course unable to see themselves finishing.

A remedy sometimes described as “the doomsday” contingency (early Keller advocates tended to be behaviorists) set a standard for completing the first several units (e.g., finish two units in the first two weeks) or students faced being dropped. This approach improved completion rates giving students a taste of the effort required. Purists might argue this type of approach was inappropriate.

Hoping for minimal effort

I conducted several studies of what I came to call effort errors (Grabe, 1982, 1994). Several of these studies involved a one-retake option for all course exams. This is not a pure mastery system, but it turned out to be a way to demonstrate the extent to which students bought into a mastery approach.

For example, if a control group and a retake available group are provided, a mastery advocate would predict that groups would be similar on the original exam and the retake group would improve the performance on the second opportunity if students chose to take it. Not so. The one-take group (traditional instruction) performed significantly better on the same initial exam. Clearly, the students who knew they had a second opportunity were not giving their best effort.

I took to identifying different types of what I would call “effort errors” – skipping the initial exam; taking a second exam, but scoring below the score on the first exam (I used a several point differences before this type of decline counted as an error); and skipping the second exam opportunity with a score of C or lower on the initial exam. More effort errors predicted lower course grades and were more common among students most in need of additional opportunities. Lower final cumulative exam scores related to more effort errors likely indicated a general lack of motivation.

Conclusion – motivation to spend additional effort cannot be assumed.

References

Arlin, M. (1984). Time variability in mastery learning. American Educational Research Journal, 21(1), 103-120.

Arlin, M. (1984b). Time, equality, and mastery learning. Review of Educational Research, 54(1), 65-86.

Grabe, M. (1982). Effort strategies in a mastery instructional system: The quantification of effort and the impact of effort on achievement. Contemporary Educational Psychology, 7(4), 327-333.

Grabe, M. (1994). Motivational deficiencies when multiple examinations are allowed. Contemporary Educational Psychology, 19(1), 45-52.