Embedded formative evaluation = mastery learning

One of the ways in which I presently see technology improving learning experiences and success involves methods of individualization that allow individuals to move forward at a rate suited to their individual aptitudes and past experiences. My thinking about this opportunity has been shaped by my previous exposure to the theory and research of mastery learning. This perspective, which I associate with Bloom, Keller and other researchers of the late 1960’s and 1970s, provides the rationale for present practice. Many ideas that were great and well-substantiated ideas at that time were difficult to implement. One way I often look at technology now is as a way to take advantage of great ideas in ways that are now practical. The use of history to claim “we tried that and it did not work” needs reexamination. It is important to ask the why question? Sometimes the why is known and ignored. This is the situation in which interesting ideas reemerge in some form based on a similar interesting past ideas, but are a waste of time and energy because the conditions of why have not been acknowledged and/or changed. For me, the argument that learning to code will develop critical thinking falls into this category. Then, there are examples in which the why should encourage a second look if the conditions of the why have changed. I see this as the case with mastery learning.

I had my class this semester read a review by Shute and Rahimi (see citation at the conclusion of this post). This article focused on recent technology-supported instruction that offered data on the importance of competence and feedback. No mention was made of mastery learning. The article used a different vocabulary which I remember as “embedded formative evaluation”. So formative evaluation is likely a familiar concept (Bloom did see his group-based mastery learning as involving formative evaluation). The idea is that technology-enabled systems can incorporate formative evaluation as a mechanism to provide feedback and determine when individuals should progress. If these new words appeal to you, I think this is great and perhaps this appeal may encourage attempts at application. What I think is unfortunate is that a vast collection of work is out there and may allow new innovators to avoid tactics that were unproductive once and will likely be so again unless the circumstances of application are changed.

Shute, V.J. & Rahimi, S. (2017).  Review of computer-based assessment for learning in elementary and secondary education. Journal of Computer Assisted Learning, 33, 1-19.

Mastery revisited (again)

Mastery learning is one the concepts I discovered early in my career and that I continue to return to as a great idea just on the edge of meaningful implementation. I see mastery learning as different from other education concepts that continue to reappear even though the efficacy of these ideas is seldom demonstrated. To me, mastery learning is an idea that has waited for a practical method for implementation. Tutoring has always been an approach to implementation responsive to individual differences in learning speed, but the cost effectiveness of tutoring has typically prevented speed of acquisition differences to be accommodated. My interest in technology from the beginning has been related to the potential for personalization. The “personal computer” offers opportunities for many forms of individualization.

Educational historian Larry Cuban has recently begun generating blog posts focused on the Personalized System of Instruction – Fred Keller’s model from the late 1960s (here and here). As a model suited to technological support, PSI offers the best model from the early days. Bloom’s more group-based approach possibly received more attention because of Blook’s conceptual framework and concepts such as formative and summative assessment, but Bloom was also simply better known as an influential figure. I try to get my grad students to consider Keller’s perspective as a better starting point for individualized instruction.


Significant innovation

Calling for innovation in education seems to be a big thing. The argument for this position would seem to be that new ideas are better or that goals have changed and adjustments must be made. There is little doubt that when it comes to the job market the world is changing, but who needs which skills to take advantage of these changes and which educational setting is best suited to helping learners develop these new skills? Are these new skills more advanced or just different? Are these skills different or do they build on traditional skills?

The innovation I see as necessary in K12 is very different from the changes advocated by others. New ideas such as coding for all, making, or project-based learning are useful and interesting, but do not address what I think is the core challenge. I think it useful to differentiate core information and skills from additional skills. In my opinion, he most significant problem is the variability in core skills that cannot be addressed by group-based instruction in the earliest grades with the consequence that more advanced students are held back and struggling students do not get the attention they need resulting in failed progress and motivational problems. The most meaningful innovation would focus on ways to individualize the development of core knowledge and skills providing the foundation for personal learner interests and more optimistic attributions for learning activities (some have taken to calling this a growth mindset).

The idea that all learners should acquire certain common skills is present at every level of education. This core may reflect essential life skills and a foundation for progress in other areas (e.g., reading, writing, math) or perhaps an expectation of the public for contributing to society  (e.g., civics/government).

I think we are at a point when schools must do multiple things in multiple ways. Perhaps a theme here might be individualization, but understanding that individualization can mean different things. The individualization in individual interests should likely be addressed by increasing options and the individualization in speed of learning core expectations due to differences in aptitude and background knowledge should be addressed through systems allowing progress when mastery is demonstrated. Grade level is not a reasonable way to think about individualization when it comes to this second category (core content). Students quickly become quite different when it comes to level of achievement. Using a group-based approach to teaching the same things to students who are advancing at different rates is far from optimal or innovative.

I often write about technology providing practical ways to implement sound educational ideas that have been extensively researched and often ignored for years. Often, the problem is one of finding a way to make the tactics employed by researchers practical in classroom settings. I recognize present technology-based implementations of individual progress systems as based in the mastery-learning research of the late 1960s. Sal Kahn is one of the few technologists who seems to recognize this connection and it took some years before he described what his group has done as a form of mastery learning.

I would like to see public schools try what is sometimes described as a “mastery” model. There are different variations, but the approach I think is most practical make use of technology to individualize core areas. The use of technology should not be understood as eliminating the importance of educator involvement. To the contrary, in these areas the technology allows educators to identify critical obstacles and to function as more of a tutor.

I see public schools ignoring such blended models and leaving them for charter schools. This is part of what I mean by talking the innovation game, but not taking on the most significant challenges. In the present political climate, I see this as a problem for public education.

Book themes

Lately, this blog has been less active than usual. I don’t want folks to give up on my posts because of this inactivity so I thought I would explain myself. I am writing as much or maybe even more than usual, but I am spending most of this time updating our Kindle textbook. Academics typically make their selections for the new academic year (Fall, 2016) late in the Spring. This means if you want your most current work to be considered, the new content must be available by then. Doing the background work and then the writing to meet this deadline has kept my pretty busy. I can wait until the summer to make adjustments to our book related online content.

Doing the background work and then the writing to meet this deadline has kept my pretty busy. I can wait until the summer to make adjustments to the online content that extends our book in keeping with the goal of using the online resources to provide the most current content possible. For example, the new ISTE standards for students should be finalized about the time of the ISTE conference and these standards should receive considerable attention in an edtech text.

One of my goals in writing a textbook is to try to bring attention to themes we see in the field. The organization of a textbook around major themes is a benefit we believe is missing in unrelated resources that can be used to organize ideas for readers. A topic that has existed in our previous work, but has not been emphasized might be described as the opportunities in using technology to individualize student learning experiences. While we prefer the term “individualize”, others might use “personalize” or perhaps some similar term. I like individualize because I can define what this means to me (no matter how others use it).

Our content is based on a cognitive perspective. Individualization is inherent in cognitive models of learning. No matter what the experiences external to each learner, common experiences or not, the external experiences are processed uniquely by each learning as a function of existing knowledge and experiences and differences in aptitude and motivation. The work of learning cannot by done by anyone else.

Our work already has a heavy focus in problem-solving and problem-based activities. Multiple problems can be offered to learners (individualization) or students can be allowed to take on problems of their own choosing (individualized or 20% time projects). Technology offers so many ways to identify, research, and propose solutions to such problems.

The newest area in which we feel technology has offered significant opportunities for individualization is computer-based instruction. Some might argue we are recognizing a use of technology that has been around even before the age of the personal computer (e.g., Plato). Of course, but the key to our claim is the phrase “significant opportunities”. CAI/CBI was never available to many learners and existed in only a rudimentary form for those who did have access. This has changed with online curriculum (e.g., Kahn Academy).Access to online curriculum seems to generate a strange reaction in so many educators. There seems to be a negative gut reaction that is difficult to understand. Mention of computer and instruction within the same phrase might encourage concerns that technology is replacing teachers or have some vague connection with political-based education issues. It all seems to depend on how you see the possibilities. Teacher as facilitator is consistent with online curricula and so is individual models of student progress.

Access to online curriculum seems to generate a strange reaction in so many educators. There seems to be a negative gut reaction that is difficult to understand. Mention of computer and instruction within the same phrase might encourage concerns that technology is replacing teachers or generate some vague connection with political-based education issues. It all seems to depend on how you see the possibilities. Teacher as facilitator is consistent with online curricula and so are individual models of student progress.

We approach individualizing instruction from a mastery perspective. The Kahn Academy, for example, offers a way to engage students in a mastery approach to learning. My interest in mastery learning as a way to meet individual needs has been a fascination throughout my career. One of my first publications back in the late 1970s and my last publication in 2014 both concerned a mastery approach to individualizing learning. The first, of course, did not involve personal computers.

I guess it is time to get back to work. Blog posts will be generated when possible and I will make an occasional effort to upgrade anyone interested on our progress in finishing our next edition.


I tend to be a topical reader. I get hooked on a topic and follow that thread until I can take it no more and then I switch to something else. My present “professional” thread concerns technology and individualization. I am also reading spy novels, but that topic is not relevant here.

I am about to finish a couple books explaining the role technology can play in disrupting education (Blended and Disrupting Class). These two books are related as Blended draws heavily on Disrupting Class. If you are interested in reading one, I would recommend Disrupting Class. I find this book has greater conceptual depth.

I will likely comment on these books a few more times in the next couple of weeks because the combination provides me plenty of ideas to address. First, a reaction to what is claimed regarding individualization.

Using my own way of conceptualizing issues, I would describe the authors as identify two reasons for individualization – mastery and learning styles. Mastery argues that learners progress at different speeds as a consequence of differences in aptitude and background knowledge. Pushing learners before they are ready creates inefficiency and frustration. The learning styles position argues learners learn in different ways.

Here is where I differ with the authors on a professional level (I worked as an educational psychologist before I retired). I strongly support the mastery perspective. I started reading this literature in the 1970s (B Bloom – group-based mastery and F Keller – Personalized System of Instruction) and published several research studies based on mastery methods. Technology may now offer more practical ways to implement some of these ideas.

When it comes to learning styles, I must say that while the idea strikes a chord with so many, research fails to support learning styles as real. Styles should not be confused with preferences. A style would be demonstrated by showing that individuals with different styles are advantaged when learning in different ways. So, group A learns significantly better with method A than B and group B learns significantly better with method B than method A. Research does not demonstrate this happens. Typically, one method is better or at least equivalent for both groups and this method should thus be the desired method of instruction. There is nothing to gain from the added complexity and cost of matching method to group.

To be clear, the lack of a trait (style) by treatment (method) interaction does not argue that only one method should ever be used. Different methods may develop different capabilities (creativity, achievement) and all learners may benefit from a mix of learning experiences.

Mixing aptitude, knowledge and learning style differences together creates a methodological challenge that is overly complex and costly to address. Individual pacing as a way to address aptitude and background knowledge differences seems a more practical methodology. Kahn Academy resources would be an example of curriculum materials consistent with the concept of individual pacing. Such resources and similar commercial offerings are available now.



Mastery learning


The theory and application of mastery learning has long been a professional interest. The origins go back to the 60s and some crude variants to even earlier models of instruction/learning. I have also presented some of these ideas in my educ psychology classes for years as an alternative way to understand learner aptitude (time to learn). In recent years I have added a component focused on the Kahn Academy. My students find Keller and Bloom to be what they call “theoretical” (even though there were clearly applications of each), but they are intrigued by Kahn. One student wanted the link today because he wanted to try the modules on biochemistry.

I have watched with interest the evolution of the Kahn Academy. By evolution, I mean the earliest version did not promote the flipped classroom (a different use of class time) and mastery learning (very similar in my thinking to Keller’s PSI). I wonder about this evolution. Is what we see now closer to the original “grand plan” or have those developing this system found that what was being developed were great ways to implement existing “big ideas”?

I do discover something new each time I prep for my presentations and visit the site to review the features. This time I discovered the separate content on programming. Educators (e.g., Maker movement) have been rediscovering programming and the Academy offers an interactive experience based in Javascript.