generative – Learning Aloud

Productive Study Behavior: Generative Learning and Retrieval Practice

I think of most of my posts as helping learners or teachers think about study behavior. I define study as what follows exposure to information, aimed at developing personal understanding and increasing the likelihood of retention and application. This perspective includes the processing experiences educators assign and the behaviors learners engage in on their own. In reality, educators tend to focus on the assignments they create and far less on what students do on their own, but this an issue I will save for another post. For those of us now operating completely on our own outside of any affiliation with a formal classroom, we study without guidance, if we make any efforts that could be described as study at all. Those providing “self-help” guidance have identified this final situation under headings such as personal knowledge management (PKM) and developing a second brain.

My occupational efforts originally focused on preparing future teachers. Now I write for practicing teachers and individuals interested in PKM-type strategies to get more value from their reading or video consumption. I try to offer specific behaviors people can implement and an understandable version of learning theory that is valid and interpretable. I have always felt it motivating to have some notion of how what you are doing is supposed to work. The perspective that fits this practical intent is called generative learning, which, as I interpret it, encourages educators and learners to view learning activities as external actions intended to promote productive internal (cognitive) behaviors. Perhaps a different way to consider this definition is to ask if you make an assignment how do you assume the activity will create productive thinking (as good a word as any in this context) in the individual who completes the assignment? What will this activity and the related thinking accomplish?

The Cognitive Behaviors – The Mental Tools

Cognitive processes are the active operations carried out in working memory to manipulate information. Here is my simplified description of the four basic mental tools:

Attend: Holding and maintaining specific ideas active in consciousness/working memory.
Link: Establishing connections between information units active in working memory or stored in LTM.
Elaborate: Generating or discovering new knowledge through the purposeful combination of active thoughts and stored memories.
Evaluate: Assessing whether a cognitive task or solution has been completed successfully.

Generative activities and retrieval practice are intended to influence these cognitive behaviors.

Generative Learning Activities

Fiorella and Mayer list eight generative activities that have received most research attention. This list groups the first four as strategies that transform information into a different representation—for example, turning text into a summary, map, drawing, or mental image. The last four involve elaborating on the material through retrieval, explanation, teaching, and practice or action. The central idea is that learning improves when students actively generate meaning through an active task.

The eight activities:

Summarizing
Mapping
Drawing
Imagining
Self-testing
Self-explaining
Teaching
Demonstrating

Recent efforts have popularized some of these activities using different labels. For example, mapping may be more familiar as concept mapping. Sketchnoting is a recent application that uses summarizing, mapping, and drawing. Taking notes might be described as summarizing, but also self-explaining. Writing to learn (writing across the curriculum) involves summarizing, explaining, and possibly teaching.

Some of these strategies change the form of the original information. A paragraph may become a summary, a set of relationships may become a map, or a verbal explanation may become a drawing. Other strategies elaborate on the material through retrieval, explanation, teaching, or action. In both cases, the learner – not just the teacher or the technology – is doing the essential work of making meaning.

This distinction matters. Giving students polished notes, slides, or an AI-generated summary may improve access to information, but it can also bypass the productive effort through which understanding develops. The goal is therefore not merely to provide better representations. It is to have students create, evaluate, and revise their own representations.

Sometimes you can use your own behavior to imagine a distinction between optional behaviors – I could do this or I could do that. Which is most generative? For example, I could highlight text I think is important as I read a book, or I could take notes from that book in my notebook. Both are external tasks, but what mental behaviors are involved? Researchers often contrast allowing students to look over (reread) learning materials versus various generative strategies – writing a summary, writing questions to be used later, listing personal examples that fit with the information to be learned.

Examples aside, I think it is valuable for educators to consider how the assignments they require or the study techniques students use encourage linking, elaboration, and evaluation. My own organization of external activities would look a little different, isolating self-testing (which I think is better described as retrieval practice) from other generative activities, but those preferences reflect how I think I can best offer a meaningful explanation to practitioners. The following section explains why.

Retrieval Practice

Yes, self-testing is included on the list of generative activities. I included it because Fiorella and Mayers did. Retrieval practice is a learning technique based on the idea that actively pulling information from memory strengthens it. The classic study making this argument involves a treatment and control condition. Both groups are exposed to new information. The control is allowed a specific amount of time to read (if text) and review this content. The treatment or retrieval practice, group is given less time to read and review, and this reduction in time is filled with closed-book questions. After a delay, the retrieval practice group, the group having shorter access to the learning and study content, demonstrates superior retention.

Retrieval practice may be a new term for you, but if you have created flashcards and then used these cards to prepare for an approaching examination, you already have engaged in retrieval practice. You practice without access to the original content (closed book). You try to remember and you typically can flip the card over after a failed effort at retrieval to check on what you could not remember. Retrieval practice has some surprising properties, including that it seems to be helpful even when retrieval fails.

Why is struggling to retrieve helpful? I have my own explanation based on a common memory experience. What do you do when struggling to remember someone’s name – say an individual you knew twenty years ago who held a specific position, such as your boss? The “tip of the tongue” phenomenon describes such situations when we cannot remember a specific thing but you feel certain if it is identified you will know what is proposed is what we were searching for. In this situation, you likely can remember other things. Can you see the person’s face? Can you remember your office and where the boss’s office was from yours? Can you remember events or stories in which you and this individual participated? What you are trying to do uses the cognitive behavior I described earlier as links. Our memories include more than items of information, but also links among these items. When we actively explore links within a short period of time, we create new connections among the items we have brought into our awareness (short-term memory) and we strengthen existing links. Future retrieval efforts will have more and stronger links to work with, helping us find what we are searching for.

Retrieving and Understanding

Obergassel and colleagues argue that while generative tasks and retrieval tasks do not encourage completely independent mental activity, generative and retrieval tasks have unique benefits in practice and it may be useful for educators and learners to ensure that both types of activities are present in the tasks assigned to learners. Simply put, generative learning activities help learners make sense of new information, and retrieval tasks ensure consolidation (I prefer retention). In practice, learners want both benefits. If they understand something immediately, but fail to retain this understanding, what is the point? The researchers therefore explored whether a combination of tasks involving generative activity AND retrieval practice would be superior to either task alone, or to neither task.

One observation I found helpful and perhaps meaningful to practicing educators was the difference between an open-book and a closed-book activity. The retrieval practice treatment used a cued recall task. Students studied a text explaining four concepts. To test recall, students typed a definition in response to a cue identifying one of the concepts, without access to the text. The generative treatment asked students to generate examples. So with access to their written text, a concept was presented and the student then had to offer a possible example of that concept.

The dependent variables in this research were retention and comprehension questions administered immediately after the study phase and after a one-week delay. The retention questions required the learners to write a definition of the four concepts covered in the study materials. The comprehension task included items in which a scenario was presented and the students were asked to create an example based on one of the concepts that fit that scenario. The other items provided examples and asked which concept fit the example.

The generative treatment, the retrievable practice group, and the treatment group receiving both all performed better than the rereading-only group. The group receiving both generative and retrieval practice tasks outperformed all other groups. When comparing the generative and retrieval practice groups. The generative group demonstrated an advantage on the immediate posttest when retrieval would have been easier.

The researchers concluded that retrieval practice and generative tasks overlap, but offer sufficiently distinct benefits that educators should provide students with both types of experience.

Boundary Conditions

Understanding options for generative and formative testing is important. Activities should suit the learning task. The “generative examples of” and “write a summary of” activities were well suited to the materials in the experiment I have just described but would not be suited say to a biochemistry class in which students were studying the Krebs Cycle. In that case, drawing and annotating a diagram of the cycle would make much more sense as a study technique.

Any external activity also costs time and mental effort. There is certainly such a thing as what students might describe as “busy work” when a specific task is inefficient. Again, consider the time required to make flashcards. This strategy seems efficient when appropriate to certain learning goals because there is benefit both in formulating the questions and easy use of the cards. The time to create and to use is minimal. This is an issue I have wondered about when it comes to a tactic such as Sketchnoting. Investing too much time in intricate artwork would be a trap that might catch some students.

Conclusion

Generative learning tasks and retrieval practice offer overlapping benefits impacting both understanding and long-term memory access. New research demonstrates that there are enough unique benefits it is worth students applying both as study strategies.

Resources

Fiorella, L., & Mayer, R. (2016). Eight Ways to Promote Generative Learning. Educational Psychology Review, 28(4), 717-741.

Obergassel, N., Renkl, A., Endres, T., Nückles, M., Carpenter, S. K., & Roelle, J. (2025). Combining generative tasks and retrieval tasks. Journal of Educational Psychology, 117(6), 980–997. https://doi.org/10.1037/edu0000949

Peer editing: Better to give than receive

I became interested in the development of writing skills through feedback focused on alternatives to the teacher or professor as editor. I worked with educators who developed writing skills and was acquainted with the time demands of providing feedback and had my own experience reviewing students’ theses and dissertations. When you have read the 200+ page dissertation of a Ph.D. candidate through a couple of drafts, you have put in some hours. I felt sorry for the English department English composition adjuncts paid a few thousand for each of 4 sections of 25 students and the time it would take to review multiple writing assignments. Still, you learn to write by writing, and feedback and rewriting in response to feedback are essential.

There are ways to provide an acceptable alternative source of feedback. Peers and now AI can critique writing, and while some more expert involvement is important, the quantity and diversity of learning activities required for skilled performance cannot consistently be monitored by instructors.

I began reviewing the research on peer editing because I was aware of the time issue faced by instructors, but also because I was interested in the role digital word processing tools could play in the feedback and revision processes. For example, Google docs offers a great way to add comments at precise locations in a document and to exchange related remarks as a document is passed back and forth between the writer and reviewer. Revision is efficient and can be explained if the editor wants to take a second look. I thought that Goodle docs offered an example of a tool that would improve the efficiency with which learners could interact with peers and then rewrite efficiently in response to comments.

As I reviewed the research literature, I came across some studies that changed my thinking on how I should advocate for peer feedback. These studies (see Cho references at the end of this post) demonstrated that peer editing also had an impact on the writing performance of the editors, and this benefit might be more important than the feedback a writer received from others. The Cho research focused on a specific population of writers generating a particular type of writing product, and understanding the focus of such research is always important in considering how and if findings might generalize to other situations. Cho focused on college students writing lab reports, i.e., the description and results of experiments performed in the lab. I know my own profession has an undergraduate course (Research Methods) with a core focus on the same type of writing task. Cho conducted several studies in which peer editing was a component. Students could either review their lab reports without feedback, with feedback, or with feedback after providing feedback on the same task to other students. The greatest difference was found when writers also provided feedback to others. When it came time to revise their own original drafts, the product they produced was judged to be superior, on average, to the products generated by those in the other groups. Moreover, statistical analysis showed that editing had a more powerful impact than having the edits of others to review.

The researchers offered two possible explanations for their findings. First, they proposed that the process of editing provides a perspective on how others might view a written product (audience effect). Writers are always told to consider their audience, but perhaps serving as the audience might provide insight into what that means for a specific written product. The other explanation involved what I would describe as a generative effect. Serving as an editor has some similarities to the research topics of writing to learn and teaching to learn. When you must externalize a position you take, this forces a concreteness and specificity you may fail to generate when just thinking about something. Having to put a position into words can lead to the understanding that you really can’t explain yourself or make you work to come up with a concrete way to express what you think.

This notion that working to improve understanding and develop proficiency seems to be raised repeatedly as educators grapple with the role AI should play in educational settings. For all of the ways AI might reduce “busy work,” there seems to be a related potential that AI provides a way to avoid the cognitive work so necessary in developing a cognitive skill. So, while AI may provide a way to provide feedback to students, there is also evidence that the work of providing feedback to others involves work that is productive both for others and for yourself. Educators face a significant challenge in communicating this reality to learners and other stakeholders.

References:

Cho, Y., & Cho, K.. 2011. Peer Reviewers Learn from Giving Comments.” Instructional Science, 39 (5), 629–643. doi:10.1007/s11251-010-9146-1

Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73-84.

Highlighting in the age of digital content

I have highlighted much of what I read for probably 50 years. I started in college, and I tried different approaches, sometimes highlighting with different colors. My preference was the slim highlighter in yellow. When I began reading using my phone, iPad, and Kindle, I learned how to highlight using these devices. My interest in educational technology led me to look more deeply into the opportunities to highlight and annotate on these devices, and you may have read what I have had to say about these tools in previous posts.

Here is the thing about highlighting. If you follow the research on the efficacy of different learning/study strategies, you soon understand that highlighting is not particularly useful. I knew this too, and I was interested in study techniques long before personal computers were a thing. I taught educational psychology to college students, and studying was a topic I hoped the students would find relevant.

There are good reviews of the research on highlighting (Dunlosky, et al, 2013) that reach the conclusion that highlighting has low utility. I think it is important to carefully understand the methodology used in the studies that investigate highlighting. What is the breadth of the perspective? In research that examines the application of note-taking, a distinction is drawn between the generative and external functions of notes. I think a similar issue applies here. The research indicates highlighting is not cognitively active and has limited generative value, but what about external storage? If it was an hour before a major test and I was trying to review the 120 pages that were assigned in my textbook, I would rather I had highlighted that book than not.

Here are some of the major findings that challenge the value of highlighting.

Highlighting may improve recall but not comprehension. (see Ponce and colleagues resource as the source for most of the comments focused on recent studies of highlighting)

Learner-generated highlighting can improve memory for the highlighted material. However, this memory boost often doesn’t extend to improved comprehension. If this distinction makes little sense, think of the difference in terms of the types of questions that might be asked to evaluate memory versus understanding. College students seem to gain more memory benefit from self-highlighting than K-12 students, potentially because they are more experienced at identifying key information. Studies focused on the importance of content that is highlighted demonstrate that college students are better at identifying core or main ideas. This makes sense for a couple of reasons. First, highlighting tends not to be encouraged among K-12 students as they use books that are not theirs. Second, college students are more experienced with the educational process and have a better feel for what content is likely to be the focus of future examinations or projects they will complete. As a consequence, and anticipating that advanced learners are likely to make use of highlighting, instruction focused on the identification of priority information is often recommended. Younger students should be asked to highlight and the type of content they designate should be evaluated.

Illusion of Mastery: Like passive rereading, looking at highlighted text can create a false sense of familiarity, leading learners to believe they know the material better than they do. This confuses familiarity with actual retrievability and understanding (Johns).

When I explored highlighting as a study technique with my students, I described a similar phenomenon. I call it the “I’ll get to that later” effect. What I proposed is that students seem to actually identify important content (these are college students), but may be challenged to understand this material. An easy way to move on and complete the reading assignment was to highlight this material, but not stop to struggle with the ideas. Later may not actually happen, or if it does, the highlighted material is then encountered out of context and less easily processed to a deeper level.

Ahrens (citations appear at the end of this post) proposes that underlining (I would assume a practice similar to highlighting) is similar to what Ahrens classifies as fleeting notes. Fleeting notes are taken to quickly capture information, and the idea of smart notes that Ahrens emphasizes focuses on the translation of fleeting notes into smart notes. A smart note can stand alone to convey meaning to the note taker and others and requires the note taker to use personal knowledge to generate a note that is meaningful now and hopefully in the future.

Highlighting of digital material may be different.

Digital reading can be different. Highlights can be exported and saved isolated from the original document. The accumulation of this once-deemed potentially useful text can be searched and examined, or potentially can become the target of an AI chat years later. This is very different than the way we highlighted journal articles or books a decade or so ago. The journals and books were stored in long rows on our office shelves, with the highlighted prose unlikely to be discovered when useful. Some books while read, were returned to the library and not highlighted in the first place.

An Edutopia article on highlighting reached a negative conclusion about the value of highlighting (it may even hinder learning) and suggested solutions that educators should explain in a way very similar to that of the difference between fleeting and permanent notes. Those who are into Personal Knowledge Management methods for taking and retaining useful notes probably recognize this distinction. Ahrens suggested these terms as a way to identify important content (fleeting notes) with the expectation that this original material will receive further processing. He suggests that students a) annotate their highlights with short summaries and personal reflections or b) generate questions related to the content they have highlighted.

The Edutopia suggestions bring me to the perspective I want to emphasize.

Technology-based reading offers advantages over paper-based reading that are seldom emphasized. I rely heavily on highlighting when I write on my Kindle or using a browser extension that allows me to highlight web content. I don’t read from paper much anymore, but when I do, I also highlight a lot. When I use my iPad or computer to read and highlight, I tend to be using tools that allow me to add annotations (actually extended additions I would prefer to describe as notes) as part of the same integrated approach. I suppose I could read from a paper source and have a notebook on my desk at the same time, but I have never actually worked in this way. These highlights and notes are part of the original documents, but can also be exported for storage and further processing.

When I used to take notes from a highlighted book or journal article, it was usually later in some process of reviewing material in preparation to write something myself. In thinking about how I work now, I propose that reading using a technology-supported environment encourages the process of creating meaningful notes earlier in the process of writing, and is often disconnected from the process of creating the end product. There is an efficiency when meaningful notes are made during the initial process of reading new content in comparison to trying to create the same context when trying to make sense of highlights or notes that simply move unprocessed words from one paper source to another after a delay.

Here is my major use of the highlights from what I have read. As an academic researcher I read many, many journal articles. For the last 15 years of my career and since, I did my academic reading on the pdfs of these articles. Like other academics, I had access to these pdfs from pretty much any journal I wanted and I used these pdfs even when I owned the journals and they were on the shelf across the office from my desk. The tools I used to keep a record of the PDFs I read (originally to access the citations for articles) and to highlight these documents changed over the years, but I generated a large collection (hundreds) of highlighted articles. In recent years, I have been able to export the highlights and annotations and store this material using a personal knowledge management tool (Obsidian on my desktop and Mem.AI online). This large collection of has become a resource I can explore, link and tag. In the past couple of years, I have been able to chat with my content using AI tools (e.g., NotebookLM and Smart Connections). The opportunity being able to interact with material I have generated over decades is a very interesting experience and for someone who writes a boon to productivity,

Given the opportunities of reading on a digital device, I think we are at a point where highlighting may have value. Under these conditions, highlighting services as a placeholder for what should be a fairly immediate generation of meaningful notes. The placeholder has two benefits — it marks and saves a location in content that offers the benefit of context should a reader need to make use of the source material later. The marked material is also isolated through highlighting, and this would seem to benefit the note-making process.

One other conclusion for the Ponce and colleagues review of highlighting studies I drew from in previous sections of this post. These authors concluded that the effectiveness of highlighting was greatly enhanced when used in conjunction with more generative learning strategies, such as note-taking or creating graphic organizers. Combining highlighting with these activities showed a notably larger effect size compared to highlighting alone

I suggest it is time to prepare secondary students for these opportunities. I also argue that educators abandon the paper is best assumption. If learning is understood as a process with initial exposure not isolated from studying and review, I cannot see how paper sources have an advantage. Learn to use a digital highlighting and annotation tool and work this tool into your knowledge generation and storage workflow.

If my position makes sense to you, you may find the series of posts I have generated on note-taking to be of value.

Summary

Highlighting has often been dismissed as an effective learning strategy. Here, I argue that this is an outdated perspective based on assumptions related to the use of paper-based content. With digital content, highlighting can be an important first step in the processing of content for comprehension and value over extended periods of time.

Sources

Ahrens, S. (2017). How to take Smart Notes: One simple technique to boost writing, learning and thinking for students, academics and nonfiction book writers

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4–58.

Johns, A. (2023). The science of reading: Information, media, and mind in modern America. In The Science of Reading. University of Chicago Press.

Ponce, H. R., Mayer, R. E., & Méndez, E. E. (2022). Effects of learner-generated highlighting and instructor-provided highlighting on learning from text: A meta-analysis. Educational Psychology Review, 34(2), 989-1024.

The Medium is the Message

Marshall McLuhan’s famous declaration “The medium is the message” never made sense to me. It sounded cool, but on the surface there was not enough there to offer much of an explanation. It seemed one of those things other people understood and used, but I did not. Perhaps I had missed the class or not read the book in which the famous phrase was explained.

The expression came up again in the book club I joined while we reading a book by Johns (The Science of Reading). A sizeable proportion of one chapter considers McLuhan’s famous proposal and provided a reference to his first use of the phrase. The original mention was a comment he made at a conference and then continued to develop.

The page is not a conveyor belt for pots of message; it is not a consumer item so much as a producer of unique habits of mind and highly specialized attitudes to person and country, and to the nature of thought itself (…) Let us grant for the moment that the medium is the message. It follows that if we study any medium carefully we shall discover its total dynamics and its unreleased powers.

Print, by permitting people to read at high speed and, above all, to read alone and silently, developed a totally new set of mental operations.

Johns’ book is about the history of the study of reading as a science with more on how reading and the methods by which reading skill is developed became a political issue. My effort to create a personal understanding of what any of this would have to do with McLuhan now is based on my consideration of different media and what McLuhan had to say specifically about reading. I have come to think about reading as a generative activity which is a topic I write about frequently. From this perspective, reading is an external task that gives priority to certain internal behaviors. In contrast to some other media, reading allows personal control of speed. A reader can take in information quickly or pause to reflect. A reader can reread. Text sometimes requires the reader to generate imagery in contrast to having imagery offered to them as would be the case with video. Reading cannot transfer a complete experience from author to reader and much is constructed by the reader based on existing knowledge. Reading has a social component. In most cases reading involves an implied interaction with an author, but also with others who have interpreted the same input and who often interact to share personal interpretations.

What McLuhan had to say about media now reminds me of the notion of affordances. Affordance refers to the potential actions or uses that an object or environment offers to an individual, based on its design and the individual’s perception of it. The term was originally coined by psychologist James J. Gibson in the context of ecological psychology to describe the possibilities for action that the environment provides. Affordances can be both obvious (like a door handle that affords pulling) or less obvious, depending on how the individual perceives and interacts with the object or environment. It is this less obvious type of affordance that applies based on expectations for texts and for how we anticipate texts to be used. Factors such as the allowances for controlling speed and pausing with a medium that is essentially static when we are not interacting with it to allow reflection are more like the obvious affordances Gibson proposes.

Those who reject a media effect

Having reached what I hope is an appropriate understanding of McLuhan’s famous insight, I realized that I have encountered a contradictory argument commonly taught within one of my fields of practice (educational technology). This controversy concerns what tends to be called the media effect.

The “media effect” refers to the idea that the medium or technology used to deliver instruction (such as television, computers, or textbooks) has a significant impact on learning outcomes. This concept suggests that different media can produce different levels of learning or change the way people learn.

This perspective was challenged by Richard Clark in his influential 1983 article, “Reconsidering Research on Learning from Media.” Clark argued that the media itself does not influence learning; rather, it is the instructional methods and content delivered through the media that determine learning outcomes. Clark famously stated, “media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition.”

Clark’s challenge to the media effect emphasized that it’s the instructional design, the way content is presented, and the interaction between learners and content that are crucial for learning, not the medium through which the instruction is delivered.

I always struggled when teaching this position. Instructional designers are expected to consider this argument, but my interpretation never allowed me to understand why this would be true. If I wanted to teach someone the cross-over dribble, wouldn’t it make more sense to begin by showing the move rather than describing it with text? I understand that each of us learns through our own cognitive actions, but how we access inputs (external representations) would seem to matter in what our cognitive behaviors have to work with. When you ask advanced students to deal with arguments such as Clark’s that challenge actions they might be prone to take, it is common to match the challenging position with a source that offers a counterargument. I paired Clark’s paper with a paper written by Robert Kozma. If you are inclined to pursue this controversy, I recommend this combination.

Does it matter?

Possibly. I think we are experiencing changes in how we experience information. Most of us experience more and more video both for entertainment and for learning. It is worth considering how we might be influenced by the medium of input. If we are trying to learn more frequently from video, how do we attempt to process the video experience in a way similar to how we can take control and process text?

References:

Clark, R. E. (1983) Reconsidering research on learning from media. Review of educational research 53 (4), 445-459.

Johns, A. (2023). The science of reading: Information, media, and mind in modern America. University of Chicago Press.

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational technology research and development, 42(2), 7-19.

Desirable Difficulty

Despite a heavy focus on cognitive psychology in the way I researched and explained classroom study tactics, I had not encountered the phrase desirable difficulty until I became interested in the handwritten vs. keyboard notetaking research. I discovered the idea when reviewing studies by Luo and colleagues and Mueller and Oppenheimer. Several studies have claimed students are better off taking notes by hand in comparison to on a laptop despite being able to record information significantly faster when using a keyboard.

Since having a more complete set of notes would seem an advantage. The combination of more notes associated with poorer performance is counterintuitive. Researchers speculated that learners who understood they had to make decisions about what they had time to record selected information more carefully and possibly summarized rather than recorded verbatim what they heard. This focus on what could be described as deeper processing seemed like an example of desirable difficulty. The researchers also proposed that the faster keyboard recording involved shallow cognitive processing.

Note: I am still a fan of more complete notes and the methodology used when demonstrating better performance from recording notes by hand needs to be carefully considered. I will comment on my argument more at the end of this post.

Desirable difficulty an idea attributed to Robert Bjork has been used to explain a wider variety of retention phenomena. Bjork suggested that retrieval strength and storage strength are distinct phenomena and learners can be misled when an approach to learning is evaluated based on retrieval strength. I find these phrases to a bit confusing as applied, but I understand the logic. Students cramming for an exam make a reasonable example. Cramming results in what may seem to be successful learning (retrieval strength), but results in poorer retention over an extended period of time (storage storage strength). Students may understand and accept the disadvantages of cramming so it is not necessary that the distinction be unrecognized by learners. In a more recent book on learning for the general public, Daniel Willingham suggests that the brain is really designed to avoid rather than embrace thinking because thinking is effortful. The human tendency is to rely on memory rather than thinking. Desirable difficulty may be a way to explain why some situations that require thinking prevent something more rote.

Increasing difficulty to improve retention

There are multiple tactics for productively increasing difficulty that I tend to group under the heading of generative learning. I describe generative activities as external tasks intended to increase the probability of productive cognitive (mental) behaviors. I suppose desirable difficulty is even more specific differentiating external tasks along a difficulty dimension. So in the following list of tasks, it is useful to imagine more and less difficult tasks. Often the less difficult task is the option learners choose to apply. In connecting these tactics with personal experience, I would recommend you consider the use of flashcards to conceptualize what would be the easier and the more challenging application. Then, move beyond flashcards to other study tactics and consider if you can identify similar contrasts.

Retrieval Practice: Testing oneself on the material rather than passively reviewing notes is considered retrieval practice. The classic empirical demonstration of the retrieval practice or the testing effect compared reviewing content versus responding to questions. Even when controlling for study time, spending some time on questions was superior. With the flashcard applications I recommended you consider, answering multiple-choice questions would be less challenging than answering short-answer questions (recognition vs recall).

Spacing (Distributed Practice): Instead of cramming, spreading out study sessions over time is more productive. This method helps improve long-term retention and understanding. Spacing allows some retrieval challenges to develop and the learner must work harder to locate the desired information in memory. See my earlier description of Bjork’s distinction between retrieval strength and storage strength.

Interleaving: Mixing different types of problems or subjects in one study session. For example, alternating between math problems and reading passages rather than focusing on one at a time. A simple flashcard version of this recommendation might be shuffling the deck between cycles through the deck. Breaking up the pattern of the review task increases the difficulty and requires greater cognitive effort.

Other thoughts

First, the concept of committing to more challenging tasks is broader than the well researched examples I provide here. Writing and teaching could be considered examples in that both tasks require an externalization of knowledge that is both generative and evaluative. It is too easy to fake it and make assumptions when the actual creation of a product is not required.

Second, desirable difficulty seems to me to be a guiding principle that does not explain all of the actual cognitive mechanisms that are involved. The specific mechanisms may vary with activity – some might be motivational, some evaluative (metacomprehension), and some at the level of basic cognitive activities. For example, creating retrieval challenges probably creates an attempt to find alternate or new connections among stored elements of information. For example, in trying to put a name with a face one might attempt to remember the circumstances in which you may have met or worked with this person and this may activate a connection you do not typically use and is not automatic. For example, after being retired for 10 years and trying to remember the names of coworkers, I sometimes remember the arrangement of our offices working my way down the appropriate hallway and this sometimes helps me recall names.

I did say I was going to return to the use of desirable difficulty as a justification for the advantage of taking notes by hand. If keyboarding allows faster data entry than handwriting, in theory keyboarding would allow more time for thinking, paraphrasing, and whatever advantage one would have when the recording method requires more time. Awareness and commitment would seem to be the issues here. However, I would think complete notes would have greater long-term value than sparse notes. One always has the opportunity to think while studying and a more complete set of notes would seem to provide the opportunity to have more external content to work with.

References:

Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.

Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971.

Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.

Willingham, D. T. (2021). Why don’t students like school?: A cognitive scientist answers questions about how the mind works and what it means for the classroom. John Wiley & Sons.

Potential conflicting benefits of your note-taking tool and approach

As I have explored and used several digital note-taking tools and examined the arguments that have been made regarding how such tools result in productivity benefits, I have identified a potential conflict in what produces more positive outcomes. The recognition of this conflict allows more purposeful execution on the part of the tool user and may better align activities with goals.

One way to identify note-taking goals is to use a long-standing approach differentiating generative and external storage benefits. This distinction was proposed long before PKM and was applied in the analysis of notes taken in classroom settings. The generative benefit proposes that the process of taking notes or sometimes of taking notes in a particular way engages our cognitive (mental) processes in ways that improve retention and understanding. External storage implies that our memory becomes less effective over time and having access to an external record (the notes) benefits our productivity. In practice (e.g., a student in a classroom) both benefits may apply, but one benefit depends on the other activity. Taking notes may not be beneficial, but to review notes one must have something to review. This is not always true as notes in one form or another can be provided or perhaps generated (for example AI identification of key ideas), but taking your own notes is by far the most common experience. In a PKM way of thinking, these two processes may function in different ways, but the classroom example should be familiar as a way to identify the theoretical benefits of note-taking.

I have written about the generative function of note-taking at length, but it is important to point out some unique specifics that apply to some digital note-taking tools. A source such as Ahrens’ Taking Smart Notes might provide the right mindset. I think of generative activities as external actions intended to produce a beneficial mental (cognitive) outcome. The idea is that external activities can encourage or change the likelihood of beneficial thinking behaviors. One way of operationalizing this perspective is to consider some of the specific activities Ahrens identified as external work resulting in such cognitive benefits. What are some of these activities? Isolating specific ideas and summarizing each as a note. Assigning tags that characterize a note. Making the effort to link notes. Periodically reviewing notes to generate retrieval practice, to reword existing notes, and to add new associations (links).

Retrieval is easier to explain. Note-taking apps with highly effective search capabilities make it easy to search and surface stored information when it might be useful. Links and tags may also be useful in this role, but search alone will often be sufficient.

What about the potential conflict?

The conflict I see proposes that some tools or approaches rely more heavily on search arguing in a way that generative processes are unnecessary.

I starting thinking about this assumption when contrasting the two note-taking systems I rely on – Mem.ai and Obsidian. While Mem.AI and Obsidian could be used in exactly the same way, Mem.ai developers argued that the built-in AI capabilities could eliminate the need to designate connections (with tags and links) because the AI capabilities would identify these connections for you. Thus when retrieving information via search, a user could use AI to also consider the notes with overlapping foci. If a user relied on this capability it would eliminate the work required to generate the connections manually created in Obsidian, but this approach would then also avoid the generative benefits of this work.

AI capabilities fascinate me so I found a way to add a decent AI capability to Obsidian. Smart Connections is an Obsidian plugin that finds connections among notes and allows a user to chat with their notes. So, I found a way to mimic Mem.ai functionality with Obsidian.

I find I have found a way to alter my more general PKM approach because of these capabilities. Rather than taking individual notes while reading, I can annotate and highlight pdfs, books, and videos and export the entire collection for each source and then bring this content into both Mem.ai and Obsidian as a very large note. Far easier than taking individual notes, but at what generative cost?

Smart Connections has added a new feature that even facilitates the use of the large note approach. Connections finds connections based on AI embeddings. An embedding is the mathematical representation of content (I would describe as weights based on what I remember of statistics). The more two notes embeddings’ weights are similar the more the notes consider similar ideas. Smart Connections used embeddings to propose related notes. Originally embeddings were generated at the note level and now at the “block” level. What this means (block level) is that Smart Connections can find the segments of a long document that have a similar focus as a selected note.

Why is this helpful? When I read long documents (pdfs of journal articles or books in Kindle), I can export a long document containing my highlights and notes generated from these documents. With Smart Connections I can then just import this exported material into Obsidian and use Smart Connections to connect a specific note to blocks of all such documents. I can skip breaking up the long document into individual notes and assigning tags and creating links.

Why is this a disadvantage? Taking advantage of this capability can be a powerful disincentive to engaging in the generative activities involved in creating and connecting individual notes the basic version of Obsidian requires.

Summary

As note-taking tools mature and add AI capabilities, it is important for users to consider how the way they use such tools can impact their learning and understanding. The tools themselves are quite flexible but can be used in ways that avoid generative tasks that impact learning and understanding. If the focus is on the retrieval of content for writing and other tasks, the generative activities may be less important. However, if you start using a tool such as Obsidian because a book such as Smart Notes influenced you, you might want to think about what might be happening if you rely on the type of AI capabilities I have described here.

References
Ahrens, S. (2022). How to take smart notes: One simple technique to boost writing, learning and thinking. Sönke Ahrens.

Writing to Learn Research – Messy

Writing to learn is one of those topics that keeps drawing my attention. I have an interest in what can be done to encourage learning and approach this interest by focusing on external tasks that have the potential to manipulate the internal cognitive (thinking) behavior of learners. My background in taking this perspective is that of an educational psychologist with a cognitive perspective. I have a specific interest in areas such as study behavior trying to understand what an educator or instructional designer can do to promote experiences that will help learners be more successful. The challenge seems obvious – you cannot learn for someone else, but you may be able to create tasks that when added to exposure to sources of information encourage productive “processing” of those experiences. We can ask questions to encourage thinking. We can engage students in discussions that generate thinking through interaction. We can assign tasks that require the use of information. Writing would be an example of such an assigned task.

Writing to Learn

Writing to learn fits with this position of an external task that would seem to encourage certain internal behaviors. To be clear, external tasks cannot control internal behavior. Only the individual learner can control what they think about and how they think about something, but for learners willing to engage with an external activity that activity may change the likelihood productive mental behaviors are activated.

I found the summary of the cognitive benefits of writing to learn useful and consistent with many of my own way of thinking about other learning strategies – external tasks that encourage productive internal behaviors. Writing based on content to be learned requires that the writer generate a personalized concrete representation at the “point of utterance”. I like this expression. To me, it is a clever way of saying that when you stare at the screen or the empty sheet of paper and must fill the void you can no longer fool yourself – you either generate something or you don’t. You must use what you know and how you interpret the experiences that supposedly have changed what you know to produce an external representation.

To produce an external product, you must think about what you already know in a way that brings existing ideas into consciousness (working memory) by following the connections activated by the writing task and newly acquired information. This forces processing that may not have occurred without the external task. Connections between existing knowledge and new information are not necessarily made just because both exist in storage. Using knowledge to write or to perform other acts of application encourages making connections.

Such attempts at integration may or may not be successful. Having something external to consider offers the secondary benefit of forced metacognition. Does what I wrote really make sense? Do the ideas hang together or do I need to rethink what I have said? Does what I have proposed fit with the life experiences (episodic memory) I have had?

Writing ends up as a generative process that potentially creates understanding and feeds the product of this understanding back into storage.

Graham, Kiuhara & MacKay, M. (2020)

In carefully evaluating and combining the results of many studies of writing to learn, these researchers intended not only to determine if the impact of writing to learn had the intended general benefit but to use the variability of writing tasks and outcomes from studies to deepen our understanding of how writing to learn encouraged learning. Surely, some activities would be more beneficial than others because of the skills and existing knowledge of learners or the specifics of the assigned writing tasks. So, the meta-analysis is asking if there is a general effect (Is writing to learn effective), and secondarily are there significant moderator variables that may help potential practitioners decide when, with whom, and how to structure writing to learn activities?

The Graham and colleagues’ research focused only on K12 learners. Potential moderator variables included grade level, content area (science, social studies, mathematics), type of writing task (argumentation, informational writing, narrative), and some others. I have a specific interest in argumentation () which is relevant here as a variable differentiating the studies because it requires a deeper level of analysis than say a more basic summary of what has been learned.

Overall, the meta-analysis demonstrated a general benefit for writing to learn (Effect size = .30). This level of impact is considered on the low end of a moderate effect. Graham and colleagues point out that the various individual studies included in the study generated great variability. A number of the studies demonstrated negative outcomes meaning in those studies the control condition performed better than the group spending time on writing to learn. The authors propose that this variability is informative as it cannot be assumed that any approach with this label will be productive. The variability also suggests that the moderator variables may reveal important insights.

Unfortunately, the moderator variables did not achieve the level of impact necessary to argue for useful insights as to how writing to learn works or who is most likely to be a priority group for this type of activity. Grade level was not significant. The topic area was not significant. The type of writing task was not significant.

Part of the challenge here is having enough studies focused on a given approach with enough consistency of outcomes to allow statistical certainty in arguing for a clear conclusion. Studies that involved taking a position and supporting that position (e.g., argumentation) produced a much larger effect size, but the statistical method of meta-analysis did not reach the level at which a certain outcome could be claimed.

One interesting observation from the study caught my attention. While writing to learn is used more frequently in social studies classrooms, the number of research studies associated with each content areas was the smallest for social studies. Think about this. Why? I wonder if the preoccupation of researchers and funding organizations with STEM is responsible.

More research is needed. I know practitioners and the general public get tired of being told this, but what else can you recommend when confronted with the messiness of much educational research? When you take ideas out of carefully controlled laboratories and try to test them in applied settings the results here are fairly typical. Humans left to their own devices as implementers of procedures and reactors to interventions are all over the place. Certainly, the basic carefully controlled research and the general outcome of meta-analysis focused on writing to learn implementation are encouraging, but as the authors suggest the variability in effectiveness means something, and further exploration is warranted.

Reference

Graham, S., Kiuhara, S. A., & MacKay, M. (2020). The effects of writing on learning in science, social studies, and mathematics: A meta-analysis. Review of Educational Research, 90(2), 179-226.