Generative learning – Page 2

Notetaking in the lab and the wild

Human behavior can be scientifically studied in the laboratory and the wild. This is the case with notetaking and other study behaviors. When politicians use the phrase “the science of learning” it can be misleading to the public because science in laboratory settings and in the wild can seemingly lead to different conclusions and related recommendations. I believe that the controversy of the “science of reading” is related to this issue, but I have greater experience with notetaking and study behavior so I will stick to explaining how this works in this more familiar area.

I have been referencing Daniel Willingham’s work a lot lately, and the following quote offers a good introduction to my point. In commenting on textbook companies building in proven study opportunities within their textbooks as aids to students, Willingham offers the following comment:

… if the readings include learning aids such as chapter outlines, chapter previews and summaries, boldface or italicized terms, or practice test questions, don’t try to use these learning aids as a replacement for reading the text. The funny thing about these features is that there’s very good research evidence that they work. Publishing companies paid to have high-quality research conducted; researchers had people read textbook chapters (with or without the learning aids), and they found that people who used the learning aids understood and remembered more than those who did not.

But the psychologists Regan Gurung and David Daniel pointed out that students “in the wild” will not necessarily use such materials the same way they were used by students in the laboratory. Gurung and Daniel suggested that some students use learning aids not to supplement the reading but to avoid it. They read the summary, look at the boldface terms, and then try to answer the practice test questions to see whether they understand enough to we skip the reading.

Willingham and other researchers (e.g., Gurung) note that educational research conducted under carefully controlled conditions may not predict applied situations. Applied situations often involve interactions as individuals make personal decisions about how learning strategies are applied. They may have different goals, different abilities, or different goals and life situations which cause them to use strategies in ways not intended or maybe not at all. Also tactics intended for the classroom situations may not encourage the development of personal skills that would be most likely used in life situations.

When I was still teaching, I sometimes contrasted attempting to do science with humans in contrast to what are often described as the “hard sciences” by note that the chemicals in a chemical reaction don’t decide if they feel like interacting.

In looking back on my own research which was conducted in applied settings I was continually frustrated by this type of issue. I focused a lot of what I did on trying to create adaptive computer-supported study environments. The idea was that a computer can offer questions related to learning goals and use student accuracy and answer confidence to identify areas of weakness and to provide direct connections to the related textbook material. The idea was to identify heat maps of more difficult material for individual learners, to provide questions related to the areas of difficulty more frequently during a study session, and even to provide access to the question related content on the screen if the student wanted. Built into the online delivery system were ways to record the amount of use, the question performance and awareness of understanding, the use of the online content and the delay following wrong answers. My frustration arose from the findings that the system was really designed to assist less capable students (lower reading ability, poorer metacognitive awareness of strengths and weaknesses) who as it turned out were far less likely to use the system and to use it in ways the research would suggest were helpful (e.g., taking advantage of the feedback following wrong answers and especially wrong answers readers thought they understood). The failed opportunity to use the system to try to recognize the lack of understanding makes a good example of what Willingham, Gurung, and others have described. Even when investing time, these learners answered question after question without taking advantage of the opportunity to process feedback.

Understanding Why Tactics Work

Those situations in which learners invest time, but do so in an inefficient way are what I find most fascinating. Motivation makes a huge difference in learning, but would seem less of an issue with these individuals. Perhaps motivation is reflected in how hard in comparison to how long a learner works. This way of thinking would seem similar to Willingham’s “Outsmart your brain” suggestion that the brain interprets easier as better. It could follow that a possible remedy would be better understanding of how a given tactic works in addition to simply learning how to perform certain tactics. Answering questions is harder than rereading but works better because answering questions requires greater effort in actively engaging memory and thinking. Taking notes is better than highlighting because taking paraphrase notes requires more cognitive thinking. Etc.

I can’t help thinking about the fascination and process-oriented debate those interested in Personal Knowledge Management have with tools and tactics in comparison to most students in formal learning settings. Perhaps this is just an impression on my part, but it seems generally to be the case. If I am correct, I think the difference is in the opportunity self-directed learners have to set personal goals and as a consequence invest time in trying to understand why differences in processes matter. The only alternative I can imagine would involve more direct instruction and how to study instruction is not emphasized or cut when resources are in short supply.

References

Daniel, David B., and Debra A. Poole. “Learning for life: An ecological approach to pedagogical research.” Perspectives on Psychological Science 4, no. 1 (2009): 91-96.

Grabe, M., & Flannery, K. (2009/2010). A preliminary exploration of on-line study question performance and response certitude as predictors of future examination performance. Journal of Educational Technology Systems, 38(4), 457-472.

Grabe, M., Flannery, K., & Christopherson, K. (2008). Voluntary use of online study questions as a function of previous minimal use requirements and learner aptitude. Internet and Higher Education. 11, 145-151.

Grabe, M. & Holfeld, B. (2014). Estimating the degree of failed understanding: a possible role for online technology. Journal of Computer Assisted Instruction. 30, 173-186.

Gurung, Regan A. R., and David B. Daniel. (2005). Evidence-Based Pedagogy: Do Pedagogical Features Enhance Student Learning? (pps. 41–55). In Best Practices for Teaching Introduction to Psychology, Dana S. Dunn and Stephen L. Chew (eds.), Mahwah, NJ: Erlbaum.

Notetaking – Your brain is lazy

My favorite writer who focuses on classroom learning is Daniel Willingham. He has a way of explaining and applying research that is both approachable and actionable. My interests and vocational focus overlap with the topics of his books allowing me to be appreciative of his insights and his creative way of communicating the mindset of educators and writers and the behaviors of both highly motivated and more casual students.

Willngham’s most recent book, Outsmart Your Brain, considers notetaking multiple times as he examines several learning challenges (the large lecture, lengthy textbook assignments, labs and other hands-on activities). Taking notes in formal educational settings can differ in important ways from the writing I do about autonomous lifelong learners involved in what is often described as Personal Knowledge Management or Building a Second Brain, but he speculates about important cognitive processes rather than just offering “here is what you should do” tactics. I assume that processes generalize and with so little research focused on learning outside of formal educational settings, the commentary I offer is largely based on using what classroom-focused researchers find that would seem to apply to learning on your own.

The meaning of Willingham’s title, “Outsmart Your Brain”, is that what seems to be an easy to accomplish tactic is often the wrong choice. He differentiates the notetaking choices made when listening to lectures and reading. In contrast to many, it should be noted that Willingham supports the lecture as an important educational strategy. It is efficient as a way to communicate information, and face-to-face efficiency seems to offer better effectiveness than recorded and distributed content. The major challenge with lectures is that we tend to speak much more rapidly than individuals can write and in a large group setting feedback to a presenter is difficult to generate and would varies greatly from listener to listener. The related issue on the part of listeners is that many are unable to sort out what should be retained in notes. Often what is written is what is understood which is understandable, but an example of doing the easier thing. He notes that collaboration or instructor-provided notes offer a solution, but proposes that these resources should be used in addition to taking notes which is a generative cognitive and thus beneficial process.

Willingham supports the researchers arguing that taking notes with pen on paper to be superior to taking notes using a digital device and as proposed in the “desirable difficulty” hypothesis proposes that the insight that more can be recorded on a keyboard provides a false sense of accomplishment. This is another example of the brain making the wrong decision. I disagree on this point and argue that Willingham ignores the opportunity a digital device can provide a written record and link audio to notes in ways that allow missed information to be re-examined. A link references the corresponding location in the audio when a note was taken. Willingham does recognize and discuss recording lectures, but discusses this opportunity as inefficient unaware I assume that the connections some apps store between notes and audio (or video) allow learners great control of how the audio is used.

Willingham discusses note-taking as a useful addition to reading recognizing that with reading the learner does not have to deal with the lack of control present when listening. The flawed option he calls out is highlighting which again offers the learner a false sense of accomplishment. He cites an interesting study in which multiple used textbooks from the same class were examined and the finding that the text selected as important varied greatly. I could not help thinking of the “most common highlighted” option available with Kindle books.

A common issue with both lectures and books is that both tend to be hierarchical, but are experienced as sequential experiences. I interpret this problem to be one that understanding is the construction of a model of how things are interrelated. Lectures and writers tend to have this model and organize what they offer accordingly, but the experience of the learner is sequential and building a hierarchical model in real time is often too demanding. Imagine an outline that is used to develop a lecture or written product and in which the product shared moves through each part of the outline from higher to lower elements as a sequence and you can imagine the issue of reconstructing the outline. Learners can rework the content they have stored in search of this structure and presents can help by offering an overview and referring back to this overview as the presentation unfolds. Willingham speculates that learners possibly read textbooks based on their experience with fiction.

Willingham proposes two additional strategies making use of notes often ignored by students. The first is the sharing and discussion of notes within small groups. Again, this is not to replace the task of taking notes, but a way to identify ideas that have been missed or misunderstood. The second is a cross-examination of notes taken from lectures and from assigned readings. Too many seem to assume that the elimination of one source is a possible opportunity, but he argues that cross-referencing sources like cross-referencing with peers allows for additional active processing.

Summary

This was intended as more than a book review, but it is a recommendation that both educators and learners read this book. Many reviewers have noted that it should be assigned reading for new college students faced with the challenge of taking more responsibility for their own learning. The notion that the brain leads us to do things in the moment that are not necessarily the best for the future is important to recognize and the assumption that taking notes or reading a book could benefit from the consideration of nonobvious strategies deserves careful consideration. When are important study skills taught and which educators are responsible for helping learners develop these skills?

Disciplinary Perspectives on Taking and Using Notes

I have found myself exploring and writing about the interrelated topics of personal knowledge management, second brains, and note-taking for the past several years. As I have spent time on these interests, it became obvious that there were multiple disciplinary perspectives on these topics. In addition, the different disciplines seem mostly oblivious to each other as indicated by the lack of cross-referencing evident in their written materials. There are sometimes references to historical connections which I will identify, but for anyone interested in these topics I would suggest there are benefits for exploring more than a single point of view.

The Perspectives

Here are the descriptive labels I have decided to use for what I claim to be different perspectives. Hopefully, the labels offer some insights into the categories I have in mind.

Academic studying – this perspective provided my personal background for this general topic. The focus of this perspective is learning in formal academic environments with the goals of the acquisition, understanding, and application of information to examinations and projects. While the general goal of education is focused on the long term and preparation for life, note-taking has a more immediate focus. I am of the opinion that the great majority of what I would describe as research is focused on topics within this category. Most of this research is based on a cognitive perspective on learning and application.
Organizational Knowledge Management – Organizations have a need to develop, preserve, and apply knowledge. For multiple practical reasons (e.g., changeover in personnel), this knowledge should be externalized for the benefit of the organization. The generation and use of this shared knowledge originate with individuals. Personal knowledge management (PKM) can be individualized or integrated with the more general needs of a given organization. Procedures for accomplishing these goals are the subject of scholarship and training in the formal programs preparing individuals for careers in organizations (e.g., business schools), but it is my impression that scholarship is less empirical than that applied by those with an academic studying perspective and more anecdotal and based in logical argumentation.
Knowledge Management Entrepreneurs – I struggled with a way to describe this perspective. It seems to me that there has been a recent and identifiable group of individuals offering self-help books and consulting expertise to those interested in Personal Knowledge Management. This category resembles the organizational knowledge management perspective but does not share the same group focus. The perspective emphasizes the collection, organization, exploration, and application of information over an extended period of time to accomplish personal goals. Of the three groups I have identified, those individuals promoting techniques and processes are the least likely engaged in what I would describe as formal scholarship.

Historical Antecedents

While not absolutely consistent, there are frequent references to similar individuals, practices, and models that can often be identified among these perspectives. Here is my own list of such sources.

Vannevar Bush’s article “As we may think” describing the manner in which individuals and organizations might use a yet-to-be-developed technology (the Memex) to take on information overload and how a knowledge worker might explore, retain, organize, and apply information.
Commonplace books are journals, diaries, or notebooks maintained by individuals. A famous historical example would be the Leonardo Di Vinci notebooks still available in different formats (Amazon source).
Luhmann’s Zettelkasten. A zettelkasten is a card-based note-taking and note-linking system now often adapted to digitization and computer applications. It did not originate with Nikolas Luhmann, but I have connected the approach with his name because his prodigious use of the system as a scholar seems the example so many use.
The encoding and external model of note-taking (e.g., Rickards & Friedman, 1978) is the basis for much of the empirical research from the academic studying perspective. It proposes that learners could possibly benefit from both the thinking required in taking notes (the encoding process) and/or by having an external record available for review (external storage). This basic differentiation has been applied to such topics as whether taking notes by hand is more or less effective than taking notes using a keyboard (encoding), the best ways to work with the external notes (e.g., retrieval practice), and individual differences in both what is stored and how what is stored is used. For example, the Cornell note-taking method is an example of a system for both taking and using notes.

Examples from the different perspectives

I have written extensively about a couple of these perspectives in previous posts so rather than repeat myself and increase the length of this post I will link to some of these earlier posts.

Academic Studying – History of Note-Taking Research, Note-taking as a Generative Activity, Cornell Notes and Beyond
Organizational Knowledge Management – this perspective is a little more challenging as I have not written about it before. Here is a source you can explore without having journal access – Towards a Co-evolution of Organizational and Personal Knowledge Management Systems. Also see Pauleen (2009) – this is the introduction to a special issue on personal knowledge management.
Knowledge Management Entrepreneurs – Creating, Storing, and Using Smart Notes, Evaluating Tech Tools for Adults

Why consideration of the different perspectives might be useful?

Having asked you to recognize the multiple perspectives that I have identified, I owe you some explanation for why I think anyone interested in taking notes should expand their awareness of the background content available on this topic. I have found a couple of personal opportunities. First, the work from the perspective of academic studying has been far more carefully evaluated and useful in answering questions of why and if specific activities work. The knowledge management entrepreneurs offer specific “how to do it” suggestions and have strongly promoted the use of technology tools in PKM. The organizational knowledge management perspective extends the note-taking and PKM for life-long learning expanding core ideas beyond the academic classroom setting.

The links I provide here should open to many other resources on the perspectives I have identified.

References not linked

Pauleen, David (2009), “Personal knowledge management: putting the ‘person’ back into the knowledge equation”, Online Information Review, vol. 33, no. 2, pp. 221–224, doi:10.1108/14684520910951177.

Rickards, J. P., & Friedman, F. (1978). The encoding versus the external storage hypothesis in note taking. Contemporary Educational Psychology, 3(2), 136-143.

The Medium is the Message

Marshall McLuhan’s famous declaration “The medium is the message” never made sense to me. It sounded cool, but on the surface there was not enough there to offer much of an explanation. It seemed one of those things other people understood and used, but I did not. Perhaps I had missed the class or not read the book in which the famous phrase was explained.

The expression came up again in the book club I joined while we reading a book by Johns (The Science of Reading). A sizeable proportion of one chapter considers McLuhan’s famous proposal and provided a reference to his first use of the phrase. The original mention was a comment he made at a conference and then continued to develop.

The page is not a conveyor belt for pots of message; it is not a consumer item so much as a producer of unique habits of mind and highly specialized attitudes to person and country, and to the nature of thought itself (…) Let us grant for the moment that the medium is the message. It follows that if we study any medium carefully we shall discover its total dynamics and its unreleased powers.

Print, by permitting people to read at high speed and, above all, to read alone and silently, developed a totally new set of mental operations.

Johns’ book is about the history of the study of reading as a science with more on how reading and the methods by which reading skill is developed became a political issue. My effort to create a personal understanding of what any of this would have to do with McLuhan now is based on my consideration of different media and what McLuhan had to say specifically about reading. I have come to think about reading as a generative activity which is a topic I write about frequently. From this perspective, reading is an external task that gives priority to certain internal behaviors. In contrast to some other media, reading allows personal control of speed. A reader can take in information quickly or pause to reflect. A reader can reread. Text sometimes requires the reader to generate imagery in contrast to having imagery offered to them as would be the case with video. Reading cannot transfer a complete experience from author to reader and much is constructed by the reader based on existing knowledge. Reading has a social component. In most cases reading involves an implied interaction with an author, but also with others who have interpreted the same input and who often interact to share personal interpretations.

What McLuhan had to say about media now reminds me of the notion of affordances. Affordance refers to the potential actions or uses that an object or environment offers to an individual, based on its design and the individual’s perception of it. The term was originally coined by psychologist James J. Gibson in the context of ecological psychology to describe the possibilities for action that the environment provides. Affordances can be both obvious (like a door handle that affords pulling) or less obvious, depending on how the individual perceives and interacts with the object or environment. It is this less obvious type of affordance that applies based on expectations for texts and for how we anticipate texts to be used. Factors such as the allowances for controlling speed and pausing with a medium that is essentially static when we are not interacting with it to allow reflection are more like the obvious affordances Gibson proposes.

Those who reject a media effect

Having reached what I hope is an appropriate understanding of McLuhan’s famous insight, I realized that I have encountered a contradictory argument commonly taught within one of my fields of practice (educational technology). This controversy concerns what tends to be called the media effect.

The “media effect” refers to the idea that the medium or technology used to deliver instruction (such as television, computers, or textbooks) has a significant impact on learning outcomes. This concept suggests that different media can produce different levels of learning or change the way people learn.

This perspective was challenged by Richard Clark in his influential 1983 article, “Reconsidering Research on Learning from Media.” Clark argued that the media itself does not influence learning; rather, it is the instructional methods and content delivered through the media that determine learning outcomes. Clark famously stated, “media are mere vehicles that deliver instruction but do not influence student achievement any more than the truck that delivers our groceries causes changes in our nutrition.”

Clark’s challenge to the media effect emphasized that it’s the instructional design, the way content is presented, and the interaction between learners and content that are crucial for learning, not the medium through which the instruction is delivered.

I always struggled when teaching this position. Instructional designers are expected to consider this argument, but my interpretation never allowed me to understand why this would be true. If I wanted to teach someone the cross-over dribble, wouldn’t it make more sense to begin by showing the move rather than describing it with text? I understand that each of us learns through our own cognitive actions, but how we access inputs (external representations) would seem to matter in what our cognitive behaviors have to work with. When you ask advanced students to deal with arguments such as Clark’s that challenge actions they might be prone to take, it is common to match the challenging position with a source that offers a counterargument. I paired Clark’s paper with a paper written by Robert Kozma. If you are inclined to pursue this controversy, I recommend this combination.

Does it matter?

Possibly. I think we are experiencing changes in how we experience information. Most of us experience more and more video both for entertainment and for learning. It is worth considering how we might be influenced by the medium of input. If we are trying to learn more frequently from video, how do we attempt to process the video experience in a way similar to how we can take control and process text?

References:

Clark, R. E. (1983) Reconsidering research on learning from media. Review of educational research 53 (4), 445-459.

Johns, A. (2023). The science of reading: Information, media, and mind in modern America. University of Chicago Press.

Kozma, R. B. (1994). Will media influence learning? Reframing the debate. Educational technology research and development, 42(2), 7-19.

Desirable Difficulty

Despite a heavy focus on cognitive psychology in the way I researched and explained classroom study tactics, I had not encountered the phrase desirable difficulty until I became interested in the handwritten vs. keyboard notetaking research. I discovered the idea when reviewing studies by Luo and colleagues and Mueller and Oppenheimer. Several studies have claimed students are better off taking notes by hand in comparison to on a laptop despite being able to record information significantly faster when using a keyboard.

Since having a more complete set of notes would seem an advantage. The combination of more notes associated with poorer performance is counterintuitive. Researchers speculated that learners who understood they had to make decisions about what they had time to record selected information more carefully and possibly summarized rather than recorded verbatim what they heard. This focus on what could be described as deeper processing seemed like an example of desirable difficulty. The researchers also proposed that the faster keyboard recording involved shallow cognitive processing.

Note: I am still a fan of more complete notes and the methodology used when demonstrating better performance from recording notes by hand needs to be carefully considered. I will comment on my argument more at the end of this post.

Desirable difficulty an idea attributed to Robert Bjork has been used to explain a wider variety of retention phenomena. Bjork suggested that retrieval strength and storage strength are distinct phenomena and learners can be misled when an approach to learning is evaluated based on retrieval strength. I find these phrases to a bit confusing as applied, but I understand the logic. Students cramming for an exam make a reasonable example. Cramming results in what may seem to be successful learning (retrieval strength), but results in poorer retention over an extended period of time (storage storage strength). Students may understand and accept the disadvantages of cramming so it is not necessary that the distinction be unrecognized by learners. In a more recent book on learning for the general public, Daniel Willingham suggests that the brain is really designed to avoid rather than embrace thinking because thinking is effortful. The human tendency is to rely on memory rather than thinking. Desirable difficulty may be a way to explain why some situations that require thinking prevent something more rote.

Increasing difficulty to improve retention

There are multiple tactics for productively increasing difficulty that I tend to group under the heading of generative learning. I describe generative activities as external tasks intended to increase the probability of productive cognitive (mental) behaviors. I suppose desirable difficulty is even more specific differentiating external tasks along a difficulty dimension. So in the following list of tasks, it is useful to imagine more and less difficult tasks. Often the less difficult task is the option learners choose to apply. In connecting these tactics with personal experience, I would recommend you consider the use of flashcards to conceptualize what would be the easier and the more challenging application. Then, move beyond flashcards to other study tactics and consider if you can identify similar contrasts.

Retrieval Practice: Testing oneself on the material rather than passively reviewing notes is considered retrieval practice. The classic empirical demonstration of the retrieval practice or the testing effect compared reviewing content versus responding to questions. Even when controlling for study time, spending some time on questions was superior. With the flashcard applications I recommended you consider, answering multiple-choice questions would be less challenging than answering short-answer questions (recognition vs recall).

Spacing (Distributed Practice): Instead of cramming, spreading out study sessions over time is more productive. This method helps improve long-term retention and understanding. Spacing allows some retrieval challenges to develop and the learner must work harder to locate the desired information in memory. See my earlier description of Bjork’s distinction between retrieval strength and storage strength.

Interleaving: Mixing different types of problems or subjects in one study session. For example, alternating between math problems and reading passages rather than focusing on one at a time. A simple flashcard version of this recommendation might be shuffling the deck between cycles through the deck. Breaking up the pattern of the review task increases the difficulty and requires greater cognitive effort.

Other thoughts

First, the concept of committing to more challenging tasks is broader than the well researched examples I provide here. Writing and teaching could be considered examples in that both tasks require an externalization of knowledge that is both generative and evaluative. It is too easy to fake it and make assumptions when the actual creation of a product is not required.

Second, desirable difficulty seems to me to be a guiding principle that does not explain all of the actual cognitive mechanisms that are involved. The specific mechanisms may vary with activity – some might be motivational, some evaluative (metacomprehension), and some at the level of basic cognitive activities. For example, creating retrieval challenges probably creates an attempt to find alternate or new connections among stored elements of information. For example, in trying to put a name with a face one might attempt to remember the circumstances in which you may have met or worked with this person and this may activate a connection you do not typically use and is not automatic. For example, after being retired for 10 years and trying to remember the names of coworkers, I sometimes remember the arrangement of our offices working my way down the appropriate hallway and this sometimes helps me recall names.

I did say I was going to return to the use of desirable difficulty as a justification for the advantage of taking notes by hand. If keyboarding allows faster data entry than handwriting, in theory keyboarding would allow more time for thinking, paraphrasing, and whatever advantage one would have when the recording method requires more time. Awareness and commitment would seem to be the issues here. However, I would think complete notes would have greater long-term value than sparse notes. One always has the opportunity to think while studying and a more complete set of notes would seem to provide the opportunity to have more external content to work with.

References:

Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.

Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971.

Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.

Willingham, D. T. (2021). Why don’t students like school?: A cognitive scientist answers questions about how the mind works and what it means for the classroom. John Wiley & Sons.

Potential conflicting benefits of your note-taking tool and approach

As I have explored and used several digital note-taking tools and examined the arguments that have been made regarding how such tools result in productivity benefits, I have identified a potential conflict in what produces more positive outcomes. The recognition of this conflict allows more purposeful execution on the part of the tool user and may better align activities with goals.

One way to identify note-taking goals is to use a long-standing approach differentiating generative and external storage benefits. This distinction was proposed long before PKM and was applied in the analysis of notes taken in classroom settings. The generative benefit proposes that the process of taking notes or sometimes of taking notes in a particular way engages our cognitive (mental) processes in ways that improve retention and understanding. External storage implies that our memory becomes less effective over time and having access to an external record (the notes) benefits our productivity. In practice (e.g., a student in a classroom) both benefits may apply, but one benefit depends on the other activity. Taking notes may not be beneficial, but to review notes one must have something to review. This is not always true as notes in one form or another can be provided or perhaps generated (for example AI identification of key ideas), but taking your own notes is by far the most common experience. In a PKM way of thinking, these two processes may function in different ways, but the classroom example should be familiar as a way to identify the theoretical benefits of note-taking.

I have written about the generative function of note-taking at length, but it is important to point out some unique specifics that apply to some digital note-taking tools. A source such as Ahrens’ Taking Smart Notes might provide the right mindset. I think of generative activities as external actions intended to produce a beneficial mental (cognitive) outcome. The idea is that external activities can encourage or change the likelihood of beneficial thinking behaviors. One way of operationalizing this perspective is to consider some of the specific activities Ahrens identified as external work resulting in such cognitive benefits. What are some of these activities? Isolating specific ideas and summarizing each as a note. Assigning tags that characterize a note. Making the effort to link notes. Periodically reviewing notes to generate retrieval practice, to reword existing notes, and to add new associations (links).

Retrieval is easier to explain. Note-taking apps with highly effective search capabilities make it easy to search and surface stored information when it might be useful. Links and tags may also be useful in this role, but search alone will often be sufficient.

What about the potential conflict?

The conflict I see proposes that some tools or approaches rely more heavily on search arguing in a way that generative processes are unnecessary.

I starting thinking about this assumption when contrasting the two note-taking systems I rely on – Mem.ai and Obsidian. While Mem.AI and Obsidian could be used in exactly the same way, Mem.ai developers argued that the built-in AI capabilities could eliminate the need to designate connections (with tags and links) because the AI capabilities would identify these connections for you. Thus when retrieving information via search, a user could use AI to also consider the notes with overlapping foci. If a user relied on this capability it would eliminate the work required to generate the connections manually created in Obsidian, but this approach would then also avoid the generative benefits of this work.

AI capabilities fascinate me so I found a way to add a decent AI capability to Obsidian. Smart Connections is an Obsidian plugin that finds connections among notes and allows a user to chat with their notes. So, I found a way to mimic Mem.ai functionality with Obsidian.

I find I have found a way to alter my more general PKM approach because of these capabilities. Rather than taking individual notes while reading, I can annotate and highlight pdfs, books, and videos and export the entire collection for each source and then bring this content into both Mem.ai and Obsidian as a very large note. Far easier than taking individual notes, but at what generative cost?

Smart Connections has added a new feature that even facilitates the use of the large note approach. Connections finds connections based on AI embeddings. An embedding is the mathematical representation of content (I would describe as weights based on what I remember of statistics). The more two notes embeddings’ weights are similar the more the notes consider similar ideas. Smart Connections used embeddings to propose related notes. Originally embeddings were generated at the note level and now at the “block” level. What this means (block level) is that Smart Connections can find the segments of a long document that have a similar focus as a selected note.

Why is this helpful? When I read long documents (pdfs of journal articles or books in Kindle), I can export a long document containing my highlights and notes generated from these documents. With Smart Connections I can then just import this exported material into Obsidian and use Smart Connections to connect a specific note to blocks of all such documents. I can skip breaking up the long document into individual notes and assigning tags and creating links.

Why is this a disadvantage? Taking advantage of this capability can be a powerful disincentive to engaging in the generative activities involved in creating and connecting individual notes the basic version of Obsidian requires.

Summary

As note-taking tools mature and add AI capabilities, it is important for users to consider how the way they use such tools can impact their learning and understanding. The tools themselves are quite flexible but can be used in ways that avoid generative tasks that impact learning and understanding. If the focus is on the retrieval of content for writing and other tasks, the generative activities may be less important. However, if you start using a tool such as Obsidian because a book such as Smart Notes influenced you, you might want to think about what might be happening if you rely on the type of AI capabilities I have described here.

References
Ahrens, S. (2022). How to take smart notes: One simple technique to boost writing, learning and thinking. Sönke Ahrens.

Does flipping the classroom improve learning?

The instructional strategy of “flipping the classroom” is one of those recommendations that seems on first consideration to make a lot of sense. The core idea hinges on the truth that classroom time with students is limited and efficient use must be made of this time. Instead of taking up a substantial amount of this time with teacher presentations, why not move the exposure to content outside of class time and use class time for more active tasks such as helping students who have problems and allowing students to engage in active tasks with other students? With easy access to tools for recording presentations and sharing recordings online, why not simply have educators share presentations with students and have students review this material before class? So, presentations were flipped from class time to settings that might have been more frequently used for homework.

This all seemed very rational. I cannot remember where I first encountered the idea, but I did purchase Flip Your Classroom (Bergman and Sams, 2012) written by the high school teachers who I believe created the concept. While I did use my blog and textbook to promote this approach, I must have always wondered. I wrote a blog post in 2012 commenting that flipping the classroom sounded very similar to my large lecture experience of presenting to hundreds of students and expecting that these students would have read the textbook before class. Again, the logic of following up an initial exposure with an anecdote-rich and expanded focus on key concepts seemed sound. However, I knew this was not the way many students used their textbooks and some probably did not even make the purchase, but I was controlling what I could control.

There have been hundreds of studies evaluating the flipping strategy and many meta-analyses of these studies. These meta-analyses tend to conclude that asking students to watch video lectures before coming to class is generally beneficial. I think many have slightly modified the suggested in-class component to expand the notion of greater teacher-student interaction to include a focus on active learning. Kapur et al (2022), authors of the meta-analysis I will focus on eventually, list the following experiences as examples of active learning – problem-solving, class discussions, dialog and debates, student presentations, collaboration, labs, games, and interactive and simulation-based learning activities.

The institution where I taught had a group very much interested in active learning and several special active learning “labs” were created to focus on these techniques. The labs contained tables instead of rows of chairs, whiteboards, and other adaptations. To teach a large class in this setting you had to submit a description of the active techniques you intended to implement. The largest classes (200+) I taught could not be accommodated in these rooms and I am not certain if I would have ever submitted a proposal anyway.

Kupar et al. (2022)

Kupar and colleagues found reason to add another meta-analysis to those already completed. While their integrated analysis of the meta-analytic papers concluded that the flipped classrooms have an advantage, Kapur and colleagues were puzzled by the great variability present among the studies. Some studies demonstrated a great advantage in student achievement for the flipped approach and some found that traditional instruction was superior. It did not seem reasonable that a basic underlying advantage would be associated with this much variability and the researchers proposed that a focus on the average effect size without consideration of the source or sources for this variability made little sense. They conducted their own meta-analysis and coded each study according to a variety of methodological and situational variables.

The most surprising finding from this approach was that the inclusion of active learning components was relatively inconsequential. Remember that the use of such strategies in the face-to-face setting was emphasized in many applications. Surprisingly, segments of lecture within the face-to-face setting were a better predictor of an achievement advantage. Despite the break from the general understanding of how flipped classrooms are expected to work, educators seemed to use these presentations to review or supplement independent student content consumption and this provided an achievement bump.

The active beneficial learning component found to make a difference involved a problem-based strategy and when the entire process began with a problem-based experience. This finding reminds me of the problem-based learning research conducted by Deanna Kuhn who also proposed that the problem-based experience start the learning sequence. Kapur used the phrase productive failure to describe the way struggling with a problem before encountering relevant background information was helpful. Kuhn emphasized a similar process without the catchy label and proposed the advantage was more a matter of the activation of relevant knowledge and guiding the interpretation of information within the presentation of content that followed.

Regarding the general perspective on the flipped model identified by Kapur and colleagues, their findings were less an indictment of the concept, but a demonstration of the lack of fidelity in implementations to the proposed advantage of using face-to-face time to interact and adjust to student needs. Increasing response to the needs of individual needs would seem beneficial and may be ignored in favor of activities that are less impactful.

References:

Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022, September). Fail, flip, fix, and feed–Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. In Frontiers in Education (Vol. 7, p. 956416). Frontiers.

Pease, M. A., & Kuhn, D. (2011). Experimental analysis of the effective components of problem?based learning. Science Education, 95(1), 57-86.

Wirkala. C. & Kuhn, D. (2011). Problem-Based Learning in K–12 Education: Is it Effective and How Does it Achieve its Effects? American Educational Research Journal, 48, 1157–1186