Could AI Replace Many of the Recommended PKM Processes?

How much of the “building” part of building a second brain is necessary? With the opportunity to direct AI at accumulated resources, how much of the linking, tagging, reviewing, and re-expressing notes is productive? Some version of this question has long bothered me. I studied classroom note-taking as an academic and became familiar with the massive research base associated with different ways to take and use notes, the cognitive and metacognitive processes associated with activities in different note-taking approaches, and individual differences in the capacity and motivation to execute specific processes effectively. While there is a great deal of speculation, multiple books, and the sharing of numerous personal processes in online posts, where is the research on PKM and Second Brains? The research on learners taking and using notes in academic settings may offer a general structure, however there would seem important differences in the use of self-directed note-taking outside of an environment that involves processes that must accomplish goals set by others to produce products or perform tasks after short time intervals (the time between exams or writing assignments).

The terminology is different. You do not find classroom-oriented research considering fleeting versus permanent notes or the ideal types of tags and links to organize notes and prepare the user for future goals and tasks that often do not exist when the notes are taken. I try to translate the proposed strategies outlined in popular books for the audience that takes notes to benefit them under these circumstances (Ahrens, Doto, and Forte). Can the research on generative cognitive processes, metacognitive comprehension monitoring, and retrieval practice serve to determine if the recommendations made by these writers are reasonable? Perhaps the strategies for creating different types of notes (fleeting vs. permanent), strategies for linking and tagging, and reviewing to find new associations are just busy work. 

Does AI provide an alternative?

MEM.AI was the first note-taking and storage tool I used that came with an embedded AI tool. The promotional material I read before investing time and money into MEM proposed that the availability of the embedded AI offered an alternative to the tags and links in a tool such as Obsidian. MEM.AI allows the manual assignment of something similar to tags and bi-directional links. The embedded AI did a couple of interesting things. As you built up a collection of notes, the AI offered suggestions based on the material you have accumulated. Similar notes were identified based on common elements of a new note and existing notes (see red box in the following image). You could link to suggested notes if you found the suggestions of value. Tags for a new note were proposed beyond any rags that were manually added (see red box in the following image) and any of these suggestions can be accepted. I have recently written about multiple ways note-taking systems can be augmented with AI, but MEM.AI is built with some of these processes assumed rather than potentially available through the creation of clever prompts. MEM.AI proposes that it creates a self-organizing workspace which seems a reasonable description.

MEM.AI has one more unique feature that seems a powerful use of the self-organizing workspace concept. There is a built-in word-processing capability called Smart Write. When you enter a forward slash / within any existing note or within an empty note, you can then enter a chat prompt. You can ask a question or make a request. With an existing note, the AI extends what already exists in what are often interesting and creative ways relying on what appears in the note you are working on and the corpus of notes you have saved. The output appears within that note and can be copied and pasted elsewhere or left as part of that note. It can be argued that this is a way to discover connections to other material more directly than what might have been manually indicated by creating a link to another specific note. 

Here is an example.

The following (Permanentnotes) is a note I created with two tags. What you can no longer see is the / and prompt that was originally inserted after the second tag. The prompt entered after the / was “Does my idea that permanent notes move more of the thinking involved in transforming notes into a final written product earlier in that writing process make sense?”  Smart Write added three paragraphs based on other stored notes.These paragraphs now appear in the original note (shown below). Not all of this content seems relevant, but connections were discovered and much of what was said could be used in writing about this concept. These additions were all based on previous content I added, but I did not have to manually find these connections.

Permanentnotes

This is a key insight. Permanent notes should be able to stand alone without reference to the original content for context. Perhaps it could be described as moving part of the writing process earlier in the transition from note-taking to a final written product.

#notetaking

#slipbox

Your thought about moving part of the writing process earlier in the transition from note-taking to a final writing product indeed makes sense and aligns with the ideas presented in the writing process model by Flower and Hayes (1981). The model emphasizes the importance of planning as a foundational step in effective writing, which involves setting goals, gathering information, and organizing ideas. By integrating permanent notes into this early stage, you essentially create a structured framework that facilitates the translation and reviewing phases of writing.

The approach of using permanent notes, akin to the slip-box system described by Ahrens, enhances this framework by allowing writers to develop a rich, interconnected repository of ideas and insights. This repository acts as an external structure that compensates for the cognitive limitations of the human brain, making it easier to retrieve and connect ideas when composing a draft. The slip-box method encourages a shift from merely capturing fleeting thoughts to actively engaging with and refining those ideas into coherent, stand-alone pieces of information.

Furthermore, this method aligns with the iterative nature of the writing process. As notes are continually reviewed and expanded, they can reveal gaps in understanding or suggest new lines of inquiry, prompting further research and reflection. This iterative engagement not only aids in the development of writing skills but also fosters deeper learning and insight generation, as noted by Ahrens. Thus, your suggestion to reposition part of the writing process earlier, in the context of note-taking and idea organization, is a strategic move that can significantly enhance both the efficiency and quality of writing.

Summary

I would argue that AI capabilities can substitute for or at least augment manual processes advocated by those arguing tags and links are an essential part of the Personal Knowledge Management process. To some extent, I have found the capabilities of the MEM self-organizing workspace to be more thorough in proposing connections than I would typically be on my own.

This brings me to an issue I consider in many applications of AI. This issue is whether the AI approach allows the user to avoid important activities that are important in and of themselves. Perhaps both the manual and AI approaches here allow a note-taker to produce comparable final written products if that was the goal. Whether the activities of exploring notes to make appropriate links and adding tags that identify key ideas encourage deeper thinking about the content seems a different issue. Again, this is difficult to know. One might make the opposite argument claiming that reviewing the text generated by the AI offers a different way to explore and find relationships within existing content and the process of considering what within this new material is relevant to the intended purpose of the prompt is a unique evaluative activity performed by the user. As I have already suggested, MEM.AI does not require that you take one approach or the other. So, explore. AI capabilities are being added to many note-taking tools and the potential is worth a look.

Loading

Desirable Difficulty

Despite a heavy focus on cognitive psychology in the way I researched and explained classroom study tactics, I had not encountered the phrase desirable difficulty until I became interested in the handwritten vs. keyboard notetaking research. I discovered the idea when reviewing studies by Luo and colleagues and Mueller and Oppenheimer. Several studies have claimed students are better off taking notes by hand in comparison to on a laptop despite being able to record information significantly faster when using a keyboard. 

Since having a more complete set of notes would seem an advantage. The combination of more notes associated with poorer performance is counterintuitive. Researchers speculated that learners who understood they had to make decisions about what they had time to record selected information more carefully and possibly summarized rather than recorded verbatim what they heard. This focus on what could be described as deeper processing seemed like an example of desirable difficulty. The researchers also proposed that the faster keyboard recording involved shallow cognitive processing.  

Note: I am still a fan of more complete notes and the methodology used when demonstrating better performance from recording notes by hand needs to be carefully considered. I will comment on my argument more at the end of this post. 

Desirable difficulty an idea attributed to Robert Bjork has been used to explain a wider variety of retention phenomena. Bjork suggested that retrieval strength and storage strength are distinct phenomena and learners can be misled when an approach to learning is evaluated based on retrieval strength. I find these phrases to a bit confusing as applied, but I understand the logic. Students cramming for an exam make a reasonable example. Cramming results in what may seem to be successful learning (retrieval strength), but results in poorer retention over an extended period of time (storage storage strength). Students may understand and accept the disadvantages of cramming so it is not necessary that the distinction be unrecognized by learners. In a more recent book on learning for the general public, Daniel Willingham suggests that the brain is really designed to avoid rather than embrace thinking because thinking is effortful. The human tendency is to rely on memory rather than thinking. Desirable difficulty may be a way to explain why some situations that require thinking prevent something more rote. 

Increasing difficulty to improve retention

There are multiple tactics for productively increasing difficulty that I tend to group under the heading of generative learning. I describe generative activities as external tasks intended to increase the probability of productive cognitive (mental) behaviors. I suppose desirable difficulty is even more specific differentiating external tasks along a difficulty dimension. So in the following list of tasks, it is useful to imagine more and less difficult tasks. Often the less difficult task is the option learners choose to apply. In connecting these tactics with personal experience, I would recommend you consider the use of flashcards to conceptualize what would be the easier and the more challenging application. Then, move beyond flashcards to other study tactics and consider if you can identify similar contrasts. 

Retrieval Practice: Testing oneself on the material rather than passively reviewing notes is considered retrieval practice. The classic empirical demonstration of the retrieval practice or the testing effect compared reviewing content versus responding to questions. Even when controlling for study time, spending some time on questions was superior. With the flashcard applications I recommended you consider, answering multiple-choice questions would be less challenging than answering short-answer questions (recognition vs recall).

Spacing (Distributed Practice): Instead of cramming, spreading out study sessions over time is more productive. This method helps improve long-term retention and understanding. Spacing allows some retrieval challenges to develop and the learner must work harder to locate the desired information in memory. See my earlier description of Bjork’s distinction between retrieval strength and storage strength. 

Interleaving: Mixing different types of problems or subjects in one study session. For example, alternating between math problems and reading passages rather than focusing on one at a time. A simple flashcard version of this recommendation might be shuffling the deck between cycles through the deck. Breaking up the pattern of the review task increases the difficulty and requires greater cognitive effort. 

Other thoughts

First, the concept of committing to more challenging tasks is broader than the well researched examples I provide here. Writing and teaching could be considered examples in that both tasks require an externalization of knowledge that is both generative and evaluative. It is too easy to fake it and make assumptions when the actual creation of a product is not required.

Second, desirable difficulty seems to me to be a guiding principle that does not explain all of the actual cognitive mechanisms that are involved. The specific mechanisms may vary with activity – some might be motivational, some evaluative (metacomprehension), and some at the level of basic cognitive activities. For example, creating retrieval challenges probably creates an attempt to find alternate or new connections among stored elements of information. For example, in trying to put a name with a face one might attempt to remember the circumstances in which you may have met or worked with this person and this may activate a connection you do not typically use and is not automatic. For example, after being retired for 10 years and trying to remember the names of coworkers, I sometimes remember the arrangement of our offices working my way down the appropriate hallway and this sometimes helps me recall names. 

I did say I was going to return to the use of desirable difficulty as a justification for the advantage of taking notes by hand. If keyboarding allows faster data entry than handwriting, in theory keyboarding would allow more time for thinking, paraphrasing, and whatever advantage one would have when the recording method requires more time. Awareness and commitment would seem to be the issues here. However, I would think complete notes would have greater long-term value than sparse notes. One always has the opportunity to think while studying and a more complete set of notes would seem to provide the opportunity to have more external content to work with. 

References:

Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J.  Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge,  MA: MIT Press.

Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971.

Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.

Willingham, D. T. (2021). Why don’t students like school?: A cognitive scientist answers questions about how the mind works and what it means for the classroom. John Wiley & Sons.

Loading

YouTube Annotation with Glasp

I take a lot of notes and have done so for years. I have tried many different tools over this time period. Social Annotation is a subcategory of these tools that allows users to share their highlights and notes. The idea is that the sharing of notes allows individuals to find resources they have not personally explored and offer their own discoveries to others. Glasp serves these purposes.

I have written about Glasp on several previous occasions. A unique capability allows built-in AI capabilities to “chat” not only with your own notes, but also the annotations stored by others.

Glasp is a combination of a Profile page that is the online location allowing access to the content you have collected (see above) and a browser extension that provides the means to highlight and annotate the content viewed within your browser. Kindle content is imported automatically. Glasp could provide the storage location for all of your notes, but I export notes to Obsidian to take advantage of more advanced features.

I don’t spend a lot of time collecting information from Youtube because most of writing is based on books and journal articles. There are exceptions when I review tutorials for software tools and want to keep track of specific tactics. I understand that others use YouTube extensively and I wanted to explore the capabilities of Glasp with this information source. The following video is my effort to describe how notes and highlights are generated from YouTube content.

Loading

Potential conflicting benefits of your note-taking tool and approach

As I have explored and used several digital note-taking tools and examined the arguments that have been made regarding how such tools result in productivity benefits, I have identified a potential conflict in what produces more positive outcomes. The recognition of this conflict allows more purposeful execution on the part of the tool user and may better align activities with goals.

One way to identify note-taking goals is to use a long-standing approach differentiating generative and external storage benefits. This distinction was proposed long before PKM and was applied in the analysis of notes taken in classroom settings. The generative benefit proposes that the process of taking notes or sometimes of taking notes in a particular way engages our cognitive (mental) processes in ways that improve retention and understanding. External storage implies that our memory becomes less effective over time and having access to an external record (the notes) benefits our productivity. In practice (e.g., a student in a classroom) both benefits may apply, but one benefit depends on the other activity. Taking notes may not be beneficial, but to review notes one must have something to review. This is not always true as notes in one form or another can be provided or perhaps generated (for example AI identification of key ideas), but taking your own notes is by far the most common experience. In a PKM way of thinking, these two processes may function in different ways, but the classroom example should be familiar as a way to identify the theoretical benefits of note-taking.

I have written about the generative function of note-taking at length, but it is important to point out some unique specifics that apply to some digital note-taking tools. A source such as Ahrens’ Taking Smart Notes might provide the right mindset. I think of generative activities as external actions intended to produce a beneficial mental (cognitive) outcome. The idea is that external activities can encourage or change the likelihood of beneficial thinking behaviors. One way of operationalizing this perspective is to consider some of the specific activities Ahrens identified as external work resulting in such cognitive benefits. What are some of these activities? Isolating specific ideas and summarizing each as a note. Assigning tags that characterize a note. Making the effort to link notes. Periodically reviewing notes to generate retrieval practice, to reword existing notes, and to add new associations (links).

Retrieval is easier to explain. Note-taking apps with highly effective search capabilities make it easy to search and surface stored information when it might be useful. Links and tags may also be useful in this role, but search alone will often be sufficient.

What about the potential conflict?

The conflict I see proposes that some tools or approaches rely more heavily on search arguing in a way that generative processes are unnecessary.

I starting thinking about this assumption when contrasting the two note-taking systems I rely on – Mem.ai and Obsidian. While Mem.AI and Obsidian could be used in exactly the same way, Mem.ai developers argued that the built-in AI capabilities could eliminate the need to designate connections (with tags and links) because the AI capabilities would identify these connections for you. Thus when retrieving information via search, a user could use AI to also consider the notes with overlapping foci. If a user relied on this capability it would eliminate the work required to generate the connections manually created in Obsidian, but this approach would then also avoid the generative benefits of this work. 

AI capabilities fascinate me so I found a way to add a decent AI capability to Obsidian. Smart Connections is an Obsidian plugin that finds connections among notes and allows a user to chat with their notes. So, I found a way to mimic Mem.ai functionality with Obsidian. 

I find I have found a way to alter my more general PKM approach because of these capabilities. Rather than taking individual notes while reading, I can annotate and highlight pdfs, books, and videos and export the entire collection for each source and then bring this content into both Mem.ai and Obsidian as a very large note. Far easier than taking individual notes, but at what generative cost?

Smart Connections has added a new feature that even facilitates the use of the large note approach. Connections finds connections based on AI embeddings. An embedding is the mathematical representation of content (I would describe as weights based on what I remember of statistics). The more two notes embeddings’ weights are similar the more the notes consider similar ideas. Smart Connections used embeddings to propose related notes. Originally embeddings were generated at the note level and now at the “block” level. What this means (block level) is that Smart Connections can find the segments of a long document that have a similar focus as a selected note. 

Why is this helpful? When I read long documents (pdfs of journal articles or books in Kindle), I can export a long document containing my highlights and notes generated from these documents. With Smart Connections I can then just import this exported material into Obsidian and use Smart Connections to connect a specific note to blocks of all such documents. I can skip breaking up the long document into individual notes and assigning tags and creating links.

Why is this a disadvantage? Taking advantage of this capability can be a powerful disincentive to engaging in the generative activities involved in creating and connecting individual notes the basic version of Obsidian requires. 

Summary

As note-taking tools mature and add AI capabilities, it is important for users to consider how the way they use such tools can impact their learning and understanding. The tools themselves are quite flexible but can be used in ways that avoid generative tasks that impact learning and understanding. If the focus is on the retrieval of content for writing and other tasks, the generative activities may be less important. However, if you start using a tool such as Obsidian because a book such as Smart Notes influenced you, you might want to think about what might be happening if you rely on the type of AI capabilities I have described here. 

References
Ahrens, S. (2022). How to take smart notes: One simple technique to boost writing, learning and thinking. Sönke Ahrens.

Loading

A use for Obsidian Unlinked Mentions

Have you had the experience of coming across an application feature and wondering why did a software designer decide to go to the trouble of creating and then shipping that feature? Somewhere I encountered a comment on an Obsidian feature called an Unlinked Mention. It took me some time to find it and then even more time in an effort to understand why it exists. I am still not certain how it is to be used and why there wouldn’t be similar features that would be more useful. I have come up with one way I find it offers some value so I will explain what seems a hack and then hope others can find my description helpful in encouraging similar or additional uses. 

Note: My description and proposed actions are based on Obsidian on a computer. Some of the actions I describe I could not get to work on my iPad. 

So, I think an unlinked mention is supposed to be understood as something like a backlink. In Obsidian when you create a link among two notes (A – B), Obsidian recognizes but does not automatically display the backlink (B-A). For a given note (A), you can get Obsidian to display any backlinks to that note using the backlinks option for the right-hand panel of the Obsidian display. For the note that is active in the middle panel, the right-hand panel should indicate linked mentions and unlinked mentions. You may have to select which you want displayed and it is possible nothing will be displayed for either option. The linked mentions are the backlinks and you can select and display the backlinked notes from this display. 

The unlinked mentions are other notes that contain the same exact phrase as you have used to title Note A. Who knew? Why? Maybe I never quite understood the power of a title or how my notes were supposed to be titled. I have tried to think about this and I still don’t get it.

Here is my hack and I think a way to take advantage of unlinked mentions. Start with a blank note and add a title likely to be used within other content you have stored within other notes. To make the effort, your word or phrase would have to be something you want to investigate. I used the word “metacognition” because this is an important concept in the applied cognitive psychology research I read and attempted to apply to educational uses of technology. I have notes about this concept, but the greatest value I found in this hack was taking advantage of all of the Kindle notes and highlights I had stored in Obsidian via Readwise. In my account, there are more than 200 books worth of notes and highlights and the content for each book is often several pages long.  I create notes myself as I read, but there is all of this additional content that may contain things I might find useful. Certainly, several of these books would contain content, especially highlights, focused on metacognition. 

Once I have my new note with the simple title “metacognition” and for this note look under unlinked mentions in the right-hand column, I now have lots of entries. At this point, my note is still blank, but I now can access many other mentions of metacognition from this list of unlinked mentions. If I select one of these mentions, a “link” button appears and if I select this button Obsidian generates a forward link in the A document and adds the A document to my blank B document as a backlink. The B note is still blank.

Here comes the hack. One of the core plugins for Obsidians is called backlink (use the gear icon from the panel on the left) and it contains a slider that will display backlinks at the bottom of a note (see following image). Now you can display backlinks on your blank note that allow access to the unlinked content you have linked. See the second image below.

The process I have described is a way to generate a collection of links on a topic that would not be available without this hack. It is the process that finds specific mentions of a concept within much larger bodies of content (the highlights from Kindle books) that I find useful. Give it a try.

Loading

Turning AI on my content

In reviewing the various ways I might use AI, I am starting to see a pattern. There are uses others are excited about that are not relevant to my life. There are possible uses that are relevant, but I prefer to continue doing these things myself because I either enjoy the activity or feel there is some personal benefit beyond the completion of a given project. Finally, there are some tasks for which AI serves a role that augments my capabilities and improves the quality or quantity of projects I am working on. 

At this time, the most beneficial way I use AI is to engage an AI tool in discussing a body of content I have curated or created as notes and highlights in service of a writing project I have taken on. There are two capabilities here that are important. First, I value the language skills of an AI service, but I want the service to use this capability only as a way to communicate with me about the content I designate. I am not certain I know exactly what this means as it would be similar to saying to an expert with whom I was interacting tell me about these specific sources without adding in ideas from sources I have not asked you to explore. Use your general background, but use this background only as a way to explain what these specific sources are proposing. What I mean is don’t add in stuff to address my prompt that does not exist within the sources I gave you.

Second, if I ask an AI service about the content I have provided, I want the service to be able to identify the source and possibly the specific material within a source that was the basis for a given position taken. Think of this expectation as similar to the expectation one might have in reading a scientific article to which the author provides citations for specific claims made. My desire here is to be able to evaluate such claims myself. I have a concern in simply basing a claim on the language of sources not knowing the methodology responsible for producing data used as a basis for a claim. For serious work, you need to read more than the abstract. Requiring a precise methodology section in research papers is important because the methodology establishes the context responsible for the generation of the data and ultimately the conclusions that are reached. Especially in situations in which I disagree with such conclusions, I often wonder if the methodology applied may explain the differences between my expectations and the conclusions reached by the author. Human behavior is complex and variables that influence behavior are hardly ever completely accounted for in research. Researchers do not really lie with statistics, but they can mislead by broad conclusions they share based on a less-than-perfect research method. There are no perfect research methods hence the constant suggestion that more research is needed. 

Several services approximate the characteristics I am looking for. I will identify three such services. I had hoped to add a fourth, but I intended to subscribe to the new OpenAI applications recently announced, but the $20 a month subscription fee necessary to use these functions was recently suspended so I will have to wait to explore these functions until OpenAI decides to expand the user base. 

The three services I have worked with include NotebookLM, Mem.ai, and Smart Connections with Obisidan. I have written about Mem.ai and Smart Connections in previous posts, so I will use NotebookLM for extended comments and then offer quick examples produced by the other two services.

NotebookLM

NotebookLM recently was made available to me so I have had less experience using this tool than the other two I have included. 

At this time you can create a notebook based on 5 pdfs or 10 text segments. There is a limit to how much content these units of information can total, but the pdfs I used were research articles so this is a lot of content. In the following image, icons for the 5 pdfs appear in the column on the left of the screen. All of the pdfs have been selected for the interaction I intend with the AI tool and a check beside each pdf indicates this is the case. The window for submitting my prompts appears at the bottom of the image.

The second image shows the results from the prompt I have submitted (I will explain my interest in interacting with these pdfs following this description). The response to this query is provided above the prompt window (I will share the text) and below this window you will see that the summary was based on 10 “citations”. Selecting one of these citations brings up the content that now appears in the left-hand column. This window shows the content of the pdf and some summary comments about that document generated by the AI. The AI also proposes additional prompts a user might want to apply to that content. 

The task I asked all three services to address was related to a recent post I wrote describing generative cognitive tasks and comparisons among the impacts of these generative activities. The pdfs were research studies I had reviewed and were the basis for that post. I had detailed familiarity with this content from both reviewing and annotating this material and writing the previous post and I use this approach of first engaging a process without AI to provide a basis for commenting on what the AI provides. The prompt that follows is what I submitted to NotebookLM to see what would be generated. I will use a similar prompt with the other two AI services/

Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities.

NoteBook response

Generative learning is a type of learning in which students are actively engaged in creating new knowledge by interacting with the material. This is in contrast to passive learning, in which students simply listen to a lecture or read a textbook.

Generative learning activities are designed to encourage students to think critically about the material and to make connections between new information and their existing knowledge.

There is a significant body of research evidence that supports the effectiveness of generative learning activities. A meta-analysis of 500 studies found that generative learning activities resulted in better learning outcomes than traditional, passive learning methods (Coffey et al., 2003).

Some of the most effective generative learning activities include:

  • Summarizing: Students are asked to summarize a text in their own words. This activity helps students to identify the main ideas of the text and to understand the relationships between them.
  • Questioning: Students are asked to generate questions about a text. This activity helps students to think critically about the text and to engage with the material in a deeper way.
  • Discussion: Students are asked to discuss a text with their classmates. This activity helps students to share their understanding of the text and to learn from each other.

The relative effectiveness of different generative learning activities depends on the specific learning goals. For example, summarizing is a good activity for helping students to remember information, while questioning is a good activity for helping students to understand the material in a deeper way.

Generative learning activities can be used in a variety of settings, including K-12 classrooms, higher education, and corporate training. They are a valuable tool for helping students to learn and remember new information in a more effective way.

***

You will note in this material that NoteBook included a reference. This was not one of the pdfs I provided for review so I thought it might have been included in the reference section of one of the pdfs. I asked Notebook to provide a full citation and it generated the following. 

Coffey, J. E., & McDonald, J. (2003). The effects of generative learning activities on student achievement: A meta-analysis. Contemporary Educational Psychology, 28(4), 240-263. doi:10.1016/j.cedpsych.2003.09.001

While this citation looks authentic and has the proper APA format, it is made up. I tried searching Google Scholar and used my library online access to search the location in Contemporary Educational Psychology and no such article exists. Google asks for user feedback and I provided the fake citation in my evaluation. 

***

Mem.AI

My use of Mem.ai and Smart Connections require a little different approach. The content stored in these services results from my exporting notes and highlights from the research articles I read. Both services are directed at my entire collection of notes so this is different from what NotebookLM could access. However, the citations indicated that the output was based on very similar source material.

Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities.

Mem.ai response

Generative learning is a process where learners actively engage with material, creating new knowledge and connections. This approach involves strategies such as summarization, teaching, self-testing, and self-explaining, which require learners to select and organize relevant information, elaborate on the material based on personal knowledge, and integrate new information with existing knowledge 1^.

Fiorella and Mayer (2016) identified eight strategies that promote generative learning, including summarization, mapping, drawing, imagining, self-testing, self-explaining, teaching, and enacting. These strategies help learners to create a product based on existing knowledge 1^.

Brod (2021) focused on age differences in generative learning strategies and emphasized the importance of producing a meaningful product that goes beyond simple paraphrasing or retelling. For instance, highlighting does not result in new content, but summarization does. Brod’s review suggests that summarizing becomes useful from high school onwards, while testing is beneficial for all ages 2^.

Chi’s (2009) framework differentiates learning activities into active, constructive, and interactive categories. Active engagement with the material can activate existing knowledge and store new input, while constructive activities involve creating a product based on existing knowledge. Interactive activities involve engaging with others to produce a product that integrates external input with personal knowledge 1^.

Fiorella (2023) proposed a new framework of sense-making that distinguishes among three sense-making modes—explaining, visualizing, and enacting—that serve unique and complementary cognitive functions. The explaining mode generalizes knowledge, the visualizing mode organizes knowledge, and the enacting mode simulates knowledge 3^.

In conclusion, generative learning involves actively engaging with the material to create new connections and deeper understanding. The effectiveness of different generative learning activities can vary based on factors such as age and the specific learning context. However, strategies such as summarization, teaching, self-testing, and notetaking with revision and elaboration can promote generative learning and enhance understanding of the material.

Summarize my notes on generative learning Tue 4/18

Brod’s generative Fri 4/14

Fiorellaet2023 Fri 10/13

1^: Summarize my notes on generative learning

2^: Brod’s generative

3^: [Fiorellaet2023](https://mem.ai/m/cqo3CLWb5tkb8syut

At the end of each paragraph in this output, you see a number. The number corresponds to the citation at the bottom. About the three links that take a user to that note, you see the names of the notes. So, this output is carefully linked to sources. I will provide an example of what my notes look like in association with the output from Smart Connections, but each note contains a citation of the article from which I created that note. 

***

Smart Connections (within Obsidian)

Obsidian contains my notes from research articles I have read and also highlights from other sources provided to Obsidian through Readwise. I tell you this again because I do not have a comparison among AI services based on exactly the same base of resources. However, again, you will note similar sources are found. Note also that the prompt is a little different to get Smart Connections to include citations. 

Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities. Include links to the individual source notes within your summary

Response:

Based on your notes, generative learning involves actively making sense of new information by mentally reorganizing and integrating it with one’s prior knowledge (Fiorella and Mayer, 2016). It is a constructive process that requires learners to select important information, reorganize it, and relate it to what they already know (Fiorellaetal2016). 

Fiorella and Mayer (2016) identified eight strategies that promote generative learning: summarizing, mapping, drawing, imagining, self-testing, self-explaining, teaching, and enacting. Summarizing involves selecting the main ideas, organizing them into a logical structure, and integrating new information with prior knowledge (Fiorellaetal2016). It is more effective for relatively simple concepts and requires training to use effectively (Fiorellaetal2016). 

Mapping and drawing also involve transforming information into different forms of representation (Fiorellaetal2016). These strategies help learners build connections and organize the material (Fiorellaetal2016). 

Self-testing, or retrieval-based learning, requires learners to actively retrieve information from memory without access to the provided materials (Fiorellaetal2016). It strengthens memory and makes it more accessible in the future (EndresHigh). 

Self-explaining involves explaining the content of a lesson to oneself during learning (Fiorellaetal2016). It promotes meaningful learning outcomes when learners engage in reflective knowledge building (Fiorellaetal2016). 

Teaching, on the other hand, involves selecting relevant information, organizing it into a coherent structure, and elaborating on it by incorporating existing knowledge (Fiorellaetal2016). It sets itself apart from other generative strategies by focusing on helping others learn (Fiorellaetal2016). 

Based on the research by Fiorella and Mayer (2016), it is more appropriate to consider each generative strategy as suitable for certain types of learning and learning contexts rather than regarding one strategy as the most effective (Fiorellaetal2016). 

In a study by Koh, Lee, and Lim (2018), the learning benefits of teaching were investigated. The results showed that teaching improved learning outcomes when it involved retrieval practice, but not when learners used a prepared script (Generative > TeachingAsRetrieval). This supports the idea that retrieval practice is an effective generative learning activity. 

Overall, generative learning activities have been shown to be effective in promoting meaningful learning outcomes. However, the effectiveness of each activity may vary depending on the complexity of the material and the training provided to learners. It is important to consider the specific learning goals and contexts when selecting and implementing generative learning strategies (Fiorellaetal2016). 

Based on your notes, here are the individual notes that provide more information on each generative learning activity:

– [[Generate1]]: Provides an overview of generative learning theory and the eight strategies identified by Fiorella and Mayer (2016).

– [[Summarizing]]: Describes the summarizing strategy and its effectiveness in selecting main ideas and creating quality summaries.

– [[Teaching]]: Discusses the goal of teaching and its distinction from other generative strategies.

– [[Self-Testing]]: Explains the self-testing strategy and its role in retrieval-based learning.

– [[Promote generative learning]]: Provides a link to the article by Fiorella and Mayer (2016) that identifies the eight strategies and discusses their effectiveness.

– [[Fiorellaetal2016]]: Summarizes the key points from Fiorella and Mayer’s (2016) article on generative learning strategies.

– [[EndresHigh]]: Discusses the importance of task differences in retrieval practice and the effects of different types of recall tasks.

– [[Yang2021]]: Highlights a meta-analysis on the effectiveness of retrieval practice in classroom learning and raises questions about the mechanism and student perception of its benefits.

***

Note: The output of Smart Connections is in markdown and the terms included in double parentheses are links that connect to the source note. So, if the summary above was opened in an app that would interpret markdown, the text within the double parentheses would appear as a link and the link would take me to a file stored on my computer. The file is named Generate1.

Here is an example of one of the original notes that was identified as source material. 

Generative learning makes sense of new information by reorganizing it and relating it to existing knowledge. This position comes from Wittrock, but is similar to other theorists (Mayer, Piaget). This specific article identified eight learning strategies that promote generative learning and provides a review of research relevant to each strategy.

[[Summarizing]]

Mapping

Drawing

Imagining

[[Self-Testing]]

Self-Explaining

[[Teaching]]

Enacting

The first four strategies (summarizing, mapping, drawing, and imagining) involve changing the input into a different form of representation.

The final four strategies (self-testing, self-explaining, teaching, and answering practice questions) require additional elaboration. 

Fiorella, L., & Mayer, R. E. (2016). Eight ways to promote generative learning. _Educational Psychology Review, 28(4), 717-741.

***

Summary

Keeping in mind my recognition that the AI of the three AI services was applied to slightly different content, I would argue that Smart Connections and Mem.ai are presently more advanced than NotebookLM. Eventually, I assume a user will be able to direct NotebookLM at a folder of files so the volume of content would be identical. Google does acknowledge that Notebook is still in the early stages and access is limited to a limited number of individuals willing to test and provide feedback. The content generated by all of the services was reasonable, but NoteBook did hallucinate a reference. 

My experience in comparing services indicates it is worth trying several in the completion of a given task. I have found it productive to keep both Smart Connections and Mem.ai around as the one I find most useful seems to vary. I do pay to use both services.

Loading

Cornell Notes and Beyond

While a research assistant at Cornell, Walter Pauk was credited with the development of the Cornell Note-taking system. Cornell notes became widely known through Pauk’s popular book “How to study in college” first published in 1962 and available through multiple editions. I checked and Amazon still carries the text.

Pauk’s approach which can be applied within a traditional notebook involves dividing a page into two columns with the right-hand column about twice as wide as the left-hand column and leaving a space across the bottom of each page for writing a summary. The idea is to take notes during a presentation in the right-hand column and later follow-up in the left-hand (often called cue column) with questions and other related comments. This second pass is supposed to follow soon after class so that other memories of the presentation are still fresh. The summary section provides a space to add just what it says – a summary of the main ideas.

Paul explained the proper way to use his system as the five Rs of note-taking. In my experience, the 5 Rs are far less well-known and yet important because they explain how the basic system is to be used. I would organize and explain the 5 Rs as follows.

During class – Record

After class – Reduce

Over time 

Recite (cover notes and see what you can recall based on cues)

Reflect (add your own ideas, extensions)

Review (review all notes each week)

While the Cornell system was designed during a different time and was suited to the technology of the day (paper and pencil), those who promote digital note-taking tools offer suggestions for applying the Cornell structure within the digital environment of the tool they promote. 

Cornell notes within Obsidian 

Cornell notes within Notion 

When I used to lecture about study skills and study behavior, I explained the Cornell system, but I would preface my presentation with the following questions. How many of you have heard of Cornell Notes? The SQ3R system? More had heard of Cornett notes and a few of SQ3R. I would then ask are any of you now using either of these systems to study my presentations or your textbook. In the thousands of students I asked, I don’t remember anyone ever raising her or his hand. To test my approach, I also asked if any student made and study note cards in their classes. The positive responses here were much more frequent. I tried to get a sense of why without much luck. I think my data are accurate and I raise this experience to get you to consider this same question. Students take notes, but don’t have a system.

I think Cornell notes are frequently proposed and taught to younger learners because the design of the note collection environment is simple and easy to describe. I wonder about how the process is communicated and perhaps more importantly implemented. The structure makes less sense if students are only intending to cram rather than frequently review. Does the learner have to “buy in” to the logic or do learners understand the logic, but just are not motivated to put in the effort? How any method is taught and understood likely has at least some impact on whether suggestions are implemented.

Understanding Cornell Notes at a deeper level

Note-taking has always been a personal interest and my posts have frequently commented on note-taking. I may have mentioned Cornell notes in a few of these posts, but my focus tends to be on a more basic level. If I am describing a system, what about specific components of that system have known cognitive benefits to learners? 

I come to the interpretations of those advocating specific study strategies from a cognitive perspective trying to analyze those strategies from this perspective. I ask what about a given study strategy seems like it makes sense given what those who study human cognition have found that benefits learning, retention, and transfer (application). What in a given study strategy could be augmented or given additional emphasis based on principles proposed by cognitive researchers? I will now try to apply this strategy to Cornell notes. I don’t know enough about Pauk’s work to know his theoretical perspective when creating this approach. For the most part, the perspective I take in my analysis has followed Pauk’s work which occurred during the 1950s. Timelines in this regard do not require that research precede practice, but there is a possibility that new research may offer new suggestions,

Topics

My comments will be organized as three topics.

  1. Stages of study behavior – how should the activities intended to benefit learning occur over time. What should be done when?
  1. Generative experiences and a hierarchy of such experiences – My explanation of a generative activity is an external activity intended to encourage a productive cognitive behavior. By hierarchy, I am pointing to research that has attempted to identify more and less effective generative activities and explain what factors are responsible for this ranking.
  1. Retrieval practice / testing effect – Research demonstrates that activities requiring the recall of stored information increases the probably of future recall and also increases understanding. Testing – free recall, cued recall, and recognition tasks – are common, but not the only or necessarily the most effective ways to engage retrieval effort.

Stages of study behavior

My personal interest in note-taking can be traced to the insights of Di Vesta and Gray. These researchers actually differentiated functions – encoding and external storage, but these processes were really centered within the stages of taking notes and then review. Encoding interpreted more broadly can occur at multiple points in time and this is my point in recognizing stages.

Pauk clearly recognized stages of study in proposing that learners function according to the 5Rs. The original notes were to be interpreted, augmented, and reviewed several times between the original recording and the immediate preparation for use. 

Luo and colleagues proposed that notetaking should be imagined as a three-stage process with a revision or update stage recognized after notetaking and before final preparation for use. In addition to recognizing the importance of following up to improve the original record, these researchers advocated for collaboration with a partner. Students do not take complete notes and the opportunity to compare notes taken with others allows for improvements. Research included in the paper points to the percentage of important ideas missed in the notes most record. The authors propose that lectures pause during presentations to provide an opportunity for comparison.

This source describes studies with college students using this pause and update method. Students were given two colored pens so additions could be identified. The pause and improve condition generated a significant achievement advantage (second study). However, this study found no benefit when comparing taking notes with a partner vs alone. Researchers looked at notes added and found few elaborations.

In an even more recent focus on multiple stages as part of a model for building a second brain, Forte described a process called distillation or progressive summarization.  In this process focused on taking notes from written sources, original content is read using an app that allows the exportation of the highlighted material. This content is first bolded and then highlighted to identify key information (progressive distillation). A summary can then be added. The unique advantage in this approach is to keep all of the layers available. One can function at different levels from the same immediate source and backtrack to a more complete level should it become necessary to recall a broader context or to take what was originally created in a different direction. 

It is possible to draw parallels here between what the Cornell system allows and what Forte proposes. The capability of reinstating context and addressing information missing from the original notes is also an advantage of the digital recording of an audio input keyed to specific notes as they are taken (see SoundNote). 

Di Vesta, F. & Gray, S. G. (1972). Listening and note taking. _Journal of Educational Psychology, 63_(1), 8-14.

Forte, T. (2022). Building a second brain: A proven method to organize your digital life and unlock your creative potential. Atria Books.

Luo, L., Kiewra, K. A., & Samuelson, L. (2016). Revising lecture notes: How revision, pauses, and partners affect note taking and achievement. Instructional Science, 44(1), 45-67.

Hierarchy of generative tasks

Again, a generative experience is an external activity intended to encourage productive activities. These productive activities may occur without any external tasks and this would be best situation because there is overhead in implementing the external tasks. However, for many learners and for most under some situations, the external tasks require cognitive activities that may be avoided or remain unrecognized as a function of poor metacognition or lack of motivation.

Many tasks initiated by a learner or educator can function as a generative function. Fiorella and Mayer (2016) have identified a list of eight general categories most educators can probably turn into specific tasks. These categories include:

  • Summarizing
  • Mapping
  • Drawing
  • Imagining
  • Self-Testing
  • Self-Explaining
  • Teaching
  • Enacting

Immediately, summarization can be identified from this list as being included in the Cornell system. Self-testing would also be involved in the way Pauk described recitation.

What I mean by a hierarchy as applied to generative activities is that some activities are typically more effective than others. 

Chi offers a framework – active-constructive-interactive – to differentiate learning activities in terms of observable overt activities and underlying learning processes. Each stage in the framework assumes the integration of the earlier stage and is assumed more productive than the earlier stage.

Active – doing something physical that can be observed. Highlighting would be another example.

Constructive – creating a **product** that extends the input based on **what is already known**. For example, summarization.

Interactive – involves interaction with another person – expert/learner, peers – to produce a product.

One insight from this scheme is that there is a stage beyond what might seem to be the upper limit of the Cornell structure (i.e., summarization). I am tempted to describe this additional level as application or perhaps elaboration. Both terms to me imply using information.  

Chi, M. T. (2009). Active?constructive?interactive: A conceptual framework for differentiating learning activities. Topics in cognitive science, 1(1), 73-105.

Fiorella, L., & Mayer, R. (2016). Eight Ways to Promote Generative Learning. Educational Psychology Review, 28(4), 717-741.

Retrieval Practice

Retrieval practice is a learning technique that involves trying to recall information from memory (see also Roediger & Karpicke). There are several reasons why retrieval practice improves future retrieval, but also understanding. First, it forces learners to actively engage with the material. This helps to create stronger connections between the information and existing knowledge. I think of retrieval as looking externally into memory to try to find something connected to what I am searching to find. This makes sense if you understand memory as a web of connections among ideas. The efforts to find specific information results in the activation and awareness of other information in order to find a connection to what is desired.Exploring retrieval not only increases the strength of connection to the desired information, but also an exploration of potentially related information resulting in new insights. 

Second, retrieval practice provides feedback on what has been learned and what needs more attention. This helps learners to identify areas where they need to improve. 

Retrieval practice is sometimes called the testing effect and asking questions or being asked questions is one way to trigger the search process (e.g., Yang and colleagues), Self testing is an activity embedded in the way Pauk imagines the use of Cornell notes. I am guessing it is also a reason the strategy of making and using flash cards is such a common study strategy. 

There are however other ways to practice retrieval. Yang and colleagues speculate that retrieval practice plays in role in the proven benefits of a learner teaching and preparing to teach. Teaching represents an important link here to the more productive levels of generative learning (see previous section). The previously mentioned hierarchy attributed to Luo and colleagues recognized the value of collaboration in reviewing notes and again the addition of sharing and discussion would represent important extensions of a personal use of any note-taking system. 

 Koh, A. W. L., Lee, S. C., & Lim, S. W. H. (2018). The learning benefits of teaching: A retrieval practice hypothesis. Applied Cognitive Psychology, 32(3), 401-410.

Luo, L., Kiewra, K. A., & Samuelson, L. (2016). Revising lecture notes: How revision, pauses, and partners affect note taking and achievement. Instructional Science, 44(1), 45-67.

Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1(3), 181-210.

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399-435.

Summary – My effort here was an attempt to cross reference what might be described as a learning system (Cornell Note) with mechanisms that might expain why the system has proven value and possibly allow the recognition of similar components present in other study systems. In addition, I have tried to emphasize that the components of a system may not be understood and applied in practice. Collaboration was suggested as a way to extend the Cornell system.

Loading