I have found myself exploring and writing about the interrelated topics of personal knowledge management, second brains, and note-taking for the past several years. As I have spent time on these interests, it became obvious that there were multiple disciplinary perspectives on these topics. In addition, the different disciplines seem mostly oblivious to each other as indicated by the lack of cross-referencing evident in their written materials. There are sometimes references to historical connections which I will identify, but for anyone interested in these topics I would suggest there are benefits for exploring more than a single point of view.
The Perspectives
Here are the descriptive labels I have decided to use for what I claim to be different perspectives. Hopefully, the labels offer some insights into the categories I have in mind.
Academic studying – this perspective provided my personal background for this general topic. The focus of this perspective is learning in formal academic environments with the goals of the acquisition, understanding, and application of information to examinations and projects. While the general goal of education is focused on the long term and preparation for life, note-taking has a more immediate focus. I am of the opinion that the great majority of what I would describe as research is focused on topics within this category. Most of this research is based on a cognitive perspective on learning and application.
Organizational Knowledge Management – Organizations have a need to develop, preserve, and apply knowledge. For multiple practical reasons (e.g., changeover in personnel), this knowledge should be externalized for the benefit of the organization. The generation and use of this shared knowledge originate with individuals. Personal knowledge management (PKM) can be individualized or integrated with the more general needs of a given organization. Procedures for accomplishing these goals are the subject of scholarship and training in the formal programs preparing individuals for careers in organizations (e.g., business schools), but it is my impression that scholarship is less empirical than that applied by those with an academic studying perspective and more anecdotal and based in logical argumentation.
Knowledge Management Entrepreneurs – I struggled with a way to describe this perspective. It seems to me that there has been a recent and identifiable group of individuals offering self-help books and consulting expertise to those interested in Personal Knowledge Management. This category resembles the organizational knowledge management perspective but does not share the same group focus. The perspective emphasizes the collection, organization, exploration, and application of information over an extended period of time to accomplish personal goals. Of the three groups I have identified, those individuals promoting techniques and processes are the least likely engaged in what I would describe as formal scholarship.
Historical Antecedents
While not absolutely consistent, there are frequent references to similar individuals, practices, and models that can often be identified among these perspectives. Here is my own list of such sources.
Vannevar Bush’s article “As we may think” describing the manner in which individuals and organizations might use a yet-to-be-developed technology (the Memex) to take on information overload and how a knowledge worker might explore, retain, organize, and apply information.
Commonplace books are journals, diaries, or notebooks maintained by individuals. A famous historical example would be the Leonardo Di Vinci notebooks still available in different formats (Amazon source).
Luhmann’s Zettelkasten. A zettelkasten is a card-based note-taking and note-linking system now often adapted to digitization and computer applications. It did not originate with Nikolas Luhmann, but I have connected the approach with his name because his prodigious use of the system as a scholar seems the example so many use.
The encoding and external model of note-taking (e.g., Rickards & Friedman, 1978) is the basis for much of the empirical research from the academic studying perspective. It proposes that learners could possibly benefit from both the thinking required in taking notes (the encoding process) and/or by having an external record available for review (external storage). This basic differentiation has been applied to such topics as whether taking notes by hand is more or less effective than taking notes using a keyboard (encoding), the best ways to work with the external notes (e.g., retrieval practice), and individual differences in both what is stored and how what is stored is used. For example, the Cornell note-taking method is an example of a system for both taking and using notes.
Examples from the different perspectives
I have written extensively about a couple of these perspectives in previous posts so rather than repeat myself and increase the length of this post I will link to some of these earlier posts.
Organizational Knowledge Management – this perspective is a little more challenging as I have not written about it before. Here is a source you can explore without having journal access – Towards a Co-evolution of Organizational and Personal Knowledge Management Systems. Also see Pauleen (2009) – this is the introduction to a special issue on personal knowledge management.
Why consideration of the different perspectives might be useful?
Having asked you to recognize the multiple perspectives that I have identified, I owe you some explanation for why I think anyone interested in taking notes should expand their awareness of the background content available on this topic. I have found a couple of personal opportunities. First, the work from the perspective of academic studying has been far more carefully evaluated and useful in answering questions of why and if specific activities work. The knowledge management entrepreneurs offer specific “how to do it” suggestions and have strongly promoted the use of technology tools in PKM. The organizational knowledge management perspective extends the note-taking and PKM for life-long learning expanding core ideas beyond the academic classroom setting.
The links I provide here should open to many other resources on the perspectives I have identified.
References not linked
Pauleen, David (2009), “Personal knowledge management: putting the ‘person’ back into the knowledge equation”, Online Information Review, vol. 33, no. 2, pp. 221–224, doi:10.1108/14684520910951177.
Rickards, J. P., & Friedman, F. (1978). The encoding versus the external storage hypothesis in note taking. Contemporary Educational Psychology, 3(2), 136-143.
How much of the “building” part of building a second brain is necessary? With the opportunity to direct AI at accumulated resources, how much of the linking, tagging, reviewing, and re-expressing notes is productive? Some version of this question has long bothered me. I studied classroom note-taking as an academic and became familiar with the massive research base associated with different ways to take and use notes, the cognitive and metacognitive processes associated with activities in different note-taking approaches, and individual differences in the capacity and motivation to execute specific processes effectively. While there is a great deal of speculation, multiple books, and the sharing of numerous personal processes in online posts, where is the research on PKM and Second Brains? The research on learners taking and using notes in academic settings may offer a general structure, however there would seem important differences in the use of self-directed note-taking outside of an environment that involves processes that must accomplish goals set by others to produce products or perform tasks after short time intervals (the time between exams or writing assignments).
The terminology is different. You do not find classroom-oriented research considering fleeting versus permanent notes or the ideal types of tags and links to organize notes and prepare the user for future goals and tasks that often do not exist when the notes are taken. I try to translate the proposed strategies outlined in popular books for the audience that takes notes to benefit them under these circumstances (Ahrens, Doto, and Forte). Can the research on generative cognitive processes, metacognitive comprehension monitoring, and retrieval practice serve to determine if the recommendations made by these writers are reasonable? Perhaps the strategies for creating different types of notes (fleeting vs. permanent), strategies for linking and tagging, and reviewing to find new associations are just busy work.
Does AI provide an alternative?
MEM.AI was the first note-taking and storage tool I used that came with an embedded AI tool. The promotional material I read before investing time and money into MEM proposed that the availability of the embedded AI offered an alternative to the tags and links in a tool such as Obsidian. MEM.AI allows the manual assignment of something similar to tags and bi-directional links. The embedded AI did a couple of interesting things. As you built up a collection of notes, the AI offered suggestions based on the material you have accumulated. Similar notes were identified based on common elements of a new note and existing notes (see red box in the following image). You could link to suggested notes if you found the suggestions of value. Tags for a new note were proposed beyond any rags that were manually added (see red box in the following image) and any of these suggestions can be accepted. I have recently written about multiple ways note-taking systems can be augmented with AI, but MEM.AI is built with some of these processes assumed rather than potentially available through the creation of clever prompts. MEM.AI proposes that it creates a self-organizing workspace which seems a reasonable description.
MEM.AI has one more unique feature that seems a powerful use of the self-organizing workspace concept. There is a built-in word-processing capability called Smart Write. When you enter a forward slash / within any existing note or within an empty note, you can then enter a chat prompt. You can ask a question or make a request. With an existing note, the AI extends what already exists in what are often interesting and creative ways relying on what appears in the note you are working on and the corpus of notes you have saved. The output appears within that note and can be copied and pasted elsewhere or left as part of that note. It can be argued that this is a way to discover connections to other material more directly than what might have been manually indicated by creating a link to another specific note.
Here is an example.
The following (Permanentnotes) is a note I created with two tags. What you can no longer see is the / and prompt that was originally inserted after the second tag. The prompt entered after the / was “Does my idea that permanent notes move more of the thinking involved in transforming notes into a final written product earlier in that writing process make sense?” Smart Write added three paragraphs based on other stored notes.These paragraphs now appear in the original note (shown below). Not all of this content seems relevant, but connections were discovered and much of what was said could be used in writing about this concept. These additions were all based on previous content I added, but I did not have to manually find these connections.
Permanentnotes
This is a key insight. Permanent notes should be able to stand alone without reference to the original content for context. Perhaps it could be described as moving part of the writing process earlier in the transition from note-taking to a final written product.
#notetaking
#slipbox
Your thought about moving part of the writing process earlier in the transition from note-taking to a final writing product indeed makes sense and aligns with the ideas presented in the writing process model by Flower and Hayes (1981). The model emphasizes the importance of planning as a foundational step in effective writing, which involves setting goals, gathering information, and organizing ideas. By integrating permanent notes into this early stage, you essentially create a structured framework that facilitates the translation and reviewing phases of writing.
The approach of using permanent notes, akin to the slip-box system described by Ahrens, enhances this framework by allowing writers to develop a rich, interconnected repository of ideas and insights. This repository acts as an external structure that compensates for the cognitive limitations of the human brain, making it easier to retrieve and connect ideas when composing a draft. The slip-box method encourages a shift from merely capturing fleeting thoughts to actively engaging with and refining those ideas into coherent, stand-alone pieces of information.
Furthermore, this method aligns with the iterative nature of the writing process. As notes are continually reviewed and expanded, they can reveal gaps in understanding or suggest new lines of inquiry, prompting further research and reflection. This iterative engagement not only aids in the development of writing skills but also fosters deeper learning and insight generation, as noted by Ahrens. Thus, your suggestion to reposition part of the writing process earlier, in the context of note-taking and idea organization, is a strategic move that can significantly enhance both the efficiency and quality of writing.
Summary
I would argue that AI capabilities can substitute for or at least augment manual processes advocated by those arguing tags and links are an essential part of the Personal Knowledge Management process. To some extent, I have found the capabilities of the MEM self-organizing workspace to be more thorough in proposing connections than I would typically be on my own.
This brings me to an issue I consider in many applications of AI. This issue is whether the AI approach allows the user to avoid important activities that are important in and of themselves. Perhaps both the manual and AI approaches here allow a note-taker to produce comparable final written products if that was the goal. Whether the activities of exploring notes to make appropriate links and adding tags that identify key ideas encourage deeper thinking about the content seems a different issue. Again, this is difficult to know. One might make the opposite argument claiming that reviewing the text generated by the AI offers a different way to explore and find relationships within existing content and the process of considering what within this new material is relevant to the intended purpose of the prompt is a unique evaluative activity performed by the user. As I have already suggested, MEM.AI does not require that you take one approach or the other. So, explore. AI capabilities are being added to many note-taking tools and the potential is worth a look.
Despite a heavy focus on cognitive psychology in the way I researched and explained classroom study tactics, I had not encountered the phrase desirable difficulty until I became interested in the handwritten vs. keyboard notetaking research. I discovered the idea when reviewing studies by Luo and colleagues and Mueller and Oppenheimer. Several studies have claimed students are better off taking notes by hand in comparison to on a laptop despite being able to record information significantly faster when using a keyboard.
Since having a more complete set of notes would seem an advantage. The combination of more notes associated with poorer performance is counterintuitive. Researchers speculated that learners who understood they had to make decisions about what they had time to record selected information more carefully and possibly summarized rather than recorded verbatim what they heard. This focus on what could be described as deeper processing seemed like an example of desirable difficulty. The researchers also proposed that the faster keyboard recording involved shallow cognitive processing.
Note: I am still a fan of more complete notes and the methodology used when demonstrating better performance from recording notes by hand needs to be carefully considered. I will comment on my argument more at the end of this post.
Desirable difficulty an idea attributed to Robert Bjork has been used to explain a wider variety of retention phenomena. Bjork suggested that retrieval strength and storage strength are distinct phenomena and learners can be misled when an approach to learning is evaluated based on retrieval strength. I find these phrases to a bit confusing as applied, but I understand the logic. Students cramming for an exam make a reasonable example. Cramming results in what may seem to be successful learning (retrieval strength), but results in poorer retention over an extended period of time (storage storage strength). Students may understand and accept the disadvantages of cramming so it is not necessary that the distinction be unrecognized by learners. In a more recent book on learning for the general public, Daniel Willingham suggests that the brain is really designed to avoid rather than embrace thinking because thinking is effortful. The human tendency is to rely on memory rather than thinking. Desirable difficulty may be a way to explain why some situations that require thinking prevent something more rote.
Increasing difficulty to improve retention
There are multiple tactics for productively increasing difficulty that I tend to group under the heading of generative learning. I describe generative activities as external tasks intended to increase the probability of productive cognitive (mental) behaviors. I suppose desirable difficulty is even more specific differentiating external tasks along a difficulty dimension. So in the following list of tasks, it is useful to imagine more and less difficult tasks. Often the less difficult task is the option learners choose to apply. In connecting these tactics with personal experience, I would recommend you consider the use of flashcards to conceptualize what would be the easier and the more challenging application. Then, move beyond flashcards to other study tactics and consider if you can identify similar contrasts.
Retrieval Practice: Testing oneself on the material rather than passively reviewing notes is considered retrieval practice. The classic empirical demonstration of the retrieval practice or the testing effect compared reviewing content versus responding to questions. Even when controlling for study time, spending some time on questions was superior. With the flashcard applications I recommended you consider, answering multiple-choice questions would be less challenging than answering short-answer questions (recognition vs recall).
Spacing (Distributed Practice): Instead of cramming, spreading out study sessions over time is more productive. This method helps improve long-term retention and understanding. Spacing allows some retrieval challenges to develop and the learner must work harder to locate the desired information in memory. See my earlier description of Bjork’s distinction between retrieval strength and storage strength.
Interleaving: Mixing different types of problems or subjects in one study session. For example, alternating between math problems and reading passages rather than focusing on one at a time. A simple flashcard version of this recommendation might be shuffling the deck between cycles through the deck. Breaking up the pattern of the review task increases the difficulty and requires greater cognitive effort.
Other thoughts
First, the concept of committing to more challenging tasks is broader than the well researched examples I provide here. Writing and teaching could be considered examples in that both tasks require an externalization of knowledge that is both generative and evaluative. It is too easy to fake it and make assumptions when the actual creation of a product is not required.
Second, desirable difficulty seems to me to be a guiding principle that does not explain all of the actual cognitive mechanisms that are involved. The specific mechanisms may vary with activity – some might be motivational, some evaluative (metacomprehension), and some at the level of basic cognitive activities. For example, creating retrieval challenges probably creates an attempt to find alternate or new connections among stored elements of information. For example, in trying to put a name with a face one might attempt to remember the circumstances in which you may have met or worked with this person and this may activate a connection you do not typically use and is not automatic. For example, after being retired for 10 years and trying to remember the names of coworkers, I sometimes remember the arrangement of our offices working my way down the appropriate hallway and this sometimes helps me recall names.
I did say I was going to return to the use of desirable difficulty as a justification for the advantage of taking notes by hand. If keyboarding allows faster data entry than handwriting, in theory keyboarding would allow more time for thinking, paraphrasing, and whatever advantage one would have when the recording method requires more time. Awareness and commitment would seem to be the issues here. However, I would think complete notes would have greater long-term value than sparse notes. One always has the opportunity to think while studying and a more complete set of notes would seem to provide the opportunity to have more external content to work with.
References:
Bjork, R.A. (1994). Memory and metamemory considerations in the training of human beings. In J. Metcalfe & A. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 185-205). Cambridge, MA: MIT Press.
Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971.
Mueller, P. A., & Oppenheimer, D. M. (2014). The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science, 25(6), 1159-1168.
Willingham, D. T. (2021). Why don’t students like school?: A cognitive scientist answers questions about how the mind works and what it means for the classroom. John Wiley & Sons.
I take a lot of notes and have done so for years. I have tried many different tools over this time period. Social Annotation is a subcategory of these tools that allows users to share their highlights and notes. The idea is that the sharing of notes allows individuals to find resources they have not personally explored and offer their own discoveries to others. Glasp serves these purposes.
Glasp is a combination of a Profile page that is the online location allowing access to the content you have collected (see above) and a browser extension that provides the means to highlight and annotate the content viewed within your browser. Kindle content is imported automatically. Glasp could provide the storage location for all of your notes, but I export notes to Obsidian to take advantage of more advanced features.
I don’t spend a lot of time collecting information from Youtube because most of writing is based on books and journal articles. There are exceptions when I review tutorials for software tools and want to keep track of specific tactics. I understand that others use YouTube extensively and I wanted to explore the capabilities of Glasp with this information source. The following video is my effort to describe how notes and highlights are generated from YouTube content.
As I have explored and used several digital note-taking tools and examined the arguments that have been made regarding how such tools result in productivity benefits, I have identified a potential conflict in what produces more positive outcomes. The recognition of this conflict allows more purposeful execution on the part of the tool user and may better align activities with goals.
One way to identify note-taking goals is to use a long-standing approach differentiating generative and external storage benefits. This distinction was proposed long before PKM and was applied in the analysis of notes taken in classroom settings. The generative benefit proposes that the process of taking notes or sometimes of taking notes in a particular way engages our cognitive (mental) processes in ways that improve retention and understanding. External storage implies that our memory becomes less effective over time and having access to an external record (the notes) benefits our productivity. In practice (e.g., a student in a classroom) both benefits may apply, but one benefit depends on the other activity. Taking notes may not be beneficial, but to review notes one must have something to review. This is not always true as notes in one form or another can be provided or perhaps generated (for example AI identification of key ideas), but taking your own notes is by far the most common experience. In a PKM way of thinking, these two processes may function in different ways, but the classroom example should be familiar as a way to identify the theoretical benefits of note-taking.
I have written about the generative function of note-taking at length, but it is important to point out some unique specifics that apply to some digital note-taking tools. A source such as Ahrens’ Taking Smart Notes might provide the right mindset. I think of generative activities as external actions intended to produce a beneficial mental (cognitive) outcome. The idea is that external activities can encourage or change the likelihood of beneficial thinking behaviors. One way of operationalizing this perspective is to consider some of the specific activities Ahrens identified as external work resulting in such cognitive benefits. What are some of these activities? Isolating specific ideas and summarizing each as a note. Assigning tags that characterize a note. Making the effort to link notes. Periodically reviewing notes to generate retrieval practice, to reword existing notes, and to add new associations (links).
Retrieval is easier to explain. Note-taking apps with highly effective search capabilities make it easy to search and surface stored information when it might be useful. Links and tags may also be useful in this role, but search alone will often be sufficient.
What about the potential conflict?
The conflict I see proposes that some tools or approaches rely more heavily on search arguing in a way that generative processes are unnecessary.
I starting thinking about this assumption when contrasting the two note-taking systems I rely on – Mem.ai and Obsidian. While Mem.AI and Obsidian could be used in exactly the same way, Mem.ai developers argued that the built-in AI capabilities could eliminate the need to designate connections (with tags and links) because the AI capabilities would identify these connections for you. Thus when retrieving information via search, a user could use AI to also consider the notes with overlapping foci. If a user relied on this capability it would eliminate the work required to generate the connections manually created in Obsidian, but this approach would then also avoid the generative benefits of this work.
AI capabilities fascinate me so I found a way to add a decent AI capability to Obsidian. Smart Connections is an Obsidian plugin that finds connections among notes and allows a user to chat with their notes. So, I found a way to mimic Mem.ai functionality with Obsidian.
I find I have found a way to alter my more general PKM approach because of these capabilities. Rather than taking individual notes while reading, I can annotate and highlight pdfs, books, and videos and export the entire collection for each source and then bring this content into both Mem.ai and Obsidian as a very large note. Far easier than taking individual notes, but at what generative cost?
Smart Connections has added a new feature that even facilitates the use of the large note approach. Connections finds connections based on AI embeddings. An embedding is the mathematical representation of content (I would describe as weights based on what I remember of statistics). The more two notes embeddings’ weights are similar the more the notes consider similar ideas. Smart Connections used embeddings to propose related notes. Originally embeddings were generated at the note level and now at the “block” level. What this means (block level) is that Smart Connections can find the segments of a long document that have a similar focus as a selected note.
Why is this helpful? When I read long documents (pdfs of journal articles or books in Kindle), I can export a long document containing my highlights and notes generated from these documents. With Smart Connections I can then just import this exported material into Obsidian and use Smart Connections to connect a specific note to blocks of all such documents. I can skip breaking up the long document into individual notes and assigning tags and creating links.
Why is this a disadvantage? Taking advantage of this capability can be a powerful disincentive to engaging in the generative activities involved in creating and connecting individual notes the basic version of Obsidian requires.
Summary
As note-taking tools mature and add AI capabilities, it is important for users to consider how the way they use such tools can impact their learning and understanding. The tools themselves are quite flexible but can be used in ways that avoid generative tasks that impact learning and understanding. If the focus is on the retrieval of content for writing and other tasks, the generative activities may be less important. However, if you start using a tool such as Obsidian because a book such as Smart Notes influenced you, you might want to think about what might be happening if you rely on the type of AI capabilities I have described here.
References Ahrens, S. (2022). How to take smart notes: One simple technique to boost writing, learning and thinking. Sönke Ahrens.
Have you had the experience of coming across an application feature and wondering why did a software designer decide to go to the trouble of creating and then shipping that feature? Somewhere I encountered a comment on an Obsidian feature called an Unlinked Mention. It took me some time to find it and then even more time in an effort to understand why it exists. I am still not certain how it is to be used and why there wouldn’t be similar features that would be more useful. I have come up with one way I find it offers some value so I will explain what seems a hack and then hope others can find my description helpful in encouraging similar or additional uses.
Note: My description and proposed actions are based on Obsidian on a computer. Some of the actions I describe I could not get to work on my iPad.
So, I think an unlinked mention is supposed to be understood as something like a backlink. In Obsidian when you create a link among two notes (A – B), Obsidian recognizes but does not automatically display the backlink (B-A). For a given note (A), you can get Obsidian to display any backlinks to that note using the backlinks option for the right-hand panel of the Obsidian display. For the note that is active in the middle panel, the right-hand panel should indicate linked mentions and unlinked mentions. You may have to select which you want displayed and it is possible nothing will be displayed for either option. The linked mentions are the backlinks and you can select and display the backlinked notes from this display.
The unlinked mentions are other notes that contain the same exact phrase as you have used to title Note A. Who knew? Why? Maybe I never quite understood the power of a title or how my notes were supposed to be titled. I have tried to think about this and I still don’t get it.
Here is my hack and I think a way to take advantage of unlinked mentions. Start with a blank note and add a title likely to be used within other content you have stored within other notes. To make the effort, your word or phrase would have to be something you want to investigate. I used the word “metacognition” because this is an important concept in the applied cognitive psychology research I read and attempted to apply to educational uses of technology. I have notes about this concept, but the greatest value I found in this hack was taking advantage of all of the Kindle notes and highlights I had stored in Obsidian via Readwise. In my account, there are more than 200 books worth of notes and highlights and the content for each book is often several pages long. I create notes myself as I read, but there is all of this additional content that may contain things I might find useful. Certainly, several of these books would contain content, especially highlights, focused on metacognition.
Once I have my new note with the simple title “metacognition” and for this note look under unlinked mentions in the right-hand column, I now have lots of entries. At this point, my note is still blank, but I now can access many other mentions of metacognition from this list of unlinked mentions. If I select one of these mentions, a “link” button appears and if I select this button Obsidian generates a forward link in the A document and adds the A document to my blank B document as a backlink. The B note is still blank.
Here comes the hack. One of the core plugins for Obsidians is called backlink (use the gear icon from the panel on the left) and it contains a slider that will display backlinks at the bottom of a note (see following image). Now you can display backlinks on your blank note that allow access to the unlinked content you have linked. See the second image below.
The process I have described is a way to generate a collection of links on a topic that would not be available without this hack. It is the process that finds specific mentions of a concept within much larger bodies of content (the highlights from Kindle books) that I find useful. Give it a try.
In reviewing the various ways I might use AI, I am starting to see a pattern. There are uses others are excited about that are not relevant to my life. There are possible uses that are relevant, but I prefer to continue doing these things myself because I either enjoy the activity or feel there is some personal benefit beyond the completion of a given project. Finally, there are some tasks for which AI serves a role that augments my capabilities and improves the quality or quantity of projects I am working on.
At this time, the most beneficial way I use AI is to engage an AI tool in discussing a body of content I have curated or created as notes and highlights in service of a writing project I have taken on. There are two capabilities here that are important. First, I value the language skills of an AI service, but I want the service to use this capability only as a way to communicate with me about the content I designate. I am not certain I know exactly what this means as it would be similar to saying to an expert with whom I was interacting tell me about these specific sources without adding in ideas from sources I have not asked you to explore. Use your general background, but use this background only as a way to explain what these specific sources are proposing. What I mean is don’t add in stuff to address my prompt that does not exist within the sources I gave you.
Second, if I ask an AI service about the content I have provided, I want the service to be able to identify the source and possibly the specific material within a source that was the basis for a given position taken. Think of this expectation as similar to the expectation one might have in reading a scientific article to which the author provides citations for specific claims made. My desire here is to be able to evaluate such claims myself. I have a concern in simply basing a claim on the language of sources not knowing the methodology responsible for producing data used as a basis for a claim. For serious work, you need to read more than the abstract. Requiring a precise methodology section in research papers is important because the methodology establishes the context responsible for the generation of the data and ultimately the conclusions that are reached. Especially in situations in which I disagree with such conclusions, I often wonder if the methodology applied may explain the differences between my expectations and the conclusions reached by the author. Human behavior is complex and variables that influence behavior are hardly ever completely accounted for in research. Researchers do not really lie with statistics, but they can mislead by broad conclusions they share based on a less-than-perfect research method. There are no perfect research methods hence the constant suggestion that more research is needed.
Several services approximate the characteristics I am looking for. I will identify three such services. I had hoped to add a fourth, but I intended to subscribe to the new OpenAI applications recently announced, but the $20 a month subscription fee necessary to use these functions was recently suspended so I will have to wait to explore these functions until OpenAI decides to expand the user base.
The three services I have worked with include NotebookLM, Mem.ai, and Smart Connections with Obisidan. I have written about Mem.ai and Smart Connections in previous posts, so I will use NotebookLM for extended comments and then offer quick examples produced by the other two services.
NotebookLM
NotebookLM recently was made available to me so I have had less experience using this tool than the other two I have included.
At this time you can create a notebook based on 5 pdfs or 10 text segments. There is a limit to how much content these units of information can total, but the pdfs I used were research articles so this is a lot of content. In the following image, icons for the 5 pdfs appear in the column on the left of the screen. All of the pdfs have been selected for the interaction I intend with the AI tool and a check beside each pdf indicates this is the case. The window for submitting my prompts appears at the bottom of the image.
The second image shows the results from the prompt I have submitted (I will explain my interest in interacting with these pdfs following this description). The response to this query is provided above the prompt window (I will share the text) and below this window you will see that the summary was based on 10 “citations”. Selecting one of these citations brings up the content that now appears in the left-hand column. This window shows the content of the pdf and some summary comments about that document generated by the AI. The AI also proposes additional prompts a user might want to apply to that content.
The task I asked all three services to address was related to a recent post I wrote describing generative cognitive tasks and comparisons among the impacts of these generative activities. The pdfs were research studies I had reviewed and were the basis for that post. I had detailed familiarity with this content from both reviewing and annotating this material and writing the previous post and I use this approach of first engaging a process without AI to provide a basis for commenting on what the AI provides. The prompt that follows is what I submitted to NotebookLM to see what would be generated. I will use a similar prompt with the other two AI services/
Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities.
NoteBook response
Generative learning is a type of learning in which students are actively engaged in creating new knowledge by interacting with the material. This is in contrast to passive learning, in which students simply listen to a lecture or read a textbook.
Generative learning activities are designed to encourage students to think critically about the material and to make connections between new information and their existing knowledge.
There is a significant body of research evidence that supports the effectiveness of generative learning activities. A meta-analysis of 500 studies found that generative learning activities resulted in better learning outcomes than traditional, passive learning methods (Coffey et al., 2003).
Some of the most effective generative learning activities include:
Summarizing: Students are asked to summarize a text in their own words. This activity helps students to identify the main ideas of the text and to understand the relationships between them.
Questioning: Students are asked to generate questions about a text. This activity helps students to think critically about the text and to engage with the material in a deeper way.
Discussion: Students are asked to discuss a text with their classmates. This activity helps students to share their understanding of the text and to learn from each other.
The relative effectiveness of different generative learning activities depends on the specific learning goals. For example, summarizing is a good activity for helping students to remember information, while questioning is a good activity for helping students to understand the material in a deeper way.
Generative learning activities can be used in a variety of settings, including K-12 classrooms, higher education, and corporate training. They are a valuable tool for helping students to learn and remember new information in a more effective way.
***
You will note in this material that NoteBook included a reference. This was not one of the pdfs I provided for review so I thought it might have been included in the reference section of one of the pdfs. I asked Notebook to provide a full citation and it generated the following.
Coffey, J. E., & McDonald, J. (2003). The effects of generative learning activities on student achievement: A meta-analysis. Contemporary Educational Psychology, 28(4), 240-263. doi:10.1016/j.cedpsych.2003.09.001
While this citation looks authentic and has the proper APA format, it is made up. I tried searching Google Scholar and used my library online access to search the location in Contemporary Educational Psychology and no such article exists. Google asks for user feedback and I provided the fake citation in my evaluation.
***
Mem.AI
My use of Mem.ai and Smart Connections require a little different approach. The content stored in these services results from my exporting notes and highlights from the research articles I read. Both services are directed at my entire collection of notes so this is different from what NotebookLM could access. However, the citations indicated that the output was based on very similar source material.
Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities.
Mem.ai response
Generative learning is a process where learners actively engage with material, creating new knowledge and connections. This approach involves strategies such as summarization, teaching, self-testing, and self-explaining, which require learners to select and organize relevant information, elaborate on the material based on personal knowledge, and integrate new information with existing knowledge 1^.
Fiorella and Mayer (2016) identified eight strategies that promote generative learning, including summarization, mapping, drawing, imagining, self-testing, self-explaining, teaching, and enacting. These strategies help learners to create a product based on existing knowledge 1^.
Brod (2021) focused on age differences in generative learning strategies and emphasized the importance of producing a meaningful product that goes beyond simple paraphrasing or retelling. For instance, highlighting does not result in new content, but summarization does. Brod’s review suggests that summarizing becomes useful from high school onwards, while testing is beneficial for all ages 2^.
Chi’s (2009) framework differentiates learning activities into active, constructive, and interactive categories. Active engagement with the material can activate existing knowledge and store new input, while constructive activities involve creating a product based on existing knowledge. Interactive activities involve engaging with others to produce a product that integrates external input with personal knowledge 1^.
Fiorella (2023) proposed a new framework of sense-making that distinguishes among three sense-making modes—explaining, visualizing, and enacting—that serve unique and complementary cognitive functions. The explaining mode generalizes knowledge, the visualizing mode organizes knowledge, and the enacting mode simulates knowledge 3^.
In conclusion, generative learning involves actively engaging with the material to create new connections and deeper understanding. The effectiveness of different generative learning activities can vary based on factors such as age and the specific learning context. However, strategies such as summarization, teaching, self-testing, and notetaking with revision and elaboration can promote generative learning and enhance understanding of the material.
Summarize my notes on generative learning Tue 4/18
At the end of each paragraph in this output, you see a number. The number corresponds to the citation at the bottom. About the three links that take a user to that note, you see the names of the notes. So, this output is carefully linked to sources. I will provide an example of what my notes look like in association with the output from Smart Connections, but each note contains a citation of the article from which I created that note.
***
Smart Connections (within Obsidian)
Obsidian contains my notes from research articles I have read and also highlights from other sources provided to Obsidian through Readwise. I tell you this again because I do not have a comparison among AI services based on exactly the same base of resources. However, again, you will note similar sources are found. Note also that the prompt is a little different to get Smart Connections to include citations.
Prompt: Use only my notes, generate a 400 word summary of generative learning activities and include what is known about the relative effectiveness of different activities. Include links to the individual source notes within your summary
Response:
Based on your notes, generative learning involves actively making sense of new information by mentally reorganizing and integrating it with one’s prior knowledge (Fiorella and Mayer, 2016). It is a constructive process that requires learners to select important information, reorganize it, and relate it to what they already know (Fiorellaetal2016).
Fiorella and Mayer (2016) identified eight strategies that promote generative learning: summarizing, mapping, drawing, imagining, self-testing, self-explaining, teaching, and enacting. Summarizing involves selecting the main ideas, organizing them into a logical structure, and integrating new information with prior knowledge (Fiorellaetal2016). It is more effective for relatively simple concepts and requires training to use effectively (Fiorellaetal2016).
Mapping and drawing also involve transforming information into different forms of representation (Fiorellaetal2016). These strategies help learners build connections and organize the material (Fiorellaetal2016).
Self-testing, or retrieval-based learning, requires learners to actively retrieve information from memory without access to the provided materials (Fiorellaetal2016). It strengthens memory and makes it more accessible in the future (EndresHigh).
Self-explaining involves explaining the content of a lesson to oneself during learning (Fiorellaetal2016). It promotes meaningful learning outcomes when learners engage in reflective knowledge building (Fiorellaetal2016).
Teaching, on the other hand, involves selecting relevant information, organizing it into a coherent structure, and elaborating on it by incorporating existing knowledge (Fiorellaetal2016). It sets itself apart from other generative strategies by focusing on helping others learn (Fiorellaetal2016).
Based on the research by Fiorella and Mayer (2016), it is more appropriate to consider each generative strategy as suitable for certain types of learning and learning contexts rather than regarding one strategy as the most effective (Fiorellaetal2016).
In a study by Koh, Lee, and Lim (2018), the learning benefits of teaching were investigated. The results showed that teaching improved learning outcomes when it involved retrieval practice, but not when learners used a prepared script (Generative > TeachingAsRetrieval). This supports the idea that retrieval practice is an effective generative learning activity.
Overall, generative learning activities have been shown to be effective in promoting meaningful learning outcomes. However, the effectiveness of each activity may vary depending on the complexity of the material and the training provided to learners. It is important to consider the specific learning goals and contexts when selecting and implementing generative learning strategies (Fiorellaetal2016).
Based on your notes, here are the individual notes that provide more information on each generative learning activity:
– [[Generate1]]: Provides an overview of generative learning theory and the eight strategies identified by Fiorella and Mayer (2016).
– [[Summarizing]]: Describes the summarizing strategy and its effectiveness in selecting main ideas and creating quality summaries.
– [[Teaching]]: Discusses the goal of teaching and its distinction from other generative strategies.
– [[Self-Testing]]: Explains the self-testing strategy and its role in retrieval-based learning.
– [[Promote generative learning]]: Provides a link to the article by Fiorella and Mayer (2016) that identifies the eight strategies and discusses their effectiveness.
– [[Fiorellaetal2016]]: Summarizes the key points from Fiorella and Mayer’s (2016) article on generative learning strategies.
– [[EndresHigh]]: Discusses the importance of task differences in retrieval practice and the effects of different types of recall tasks.
– [[Yang2021]]: Highlights a meta-analysis on the effectiveness of retrieval practice in classroom learning and raises questions about the mechanism and student perception of its benefits.
***
Note: The output of Smart Connections is in markdown and the terms included in double parentheses are links that connect to the source note. So, if the summary above was opened in an app that would interpret markdown, the text within the double parentheses would appear as a link and the link would take me to a file stored on my computer. The file is named Generate1.
Here is an example of one of the original notes that was identified as source material.
Generative learning makes sense of new information by reorganizing it and relating it to existing knowledge. This position comes from Wittrock, but is similar to other theorists (Mayer, Piaget). This specific article identified eight learning strategies that promote generative learning and provides a review of research relevant to each strategy.
[[Summarizing]]
Mapping
Drawing
Imagining
[[Self-Testing]]
Self-Explaining
[[Teaching]]
Enacting
The first four strategies (summarizing, mapping, drawing, and imagining) involve changing the input into a different form of representation.
The final four strategies (self-testing, self-explaining, teaching, and answering practice questions) require additional elaboration.
Fiorella, L., & Mayer, R. E. (2016). Eight ways to promote generative learning. _Educational Psychology Review, 28(4), 717-741.
***
Summary
Keeping in mind my recognition that the AI of the three AI services was applied to slightly different content, I would argue that Smart Connections and Mem.ai are presently more advanced than NotebookLM. Eventually, I assume a user will be able to direct NotebookLM at a folder of files so the volume of content would be identical. Google does acknowledge that Notebook is still in the early stages and access is limited to a limited number of individuals willing to test and provide feedback. The content generated by all of the services was reasonable, but NoteBook did hallucinate a reference.
My experience in comparing services indicates it is worth trying several in the completion of a given task. I have found it productive to keep both Smart Connections and Mem.ai around as the one I find most useful seems to vary. I do pay to use both services.
Manage Consent
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.
You must be logged in to post a comment.