Generative learning – Page 3

Potential conflicting benefits of your note-taking tool and approach

As I have explored and used several digital note-taking tools and examined the arguments that have been made regarding how such tools result in productivity benefits, I have identified a potential conflict in what produces more positive outcomes. The recognition of this conflict allows more purposeful execution on the part of the tool user and may better align activities with goals.

One way to identify note-taking goals is to use a long-standing approach differentiating generative and external storage benefits. This distinction was proposed long before PKM and was applied in the analysis of notes taken in classroom settings. The generative benefit proposes that the process of taking notes or sometimes of taking notes in a particular way engages our cognitive (mental) processes in ways that improve retention and understanding. External storage implies that our memory becomes less effective over time and having access to an external record (the notes) benefits our productivity. In practice (e.g., a student in a classroom) both benefits may apply, but one benefit depends on the other activity. Taking notes may not be beneficial, but to review notes one must have something to review. This is not always true as notes in one form or another can be provided or perhaps generated (for example AI identification of key ideas), but taking your own notes is by far the most common experience. In a PKM way of thinking, these two processes may function in different ways, but the classroom example should be familiar as a way to identify the theoretical benefits of note-taking.

I have written about the generative function of note-taking at length, but it is important to point out some unique specifics that apply to some digital note-taking tools. A source such as Ahrens’ Taking Smart Notes might provide the right mindset. I think of generative activities as external actions intended to produce a beneficial mental (cognitive) outcome. The idea is that external activities can encourage or change the likelihood of beneficial thinking behaviors. One way of operationalizing this perspective is to consider some of the specific activities Ahrens identified as external work resulting in such cognitive benefits. What are some of these activities? Isolating specific ideas and summarizing each as a note. Assigning tags that characterize a note. Making the effort to link notes. Periodically reviewing notes to generate retrieval practice, to reword existing notes, and to add new associations (links).

Retrieval is easier to explain. Note-taking apps with highly effective search capabilities make it easy to search and surface stored information when it might be useful. Links and tags may also be useful in this role, but search alone will often be sufficient.

What about the potential conflict?

The conflict I see proposes that some tools or approaches rely more heavily on search arguing in a way that generative processes are unnecessary.

I starting thinking about this assumption when contrasting the two note-taking systems I rely on – Mem.ai and Obsidian. While Mem.AI and Obsidian could be used in exactly the same way, Mem.ai developers argued that the built-in AI capabilities could eliminate the need to designate connections (with tags and links) because the AI capabilities would identify these connections for you. Thus when retrieving information via search, a user could use AI to also consider the notes with overlapping foci. If a user relied on this capability it would eliminate the work required to generate the connections manually created in Obsidian, but this approach would then also avoid the generative benefits of this work.

AI capabilities fascinate me so I found a way to add a decent AI capability to Obsidian. Smart Connections is an Obsidian plugin that finds connections among notes and allows a user to chat with their notes. So, I found a way to mimic Mem.ai functionality with Obsidian.

I find I have found a way to alter my more general PKM approach because of these capabilities. Rather than taking individual notes while reading, I can annotate and highlight pdfs, books, and videos and export the entire collection for each source and then bring this content into both Mem.ai and Obsidian as a very large note. Far easier than taking individual notes, but at what generative cost?

Smart Connections has added a new feature that even facilitates the use of the large note approach. Connections finds connections based on AI embeddings. An embedding is the mathematical representation of content (I would describe as weights based on what I remember of statistics). The more two notes embeddings’ weights are similar the more the notes consider similar ideas. Smart Connections used embeddings to propose related notes. Originally embeddings were generated at the note level and now at the “block” level. What this means (block level) is that Smart Connections can find the segments of a long document that have a similar focus as a selected note.

Why is this helpful? When I read long documents (pdfs of journal articles or books in Kindle), I can export a long document containing my highlights and notes generated from these documents. With Smart Connections I can then just import this exported material into Obsidian and use Smart Connections to connect a specific note to blocks of all such documents. I can skip breaking up the long document into individual notes and assigning tags and creating links.

Why is this a disadvantage? Taking advantage of this capability can be a powerful disincentive to engaging in the generative activities involved in creating and connecting individual notes the basic version of Obsidian requires.

Summary

As note-taking tools mature and add AI capabilities, it is important for users to consider how the way they use such tools can impact their learning and understanding. The tools themselves are quite flexible but can be used in ways that avoid generative tasks that impact learning and understanding. If the focus is on the retrieval of content for writing and other tasks, the generative activities may be less important. However, if you start using a tool such as Obsidian because a book such as Smart Notes influenced you, you might want to think about what might be happening if you rely on the type of AI capabilities I have described here.

References
Ahrens, S. (2022). How to take smart notes: One simple technique to boost writing, learning and thinking. Sönke Ahrens.

Does flipping the classroom improve learning?

The instructional strategy of “flipping the classroom” is one of those recommendations that seems on first consideration to make a lot of sense. The core idea hinges on the truth that classroom time with students is limited and efficient use must be made of this time. Instead of taking up a substantial amount of this time with teacher presentations, why not move the exposure to content outside of class time and use class time for more active tasks such as helping students who have problems and allowing students to engage in active tasks with other students? With easy access to tools for recording presentations and sharing recordings online, why not simply have educators share presentations with students and have students review this material before class? So, presentations were flipped from class time to settings that might have been more frequently used for homework.

This all seemed very rational. I cannot remember where I first encountered the idea, but I did purchase Flip Your Classroom (Bergman and Sams, 2012) written by the high school teachers who I believe created the concept. While I did use my blog and textbook to promote this approach, I must have always wondered. I wrote a blog post in 2012 commenting that flipping the classroom sounded very similar to my large lecture experience of presenting to hundreds of students and expecting that these students would have read the textbook before class. Again, the logic of following up an initial exposure with an anecdote-rich and expanded focus on key concepts seemed sound. However, I knew this was not the way many students used their textbooks and some probably did not even make the purchase, but I was controlling what I could control.

There have been hundreds of studies evaluating the flipping strategy and many meta-analyses of these studies. These meta-analyses tend to conclude that asking students to watch video lectures before coming to class is generally beneficial. I think many have slightly modified the suggested in-class component to expand the notion of greater teacher-student interaction to include a focus on active learning. Kapur et al (2022), authors of the meta-analysis I will focus on eventually, list the following experiences as examples of active learning – problem-solving, class discussions, dialog and debates, student presentations, collaboration, labs, games, and interactive and simulation-based learning activities.

The institution where I taught had a group very much interested in active learning and several special active learning “labs” were created to focus on these techniques. The labs contained tables instead of rows of chairs, whiteboards, and other adaptations. To teach a large class in this setting you had to submit a description of the active techniques you intended to implement. The largest classes (200+) I taught could not be accommodated in these rooms and I am not certain if I would have ever submitted a proposal anyway.

Kupar et al. (2022)

Kupar and colleagues found reason to add another meta-analysis to those already completed. While their integrated analysis of the meta-analytic papers concluded that the flipped classrooms have an advantage, Kapur and colleagues were puzzled by the great variability present among the studies. Some studies demonstrated a great advantage in student achievement for the flipped approach and some found that traditional instruction was superior. It did not seem reasonable that a basic underlying advantage would be associated with this much variability and the researchers proposed that a focus on the average effect size without consideration of the source or sources for this variability made little sense. They conducted their own meta-analysis and coded each study according to a variety of methodological and situational variables.

The most surprising finding from this approach was that the inclusion of active learning components was relatively inconsequential. Remember that the use of such strategies in the face-to-face setting was emphasized in many applications. Surprisingly, segments of lecture within the face-to-face setting were a better predictor of an achievement advantage. Despite the break from the general understanding of how flipped classrooms are expected to work, educators seemed to use these presentations to review or supplement independent student content consumption and this provided an achievement bump.

The active beneficial learning component found to make a difference involved a problem-based strategy and when the entire process began with a problem-based experience. This finding reminds me of the problem-based learning research conducted by Deanna Kuhn who also proposed that the problem-based experience start the learning sequence. Kapur used the phrase productive failure to describe the way struggling with a problem before encountering relevant background information was helpful. Kuhn emphasized a similar process without the catchy label and proposed the advantage was more a matter of the activation of relevant knowledge and guiding the interpretation of information within the presentation of content that followed.

Regarding the general perspective on the flipped model identified by Kapur and colleagues, their findings were less an indictment of the concept, but a demonstration of the lack of fidelity in implementations to the proposed advantage of using face-to-face time to interact and adjust to student needs. Increasing response to the needs of individual needs would seem beneficial and may be ignored in favor of activities that are less impactful.

References:

Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022, September). Fail, flip, fix, and feed–Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. In Frontiers in Education (Vol. 7, p. 956416). Frontiers.

Pease, M. A., & Kuhn, D. (2011). Experimental analysis of the effective components of problem?based learning. Science Education, 95(1), 57-86.

Wirkala. C. & Kuhn, D. (2011). Problem-Based Learning in K–12 Education: Is it Effective and How Does it Achieve its Effects? American Educational Research Journal, 48, 1157–1186

Flashcard Effectiveness

This post is a follow-up to my earlier post promoting digital flashcards as an effective study strategy for learners of all ages. In that post, I suggested that at times educators were anti rote learning assuming that strategies such as flashcards promoted a shallow form of learning that limited understanding and transfer. While this might appear to be the case because flashcards seem to involve a simple activity, the cognitive mechanisms that are involved in trying to recall and reflect on the success of such efforts provide a wide variety of benefits.

The benefits of using flashcards in learning and memory can be explained through several cognitive mechanisms:

1. Active Recall: Flashcards engage the brain in active recall, which involves retrieving information from memory without cues (unless the questions are multiple-choice). This process strengthens the memory trace and increases the likelihood of recalling the information later. Active recall is now more frequently described as retrieval practice and the benefits as the testing effect. Hypothesized explanations for why efforts to recall and even why efforts to recall that are not successful are associated not only with increased success at recall in the future but also broader benefits such as understanding and transfer offer a counter to the concern that improving memory necessarily is a focus on rote. More on this at a later point.

2. Spaced Repetition: When used systematically, flashcards can facilitate spaced repetition, a technique where information is reviewed at increasing intervals. This strengthens memory retention by exploiting the psychological spacing effect, which suggests that information is more easily recalled if learning sessions are spaced out over time rather than crammed in a short period.

3. Metacognition: Flashcards help learners assess their understanding and knowledge gaps. Learners often have a flawed perspective of what they understand. As learners test themselves with flashcards, they become more aware of what they know and what they need to focus on, leading to better self-regulation in learning

4. Interleaving: Flash cards can be used to mix different topics or types of problems in a single study session (interleaving), as opposed to studying one type of problem at a time (blocking). Interleaving has been shown to improve discrimination between concepts and enhance problem-solving skills.

5. Generative Processing: External activities that encourage helpful cognitive behaviors is one way of describing generative learning. Responding to questions and even creating questions have been extensively studied and demonstrate achievement benefits.

Several of these techniques may contribute to the same cognitive advantage. These methods (interleaving, spaced repetition, recall rather than recognition) increase the demands of memory retrieval and greater demands force a learner to move beyond rote. They must search for the ideas they want and effortful search activates related information that may provide a link to what they are looking for. An increasing number of possibly related ideas become available within the same time frame allowing new connections to be made. Connections can be thought of as understanding and in some cases creativity.

This idea of the contribution of challenge to learning can be identified in several different theoretical perspectives. For example, Vygotsky proposed the concept of a Zone of Proximal Development that position ideal instruction as challenging learners a bit above their present level of functioning, but within the level of what a learner could take on with a reasonable change of understanding. A more recent, but similar concept proposing the benefits of desirable difficulty came to my attention as the explanation given for why taking notes on paper was superior to taking notes using a keyboard. The proposal was that keyboarding is too efficient forcing learners who record notes by hand to think more carefully about what they want to store. Deeper thought was required when the task was more challenging.

Finally, I have been exploring researchers studying the biological mechanism responsible for learning. As anyone with practical limits on my time, I don’t spend a lot of time reviewing the work done in this area. I understand that memory is a biological phenomenon and cognitive psychologists do not focus on this more fundamental level, but I have also yet to find insights from biological research that required I think differently about how memory happens. Anyway, a recent book (Ranganath, 2024) proposes something called error-driven learning. The researcher eventually backs away a bit from this phrase suggesting that it does not require you to make a mistake but happens whenever you struggle to recall.

The researcher proposes that the hippocampus enables us to “index” memories for different events according to when and where they happened, not according to what happened. The hippocampus generates episodic memories. by associating a memory with a specific place and time. As to why changes in contexts over time matter, memories stored in this fashion become more difficult to retrieve. Activating memories with spaced practice both creates an effortful and more error-prone retrieval, but if successful offers a different context connection. So, spacing potentially offers different context links because different information tends to be active in different locations and times (note other information from what is being studied would be active) and involves retrieval practice as greater difficulty involves more active processing and exploration of additional associations. I am adding concepts such as space and retrieval practice from my cognitive perspective, but I think these concepts fit very well with Ranganath’s description of “struggling”.

I have used the term episodic memory in a little different way. However, the way Rangath describes changing contexts over time seems useful as an explanation for what has long been appreciated as the benefit of spaced repetition in the development of long-term retention and understanding.

When I taught educational psychology memory issues, I described the difference between episodic and declarative memories. I described the difference as similar to the students’ memory for a story and the memory for facts or concepts. I proposed that studying especially trying to convert the language and examples of the input (what they read or heard in class) into their own way of understanding with personal examples that were not part of the original content they were trying to process was something like converting episodic representations (stories) into declarative representations linked to relevant personal episodic elements (students’ own stories). This is not an exact representation of human cognition in several ways. For example, even our stories are not exact and are biased by past and future experiences and can change with retelling. However, it is useful as a way to develop what might be described as understanding.

So, to summarize, memory tasks, even what might seem to be simple ones such as might be the case with basic factual flashcards can introduce a variety of factors conducive to a wide variety of cognitive outcomes. The assumption that flashcards are useful only for rote memory is flawed.

Flashcard Research

There is considerably more research on the impact of flashcards that I realized and some recent studies that are specific to digital flashcards.

Self-constructed or provided flashcards – When I was still teaching the college students I say using flashcards were obviously using paper flashcards they had created. My previous post focused on flashcard tools for digital devices. As part of that post, I referenced sources for flashcards that were prepared by textbook companies and topical sets prepared by other educators and offered for use. I was reading a study comparing premade versus learner-created flashcards (description to follow) and learned that college students are now more likely to use flashcards created by others. I guess this makes some sense considering how digital flashcard collections would be easy to share. The question then is are questions you create yourself better than a collection that covers the material you are expected to learn.

Pan and colleagues (2023) asked this question and sought to answer it in several studies with college students. One of the issues they raised was the issue of time required to create flashcards. They controlled the time available for the treatment conditions with some participants having to create flashcards during the fixed amount of time allocated for study. Note – this focus on time is similar to the retrieval practice studies using part of the time in the study phase for responding to test items while others were allowed to study as they liked. The researchers also conducted studies in which the flashcard group created flashcards in different ways – transcription (typing the exact content from the study material), summarization, and copy and pasting. The situation investigated here seems similar to note-taking studies comparing learner-generated notes and expert notes (quality notes provided to learners). With both types of research, one might imagine a generative benefit to learners in creating the study material and a completeness/quality issue. The researchers did not frame their research in this way, but these would be alternative factors that might matter.

The results concluded that self-generated flashcards were superior. They also found that copy-and-paste flashcards were effective which surprised me and I wonder if the short time allowed may have been a factor. At least, one can imagine using copy and paste as a quick way to create the flashcards using the tool I described in my previous flashcard post.

Three-answer technique – Senzaki and colleagues (2017) evaluated a flashcard technique focused on expanding the types of associations used in flashcards. They proposed their types of flashcard associations based on the types of questions they argued college students in information-intensive courses are asked to answer on exams. The first category of test items are verbatim definitions for retention questions, the second are accurate, paraphrases for comprehension questions, and the third are realistic examples for application questions. Their research also investigated the value of teaching students to use the three response types in comparison to requesting they include these three response types.

The issue of whether students who use a study technique (e.g., Cornell notes, highlighting) are ever taught how to use a study strategy why it might be important to apply the study in a specific way) has always been something I have thought was important.

The Senzaki and colleagues research found their templated flashcard approach to be beneficial and I could not help seeing how the Flashcard Deluxe tool I described in my first flashcard post was designed to allow three possible “back sides” for a digital flashcard. This tool would be a great way to implement this approach.

AI and Flashcards

So, while learner-generated flashcards offer an advantage, I started to wonder about AI and was not surprised to find that AI-generated capabilities are already touted by companies providing flashcard tools. This led me to wonder what would happen if I asked AI tools I use (ChatGPT and NotebookLM) to generate flashcards. One difference I was interested in was asking ChatGPT to create flashcards over topics and NotebookLM to generate flashcards focused on a source I provided. I got both approaches to work. Both systems would generate front and back card text I could easily transfer to a flashcard tool. I found that some of the content I decided would not be particularly useful, but there were plenty of front/back examples I thought would be useful.

The following image shows a ChatGPT response to a request to generate flashcards about mitosis.

This use of AI used NotebookLM to generate flashcards based on a chapter I asked it to use as a source.

This type of output could also be used to augment learner-generated cards or could be used to generate individual cards a learner might extend using the Senzaki and colleagues design.

References

Pan, S. C., Zung, I., Imundo, M. N., Zhang, X., & Qiu, Y. (2023). User-generated digital flashcards yield better learning than premade flashcards. Journal of Applied Research in Memory and Cognition, 12(4), 574–588. https://doi-org.ezproxy.library.und.edu/10.1037/mac0000083

Ranganath, C. (2024). Why We Remember: Unlocking Memory’s Power to Hold on to What Matters. Doubleday Canada.

Senzaki, S., Hackathorn, J., Appleby, D. C., & Gurung, R. A. (2017). Reinventing flashcards to increase student learning. _Psychology Learning & Teaching, 16(3), 353-368.

Writing to Learn Research – Messy

Writing to learn is one of those topics that keeps drawing my attention. I have an interest in what can be done to encourage learning and approach this interest by focusing on external tasks that have the potential to manipulate the internal cognitive (thinking) behavior of learners. My background in taking this perspective is that of an educational psychologist with a cognitive perspective. I have a specific interest in areas such as study behavior trying to understand what an educator or instructional designer can do to promote experiences that will help learners be more successful. The challenge seems obvious – you cannot learn for someone else, but you may be able to create tasks that when added to exposure to sources of information encourage productive “processing” of those experiences. We can ask questions to encourage thinking. We can engage students in discussions that generate thinking through interaction. We can assign tasks that require the use of information. Writing would be an example of such an assigned task.

Writing to Learn

Writing to learn fits with this position of an external task that would seem to encourage certain internal behaviors. To be clear, external tasks cannot control internal behavior. Only the individual learner can control what they think about and how they think about something, but for learners willing to engage with an external activity that activity may change the likelihood productive mental behaviors are activated.

I found the summary of the cognitive benefits of writing to learn useful and consistent with many of my own way of thinking about other learning strategies – external tasks that encourage productive internal behaviors. Writing based on content to be learned requires that the writer generate a personalized concrete representation at the “point of utterance”. I like this expression. To me, it is a clever way of saying that when you stare at the screen or the empty sheet of paper and must fill the void you can no longer fool yourself – you either generate something or you don’t. You must use what you know and how you interpret the experiences that supposedly have changed what you know to produce an external representation.

To produce an external product, you must think about what you already know in a way that brings existing ideas into consciousness (working memory) by following the connections activated by the writing task and newly acquired information. This forces processing that may not have occurred without the external task. Connections between existing knowledge and new information are not necessarily made just because both exist in storage. Using knowledge to write or to perform other acts of application encourages making connections.

Such attempts at integration may or may not be successful. Having something external to consider offers the secondary benefit of forced metacognition. Does what I wrote really make sense? Do the ideas hang together or do I need to rethink what I have said? Does what I have proposed fit with the life experiences (episodic memory) I have had?

Writing ends up as a generative process that potentially creates understanding and feeds the product of this understanding back into storage.

Graham, Kiuhara & MacKay, M. (2020)

In carefully evaluating and combining the results of many studies of writing to learn, these researchers intended not only to determine if the impact of writing to learn had the intended general benefit but to use the variability of writing tasks and outcomes from studies to deepen our understanding of how writing to learn encouraged learning. Surely, some activities would be more beneficial than others because of the skills and existing knowledge of learners or the specifics of the assigned writing tasks. So, the meta-analysis is asking if there is a general effect (Is writing to learn effective), and secondarily are there significant moderator variables that may help potential practitioners decide when, with whom, and how to structure writing to learn activities?

The Graham and colleagues’ research focused only on K12 learners. Potential moderator variables included grade level, content area (science, social studies, mathematics), type of writing task (argumentation, informational writing, narrative), and some others. I have a specific interest in argumentation () which is relevant here as a variable differentiating the studies because it requires a deeper level of analysis than say a more basic summary of what has been learned.

Overall, the meta-analysis demonstrated a general benefit for writing to learn (Effect size = .30). This level of impact is considered on the low end of a moderate effect. Graham and colleagues point out that the various individual studies included in the study generated great variability. A number of the studies demonstrated negative outcomes meaning in those studies the control condition performed better than the group spending time on writing to learn. The authors propose that this variability is informative as it cannot be assumed that any approach with this label will be productive. The variability also suggests that the moderator variables may reveal important insights.

Unfortunately, the moderator variables did not achieve the level of impact necessary to argue for useful insights as to how writing to learn works or who is most likely to be a priority group for this type of activity. Grade level was not significant. The topic area was not significant. The type of writing task was not significant.

Part of the challenge here is having enough studies focused on a given approach with enough consistency of outcomes to allow statistical certainty in arguing for a clear conclusion. Studies that involved taking a position and supporting that position (e.g., argumentation) produced a much larger effect size, but the statistical method of meta-analysis did not reach the level at which a certain outcome could be claimed.

One interesting observation from the study caught my attention. While writing to learn is used more frequently in social studies classrooms, the number of research studies associated with each content areas was the smallest for social studies. Think about this. Why? I wonder if the preoccupation of researchers and funding organizations with STEM is responsible.

More research is needed. I know practitioners and the general public get tired of being told this, but what else can you recommend when confronted with the messiness of much educational research? When you take ideas out of carefully controlled laboratories and try to test them in applied settings the results here are fairly typical. Humans left to their own devices as implementers of procedures and reactors to interventions are all over the place. Certainly, the basic carefully controlled research and the general outcome of meta-analysis focused on writing to learn implementation are encouraging, but as the authors suggest the variability in effectiveness means something, and further exploration is warranted.

Reference

Graham, S., Kiuhara, S. A., & MacKay, M. (2020). The effects of writing on learning in science, social studies, and mathematics: A meta-analysis. Review of Educational Research, 90(2), 179-226.

Use EdPuzzle AI to generate study questions

This post allows me to integrate my interest in studying, layering, questions, and using AI as a tutor. I propose a specific use of EdPuzzle, a tool for adding (layering) questions and notes to videos, be used as a study tool. EdPuzzle has a new AI feature that allows for the generation and insertion of open-ended and multiple-choice questions. So an educator interested in preparing videos students might watch to prepare for class could prepare a 15 minute mini-lecture and then use EdPuzzle to layer questions on this video and assign the combination of video and questions to students to be viewed before class. Great idea.

The AI capability was added to make the development and inclusion of questions less effortful. Or, the capability could be used to add some questions that educators could embellish with questions of their own. I propose a related, but different approach I think has unique value.

How about instead of preparing questions for students, allow students to use the AI generation tool to add and answer themselves or with peers.

Here is where some of my other interests come into play. When you can interact with AI that can be focused on assigned content you are to learn, you are using AI as a tutor. Questions are a part of the tutoring process.

What about studying? Questions have multiple benefits in encouraging productive cognitive behaviors. There is such a thing as a prequestioning effect. Attempting to answer questions before you encounter related material is a way to activate existing knowledge. What do you already know? Maybe you cannot answer many of the questions, but just trying makes you think of what you already know and this activated knowledge improves understanding as you then process assigned material. Postquestions are a great check on understanding (improving metacognition and directing additional study) and attempting to answer questions involves retrieval practice sometimes called the testing effect. For most learners, searching your memory for information has been proven to improve memory and understanding beyond what just studying external information (e.g., your notes) accomplishes.

I have described EdPuzzle previously, here are some additional comments about the use of the generative question tool.

After you have uploaded a video to EdPuzzle. You should encounter the opportunity to edit. You use edit to crop the video and to add notes and questions. The spots to initiate editing and adding questions are shown in the following images. When using AI to add questions, you use Teacher Assist – Add Questions.

After selecting Add Questions, you will be given the option of adding Open ended or Multiple Choice questions. My experience has been that unless your video includes a good deal of narration, the AI will generate more Open Ended than Multiple Choice questions. If you want to emphasize MC questions, you always have the option of adding questions manually.

Responding to a question will look like what you see in the following image. Playing the video will take the student to the point in the video where a question has been inserted and then stop to wait for a response.

When an incorrect response is generated to a MC question, the error will be identified.

EdPuzzle allows layered videos to be assigned to classes/students.

Anyone can explore EdPuzzle and create a few video lessons at no cost. The pricing structure for other categories of use can be found at the EdPuzzle site.

One side note: I used a video I created fitting the potential scenario I described of an educator preparing content for student use. However, I had loaded this video to YouTube. I found it difficult to download this video and finally resorted to the use of ClipGrab. I am unclear why I had this problem and I understand that “taking” video from some sources can be regarded as a violation of copyright. I know this does not apply in this case, but I did not want to mention this issue.

References:

Pan, S. C., & Sana, F. (2021). Pretesting versus posttesting: Comparing the pedagogical benefits of errorful generation and retrieval practice. Journal of Experimental Psychology: Applied, 27(2), 237–257.

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. _Psychological Bulletin_, _147_(4), 399-435.

Content focused AI for tutoring

My explorations of AI use to this point have resulted in a focus on two applications – AI as tutor and AI as tool for note exploration. Both uses are based on the ability to focus on information sources I designate rather than allowing the AI service to rely on its own body of information. I see the use of AI to interact with the body of notes I have created as a way to inform my writing. My interest in AI tutoring is more related to imagining how AI could be useful to individual students as they study assigned content.

I have found that I must use different AI services for these different interests. The reason for this differentiation is that two of the most popular services (NotebookLM and OpenAI’s Custom GPTs) limit the number of inputs that can be accessed. I had hoped that I could point these services at a folder of notes (e.g., Obsidian files) and then interact with this body of content. However, both services presently allow only a small number of individual files (10 and perhaps 20) can be designed as source material. This is not about the amount of content as the focus of this post involves using these two services to interact with a single file of 27,000 words. I assume in a year the number of files will be less of an issue.

So, this post will explore the use of AI as a tutor applied to assigned content as a secondary or higher ed student might want to do. In practice, what I describe here would require that a student would have access to a digital version of assigned content not protected in some way. For my explorations, I am using the manuscript of a Kindle book I wrote before the material was converted to a Kindle book. I wanted to work with a multi-chapter source of a length students might be assigned.

NotebookLM

NotebookLM is a newly released AI service from Google. The AI prompts can be focused on content that is available in Google drive or uploaded to the service. This service is available at no cost, but it should be understood that this is likely to change when Google is ready to offer a more mature service. Investing time in this service rather than others allows the development of skills and the exploration of potential, but in the long run some costs will be involved.

Once a user opens NotebookLM and creates a notebook (see red box surrounding new notebook), external content to be the focus of user prompts can be added (second image). I linked Notebook to the file I used in preparation for creating a Kindle book. Educators could create a notebook on unprotected content they wanted students to study.

The following image summarizes many essential features used when using NotebookLM. Starting with the right-hand column, the textbox near the bottom (enclosed in a red box) is where prompts are entered. The area above (another red box) provides access to content used by the service in generating the response to a prompt. The large area on the left-hand side displays the context associated with one of the areas referenced with the specific content used highlighted.

Access to a notebook can be shared and this would be the way an educator would provide students access to a notebook prepared for their use. In the image below, you will note the icon (at the top) used to share content, and when this icon is selected, a textbox for entering emails for individuals (or for a class if already prepared) appears.

Custom GPTs (OpenAI)

Once you have subscribed to the monthly payment plan for ChatGPT – 4, accessing the service will bring up a page with the display shown below. The page allows access to ChatGPT and to any custom GPTs you have created. To create a Custom GPT you select Explore and then select Create a GPT. Describing the process of creating a GPT would require more space than I want to use in this post, but the process might best be described as conversational. You basically interact by describing what you are trying to create and you upload external resources if you want prompts to be focused on specific content. Book Mentor is the custom GPT I created for this demonstration.

Once created, a GPT is used very much in the same way a NotebookLM notebook is used. You use the prompt box to interact with the content associated with that GPT.

What follows are some samples of my interactions with the content. You should be able to see the prompt (Why is the word layering used to describe what the designer does to add value to an information source?)

Prompts can generate all kinds of ways of interaction (see a section below that describes what some of these interactions might be). One type I think has value in using AI as a tutor is to have the service ask you a question. An example of this approach is what is displayed in the following two images. The first image describes a request for the service to generate a multiple-choice question about generative activity which I then respond (correctly) and receive feedback. The second image shows the flexibility of the AI. When responding to the question, I thought a couple of the responses could be correct. After I answered the question and received feedback, I then asked about an answer I did not select wondering why this option could not also be considered correct. As you see in the AI reply, the system understands my issue and acknowledges how it might be correct. This seems very impressive to me and demonstrates that the interaction with the AI system allows opportunities that go beyond self-questioning.

Using AI as tutor

I have written previously about the potential of AI services to interact with learners to mimic some of the ways a tutor might work with a learner. I make no claims of equivalence here. I am proposing only that tutors are often not available and an AI system can challenge a learner in many ways that are similar to what a human tutor would do.

Here are some specific suggestions for how AI can be used in the role of tutor.

Summary

This post describes two systems now available that allow learners to work with assigned content that mimics how a tutor might work with a student. Both systems would allow a designer to create a tool focused on specific content that can be shared. ChatGPT custom GPTs require that those using a shared GPT have an active $20 per month account which probably means this approach would not presently be feasible for common application. Google’s Notebooks can be created at no cost to the designer or user, but this will likely change when Google decides the service is beyond the experimental stage. Perhaps the capability will be included in present services designed for educational situations.

While I recognize that cost is a significant issue, my intent here is to propose services that can be explored as proof of concept and those educators interested in AI opportunities might explore future productive classroom applications of AI.

Evaluating tech tools for adult learning

I feel comfortable writing about learning in educational environments. I have reviewed many instructional and learning strategies, read applied studies intended to evaluate the efficacy of these strategies, and read a substantial amount of the basic cognitive research potentially explaining the why of the applied investigations. In a small way, I have contributed to some of this research.

As my life circumstances have changed, I have begun exploring related, but unfamiliar topics. In retirement, I am by definition no longer playing an active role as a salaried educator or researcher. I retain the opportunity to access the scholarly literature as an emeritus faculty member, but I can no longer engage as a researcher. These changes led to a different perspective. I have become more interested in other folks like me who are still interested in learning and how they go about responding to such interests.

As I have contemplated this situation, it has become clear that this situation is not a matter of age. While it was very important for me to constantly learn while I was working, I don’t think I spent much time considering how I should best go about it. There was work to be done and despite my own focus on education, I did little to consider the strategies of my own learning.

I began to think more deeply about self-directed learning, adult learning, or whatever else might be the current way to describe this situation when I began participating in a book club that has as one interest Personal Knowledge Management (PKM) and the technology tools that can be applied when committed to implementing this concept. For those who are unfamiliar with PKM, one way to gain insight would be to read a couple of the self-help books explaining views on this topic and describing techniques argued to be useful in achieving goals consistent with the general idea of Personal Knowledge Management.

Sonke Ahrens How to Take Smart Notes: One Simple Technique to Boost Writing, Learning and Thinking

Tiago Forte Building a Second Brain: A Proven Method to Organize Your Digital Life and Unlock Your Creative Potential

There is plenty of specific information available from such books and other online resources regarding how to study topics for understanding and retention. It is easy to locate tutorials for online services and apps to implement these strategies. There seem to be hundreds of posts on Medium, Substack, and YouTube with titles like “My Obsidium Workflow”, “I Switched From OneNote to Notion and Can’t Believe My New Productivity”, and “All of the Notetaking Apps in One Post”. There must be something people want to understand and evaluate here. When i dig deeper there are some logical arguments proposed to justify techniques digital tools enable such as the creation of permanent and atomic notes, linking notes, and progressive summarization and I can sometimes associate these techniques with cognitive concepts I knew such as generative learning, spaced repetition, and retrieval practice.

What I finally decided I was missing was the type of applied research I found readily available when specific study techniques are proposed for classroom use. Learning and studying over time is not really what is studied in K12 and postsecondary education. What students know is studied over time, but not frequently how different methods of study influence the development of skills and knowledge. Differences in what studies can do on the next exam or at the end of a course are typically the focus. This seems different from the goal of evaluating learner-guided activities to develop knowledge and skills over many years.

The time frame is not the only difference. Some of the strategies for school and adult independent note-taking are similar on the surface but different enough to warrant additional research. Note-taking, sometimes even described as note-making to differentiate the processes by advocates of some PKM methods, is a good example. In the Smart Note approach, isolating specific concepts such as individual notes written with enough context to be interpretable over time and then linking these individual notes to other notes by way of multiple links is quite different from how students take and make use of notes. The note-taking tools are different, the goals are different, and the mechanisms of creating and then acting on the written record are different. I want to know if the mechanics of these differences are actually useful. Controlled comparisons would be interesting, but so would studies examining how adults familiar with these approaches make use of what the tools allow over time, if they actually do. Do learners working for their own purposes stick with what the logic proposed for the use of a learning tool or do they modify the ideal approach to something that is simpler and less cognitively demanding? Formal research methods have proven useful in understanding study strategies proposed for classroom-associated use but should be repeated in evaluating self-directed adult learning.

I don’t think much if any of the type of formal research I propose exists. At least, I have not been able to locate this work. Maybe the payoff for such effort just is not there. Maybe there is a lack of grant support to fund academic research, but we academics are still interested in topics that seldom bring funding. There is a payoff available to those who develop tools and services in the form of subscriptions and for those writing self-help books that attract attention in the form of sales.

As I consider what it would take to work on these topics, I can imagine the challenges researchers would face. How would you collect data and how would you assure privacy when the tools used are often associated with work? How would you get individuals to participate in studies? What would individuals be willing to provide if you wanted to evaluate the effectiveness of the technique employed? I at least would hope individuals might be willing to provide information about the tools they used, how long they have used these tools, and how they have used the tools and perhaps changed their patterns of use over time.

Adults continually have learning tasks to keep up with vocational demands and for personal growth. We are told that rapid advancements in so many areas and so many information sources learning and learning to learn using technology would seem of increasing value. Perhaps by explaining my observations I can interest those still involved as active researchers. It is also possible I am missing a body of research that would address my interests. If this is the case, I would welcome suggestions.