An AI LLM Can Identify Important Missing Content In Student Notes

There seems to be a resurgence of research interest in note-taking and note-use. One source of new work was the controversy over whether students should take handwritten notes or use their laptops or tablets. A second focus has been generated by what I would describe as “out of classroom” notetaking described by those promoting personal knowledge management (PKM) and interactions with a second brain. 

I have had a long-term interest in notetaking and studying and recent activity has caused me to review old sources in light of newer activity. Possible connections between notes taken and stored digitally and the capabilities of AI tools seem worth exploring.

I still understand note-taking and note use from a decade old model that argues learning with and from notes can be outlined in terms of an immediate generative benefit of taking notes and by then using the external storage function of notes for review and study. A generative benefit implies that taking notes provides an advantage over simply listening and reading. The external task prompts the note-taker to identify important ideas because it is not practical to record everything, and then cognitively acts on the selected information to personalize the original content through summarization and connections to existing knowledge. Of course, there are many strategies for doing this, as well as individual differences and less active approaches, such as simply writing down what the instructor displays on his/her PowerPoint slides.

One topic relevant to how I imagine AI being used in interaction with student notes is work that examines how effectively learners have recorded important ideas. How many ideas find their way into the notes and are these ideas important ideas? Here you find some interesting controversies. Early work on note-taking in actual classroom settings found a substantial correlation between the volume of content students recorded and their exam performance (Nye and colleagues, 1984). I want to emphasize the word “naturalistic” here because, as the authors of this study claim, so much of the research has been conducted in short-term laboratory studies involving short segments of content and immediate examinations – not the situation students face in practice. I have made the same complaint in a recent post

A recent controversy related to what might be described as the “more is better” position is the handwriting vs. keyboarding controversy. The issue here is that keyboarding allows for more input, but handwriting appears to lead to more generative activity because the slower process encourages greater selectivity and summarization. My observation is that selection and reworking of notes does not have to happen in real time and having more content to work with during review and study makes such reworking more productive. 

This brings me to a second observation. While it seems possible that learners would not record information they think they already know, for one reason or another, students fail to record a sizeable proportion of important information, and what is not recorded can be related to specific later failures of recall and application on learning measures. It is difficult to study what you don’t know and what was not recorded (e.g., Kiewra, 1985; Lawson & Mayer, 2024).

Expert Notes and the AI Alternative

One solution to the many important ideas that students miss, especially less capable students, is to provide what are often called “expert notes” – essentially notes provided by the instructor or taken by an advanced student (Kiewra, 1985). This is a topic I explored in my own research with my graduate students (e.g., Grabe and colleagues, 2005). To be clear, the idea is not to substitute expert notes for student notes because of the generative value of taking notes, but to provide a second source learners can use to “fill in the blanks” for what they missed or perhaps did not understand well enough to even attempt recording. 

There are practical issues with providing quality notes to students. One issue that interested me in the citation I provided was the relationship between providing quality notes and class attendance. You do want students to attend class, but the reality is that there is also the question of what can be done for those students who miss class for legitimate reasons. Second, the generative effect of taking notes describes note-taking itself as a learning experience, so you really don’t want to discourage student note-taking. 

I have been exploring a use of AI that seems interesting and efficient in a couple of ways. It is relatively easy to prompt AI to identify the “important ideas in a lecture”, to compare this material with the notes taken by a learner, and to generate an output of the important ideas not contained in the student notes. This approach is efficient for students because it does not require reviewing the entirety of what might be considered “expert notes.” It also builds on rather than eliminates the student’s generative effort to create their own notes.

To explore how this might work, I used my current AI service to run the following prompt. The inputs included, first, the transcript of a source podcast on why climate change can be responsible for both drought and flooding, and, second, the notes I took while listening to the podcast. The original source contained approximately 2500 words and as a podcast required approximately 15 minutes. I attempted to play the role of students and created a set of notes while listening. I admit that I was surprised to learn a few things about how climate change works even though the material was prepared at the high school level. I purposely ignored comments that appeared near the end of the podcast that focused on fire danger and health concerns as a way to approximate what notes might look like if a student became distracted or cut out of class early. 

Prompt: I am providing two text segments. I want you to identify the important content from the first segment. I then want you to compare this important content with the content in the second segment and identify the important ideas in the first segment that are not included in the second.

I have included, as an Appendix, both the important ideas identified by the AI tool and the list of “important ideas” in the presentation that the AI tool said were not included in my notes. 

The approach identified and provided missing comments on both wildfires and health issues, which was my proof of concept when it came to responding to the problem of students missing important ideas, but it also included additional information that the comparison of the original content and my notes did not. So, when I compared the summary with my notes, the comparison in response to the prompt identified content I purposefully missed and other items I heard but did not record. This second body of information was much larger than I would have anticipated and fits existing research concluding even better students (apologies if this sounds conceited) fail to record more than one might expect. It is hard to know if this is an actual problem or simply a difference in standards. I have no idea whether the prompt could be altered to change how the AI tool interpreted “important content.” 

One interesting item – there is at least one item in the comparison between the AI summary of important ideas and my notes identified as missing that actually does not appear in the AI summary but was included in the full transcript (The claim that the most destructive floods now occur about three times as often as before). 

Final Comments

It is likely not a surprise that AI creates a summary of content, as this is a commonly offered service now often available as an alternative to asking readers or listeners to review all of the original raw content. I am not certain if I explored different wordings of my prompts if a request for the “important ideas” would be much different from a summary.

I have considered whether a summary or review of the most important information would offer essentially the same practical value as the second step I have AI to provide. The amount of content would differ. The summary I provide as an appendix is about a third of what would be generated from a traditional fifty-minute college lecture. I do like the idea of identifying what a student missed, which may have unique value in prompting students to consider their note-taking process. Helping students identify important information they missed also keeps the focus on student responsibility for the generative learning opportunity in making their own notes.

Sources:

Grabe, M., Christopherson, K., & Douglas, J. (2005). Providing Introductory Psychology Students Access to Online Lecture Notes: The Relationship of Note Use to Performance and Class Attendance. Journal of Educational Technology Systems, 33(3), 295-308.

Kiewra, K.A. (1985). Providing the instructor’s notes: an effective addition to student notetaking. Educational Psychologist, 20(1), 33-39. https://doi.org/10.1207/s15326985ep2001_5

Nye, P.A., Crooks, T.J., Powley, M., & Tripp, G. (1984). Student note-taking related to university examination performance. Higher Education, 13(1), 85-97. https://doi.org/10.1007/BF00136532

Appendix:

Important content in Segment 1

  • Climate change is intensifying both droughts and heavy rainfall/flooding.
  • A warmer atmosphere holds more water vapor, while hotter land surfaces evaporate moisture faster:
    • This dries land and worsens drought.
    • It also creates the potential for more intense downpours once rain occurs.
  • The most serious change may be in extreme storms, rather than in average rainfall or average storms.
  • Existing infrastructure is poorly suited to extreme weather, especially because roads, roofs, parking lots, and other impervious surfaces increase runoff and flooding.
  • Flood risk is not limited to coasts and rivers; intense rainfall can cause serious stormwater flooding inland.
  • Communities can reduce flood impacts through:
    • Updated emergency plans and warning systems
    • Recovery resources
    • Green space, trees, grass, and rain gardens
    • Reducing impervious surfaces, including through greater public transportation
  • Climate impacts vary by region:
    • New England’s warming coastal ocean can fuel stronger winter storms.
    • Subtropical regions, including the U.S. Southwest, are generally expected to become drier or more arid.
    • Some areas may not follow this pattern; for example, the Sahara may become wetter.
    • Monsoon regions may receive rain in fewer, more intense events, harming people and agriculture.
  • Water-cycle changes also worsen other hazards:
    • Drought can increase wildfire risk.
    • Alternating wet and dry periods can grow vegetation and then dry it into wildfire fuel.
    • Humidity makes heat waves more dangerous because it prevents sweat from cooling the body effectively.
  • Adaptation is necessary, but reducing fossil-fuel use and climate pollution is also essential to limit future worsening.
  • Community discussion and engagement are presented as meaningful individual responses to climate change.

Important ideas from Segment 1 not included in Segment 2 (my own notes)

  1. Examples and evidence of current extremes
    • California reservoir lows, Fort Lauderdale flooding, and Brooklyn’s extreme rainfall.
    • The claim that the most destructive floods now occur about three times as often as before.
  2. The distinction between average conditions and extremes
    • The largest storms are becoming stronger even if the average storm is not necessarily changing as much.
    • The primary concern is extreme downpours, not merely average rainfall.
  3. Detailed flooding and infrastructure discussion
    • Impervious surfaces such as roofs, roads, sidewalks, and parking lots worsen runoff.
    • Inland communities face stormwater-flooding risks even when they are far from rivers or coasts.
    • Specific adaptation measures: rain gardens, trees, green space, early-warning systems, emergency planning, recovery resources, and public transportation.
  4. New England’s regional climate risks
    • Rapid warming of nearby ocean waters.
    • Greater land–ocean temperature contrast in winter that can help fuel nor’easters, blizzards, and hurricane-force winds.
  5. Regional variation and uncertainty
    • Climate impacts do not follow identical rules everywhere because continents and local wind systems complicate global patterns.
    • The Sahara may become wetter despite the broader tendency for many subtropical dry regions to dry further.
    • Monsoon rainfall may become more variable and concentrated into fewer, more intense events.
  6. Wildfire impacts
    • Drier conditions make vegetation easier to ignite and allow fires to spread.
    • Wet-to-dry cycles can build vegetation and then convert it into highly flammable fuel.
  7. Humidity and heat-wave danger
    • Humid air makes sweating less effective, making humid heat more dangerous than dry heat.
  8. Mitigation and social response
    • The need to transition away from fossil fuels and reduce climate pollution.
    • The message that some worsening is already expected, but further escalation can still be limited.
    • Encouragement for people to discuss climate change within their communities to build understanding and counter despair.

Loading

What is generative learning?

Many of the recommendations I make for classroom and even nonschool-affiliated learning strategies are based in my understanding of generative learning. I have described a specific activity as generative in previous things I have written, but I don’t think I have ever made the effort to provide what I mean by generative. I decided I would give this background now both to explain what the term implies to me and to have something I can refer to in the future.

My applied work in educational psychology is based in cognitive psychology. Cognition is just a way of understanding thinking. Unless someone is really interested in digging into the field, I think it helps if I make an effort to translate some of the core ideas. There is always a danger making the complex simple is a bad idea and my efforts at simplification are off target, but I do it anyway. Think of thinking in terms of mental actions. Assume that learners have at their disposal mental actions they can use to accomplish the thinking and learning tasks they encounter. Learners may differ in which actions are selected to tackle a given task, how skillfully the tools are applied, and how effectively they evaluate the outcome of tool application to determine whether or not more needs to be done.  

Here are four actions with a description of the task to which each would  typically be applied:  

  • Attend – maintain certain ideas in consciousness (also called working memory)
  • Find and retrieve – locate what is already stored (also known as long-term memory) and attend to this content
  • Link – establish connections between information units stored in long-term memory  or that content active in working memory
  • Elaborate – create or discover new knowledge from the logical and  purposeful combination of active and stored memory components  
  • Evaluate – determine whether a cognitive task has been completed  successfully 

We can often take control and apply these activities without assistance, but motivation or lack of awareness of what activities might be useful can result in important activities not happening. Generative activities (Wittrock, 1974, 1990) are external to the internal mental activities of the learner but can make predictable internal activities more likely to occur. Questions about something a student is trying to learn make a good example. A question is external to the thinking of a learner. However, if I ask a question and you cannot answer, attempting to answer this question should have required you to evaluate your understanding. In attempting to answer my question, you have also probably made the effort to find and retrieve information. One related thing to consider – generative activities may encourage activities that are redundant with activities a learner have initiated on her own. This probably does no harm, but it also might be described as busy work. Cognitive activity is always the mental work of the learner with others only able to manipulate such behaviors indirectly and with less precision than a competent and motivated learner could do for themself.

What are some examples of generative activities? Fiorella and Mayer (2016) have identified a list of eight general categories most educators can probably turn into specific tasks. These categories include:

  • Summarizing
  • Mapping
  • Drawing
  • Imagining
  • Self-Testing
  • Self-Explaining
  • Teaching
  • Enacting

Summarizing – To summarize, students think about what they have just learned and then rephrase the most important information in their own words.

Mapping – Mapping is the process of converting words into a visual representation. Mind maps, tables, diagrams, and graphs are all common examples. 

Drawing – Drawing is a great way to help your students learn more deeply about the material you are teaching. When students draw, they have to think about what information to include, what to leave out, and how to best represent it visually. 

Imagining – Forming a mental representation of new information is surprisingly beneficial for learning. An example is tasking your students to imagine the process of digestion by creating mental pictures of each step.

Self-testing – Self-testing is a highly effective learning method. Educators likely recognize that retrieval practice (self-testing) is presently receiving a lot of attention. Some examples of self-testing include using flashcards and quizzes.

Self-explaining – Self-explaining requires students to recall new information and explain it in their own words. This helps students to understand the material better and to avoid simply repeating back what they have read or heard.

Teaching – Peer teaching is another active strategy requiring the recall and translation of what has been learned to present to others. Teaching involves preparation, delivery, and interaction related to the content to be learned. Most educators intuitively appreciate the unique requirements of teaching and recognize that learning for the self and to inform others involve different activities. 

Enacting – I think demonstrating is an acceptable way to explain what the researchers meant by enacting. 

Generative learning is a powerful approach to education that encourages learners to actively engage with the material, creating new knowledge and connections. This method, grounded in the work of Fiorella and Mayer (2016), and Brod (2021), among others, is centered around the idea that learning is not a passive process, but an active one that involves the learner in the creation of their own understanding.

The strategies I have listed require learners to select and organize relevant information, elaborate on the material based on personal knowledge, and integrate new information with existing knowledge.

Summarization, for instance, involves concisely stating the main ideas from a lesson in one’s own words. This goes beyond copying words or phrases verbatim from the lesson; rather, it involves selecting the most relevant information from the lesson, organizing it into a coherent structure such as an outline, and integrating it with students’ prior knowledge.

Teaching involves selecting the most relevant information to include in one’s explanation, organizing the material into a coherent structure that can be understood by others, and elaborating on the material by incorporating one’s existing knowledge.

Generative learning is not just about the creation of new content. Brod (2021) emphasizes that generative learning requires the production of a meaningful product that goes beyond the information that is an input. This means that activities like highlighting, which do not result in new content, are not considered generative.

Generative learning strategies are not just for students. They can be used by anyone looking to deepen their understanding of a topic. For example, if you’re reading a book or article, try summarizing the main points in your own words, or explaining the concepts to someone else. You might be surprised at how much more you understand the material!

Fiorella and Mayer (2016) offer one additional observation related to these eight types of activity. Four strategies (summarizing, mapping, drawing, and imagining) involve changing the input into a different form of representation.

The other four strategies (self-testing, self-explaining, teaching, and answering practice questions) require additional elaboration. This distinction contrasts ”knowledge-building” and ” knowledge-telling” (e.g., Roscoe and Chi, 2007). Knowledge telling is regarded as the weak form involving a restatement of what is known with limited activation of other existing knowledge (e.g., attempts to generate examples from personal experience) and less extensive monitoring of understanding. In knowledge-building, the strong form, the learner adds to core ideas from existing personal knowledge and in doing to reflects on the core ideas in greater depth resulting in more effective comprehension monitoring.

One additional comment about the eight categories is that the categories were explained by the scholars identifying this category system in terms what the learner could do. While learners could certainly decide to do these things without guidance, it is probably more likely that these external tasks are recommended or assigned by an educator. 

What I have described to this point is how I would likely cover this topic in an educational setting. This approach would be designed to be true to what I believe to be the origins of the ideas and learners can then apply what they find useful. Given this background, my own research and practice have both focused on a subset of this list of activities and have taken the general idea of using external tasks to encourage desirable mental activities to recommend activities that share characteristics with the tasks mentioned. I have focused on questions, summarization, teaching, and self-explaining and proposed applications that have included peer tutoring and collaborative notetaking, writing across the curriculum, computer-enabled study environments that involve testing associated with accuracy prediction and data collection that feeds the identification of specific areas needed more work back to students, and the technology-based collection and exploration of notes over extended periods of time to improve personal productivity (smart notes and personal knowledge management). Thinking of external activities that efficiently encourage important cognitive activities has proven a productive way to both think about learning and what tasks may be helpful in helping students learn.

References:

Brod, G. (2021). Generative learning: Which strategies for what age? Educational Psychology Review, 33(4), 1295-1318.

Fiorella, L., & Mayer, R. E. (2016). Eight ways to promote generative learning. Educational Psychology Review, 28(4), 717-741.

Roscoe, R. D. & Chi, M.T. (2007). Understanding tutor learning: Knowledge-building and knowledge-telling in peer tutors’ explanations and questions. Review of Educational Research, 77, 534-574.

Wittrock, M.C. (1974). Learning as a generative process. Educational Psychologist, 11, 87-95.

Wittrock, M.C. (1990). Generative processes of comprehension. Educational Psychologist, 24, 345-376.

Loading