An AI LLM Can Identify Important Missing Content In Student Notes

There seems to be a resurgence of research interest in note-taking and note-use. One source of new work was the controversy over whether students should take handwritten notes or use their laptops or tablets. A second focus has been generated by what I would describe as “out of classroom” notetaking described by those promoting personal knowledge management (PKM) and interactions with a second brain. 

I have had a long-term interest in notetaking and studying and recent activity has caused me to review old sources in light of newer activity. Possible connections between notes taken and stored digitally and the capabilities of AI tools seem worth exploring.

I still understand note-taking and note use from a decade old model that argues learning with and from notes can be outlined in terms of an immediate generative benefit of taking notes and by then using the external storage function of notes for review and study. A generative benefit implies that taking notes provides an advantage over simply listening and reading. The external task prompts the note-taker to identify important ideas because it is not practical to record everything, and then cognitively acts on the selected information to personalize the original content through summarization and connections to existing knowledge. Of course, there are many strategies for doing this, as well as individual differences and less active approaches, such as simply writing down what the instructor displays on his/her PowerPoint slides.

One topic relevant to how I imagine AI being used in interaction with student notes is work that examines how effectively learners have recorded important ideas. How many ideas find their way into the notes and are these ideas important ideas? Here you find some interesting controversies. Early work on note-taking in actual classroom settings found a substantial correlation between the volume of content students recorded and their exam performance (Nye and colleagues, 1984). I want to emphasize the word “naturalistic” here because, as the authors of this study claim, so much of the research has been conducted in short-term laboratory studies involving short segments of content and immediate examinations – not the situation students face in practice. I have made the same complaint in a recent post

A recent controversy related to what might be described as the “more is better” position is the handwriting vs. keyboarding controversy. The issue here is that keyboarding allows for more input, but handwriting appears to lead to more generative activity because the slower process encourages greater selectivity and summarization. My observation is that selection and reworking of notes does not have to happen in real time and having more content to work with during review and study makes such reworking more productive. 

This brings me to a second observation. While it seems possible that learners would not record information they think they already know, for one reason or another, students fail to record a sizeable proportion of important information, and what is not recorded can be related to specific later failures of recall and application on learning measures. It is difficult to study what you don’t know and what was not recorded (e.g., Kiewra, 1985; Lawson & Mayer, 2024).

Expert Notes and the AI Alternative

One solution to the many important ideas that students miss, especially less capable students, is to provide what are often called “expert notes” – essentially notes provided by the instructor or taken by an advanced student (Kiewra, 1985). This is a topic I explored in my own research with my graduate students (e.g., Grabe and colleagues, 2005). To be clear, the idea is not to substitute expert notes for student notes because of the generative value of taking notes, but to provide a second source learners can use to “fill in the blanks” for what they missed or perhaps did not understand well enough to even attempt recording. 

There are practical issues with providing quality notes to students. One issue that interested me in the citation I provided was the relationship between providing quality notes and class attendance. You do want students to attend class, but the reality is that there is also the question of what can be done for those students who miss class for legitimate reasons. Second, the generative effect of taking notes describes note-taking itself as a learning experience, so you really don’t want to discourage student note-taking. 

I have been exploring a use of AI that seems interesting and efficient in a couple of ways. It is relatively easy to prompt AI to identify the “important ideas in a lecture”, to compare this material with the notes taken by a learner, and to generate an output of the important ideas not contained in the student notes. This approach is efficient for students because it does not require reviewing the entirety of what might be considered “expert notes.” It also builds on rather than eliminates the student’s generative effort to create their own notes.

To explore how this might work, I used my current AI service to run the following prompt. The inputs included, first, the transcript of a source podcast on why climate change can be responsible for both drought and flooding, and, second, the notes I took while listening to the podcast. The original source contained approximately 2500 words and as a podcast required approximately 15 minutes. I attempted to play the role of students and created a set of notes while listening. I admit that I was surprised to learn a few things about how climate change works even though the material was prepared at the high school level. I purposely ignored comments that appeared near the end of the podcast that focused on fire danger and health concerns as a way to approximate what notes might look like if a student became distracted or cut out of class early. 

Prompt: I am providing two text segments. I want you to identify the important content from the first segment. I then want you to compare this important content with the content in the second segment and identify the important ideas in the first segment that are not included in the second.

I have included, as an Appendix, both the important ideas identified by the AI tool and the list of “important ideas” in the presentation that the AI tool said were not included in my notes. 

The approach identified and provided missing comments on both wildfires and health issues, which was my proof of concept when it came to responding to the problem of students missing important ideas, but it also included additional information that the comparison of the original content and my notes did not. So, when I compared the summary with my notes, the comparison in response to the prompt identified content I purposefully missed and other items I heard but did not record. This second body of information was much larger than I would have anticipated and fits existing research concluding even better students (apologies if this sounds conceited) fail to record more than one might expect. It is hard to know if this is an actual problem or simply a difference in standards. I have no idea whether the prompt could be altered to change how the AI tool interpreted “important content.” 

One interesting item – there is at least one item in the comparison between the AI summary of important ideas and my notes identified as missing that actually does not appear in the AI summary but was included in the full transcript (The claim that the most destructive floods now occur about three times as often as before). 

Final Comments

It is likely not a surprise that AI creates a summary of content, as this is a commonly offered service now often available as an alternative to asking readers or listeners to review all of the original raw content. I am not certain if I explored different wordings of my prompts if a request for the “important ideas” would be much different from a summary.

I have considered whether a summary or review of the most important information would offer essentially the same practical value as the second step I have AI to provide. The amount of content would differ. The summary I provide as an appendix is about a third of what would be generated from a traditional fifty-minute college lecture. I do like the idea of identifying what a student missed, which may have unique value in prompting students to consider their note-taking process. Helping students identify important information they missed also keeps the focus on student responsibility for the generative learning opportunity in making their own notes.

Sources:

Grabe, M., Christopherson, K., & Douglas, J. (2005). Providing Introductory Psychology Students Access to Online Lecture Notes: The Relationship of Note Use to Performance and Class Attendance. Journal of Educational Technology Systems, 33(3), 295-308.

Kiewra, K.A. (1985). Providing the instructor’s notes: an effective addition to student notetaking. Educational Psychologist, 20(1), 33-39. https://doi.org/10.1207/s15326985ep2001_5

Nye, P.A., Crooks, T.J., Powley, M., & Tripp, G. (1984). Student note-taking related to university examination performance. Higher Education, 13(1), 85-97. https://doi.org/10.1007/BF00136532

Appendix:

Important content in Segment 1

  • Climate change is intensifying both droughts and heavy rainfall/flooding.
  • A warmer atmosphere holds more water vapor, while hotter land surfaces evaporate moisture faster:
    • This dries land and worsens drought.
    • It also creates the potential for more intense downpours once rain occurs.
  • The most serious change may be in extreme storms, rather than in average rainfall or average storms.
  • Existing infrastructure is poorly suited to extreme weather, especially because roads, roofs, parking lots, and other impervious surfaces increase runoff and flooding.
  • Flood risk is not limited to coasts and rivers; intense rainfall can cause serious stormwater flooding inland.
  • Communities can reduce flood impacts through:
    • Updated emergency plans and warning systems
    • Recovery resources
    • Green space, trees, grass, and rain gardens
    • Reducing impervious surfaces, including through greater public transportation
  • Climate impacts vary by region:
    • New England’s warming coastal ocean can fuel stronger winter storms.
    • Subtropical regions, including the U.S. Southwest, are generally expected to become drier or more arid.
    • Some areas may not follow this pattern; for example, the Sahara may become wetter.
    • Monsoon regions may receive rain in fewer, more intense events, harming people and agriculture.
  • Water-cycle changes also worsen other hazards:
    • Drought can increase wildfire risk.
    • Alternating wet and dry periods can grow vegetation and then dry it into wildfire fuel.
    • Humidity makes heat waves more dangerous because it prevents sweat from cooling the body effectively.
  • Adaptation is necessary, but reducing fossil-fuel use and climate pollution is also essential to limit future worsening.
  • Community discussion and engagement are presented as meaningful individual responses to climate change.

Important ideas from Segment 1 not included in Segment 2 (my own notes)

  1. Examples and evidence of current extremes
    • California reservoir lows, Fort Lauderdale flooding, and Brooklyn’s extreme rainfall.
    • The claim that the most destructive floods now occur about three times as often as before.
  2. The distinction between average conditions and extremes
    • The largest storms are becoming stronger even if the average storm is not necessarily changing as much.
    • The primary concern is extreme downpours, not merely average rainfall.
  3. Detailed flooding and infrastructure discussion
    • Impervious surfaces such as roofs, roads, sidewalks, and parking lots worsen runoff.
    • Inland communities face stormwater-flooding risks even when they are far from rivers or coasts.
    • Specific adaptation measures: rain gardens, trees, green space, early-warning systems, emergency planning, recovery resources, and public transportation.
  4. New England’s regional climate risks
    • Rapid warming of nearby ocean waters.
    • Greater land–ocean temperature contrast in winter that can help fuel nor’easters, blizzards, and hurricane-force winds.
  5. Regional variation and uncertainty
    • Climate impacts do not follow identical rules everywhere because continents and local wind systems complicate global patterns.
    • The Sahara may become wetter despite the broader tendency for many subtropical dry regions to dry further.
    • Monsoon rainfall may become more variable and concentrated into fewer, more intense events.
  6. Wildfire impacts
    • Drier conditions make vegetation easier to ignite and allow fires to spread.
    • Wet-to-dry cycles can build vegetation and then convert it into highly flammable fuel.
  7. Humidity and heat-wave danger
    • Humid air makes sweating less effective, making humid heat more dangerous than dry heat.
  8. Mitigation and social response
    • The need to transition away from fossil fuels and reduce climate pollution.
    • The message that some worsening is already expected, but further escalation can still be limited.
    • Encouragement for people to discuss climate change within their communities to build understanding and counter despair.

Loading

Leave a Reply