There are multiple ways to think about what using AI in writing could mean. Here is one approach. As I have studied and used AI in writing, I have noticed what I think is a continuum of categories that I can operationalize based on different AI tools and how they move from information to written product. Most of these tools are more flexible than I describe here, but I think the distinctions I identify make sense.
The products I am referring to range from a completed blog to ideas or an outline I might use to guide my own writing. Here is one example.
Assume my goal is to write a 1000-word blog post on how taking notes by hand versus using a keyboard influences student learning and understanding. Here are various approaches I might take and the tools/services I would use.
ChatGPT or Claude. Submit the goal just stated to an AI tool as a prompt and accept the output as the written product. In this case, the input information is the knowledge base stored as the model within the system and the mechanism the system uses for translating that knowledge into a product. Beyond submitting the prompt, I need to do nothing else to generate an output.
Recall.ai. This approach uses RAG to designate information sources that an AI prompt then uses to generate a product. Given my goal, I might submit, say, 15 PDFs of journal articles relevant to the topic I have in mind. The AI tool then creates a summary (I can control brief vs extended) of each article. Once completed, I submit the prompt explaining my general goal. The one thing I control is the information input to the system. I don’t have to read the documents and the AI system creates an output based on the designated content based on the request identified in the prompt.
Obsidian and AI plugin. I read, highlight, and annotate 15 PDFs of journal articles on the topic of interest. Highlights and annotations are first exported from the tool I have used to read these articles and are then stored as a note or notes within Obsidian. The AI plugin in Obsidian then uses the same prompt identified earlier to generate the written product. In this approach, I have selected the articles, designated information I think is relevant (highlights), and added my own insights as annotations. The AI uses the information I designated as relevant and my annotations to generate an output.
Obsidian to Karpathy Wiki to Product. I read, highlight, and annotate 15 PDFs of journal articles on the topic of interest. Highlights and annotations are exported from the tool I used to read these articles and stored as notes in Obsidian. AI is now used in two distinct stages. First, the AI runs on the content extracted from the original PDFs to identify key concepts, entities (e.g., researchers), and summaries of individual sources. The concepts, entities and summaries are linked by AI to establish relationships – e.g., which researchers conducted which studies, which studies reached a given conclusion or contradicted a given conclusion. The second use of AI applies the goal prompt to this more organized information to produce the final product. (Note: The goal of this process is really to be able to add new resources over time and not have to process the entire batch of inputs each time a new prompt is submitted. This approach offers an efficiency and cost advantage. However, I am focused here on initially identifying themes and concepts, then using them as distinct inputs for the final product as a different approach and possible parallel to how human writers function.)
Obsidian. I read, highlight, and annotate 15 PDFs of journal articles on the topic of interest. I first export highlights and annotations from the tool I used to read these articles and store them as notes within Obsidian. Once stored, these notes are tagged, linked, and new related notes are generated by thinking about the content already stored. I write the final product to meet my proposed goal using this content. Obviously, there is no AI use at this end of the continuum I have constructed but I wanted my list to move from total AI to total human.
I have used most of these approaches, except for simply posting something written without any input beyond my prompt. I wonder objectively which of these approaches produces the product of the highest quality, but that is likely a nuanced question and I will leave it for others to explore for now.
I frequently write about AI applications in education, so when I find a relevant research study, I pay close attention. It is not that there is a lack of attention to AI among educators; there are plenty of suggestions for classroom applications and data on educators’ and students’ practices. What interests me most are manipulative research studies that test the impact of specific uses of AI on achievement. The following is a description of one such study and a critique focused on how educators may misinterpret what they think the study shows about classroom applications. The concerns I emphasize were explained in detail in a previous post.
Effects of LLM use and note-taking on reading comprehension and memory
Study behavior has been a professional interest for years. Generative activities, external tasks such as answering questions or taking notes that potentially improve the efficiency or even application of beneficial cognitive (mental) behaviors, have been the framework I have used as a way to offer learners ways to improve understanding and retention. The interaction between learners and AI offers multiple ways a learner can use an LLM to create generative experiences for themselves. For example, a learner can request that an LLM present and evaluate their responses to questions about content submitted as part of a prompt or even just the description of a topic of study (e.g., Ask me five multiple-choice questions about operant conditioning one at a time and provide feedback after my response to each question).
Pia Kreijkes and colleagues wanted to evaluate how high school students would apply an LLM while reading an assignment and whether their actions would be related to multiple measures of understanding and retention. To put the impact of the AI activities in some type of perspective, the researchers decided to compare the impact of the AI activities with a more traditional generative strategy – taking notes. In their study, hundreds of high school students were assigned a reading task that would take between ten and fifteen minutes under two of three treatment conditions – take notes while reading, interact with an LLM while reading, or do both. This is called a within-subjects design, as each participant engaged in two of the treatment conditions, allowing for the elimination of some of the variability (error) due to differences (e.g., reading ability) among participants. It is a weird design in that in most within-S studies each participant engages in all of the treatment conditions and why this two-of-three approach was used I cannot explain. Anyway, as a research design, the number of participants and the within-S design lend high confidence to the results.
Three days later, participants responded to items testing their retention and understanding and a request to free-recall as much information as they could.
The results showed that the note-taking treatment and the combination of LLM and note-taking treatments scored higher than the LLM-only treatment. The researchers concluded that the benefit of note-taking over LLM use tended to be stronger and was observed across all measures of learning, whereas the benefit of LLM and Notes was observed for literal retention and comprehension but not for free recall.
The researchers attempted to qualitatively analyze the prompts students generated with the LLM. The most common prompt emphasized seeking additional information and deeper understanding rather than information condensation or study and memory help.
This is a very basic description, but I am interested in what readers might assume from it. Consider what you think this study suggests for secondary classrooms. I will offer my own takeaway after making a few specific comments on this study.
Consider the Methodology Carefully When Using Findings to Guide Actions
Here are some issues I have with the methodology:
Why is there no actual control group (reading only)? The sample size certainly would allow the existing treatments to be compared to a read-only treatment. This may matter because of other issues I have with the methodology.
Why is there no opportunity for review or what I would describe as study? There is a small, well-known generative benefit to taking notes as a way to process input. However, there is a much larger benefit of using notes post-exposure, but before the assessment of retention and understanding. I have not read studies on LLM interaction that report immediate benefits. One might assume this would be there, but it did not seem to be in this study, given the specific content studied. I would be more interested in LLM benefits as part of review or as a way to get additional insights as the original memories have faded.
The three-day delay seems short if one is interested in typical classroom assessments or in generalizing to other out-of-classroom uses of notes and LLMs.
The content seems very short – you can both read and process in between 10 and 15 minutes.
My takeaway. Here is the issue I think the study suggests. This might surprise you. I think it is impressive that high school students, when given access to an AI tool, use it to improve their understanding through additional information or a different way of explaining something. It was not used to make the learning task easier – some version of a summary so I don’t have to read it all approach.
One guess about why so many potential improvements may have been ignored. Why not use longer reading passages? Why not a longer delay and opportunities to review? Why not an expansion of the treatment conditions or a more powerful within-subjects approach in which subjects experienced all treatments. My guess is that to get schools to allow researchers access to students, schools wanted an approach that was quick and over quickly. In my experience, this is understandable, but also a severe limitation on research that would provide the most useful information for applications.
What I think should be the focus on both note-taking and student use of LLMs
As I alluded to above, the greatest benefits to learning and understanding come from what learners do after initial exposure to experiences. One way to think about this is to describe such activities collectively as studying. Simple strategies such as retrieval practice and interleaving have powerful and consistent benefits. There are plenty of strategies for working with personal notes over time that offer variations on these proven learning strategies and some exciting opportunities for student use of LLMs. Students can have an LLM ask them questions and more importantly evaluate responses and provide feedback and interaction based on the feedback. The LLM can function in ways as a tutor or even be used by small study groups as a tutor.
It is easy to imagine research on spontaneous use of such study possibilities or structured approaches that would guide student use of note taking, LLMs, or potentially the combination. As I have suggested, the logistics of implementing such applied research is challenging, but I don’t think the short-term, quick experiments provide the level of authenticity that is needed.
Kreijkes, P., Viktor Kewenig, Martina Kuvalja, Mina Lee, Jake M. Hofman, Sylvia Vitello, Abigail Sellen, Sean Rintel, Daniel G. Goldstein, David Rothschild, Lev Tankelevitch, Tim Oates. (2026). Effects of LLM use and note-taking on reading comprehension and memory: A randomised experiment in secondary schools. Computers & Education, (243), 1-24. (Article 105514)
‘
Posted inUncategorized|TaggedAI, studying|Comments Off on Reading With LLM Access: It’s Not About Studying
I have been reading Columbia University’s legal scholar Timothy Wu’s Age of Extraction for a second time. I highly recommend this book for those interested in an analysis of the big online platforms (e.g., Google, Amazon, Facebook). With the passage of time and the new life experiences that occur, a second reading can allow different insights and new topics for sharing. I first read Wu’s book soon after it was published and at a time I was also concerned about “free” social media and the damage done by manipulative algorithms. I was exploring AI, but like Wu made few connections beyond noting AI companies like social media were dominated by large platform companies. It is these insights into AI platforms and possible alternatives I want to explore here.
Wu’s book is partly about the history of our opportunities online and how the original opportunities for individuals to independently share their insights and explore commercial opportunities were eventually taken over by a few major platforms. This transition was explained in terms of first the opportunities platforms provided, first free everything, and then the transition to the manipulation of time and attention in a way that made money for platforms and for the associated boards and stock holders. So many have been captured because of the network effect (I can’t leave because my friends are all there) and while little can be done about this for the majority, Wu offers suggestions for those who are interested in alternatives.
Is AI different or is it still too early?
Upon my second reading, I am more thoroughly steeped in the short history of AI and the opportunity to explore the possibilities both for those in the education community and for myself. AI platforms don’t seem to have followed the same pattern as social media. AI did not grow from the initial interests and activities of hobbyists and individual explorers toward the integration of everyone by the platforms making everything easier and more powerful. The ideas may have originated with the community of academics exploring various approaches to thinking machines over the decades, but the present large language models required the huge financial resources of major corporations and their opportunity to raise massive sums of money necessary to both explore and build the models we can now access. Even the massive budgets of these companies may not be able to feed the beast that is emerging as the way to develop this field. Governments may need to be involved to limit the redundancy that is at the core of the typical capitalistic approach. I explore the ideas of those who believe China has an advantage because the powers that be there can require a more integrated and less competitive approach.
The situation with AI has changed a bit. Progress has been made to the point at which individuals and smaller organizations can create and obtain value from the application of smaller models on equipment as inexpensive as a Mac Mini. These opportunities have captured my time and treasure for the past few months. Is it possible we are witnessing a reversal of the typical platform dominant pattern Wu describes?
Here is a short description with a few more details describing Wu’s platform model.
Platforms begin by offering genuine convenience. Leading AI companies offer cheap or free assistants, APIs, coding tools, and cloud-hosted models that lower barriers for developers and users.
Dependency follows scale and network effects. A company can become hard to leave once an organization has built workflows, data pipelines, fine-tuning, agents, evaluations, and staff skills around its models and APIs.
The platform becomes a gatekeeper. AI firms and their cloud partners can control access to compute, model capabilities, distribution channels, app ecosystems, and sometimes the terms under which others can build businesses.
Extraction comes after lock-in. This could appear as higher API prices, usage metering, restrictive contracts, preferential access for the platform’s own products, data capture, or fees are imposed on developers and businesses that rely on a given platform.
Concentrated power shapes the whole economy. If a small group controls the main models, chips, cloud infrastructure, and routes to users, it can influence which firms survive, what work is automated, and who receives AI-generated productivity gains. The power to influence government decisions when AI platforms reach a scale of influencing the economy are possible. Note the present argument that these platforms must be supported to match or exceed advances in other countries.
The process sounds scary when described like this. For me, we are already well on our way.
My personal interest has been in the application of AIs analysis and writing skills to the content I have abstracted from my personal reading and augmented with my thoughts as notes. To do this, I must first upload my collected content (a process typically called injestion) and then interact with this content searching for insights and ways of expressing what such insights might reveal. I am much less interested in interacting with AI based on the existing models than the application to my content. This is most frequently described as Retrieval Augmented Generation (RAG).
As long as you can trade time for speed and power, RAG applications can now be executed on your own equipment. While not the equivalent of similar approaches using more powerful platform models, the output is certainly useful to my personal work flow and I assume will only become more so if these efforts on personal AI are allowed to continue.
The issue is whether this process will be allowed to flourish. Will the companies developing these smaller models start selling them or not release them at all? Wu doesn’t address such innovations, but the pressure to commercialize that he describes would make this prediction. While these smaller models are available from several sources, I am presently using QWEN which is made available by a Chinese company. I suppose this will eventually raise concerns (see the following comment about dangerous uses), but after some trial and error and suggestions from others models from this company just seem to work better for what I am trying to do.
One thing I already see that worries me is the concern that effective AI outside of the control of big corporations opens up the possibility that dangerous actors will use these less controlled tools to create dangerous biological weapons or engage in hacking activities making our reliance on online resources more dangerous. The approach of raising concerns of hypothetical dangers seems a common corporate strategy – typically it is about children and who can best protect them. I see this more about what we believe is the best approach to experimentation and improvement. Present approaches cost out individuals, small companies, and even university researchers. It seems about who will be able to convince Congress of what?
Source:
Wu, T. (2025). The age of extraction: How tech platforms conquered the economy and threaten our future prosperity. Knopf
Posted inUncategorized|Comments Off on AI’s Future: Bigger and Bigger, Smaller and More Personal, or Both
The present political administration has staffed key government agencies with individuals who are more loyal to the administration than to the traditional responsibilities of the agencies they now head. In response, multiple climate scientists have banded together to offer their own site for current research in their field.
A good proportion of what I write is based on highlights and notes I have stored in Obsidian. Collaborative notetaking has always been a personal interest. There is a way to share Obsidian notes directly, but because this requires your content to actively use a server located off your own computer, it requires a subscription. I have no objection to supporting developers through subscriptions, but I have reached my personal limit on how many subscriptions I can handle. There are other ways.
In a previous post, I described Glasp, which is a site for highlighting and annotating with built-in AI. The AI is referred to within the service as an AI clone – implying that you can prompt your own documentation. There is also a “find others with similar interests” feature that takes you to the notes of others willing to share, and you can prompt their content in a similar way.
One of my more popular posts explained how to export Obsidian content to NotebookLM. It turns out NotebookLM notebooks can be shared, so I decided I would share my content for anyone who wants to use it. I am an educational technologist, mostly now exploring note-taking, AI applications, and generative processes to improve retention and understanding. You may or may not be interested in these areas, but if you are a NotebookLM user you might just want to explore this sharing option. NotebookLM is a great tool for interacting with content using AI and perhaps even creating a podcast based on queries of this content. You cannot add to or edit someone else’s content, which I wouldn’t want, but a user is free to explore your shared content using the “read only” features of NotebookLM.
Fitting with my personal interests, the opportunities an educator might find for collecting specific references he/she wants students to explore just seems like a useful application.
A very brief tutorial
A screen capture of the topic of an open Notebook appears below. The process of sharing this notebook is initiated from the share button at the top of the browser window.
Selecting the share button opens an overlay with multiple options. My intent is to offer public access to explore my notes. NotebookLM then returns a URL others can use.
Andrew Karpathy has identified an issue with the way those of us who make use of AI to interact with content we have collected (RAG- retrieval-augmented generation). Each time we submit a prompt to query our designated content, the AI resets and reprocesses it to address our query. As I will eventually demonstrate, this repeated input processing can be significantly demanding, and if you pay by the token or have a plan that has a token limit, this repetitive processing can be quite expensive. Karpathy proposed that you have a reasonable understanding of what you generally want to get from your curated content, you can find and export this content once and then focus on an organized wiki (Karpathy’s term) of the types of information you anticipated would be of value. Karpathy’s concept and its multiple variations have generated considerable attention, including interest in running them on personal computers. It is not my intent to focus on the how-tos of the LLM-wiki process. I am intrigued and I did purchase a mac-mini I can dedicate to exploring this process. The popularity of this idea has made purchasing Mac Minis very popular and the machine I purchased in April is not scheduled for delivery until mid-July. I take this as an indication of the potential usefulness of this approach.
What I want to document here is the inefficiency of RAG. Unless you are a heavy AI user and unless you make frequent use of AI to explore a large pool of content that you designate, you are unlikely to encounter this challenge. I don’t presently have a $20 a month account. Instead, I have a lower-max account (Abacus.AI) and pay for several accounts that allow me to upload content for exploration (Mem.ai, Recall.ai). The situation that allowed me to carefully analyze token use was the API plugin I used to apply AI to content I have stored and organized in Obsidian. This plugin accesses Claude via an API and a “pay as you use” plan. Anthropic provides detailed usage data, enabling interesting analyses.
Claude via Obsidian
I have used Obsidian to store notes and highlights I generate from my reading for several years and I have collected a sizeable body of content. The folder on note-taking I am using in the following demonstration contains 132 notes totaling 38,208 words. The following demonstration takes the following approach. The data available from my use of Claude to prompt this folder with prompts is saved by the day so for purposes of this little experiment, I issued one prompt a day.
Prompts by the day:
Day 1: Under what circumstances is notetaking a generative activity
Day 2: What have research studies shown regarding the relative effectiveness of taking notes by hand or using a keyboard?
Day3: Using my notes, generate separate responses to the following prompts. Under what circumstances is notetaking a generative activity? What have research studies shown regarding the relative effectiveness of taking notes by hand or using a keyboard?
Logic of this approach. The data retained by Anthropic for use of Claude (4.6 in this case) is totaled daily. In addition to the charge to my account (one chart), a daily token total is provided, along with a running total of Tokens In and Tokens Out. By tracking the totals in combination with the charts provided, it is possible to demonstrate the problem others have demonstrated with AI and RAG – the processing required for inputting the content is required each time the system is used hence asking for a response to the same two requests following a single input will save a considerable number of tokens (i.e., day 3 token use will be nearly the same as either day 1 or day 2). So, the cost and token usage on day 3 were only slightly higher than on either day 1 or day 2.
Summary
I must admit I was surprised at the cost of asking an AI tool to focus on content a user provides instead of asking a similar question to be answered from that tool’s own stored “knowledge”. These seemed basic requests someone might make many times a day in exploring with AI, but the focus on content the tool must first tokenize adds substantially to the cost. Given this cost, the Karpathy wiki strategy makes a lot of sense. At present as a temporary strategy, I store the output from my Obsidian-related AI queries in a separate Obsidian folder and I can then target this folder for a lower Input Token cost.
One more thing. Claude’s response to my query was great and would have been quite helpful had I been focused on creating a post. The structure of the output was, I would say, insightful. I have appended the response to my query about taking notes by hand versus on a digital device (see below).
*****
Notetaking – Hand vs Keyboard
Hand vs. Keyboard Note-Taking: What Your Notes Show
Your notes contain extensive coverage of this topic across multiple studies. Here is a synthesis of the key findings:
The Foundational Study
The landmark study by Mueller & Oppenheimer (2014), documented in Pen is mightier than the keyboard, found that keyboard users recorded more content but that longhand note-takers performed better, particularly on conceptual understanding items. In Study 3, which included a week delay with note review, differences favoring handwritten notes emerged specifically when review was allowed.
The Generative Processing Explanation
Several of your notes describe the primary theoretical explanation for a longhand advantage. Because typing speed (~33 wpm) exceeds handwriting speed (~22 wpm), as noted in transcription speeds, longhand note-takers are forced to summarize and paraphrase rather than transcribe verbatim. This engages generative processing—connecting new information to prior knowledge and reorganizing ideas—which promotes deeper learning. As described in Luo, Linlin, Kenneth A. Kiewra, Abraham E. Flanigan, and Markeya S. Peteranetz2018:
“Laptop note takers had a transcription orientation and recorded mainly verbal information… whereas longhand note takers had a generative orientation and used a combination of verbal and spatial note-taking strategies.”
The Luo et al. (2018) study further found that keyboard users recorded fewer visual notes (charts, graphs) and that the process function was more important for keyboard notes while the product function (review) was more important for longhand notes.
Contradictory and Nuanced Findings
Your notes reveal that the research is far from settled:
The 2022 Meta-Analysis (2022 Notetaking Meta-Analysis) by Voyer et al. analyzed 36 articles and found no overall effect for notetaking approach. The authors speculated that studies showing a longhand benefit may have been confounded by distraction opportunities on digital devices.
Urry et al. (2021) (Urry.et.al) replicated Mueller & Oppenheimer and found inconsistent results. Laptop users took more notes, and more notes were related to better test performance. Importantly, Urry differentiated quantity from verbatim overlap—quantity was positively related to performance while proportion of verbatim overlap was negatively correlated.
Note completeness research (NoteCompleteness) challenges the longhand advantage by showing that the volume and completeness of notes influence retention. Bui et al. (2013) found that students with access to more complete notes (as produced by keyboards) demonstrated better performance, directly contradicting the Mueller & Oppenheimer interpretation.
The Revision Factor
Your notes highlight that the comparison changes substantially when revision is introduced:
Flanigan et al. (2023) (Flaniganetal2023) found that among note revisers, longhand note-takers added three times as many complete ideas during revision pauses as computer note-takers, and scored more than half a letter grade higher on posttests. However, the advantage emerged primarily through the revision process rather than initial note-taking alone.
Cojean & Grand (2024) (Cojean&Grand2024) found that computer note-takers were more likely to reformat their notes after class, suggesting they compensate for shallow initial encoding through later revision—a finding that complicates simple hand-vs-keyboard comparisons.
Research with Children
Your note childrennotes describes Horbury & Edmonds (2021), which is notable as one of the few studies with children (ages 10-11). They found conceptual understanding was superior when taking notes by hand, but unlike adult studies, there were no significant differences in the amount of content recorded. The authors speculated that children’s lesser keyboarding experience may have produced greater cognitive load when typing.
Critical Perspectives from Your Notes
Your notes contain several important critiques of this research area:
The generative argument has a logical flaw: As you note in transcription speeds, the logic assumes “a strange lack of awareness on the part of the learner. It would seem that the keyboarding learner is oblivious to the benefits of generative processing and goes with what is easier, but less effective.”
Modification can equalize outcomes: Your AI response to my question challenging the Luo study that assumed the notes themselves could not result in generative activity argues persuasively that generative processing can be shifted to the review phase, that laptop notes provide a richer base for modification, and that laptop affordances enable superior reorganization. The conclusion: “The medium matters less than the strategy.”
Ecological validity concerns: Multiple notes flag that most studies use brief delays and immediate testing, which differs significantly from actual classroom conditions where review may occur days or weeks later (Luo, et al 2018, Flaniganetal2023).
Summary
Finding
Support
Longhand produces more generative/paraphrased notes
Consistent across studies
Keyboard produces more complete/voluminous notes
Consistent across studies
Longhand leads to better learning outcomes
Mixed—meta-analysis shows no overall effect
Note completeness predicts achievement
Supported, which favors keyboard
Longhand advantage strongest with note review
Supported in some studies, not replicated in others
Revision can compensate for shallow keyboard encoding
Supported by modification research
The overall picture from your notes is that the hand-vs-keyboard question is far more nuanced than popular accounts suggest. The medium matters less than the strategy the learner employs, and the opportunity for revision and post-processing may be the most important factor in determining which approach is more effective.
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it.
You can revoke your consent any time using the Revoke consent button.
You must be logged in to post a comment.