I have spent a significant amount of time over the past week or so developing a Karpathy wiki based on a large portion of my Obsidian notes. This process began in April when I decided to purchase a Macintosh Mini I intended to devote exclusively to the exploration of AI on the desktop. I was a bit slow in making this purchase and it took until last week to receive my purchase. My tardiness cost several hundred dollars more than it would have a few months ago.
I was motivated to invest and explore this area for two reasons. First, my main interest in AI continues to focus on retrieval-augemented generation (RAG) of the notes and highlights I have collected to serve as a foundation for my writing projects. As I have used AI plugins to interact with the content I have stored and organized in Obsidian, I discovered that the API-based services for interacting with these notes are relatively expensive because the process of first feeding the notes to the AI service must be repeated each time a session is initiated. Karpathy proposed that AI could be used to create a wiki based on the concepts and connections in a collection of source material, either once (when the collection was created) or as each new item of content was added, and this wiki could then be the focus of future explorations, reducing the cost due to the repeated input of the same content to the AI service.
My second motivator was personal curiosity, sparked by the many posts promoting the potential of AI tools and models that could run on personal hardware, avoiding the costs and scrutiny associated with using online services from major AI companies. The proposal was that many common uses of AI no longer required access to $20 or $200-a-month subscription services.
I understand that another tutorial or “how I did it” post may be required at this point, but I read a post explaining that getting started with self-hosting LLMs will not be easy as the posts newbies are likely to read make it sound, and a good deal of exploration and personalization will be required. The message was intended not to be discouraging but to communicate that “don’t give up, you should be able to get it to work.” This was pretty much my experience, and I thought it worthwhile to explain the issues I encountered and why I had to make adjustments to my specific situation. My experiences with tech since the mid 1980s have kind of gone this way.
So, I have two Mac Minis now, and the first challenge was how to connect both to the same large monitor so I can switch back and forth as required by a general-use and a specific-use computer arrangement. I knew I would have to purchase a KVM (keyboard, video, mouse), but I had not considered that my current setup uses a Bluetooth mouse and keyboard. More specifically, Apple’s Magic Keyboard and Mouse are not intended to be linked to more than one device. You charge your Magic keyboard with a USB cable, so the cable can be used as it has long been used to connect to a computer. You also charge your Magic Mouse, but the cable is inserted on the bottom of the mouse, preventing it from being used while it is being charged. Solution – purchase a mouse with a cable. The first challenge is overcome.
My plan was to use the Obsidian Karpathy LLM wiki plugin because this seemed the most efficient way to create a working system. The plugin’s setup allows selecting multiple AI sources, including subscription services. I did use Anthropic’s Claude API when I was having difficulty getting either of the two local options (Ollama or LMStudio) to work. Claude worked great, but adding one new source document cost 70 cents. My present collection is close to 300 note files, and the work the AI does increases as the complexity of the wiki increases so I treated the success as a sign the struggles I was experiencing could eventually be overcome.
When using Ollama, I was experiencing a consistent problem with some, but not all of the note files the AI was ingesting to build the wiki. I spent a considerable amount of time over several days comparing the files that could and could not be processed and I never did find a difference. It wasn’t the length, the presence of specific markdown tags, the tool I had used to create the original markdown file, or any other variable I could imagine. Nothing. However, the problem was consistent. The same files, time after time, would either work or fail.
My typical strategy in such situations is to ask questions of the Internet. One proposal was that the JSON history had become corrupted. The solution was to reveal the invisible files (the .files and folders) and delete these files. New files would be generated when the Obsidian app was next launched. This was done without consequence.
One issue I encountered was that the models displayed as options within Ollama did not contain the model (qwen2.5) I had found recommended when I read the descriptions of others. I searched how to add other models to Ollama and found it could be done with a terminal command (ollama pull <model name>. Now qwen2.5 appeared. Qwen3.6 was originally listed and I assumed there would be little difference, but for some reason, I was wrong, and the system worked with qwen2.5.
Without going into details because others have already provided tutorials, you first add and install the Karpathy LLM Wiki plugin for Obsidian. The gear icon associated with this community plugin provides a “fill in the blank” form where you enter information linking Obsidian to the AI online service or local option you want to use.
The wiki construction process is controlled by the commands that appear in the Obsidian command list when the Karpathy plugin is installed.
So, you start Ollama and select the model you will use in Obsidian. Start Obsidian and select the command to ingest a file or folder and be patient. Eventually, your wiki will be generated, and you can query the wiki rather than the source files. The right-hand column displays a response to a prompt.
So, I was able to generate a wiki based on more than 150 of my notes. In examining some of the components of the wiki I did find some weird artifacts. There were some with Chinese characters. I happen to be listening to LeoLaporte talking about different AI models and he said that qwen originated in China. It then made sense to me that the model might translate some of the Chinese names in my article summaries and include their Chinese translations (no idea if that is actually what happened). I also found some md pages with titles, but no content.
When I used the Karpathy command to submit queries, I found the quality of the responses to vary. Some made sense and some ignored sources I knew existed and were central to what I expected. Rewording of the query in ways I thought the model should have understood as equivalent sometimes resulted in the response I expected.
The following comparison (first from querying my new wiki, and the second from using Claude with my original source files) demonstrates fairly substantial differences. As you can tell from a quick skim, there are significant differences in the eloquence and completeness of the responses. The sample responses are presented as an Appendix.
Summary
Each user is likely to have a unique experience when applying a local AI system to their content to generate a Karpathy-style wiki. Expecting that it will take some tinkering is good advice even when following instructions or “how to do it” videos on YouTube. My effort here was to describe what this tinkering experience looked like.
After at least getting the system to work, I was able to compare my first successful effort with what I generated for 44 cents using Claude. The differences were substantial.
I assume there are many more issues to explore. I am currently repeating the process with a different script that instructs the AI on what to do when creating the wiki. If that effort produces substantial improvements, I will have to write another post.
*******
Response of local model to wiki
Summary of Research Comparing Handwritten Notes vs. Computer Notes
According to the literature summarized from various studies, there are several key findings regarding the comparison between handwritten notes and notes taken on computers:
- Repetition in Note-Taking:
– A study by Bui et al. 2015 suggests that repeated exposure to material enhances learning outcomes, especially when coupled with effective note-taking strategies.
- Handwritten Notes and Memory Retention:
– Longhand notes are described as more paraphrased and containing more signals and images, which can lead to higher text-related achievement when reviewed (from Longhand Notes).
– Lecture Note-Taking Strategies indicate that students who take notes by hand outperform those who use digital devices, as handwritten notes require a higher level of processing and engagement.
- Editing and Revisions:
– Research by Mueller and Oppenheimer (2014) found that while typing is faster, it leads to less cognitive effort and thus less retention due to the lack of engagement required in handwriting notes.
– Follow-up revisions are crucial as they enhance memory retention. According to a study cited by AI response, revising digital notes can mitigate some of the disadvantages of digital note-taking, improving long-term recall and academic performance (from AI Response to Challenging Luo Study).
- Professional Note-Taking:
– Additionally, in the context of professional training, effective note-taking is vital for future practitioners (from “Notetaking in Professions”).
References
- Longhand Notes — Core mechanism explaining why handwritten notes are more beneficial.
- Bui et al. 2015 — Effects of Repetition on Note-Taking and Recall Strategies in College Lectures.
- “Notetaking in Professions”. Stacy and Cain, American Journal of Pharmaceutical Education (2015) — Importance of note-taking skills for professional training.
- Mueller and Oppenheimer (2014) — Pen and Pad versus Laptop for Note-Taking.
- AI Response to Challenging Luo Study — How Note Modification Research Might Counter Luo et al. (2018)
*******
Paste from Claude plugin within Obsidian
Handwritten vs. Computer Note-Taking: What Your Notes Say
Your vault contains extensive research on this topic, and the picture that emerges is more nuanced and contested than the popular narrative suggests. Here is a synthesis of what your notes reveal:
The Foundational Study
The most influential study is Mueller & Oppenheimer (2014), documented in Pen is mightier than the keyboard. Their key finding was that faster keyboard transcription led to more verbatim, less generative notes, while the slower speed of handwriting forced summarization and paraphrasing. Notably, their Study 3 found no differences when note review was not allowed, but longhand was superior when review was allowed—a surprising result.
The Generative Processing Argument
Several of your notes outline the theoretical rationale:
- Translation hypothesis: Writing adds a distinctive encoding channel beyond just listening (Luo, Linlin, Kenneth A. Kiewra, Abraham E. Flanigan, and Markeya S. Peteranetz2018)
- Generative hypothesis: Handwriting’s slower speed forces paraphrasing, organization, and integration with prior knowledge, leading to deeper processing (Depth and Generative, EncodingEffect)
- As noted in transcription speeds, adults type at ~33 wpm and write longhand at ~22 wpm, meaning handwriters must be more selective
Your note in transcription speeds raises a sharp critique of this logic: it “suggests a strange lack of awareness on the part of the learner. It would seem that the keyboarding learner is oblivious to the benefits of generative processing and goes with what is easier, but less effective.”
Contradictory and Inconsistent Findings
Your notes consistently emphasize that research findings in this area are contradictory:
- Urry.et.al documents a major replication of Mueller & Oppenheimer that found inconsistent results. Urry found laptop users took more notes and more notes related to better performance. Critically, they differentiated quantity from verbatim overlap—quantity was positively related to performance while verbatim proportion was negatively correlated.
- Luo, et al 2018 found that when notes were recorded but not reviewed (process function), laptop was actually better for image-related learning. Only when notes were reviewed (product function) did longhand show an advantage. The authors themselves note that “findings in this area of research are often contradictory and uncontrolled variables may be responsible.”
- Flaniganetal2023 found that, surprisingly, longhand and computer note takers recorded a similar number of ideas during the lesson, contradicting the expected computer advantage. However, longhand revisers added three times as many complete ideas during revision pauses.
The Note Completeness Counter-Argument
Your notes in NoteCompleteness and Colllaborative Notes document an important counter-narrative: note completeness predicts achievement, and computers produce more complete notes. As noted:
“The studies of note volume and completeness would seem to challenge the argument that taking notes on a computer is a liability because more notes are taken implying less thinking about the content.”
Bui et al. (2013) found that students with access to their more complete (computer-generated) notes demonstrated better performance, directly contradicting Mueller & Oppenheimer’s interpretation.
The Revision Factor
Multiple notes highlight that the handwriting vs. computer debate may miss the most important stage entirely—revision:
- Three stage model of note-taking proposes that effective note-taking is a three-stage process: recording ? revision ? review, and most studies ignore the revision stage.
- Cojeanatal2024 found that computer note-takers were more likely to reformat and revise their notes after class, potentially compensating for less generative initial encoding.
- Flaniganetal2023 found longhand revisers added more ideas during revision pauses, but this study used interspersed pauses—an artificial design unlikely to reflect real practice.
- Cohen2013ActiveSummary demonstrated that note restructuring significantly improved exam scores regardless of medium, suggesting what you do after taking notes matters more than how you take them.
The Children’s Research
Your note childrennotes documents a unique study (Horbury & Edmonds, 2021) with 10–11-year-olds that found:
- Conceptual understanding was superior with handwritten notes
- Unlike adult studies, there were no significant differences in note quantity between conditions
- The authors speculated that children’s limited keyboarding experience may have increased cognitive load when typing, a different mechanism than proposed for adults
Your AI-Generated Synthesis
Your note AI response to my question challenging the Luo study that assumed the notes themselves could not result in generative activity offers a compelling reframing:
| Luo et al. Conclusion | Modification Research Counter |
| Laptop notes are transcription-oriented | Generative processing can be added during revision |
| Longhand notes are better review products | Laptop notes can be transformed into superior products through revision |
| Laptop notes lack images and signals | These can be added easily using laptop affordances |
| Reviewing longhand notes yields higher achievement | Studies only tested passive review, not active modification |
| Longhand is generally preferable | The medium matters less than the strategy |
Overall Conclusions From Your Notes
- The popular narrative favoring handwriting is overstated. Replication attempts and meta-analyses show inconsistent results.
- The mechanism matters: Quantity of notes is positively related to achievement; verbatim overlap is negatively related. These are separate variables that should not be conflated.
- Most studies use artificial conditions (immediate testing, brief delays, no real revision opportunities) that don’t reflect actual academic practice.
- Revision and post-processing may be more important than the initial recording medium, and computers offer superior affordances for revision.
- The real issue is strategy, not medium. Untrained students default to transcription on keyboards and forced summarization by hand—but training could equalize outcomes.
- Note completeness matters for the storage function, and the storage function is generally more important than the encoding function (ExternalStorage).
You must be logged in to post a comment.