AI tutoring based on retrieval generated augmentation

Here is a new phrase to add to your repertoire – retrieval generated augmentation (RAG). I think it is the term I should have been using to explain my emphasis in past posts to my emphasis on focusing AI on notes I had written or content I had selected. Aside from my own applications, the role for retrieval generated augmentation I envisioned is as an educational tutor or study buddy.

RAG works in two stages. The system first retrieves information from a designated source and then uses generative AI to take some requested action using this retrieved information. So, as I understand an important difference, you can interact with a large language model based on the massive corpus of content on which that model was trained or you can designate specific content to which the generative capabilities of that model will be applied. I don’t pretend to understand the specifics, but this description seems at least to be descriptive. Among the benefits is a reduction in the frequency of hallucinations. When I propose using AI tools in a tutoring relationship with a student, suggesting to the tool that you want to focus on specific information sources seems a reasonable approximation to some of the benefits a tutor brings. 

I have tried to describe what this might look like in previous posts, but it occurred to me that I should just record a video of the experience so those with little experience might see for themselves how this works. I found trying to generate this video an interesting personal experience. It is not like other tutorials you might create in that it is not possible to carefully orchestrate what you present. What the AI tool does cannot be perfectly predicted. However, trying to capture the experience as it actually happens seems more honest.

A little background. The tool I am using in the video is Mem.ai. I have used Mem.ai for some time to collect notes on what I read so I have a large collection of content I can ask the RAG capabilities of this tool to draw on. To provide a reasonable comparison to how a student would study course content, I draw some parallels based on the use of note tags and note titles. Instead of using titles and tags in the way I do, I propose a student would likely take course notes and among the tags label notes for the next exam with something like “Psych1” to indicate a note taken during the portion of a specific course before the first exam to which that note might apply. I hope the parallels I explain make sense.

Loading

Ranked imperfect actions to fix big tech

I recently listened to part of the Senate hearing focused on Big Tech and online child abuse. While recognizing multiple complaints, many legitimate, focused on Big Tech, I am beginning to have a small level of annoyance with the political committees. My reaction to the committee attitude is often – OK, you have identified a problem. You don’t seem to have a solution you want to put forward in legislation beyond complaining about Big Tech and expecting them to come up with a solution. I am not a fan of large technology companies but the situation always seems complicated to me with conflicting values generating complaints and accusations. Since I assume Congress can impose regulations, I want to see the specific solutions they threaten to impose and give everyone a chance to react.

The following is a challenge I sometimes give myself. Here is the problem. I understand some of the issues and ask myself what would I have Congress require. Then, I try to consider what the consequences would be and what complaints would be advanced. I have come up with an ordered list of my recommendations, but first I want to identify some of the common counter-arguments I have considered.

Challenges that are barriers to many potential solutions:

  1. Big is necessary for many services. The most obvious example I can think of would be AI. I know that even major universities cannot independently develop and evaluate the powerful models of big tech companies because of the cost. University researchers do have a more independent view I would like to see in important conversations but doing research and development is difficult. Mandated regulations are easier for big companies to satisfy limiting important competition. Added responsibilities are easier for existing companies with a large user base and solid revenue to address added requirements. Fear of legal action is also more threatening to a company without the cash flow to hire legal counsel and the funds to pay penalties. 
  2. I see anonymity as a problem because it allows big tech to be blamed for the behavior of bad actors. Rather than going directly after bad actors demands are made of the platforms the bad actors use. This is the issue addressed by Section 230, but 230 is under attack. Anonymity is defended because it is argued some users need to have their identities protected – children with personal concerns that their parents reject, women hiding from abusive partners, citizens living under oppressive rulers, etc.
  3. Free speech and who gets to interpret what free speech allows is a concern. Given how some politicians use the free speech argument to defend their own behavior trying to regulate harassment and bullying among citizens seems to be problematic. If teens call a peer fat is that free speech?
  4. What would actually change? dana boyd (Microsoft researcher and expert on online adolescent behavior) makes the following argument concerning how politicians see many problems. She contends that most problems have multiple causes with online behavior being one. She argues that the nature of our culture creates an environment such that whatever would be done about Big Tech would result in very little change in the challenges of adolescence.
  5. We as a society become locked into a model we mistakenly understand to be free. We are wrong because we pay for many online services by our assumed attention to ads, but it is the perception of free that is the problem. 
  6. Equity – those able to pay for services are less vulnerable in that we can exercise greater control over known threats.

I do not claim that my ordered list of possible actions resolves what I think of as conflicting challenges, but I wanted to recognize these challenges that must be recognized. You may have others you would add to the list.

What I mean by an ordered list is that unless items at the top of the list are addressed, items lower on the list will either be difficult to implement or implementation will be less effective.

I am going to try to present items on my ordered list using a specific structure. In some cases, this is more challenging than in others. I first intend to present a problem, then identify a mechanism that enables the problem (flaw), and finally propose a concrete action that could be taken.

Problem Onelack of competition limiting options that could be based on what users believe to address a personal need. Flaw – the network effect locks individuals into an inferior service because that is where colleagues are. Solution – depending on the service politicians could require that interoperability and easy movement of personal data be provided. 

Note: Cory Doctorow makes the argument that many of the nation’s problems will not likely be addressed without better knowledge and communication. I am simply extending his argument to suggest that until online participants have options that address their personal needs and concerns meaningful change will be far less likely. Interoperability and easy movement of data are practical ways to challenge the perception of being locked in.

Problem TwoCollection of personal information. Flaw – the revenue model of big tech is largely based on the collection of personal information to allow targeted ads and communication. Companies pay far more for personalized rather than random access and this higher revenue is what allows for the infrastructure, research and development, high salaries and return to investors necessary to maintain big tech. Solution – the best solution would be to deny or limit the collection of personal information. Making third-party cookies illegal would be helpful, but denying the use of ads and forcing a subscription-based revenue approach would be a better solution. Ad blocking at present is a gray ethical area as it provides no alternative to support work. Moving the financial model to subscriptions would be ideal. Requiring companies to provide ad-supported and ad-free options makes some sense, but presents an equity issue in that more vulnerable people would likely accept what would be an objectionable situation. BTW – my selection of my first priority was related to the equity issue and the importance of meaningful alternatives.

3. Problem – big is necessary for R&D. Flaw – the R&D function of higher education does not allow the budget to contribute to many of the issues now addressed by big tech. Solution – Fund a competitive consortium of higher ed institutions to focus on R&D issues ignored by big tech and evaluate strategies used by big tech for consequences that may be damaging.

4. Problem – holding tech companies accountable for the actions of anonymous bad actors that use tech platforms. Flaw – individuals responsible for actions cannot be identified because of anonymity. Solution – require verification of identity for use. Companies could maintain this information and release it when required by authorities. Issues – increased cost and time, distrust of those who control and access identify data.

5. Problem – online services are addictive increasing dependency even when experiences are negative. Flaw – companies are motivated to increase the amount of use to show more ads. Solution – eliminate easy sources of reward (likes, shares) in favor of more effortful reactions (comments). Note – moving to a subscription model would remove the incentive for showing more ads – see problem 2.

So this is my list. I think what I propose is concrete and that is important in comparison to blaming and failure to define actual options. I invite others to react and to generate concrete actions they think could be implemented. 

*****

Like many focused on educational uses of technology, I spend some time exploring AI applications. When sharing AI-generated content, I intend to differentiate what I have written from what I have used AI to create.

I asked ChatGPT to comment on the advantages and disadvantages using moving from ad-supported content and services to a subscription model. I thought the response was worth sharing.

The shift towards subscription models over targeted advertising is a transformative approach that promises a host of benefits, albeit with some trade-offs. Here’s a balanced view on how requiring a subscription instead of relying on targeted ads could shape the future of content consumption, user privacy, and business models.

Pros:

  1. Enhanced Privacy and Security: The most significant advantage of a subscription-based model is the protection of user privacy. Targeted advertising relies on collecting vast amounts of personal data to tailor ads, raising concerns about data misuse and privacy breaches. A subscription model eliminates the need for data harvesting for advertising purposes, thereby enhancing user privacy and reducing the risk of data breaches.
  2. Improved User Experience: Subscriptions can lead to a cleaner, more enjoyable user experience, free from the clutter and intrusion of targeted ads. This can increase user satisfaction and engagement, as content is no longer interrupted by irrelevant or annoying advertisements.
  3. Predictable Revenue Streams: For content creators and platforms, subscriptions provide a stable, predictable source of revenue. This can be especially beneficial for smaller creators or niche platforms, which might struggle to attract advertising dollars but have a dedicated user base willing to pay for quality content.
  4. Encourages Quality Content: A subscription model incentivizes the creation of high-quality, valuable content. Since revenue is directly tied to user satisfaction and willingness to pay, there is a stronger focus on producing content that meets or exceeds user expectations.

Cons:

  1. Barrier to Entry: A subscription fee can act as a barrier to entry, potentially alienating users who are unwilling or unable to pay. This could limit the audience size, particularly in markets where consumers are accustomed to free content supported by ads.
  2. Subscription Fatigue: As more platforms adopt subscription models, users may experience subscription fatigue, feeling overwhelmed by the number of subscriptions they need to manage and pay for. This could lead to a churn rate increase as users prioritize their subscriptions based on value and budget constraints.
  3. Reduced Reach for Advertisers: For advertisers, subscription models mean reduced access to potential customers. Targeted advertising allows brands to reach specific demographics with precision, but a subscription model limits this reach, potentially impacting sales and marketing strategies.
  4. Financial Risk for Creators: Switching to a subscription model presents financial risks, especially if the existing audience is not willing to convert from free to paid content. This could lead to a temporary or permanent dip in revenue during the transition period.

In conclusion, while the shift towards subscription models offers promising solutions to many concerns associated with targeted advertising, it is not without its challenges. Balancing user privacy, content quality, and financial sustainability will be key to successfully implementing this model.


Loading

New Project

I am trying something different with one of my longer writing projects.. My wife and I had a 15 year run with a commercial textbook for the “technology for teachers” undergraduate teacher preparation course. Fifteen years translates as 5 editions of the book. This course does not generate the review of a large lecture course (e.g., Introduction to Psychology), but there is less competition in the area of our book and we did well financially. 

As we gained a lot of experience allowing us to analyze the textbook industry and the niche in which we published, we became very aware of the backlash against textbook costs (ours sold for $140 to students) and began to identify issues a traditional textbook for this niche could not address.

We came up with a plan to publish a much shorter version we called a Primer and wanted to match this with online resources. We proposed a $29 Primer and intended to serve the online content ourselves. I still think some of our arguments for this approach make sense. For example, those intending to teach high school and early elementary have very different interests in what to do with technology. Why not provide the basics in a Primer and then a larger variety of content for specific content areas and grade levels online? Technology is a field that moves quickly and keeping content current is a tremendous challenge. Not only did we publish once every three years, but 9-12 months were set aside to generate the next edition. You see the time lag that is created. Why not write online continually to keep a given textbook current? 

Textbook companies think differently about their relationship with the authors they hire. A proposal such as paying someone to write continuously does not make sense to them even though they might appreciate the issue of keeping content current. They typically have a couple of books in a niche and their field reps encourage the adoption of the most recent book in a niche. This is more because of the used book market than the issue of currency and the issue of a general approach rather than what would be best for a given book is the perspective they take. At the time (this has changed since), combining online content with a physical product was also a foreign idea that did not translate as easily into income. 

Anyway, we agreed to go our separate ways and were given our copyright back so we could pursue our interests with another company or with an outlet such as Kindle. 

We continue to offer a version of this textbook through Amazon. I developed a second resource (Layering for learning) which was not really a full-length textbook, but concentrated on specific online services I proposed educators could use to make more effective use of web pages and online videos. It is this second “book” that I have decided to take in a different direction.

My professional writing activities have long been mostly a hobby. We made our money on our original textbook, but now my work is mostly about exploring topics in online publishing. Instead of $140, the online textbook sells for $9. Same basic book. I think it appropriate content that takes considerable time to create be treated as having value and I have always require some payment for my professional work even if mostly symbolic. So, what other outlet and approach can I explore as an alternative to Amazon?

Here is my new project. I am updating my layering book and serializing it on Medium. If you have not used Medium directly, you may have encountered work offered on Medium through a search engine. Sometimes you could read what the search engine found and sometimes you may have found that the content was behind a paywall. There are two competitors in this space – Substack and Medium. With Substack, if an author wants to be compensated for her work, she requires readers to subscribe to her work for a price. A reader makes a specific commitment (usually $5 a month or so) to specific authors. With Medium, you pay a subscription fee ($50 a year) and then read whatever you want from as many authors as you want. Medium takes a cut and then allocates the rest to the authors based on several variables they use to define value. Like other social outlets for the vast majority of writers, you receive little money (I hope to make enough to cover my own Medium subscription fee). I think of it as a way to keep score. Do people find what I write interesting and of value? What are the options for those who generate the kind of content I create and how do different options compare? 

If you are not a Medium users, I think you are allowed three free reads a month and the Introduction to my serialized book is explained in greater detail. 

Loading

Why not ask for help? Have the benefits of technology-augmented studying been demonstrated?

I have written posts for Medium for a few months now. It is clear that some of my most popular posts concern note-taking and personal knowledge management. I have a history with the topic of note-taking having conducted research with college students based in my more general background in the cognitive processing of learning. When I most it is often about evaluating specific digital note-taking practices or knowledge management concepts based on basic cognitive principles. What about how learning works justifies a specific practice the self-help authors advocating for smart/atomic notes or second brain recommends.

I asked Dall-E to help come up with an image depicting the type of learning I had in mind. My prompt asked for an image showing an adult using a computer and note-taking to learn a skill that was something they had not studied in school. I decided I needed something specific so I requested an image focused on learning to bake bread.

As I have explained in some of these posts, I think some claims made for digital note-taking lack empirical support in the context to which the self-help writers propose their tactics apply. 

A couple of observations about the framework within which nearly all (maybe all) existing research was conceptualized. The research I am familiar with focuses on learning within a formal educational setting. Whether it be middle school or graduate students, note-taking is largely a practice to deal with information inputs that are determined by others with the goals for the learner being storing, understanding, and applying this information to examinations, projects, and papers assigned by others. The time frame with perhaps the exception of licensing exams or graduate preliminary examinations are weeks and at most a few weeks in length. Proposals such as Ahren’s Smart Notes or Forte’s Second Brain propose unique tactics and imagine the use of notes over an extended period. Implications of these differences do not seem to be tested or at least are not examined directly by existing research. 

The vocabulary of multiple authors proposing new systems and tactics can be an issue by itself. I am trying to understand the difference between smart notes, atomic notes, and permanent notes. For example, Ahrens titles his book Smart Notes, but then describes fleeting notes, literature notes, and permanent notes. There is a process here – fleeting notes can become permanent notes through a personalization process similar to what Forte in this book about the second brain called progressive summarization. I threw in personalization because that is what I call the process of rephrasing and emphasizing based on what the learner knows (again similar to certain properties of progressive summarization). I think I should be able to apply labels if I think my label communicates meaning more clearly. 

What am I looking for? I am searching for research literature that examines tactics used with these digital services as applied to learner-determined goals. Starting from a long-standing and nuanced literature defining cognitive benefits associated with note-taking, note-reviewing, highlighting, basic memory, and application what can be understood about self-directed learning? What basic descriptive data are available on the common use of the various features of the affordances of digital services? What types of notes do users actually create? Do users make use of tags and links when they attempt to use the notes they have created or do they simply search? Are notes reviewed periodically and new connections found as recommended by the self-help gurus? 

I have tried the various tools scholars use to explore the literature (Research Rabbit, Elicit, Google Scholar, Litmaps, etc.) with no luck. All I need is one or two quality studies of the type I have in mind and finding related work should be easy. Before I give up completely and decide advances in this area will proceed by logic and salespersonship, I decided maybe I should just ask for help. Maybe the wisdom of the crowd really exists. If you think you can provide a lead please do so. I am not putting down those who just imagine strategies for learning they think are unique and creative, but at some point I want to see the data. Am I missing something or is there just nothing there? If there is nothing there, why is this the case?

References

Ahrens, S. (2022). _How to take smart notes: One simple technique to boost writing, learning and thinking_. Sönke Ahrens.

Forte, T. (2022). Building a second brain: A proven method to organize your digital life and unlock your creative potential. Atria Books.

Loading

Sharing the exhaust of reading

I had the opportunity yesterday to do a product review of the online note-taking service Mem.ai with a company representative. I make regular use of Obsidian (with the addition of the Smart Connections plugin) and Mem.ai while reading to do the same things – take, organize, and connect notes and then interact with the notes using AI. I do this as a way to explore two popular services so I can write about their similarities and differences.

The rep had sent me set of questions and then we interacted for what turned out to be a substantial amount of time in reaction to these questions. The question that prompted this post was “What modifications would I like to see Mem.ai add in the future?”

Open.ai is a great service and worth the subscription price, but I would guess any user familiar with other products would be aware of some features they wished any one of the services would add. With Mem.ai. I would like to have access to a feature that would be a way to share portions of my note collection with others. Mem has a pricing level for teams and a way to share any given note within the team, but this is not a way to share with other audiences.

To create this post, I decided I would try to describe this interest and how other services approach this goal. I have previously described these services in detail and I have tried to embed links here that will take you to these descriptions. Here, what I have in mind is emphasizing a way for users of note-taking and organization tools to offer parts of their total collection to others and ideally others who have not invested in the same service. For example, a university history professor might have a massive collection of notes and she may find it valuable to share selected notes with students in different classes. Could this be done efficiently without the students having to purchase the same service?

Hypothes.is 

I became personally familiar with the sharing of notes using the free service Hypothes.is. Hypothes.is allows the sharing of notes to designated groups and the sharing of public annotations to other users who happen on the same online source document. Users add a free extension to any one of several different browsers to create personal annotations for personal use or to participate in groups. This is a base-level product with fewer features than my interests require (e.g., linking, AI analysis), but this service is a great application for classroom use. I first wrote about Hypothes.is in 2017. Here is a more recent description (here is a video). 

Diigo

Diigo has also been around for some time and I originally thought of Diigo as a social bookmarking site. I have written a lengthy description of Diigo so I will not describe in detail here. My public Diigo account is still active and available if you want to take a look. There is a limited free version of Diigo and a free version for educators (pricing chart). The version for educators has a few limitations (e.g., the number of pdfs that can be annotated), but should work well if what you want is to share annotations and links to source sites with students. Diigo is beginning to show its age and does not offer newer features found in Personal Knowledge Management systems such as links among notes or direct access to AI.

Obsidian

Technically, Obsidian stores its data on your machine. Great for security, but a challenge if you want to get at your content from a different machine. I make use of Apple’s iCloud which functions as online external storage to get at my content from multiple devices. I know others use DropBox in the same way. I also know that you can share a DropBox folder with others and this would allow others with access to work with your notes, but this is not the way Obsidian is intended to work and there certainly are security issues. I want to offer read-only access to a designated subset of my notes.

Obsidian does offer an approved way to allow others access. Obsidian Publish is an $8-a-month add-on that would do pretty much what I want to do. Given the infrastructure requirements of providing this approach, the price is certainly reasonable and I should just add publish to the list of my subscriptions. As I write this resistance seems futile, but I wonder if there would be much interest. 

Mem.AI

Mem.AI is an online knowledge management system. The price for an individual user is $8.33 a month for the yearly subscription plan (pricing options). There is a team option for a higher price that is designed for collaborative work (see my initial comments). Mem.ai is a great product for personal use but does not offer the public sharing I would like to see. There is a way to share an individual mem (example), but this is not what I am focusing on here.

MEMEX Garden

For those who follow the personal knowledge management space, MEMEX Garden (Memex Go for portable devices) is a service they probably missed. I paid upfront with a promise of a couple of years fee forgiveness, but I think the monthly subscription price is $6 a month for the yearly plan. MEMEX is positioned within that workflow space before a tool such as Obsidian and can be set up to feed highlights and annotations into Obsidian and some other PKM tools. I mention MEMEX here because it has a sharing capability close to what I would like to see included in my perfect product. 

When you take notes or highlight while reading with this tool, the resulting online entry consists of your notes, highlights, tags, and a link to the source. You can also add notes to a theme. A theme can be shared. The image below shows what a theme looks like and another user would experience when making use of a shared theme.  

When someone accesses a shared theme, they can link to the original article, view notes/highlights (first image that follows), or get an AI-generated summary of the article (second image that follows).

The way I imagine this being useful to others is to first construct themes to share. Then, offer the link to these theme to others. Those individuals could scan the titles listed within a theme and then use the AI feature to identify sources they might find useful. Finally, they would access the selected source documents for their detailed reading.

I have generated an example of what this looks like based on several themes I follow. Note: the AI summaries use OpenAI and will hit a limit. The summarization gets shut off when the limit I have set is reached. If you do not see the AI tool and want to give it a try, try a different browser. For example, it shows up for me in Chrome, but not Brave. 

Summary

I get to the end of this post and upon rereading I find that I have mixed two similar potential social services – social bookmarking and whatever Obsidian and Mem.ai are. Both allow the sharing of highlights and notes, but the ideas differentiated from the original content should be different. In Ahern’s book Smart Notes, he proposed that the notes saved in Obsidian or in Luhman’s Zettelkasten be sufficient to be a meaningful representation of an idea on their own. I interpreted this idea as having sufficient information it was not necessary to have the original context to communicate a useful idea. Highlights and margin notes do not meet this standard and are more likely what is produced when using a tool that both presents the original content and accepts the additions of a reader (e.g., Diigo, Hypothes.is, Memex). 

The idea of sharing the ideas generated while reading appeals to me whatever the form the shared information takes. Here, I made an attempt to comment on several services/tools suited to this goal. I use several of these tools and have not found one that matches what I would like to see in a single tool. Perhaps you will find one that fits your interests.

Loading

Use EdPuzzle AI to generate study questions

This post allows me to integrate my interest in studying, layering, questions, and using AI as a tutor. I propose a specific use of EdPuzzle, a tool for adding (layering) questions and notes to videos, be used as a study tool. EdPuzzle has a new AI feature that allows for the generation and insertion of open-ended and multiple-choice questions. So an educator interested in preparing videos students might watch to prepare for class could prepare a 15 minute mini-lecture and then use EdPuzzle to layer questions on this video and assign the combination of video and questions to students to be viewed before class. Great idea. 

The AI capability was added to make the development and inclusion of questions less effortful. Or, the capability could be used to add some questions that educators could embellish with questions of their own. I propose a related, but different approach I think has unique value.

How about instead of preparing questions for students, allow students to use the AI generation tool to add and answer themselves or with peers. 

Here is where some of my other interests come into play. When you can interact with AI that can be focused on assigned content you are to learn, you are using AI as a tutor. Questions are a part of the tutoring process.

What about studying? Questions have multiple benefits in encouraging productive cognitive behaviors. There is such a thing as a prequestioning effect. Attempting to answer questions before you encounter related material is a way to activate existing knowledge. What do you already know? Maybe you cannot answer many of the questions, but just trying makes you think of what you already know and this activated knowledge improves understanding as you then process assigned material. Postquestions are a great check on understanding (improving metacognition and directing additional study) and attempting to answer questions involves retrieval practice sometimes called the testing effect. For most learners, searching your memory for information has been proven to improve memory and understanding beyond what just studying external information (e.g., your notes) accomplishes.

I have described EdPuzzle previously, here are some additional comments about the use of the generative question tool. 

After you have uploaded a video to EdPuzzle. You should encounter the opportunity to edit. You use edit to crop the video and to add notes and questions. The spots to initiate editing and adding questions are shown in the following images. When using AI to add questions, you use Teacher Assist – Add Questions.

After selecting Add Questions, you will be given the option of adding Open ended or Multiple Choice questions. My experience has been that unless your video includes a good deal of narration, the AI will generate more Open Ended than Multiple Choice questions. If you want to emphasize MC questions, you always have the option of adding questions manually.

Responding to a question will look like what you see in the following image. Playing the video will take the student to the point in the video where a question has been inserted and then stop to wait for a response. 


When an incorrect response is generated to a MC question, the error will be identified.

EdPuzzle allows layered videos to be assigned to classes/students. 

Anyone can explore EdPuzzle and create a few video lessons at no cost. The pricing structure for other categories of use can be found at the EdPuzzle site. 

One side note: I used a video I created fitting the potential scenario I described of an educator preparing content for student use. However, I had loaded this video to YouTube. I found it difficult to download this video and finally resorted to the use of ClipGrab. I am unclear why I had this problem and I understand that “taking” video from some sources can be regarded as a violation of copyright. I know this does not apply in this case, but I did not want to mention this issue.

References

Pan, S. C., & Sana, F. (2021). Pretesting versus posttesting: Comparing the pedagogical benefits of errorful generation and retrieval practice. Journal of Experimental Psychology: Applied, 27(2), 237–257.

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. _Psychological Bulletin_, _147_(4), 399-435.

Loading

The Advantage of Paper? Why?

Happy New Year. My final post of the year will return to one of the issues I have written about several times during this past year. I would describe the issue as seeking an answer to the questions “Does reading from a screen reduce understanding immediately and reduce the development of reading skills long term?” At one level, I know what I think personally. I read nearly everything from a screen because what I am doing is more than reading. I am using the digital advantage of reading from a device in the process of taking notes and annotating I see as beneficial in the long run. I have made my own decision regarding my behavior in what I consider a logical way. The issue of skill development is a different issue. I see the amount of time our grandkids spend on their devices mostly watching video. I really don’t know if this preoccupation with screen-based information is damaging to their development of reading and thinking skills. I do think it is an important issue that deserves attention.

I just finished working my way through a new meta-analysis related to this issue (Altamura and colleagues). Like so many articles I have read on the topic, the results are troubling as related to the concern for younger readers. However, the results because of the limitations of the methodology employed are open to questions and alternative interpretations. I will describe the review as best I can and as always invite those interested in this issue to read the original document for themselves. 

The authors begin their approach by noting a widely accepted relationship between “out of classroom” reading to reading skill development. The relationship is proposed based on a positive spiral. As younger readers are exposed to gradually more demanding texts, they develop improved skills important to reading comprehension. These improved skills make possible successful understanding of even more demanding material which tends to be more informative and enjoyable and the upward spiral of capabilities continues. This relationship relates to other literacy correlates such as having access to more reading material in the home and being read to more frequently. 

With an interest in digital reading, the researchers decide to review literature relating screen-based reading to reading proficiency as readers age. Does the same relationship between reading digital content and reading skill hold developmentally for what they define as recreational reading? With their definition of recreational reading comes the first methodological issue? They include pretty much any text-based experience one might have on a digital device – text messages, social media interactions of any type, blogs, and digital books. Obviously, there is not a meaningful equivalent to social media and text in previous studies of the relationship between text consumption and reading proficiency. The researchers did code from the difference between these short and long form samples of text in their statistical approach and this distinction will end up being important in my comments at a later point in this post.

The researchers propose they are testing two competing hypotheses. The displacement hypothesis suggests digital text is a replacement for text on paper and the shallowing hypothesis suggests that shorter segments of text so frequently available in digital environments require less in processing skill and encourage scanning and skimming. Shallowing may prevent critical skills from being applied and developed. 

The researchers summarize their results as follows:

This relationship is significantly moderated by the reader’s educational stage. At early stages (primary and middle school) negative relationships are observed between leisure digital reading and text comprehension, while at later stages (high school and university) the relationship turns positive. 

While the pattern of reading activity and comprehension skill differs from what is found with paper-based text, neither of the hypotheses was cleanly supported. This was the case because age was the only moderator variable achieving significance and the relationship was not consistent. The differentiation of the type of reading material was not significant. That is, the division between time spent on the more purely social and short content versus longer content was not a significant predictor of skill differences. The researchers suggest that studying the combination of screen and paper could be important. In other words, a focus on digital reading ignores other reading the participants might have done with text on paper which would likely have been longer-form reading. 

So, this paper can be added to others (e.g., Delgado at colleagues) that offer concerns especially for younger readers. As I have suggested in other posts, this question is difficult for researchers to address because important variables are difficult to control with carefully controlled research studies. The developmental nature of reading skill is not consistent with an experimental approach that would control a treatment such as whether individuals read from paper or screen over an extended period of time. I question how much “reading” children actually do from screens. Few are allowed or can access social media services as a matter of personal safety. A more significant issue would seem to me to be whether reading is being replaced by watching. Research typically fails to provide actual quantification of the amount of time individuals are exposed to text. Without total exposure perhaps differentiated as screen vs paper, what conclusions are possible?

References

Altamura, L., Vargas, C., & Salmerón, L. (2023). Do New Forms of Reading Pay Off? A Meta-Analysis on the Relationship Between Leisure Digital Reading Habits and Text Comprehension. Review of Educational Research, 00346543231216463.

Delgado, P., Vargas, C., Ackerman, R., & Salmerón, L. (2018). Don’t throw away your printed books: A meta-analysis on the effects of reading media on reading comprehension. Educational Research Review, 25, 23-38.

Loading