Category: Generative learning

Engagement with a task with the intent of improving understanding, retention, or application of the information to which that task is applied.

AI augmented thinking

I have become quite interested in the history of attempts to use technology to support personal productivity. Rather than begin at the stage of visionary speculation (e.g., Vannevar Bush’s Memex), I will use my own history of using digital technology to generate written products. A point to consider from the beginning of this description – what follows depends on my own recollection of the features of the digital tools I have used which may be incomplete or flawed. I am more certain of the chronology of tools.

Most of my academic work involved reading what others had already written and using my understanding of these inputs to guide my own research and writing. In a career that covered 40+ years, this involved reviewing hundreds of books and thousands of journal articles in order to produce a number of research articles, a couple of books, and more recently thousands of blog posts. Technology played a role of even my early writing if you count learning a markup language that was used to generate a dissertation initially stored on 80 column computer cards. Let’s ignore the next decade or so and skip ahead to time associated with the availability of the personal computer.

I would like to focus on what has come to be described as a second brain. My early research interest in classroom note taking has always caused me to use the description of “external storage” which was accurate at the time, but now seems too narrow. I now see the use of digital tools as striving for more than just storage. I don’t like term second brain because the phrase is too ambiguous. I think in terms of a verb – externalization. Digital tools may result in an internal record, but getting to what is stored externally is also important and understanding the value of the process or work flow enabled by digital tools is very important. This work flow is also external.

Looking back and looking forward

In preparing my comments on this topic, I have considered several ways to explain both my personal experiences and how I see progress in using technology to facilitate knowledge accumulation and the creative process. Looking back and looking forward popped into my head as a way to explain an important aspect of how the use of technology has evolved within this domain.

In my own career, my early work typically began with general reading of books and academic journals. There were certain topics I emphasized, but I knew it was wise to at least broaden what I read to topics I might cover in the classes I taught. In this exploration, I took notes on what I decided was useful and/or important information. Once in a while, I encountered an idea that really intrigued me and I wanted to make certain I would add to the topics I talked about with students or perhaps I wanted to incorporate into my research and writing activities. 

These more unique discoveries often motivated me to look back. The author typically offered an idea I thought was important, but also tied it back in some way to ideas recorded in other documents. To be cited, these earlier documents were older. I used the reference section of the document I was reading to identify these sources and could then locate and read the documents published at an earlier date. This process might continue through several iterations until I either ran out of time or the historical content seemed less useful.

Looking back enabled a certain kind of linking. It helped me see how ideas built on other ideas and often how ideas diverged as new information was discovered. The literature itself has built in linkages and by following these connections I could built my own understanding and sometimes generate an external representation of this understanding.

The limitation of looking back while helpful was that it did allow an easy way to look forward. I could make crude efforts such as trying to find newer work the authors I read had generated, but this was not an easy process. If you are familiar with Google Scholar, you are familiar with a technological innovation that allows a form of looking forward. Google Scholar accumulates the citations from published work and provides a list of articles that cite an article you target with a search. This collection of citations offers a way to consider how work that follows the searched article used the information in the article you identified. What applications were attempted and how did they work? What limitations were considered and were these limitations proven to be valid. Have some of the core ideas been extended in useful ways? Now, I had to do the work of reading the new material I thought might be informative and often draw my own connections and conclusions, but at least the web of citations Google Scholar makes available is a way to start. 

A twist on my way of thinking might be understand this as a social system. It is not a purposeful system as might be involved when a group of individuals works with each other to contribute to a summary valued by all. There are now digital tools for such efforts. It is simply a way to cross reference connections others have observed.

Technology-enabled discovery functions involving methods beyond the example of looking forward I have just described are the area in which I see most innovation occurring.

Technology supported thinking

Supported is the key word here. We still do the thinking, but technology can allow supports that compensate for some important limitations of the cognitive system. Retrieval makes a good example. We often know things were are unable to recall when information would be useful. The search features of technology tools often can substitute for our own efforts at retrieval.

The developments I want to describe can be understood as the evolution of reference managers. By evolution, I mean tools originally designed to store, organize, search, and export the citations stored as references have become much more and now have become external environments within which the user can think by discovering new information to expand how existing ideas are understood and gain understanding by summarizing and speculating about what others have proposed.

A technological reference manager was originally a way to store references either by the entry or importing of citations. Once entered, references could be augmented by tags, annotations, and perhaps the abstract of the original document and connections to the full document as a pdf. Having this content in a digital format allows retrieval by search and the surfacing of other stored documents containing the same search phrase or tag. Aside from the value of just having such information in a form allowing easy retrieval, reference managers saved users a lot of work by allowing citations to be output in a format that could be used as a reference section for written outputs. As an example, I used EndNote for many years. I don’t mean to imply that EndNote is a primitive reference manager as tools of this type have become more powerful over time.

I would argue that an important extension of such reference managers occurred when the tools encouraged users to write earlier. If you consider the role of tools in the process of moving beyond storage and retrieval more toward personal application, writing earlier means that a tool is used closer in time to the original exposure (reading, listening) to record personal insights, interpretations and possible applications. So, I might collect references over many years in anticipation of eventually using the sources in writing something original. Rather that wait until I want to write something to review and then trying to find ideas in my digital database of resources, I now create summaries of my ideas upon initial reading which may make later application of the content still embedded in multiple documents (perhaps highlighted) much more efficient. When I am initially reading something, the context is right there making it easier to personalize ideas I might have. I cannot necessarily anticipate how ideas will eventually be used but I can avoid much of the time and effort required to reread what might or might not be useful to get to the point of reactivating an understanding of a primary source. There is also cognitive value in generating personal summaries for understanding and transfer and while such summaries could be stored external to the type of tools I am describing here, connecting such summaries with the citation and pdfs offers some advantages for retrieval and contextualization.

I use the following tools to add annotations, notes, and highlighting to pdfs (both are Apple tools). There are many similar tools. 

Bookends

Highlights

I extract some of my summarizations and organize them in Obsidian. 

Discovery seems to be moving to AI

I would recommend any of the tools I have mentioned to anyone wanting to keep track of useful sources they have discovered. I am now going to describe some opportunities that are attempts to extend the cognitive benefits of what I have described using artificial intelligence (AI).

When I described Google Scholar as forward looking, I was describing a service that identifies other sources that are related in some unstated way to an earlier source. What if the discovery of associated content could be identified in other ways? Our own memory often works through association. One idea makes us think of something else and sometimes noticing this connection turns out to be very useful. Perhaps it is a connection we have not considered before. The AI applications I am describing here attempt to do something similar. As I understand the process, the AI creates summaries that are stored and then attempts to locate similarities across other generated summaries. In some of these cases I have explored, the units of association are smaller than an entire document. You can read a summary of one approach and determine if my interpretation is at least close. Such possible connections may exist unnoticed in summaries you have already stored yourself or perhaps in summaries generated by others working on the same issue. Once discovered, you can consider the possible connection and determine if you think there is something valuable in the relationships you explore.

Here are some of the efforts using AI I have been exploring. It is too early for me to offer personal comments about the usefulness of these tools as effective use would seem to require I create a significant amount of stored content the systems can use to identify connections.

DevonThink

Mem X

Semantic scholar/reader

These examples are available for exploration. Mem X strikes me as something I might pay to use ($10 a month) after an exploration phase. The Mem X note-taking app has an advanced feature called smart search that allows what the developers call serendipity. The purpose of this feature which seems the main differentiator from the free Mem is this capability of knowledge discovery (among teams) and rediscovery for each individual. Semantic Scholar (wikipedia description) is available now and Semantic Reader is under development with some examples available for exploration.

One final comment. The final tools I list and the more general common on AI refer to tools that support the work of thinking. The distinction between support and thinking itself is important. I doubt that traditional sresearch will surface evaluating the value of such support. Unlike the work I studied years ago that evaluated the value of taking notes (the generative function) and external storage (the value of consulting these notes at a later time), the value of suggested relationships among ideas would be difficult to investigate in a controlled fashion. This is likely to be a topic that will rely on anecdotal reports from those trying something out and if the reported experience is positive the investment of time to see if the tool is helpful to you. I don’t think we are really to the point yet that even the anecdotal recommendations are really available. My purpose is proposing that such tools and the related explorations by individuals are underway. You can join the exploration if you are so inclined. 

Loading

Learners need to learn to read from both paper and the screen

Cohn (2021) argues that we read in different circumstances for different purposes and yet we tend to teach reading as if reading is a single skill. Learners would be better served if they were to be guided to explore the different types of reading they do and what tools and tactics would be best suited to these different circumstances.

This idea is important. We read for many different purposes. Do we think about which approach best serves a given purpose? As an adult thinking about my own reading behavior, I can see this complexity. I very seldom purchase a paper product for reading. There are still a wide variety of ways in which I read digital content. I read some online content and by that fact that are reading this post, so do you. How about books? I read digital books when I read to acquire information especially when I intend to store specific information for later use. I listen (audiobooks) when I read for pleasure. The read to learn versus read for pleasure is a common distinction many recognize, but college study skill experts suggest that students often struggle with acting upon this distinction in their efforts to learn from their textbooks.

If we are educators, do we consider factors as basic as the physical circumstances that impact how and when their students or themselves read. Do we want to read in a coffee shop, a library, or at our desk? When do we want absolute quiet and when is some music or coffee shop banter in the background welcomed? Perhaps the noise in a coffee shop distracts us from time to time and these interruptions provide the signal to reflect on what we are reading rather than continuing to plow ahead. We may ignore the physical realities of reading that some students must consider. Perhaps some must read on their phones on the bus or train because this is when they have the opportunity to work on class assignments. Perhaps their phone is the only device they have that can be applied in these circumstances. 

How we understand what reading involves matters. Cohn (2021) offers a set of reading goals that may or may not be accepted by the reader. The final purpose she describes is creativity. Her definition is a little different than the way I tend to think of the concept, but she proposes creativity involves the understanding that reading should result in the building of new knowledge. She argues that when we read we may not see the benefit of creating something after we read as if reading should be enough. My take on this expectation brings to mind distinction between reading for understanding and retention and pleasure. Extra effort is obviously involved when the goal is creating something even when this is not a written product. Do others not think in this way?

Chon argues that most readers and writers understand that reading and writing are knowledge transmission acts, but proposes that they should be understood as knowledge construction. This difference encourages additional processing and the utilization of additional tools. This is where instruction in the use of such tools comes in. Do teachers teach the application of such tools? Which teachers and in which subject areas? Note-taking is one activity that recognizes the connection of ideas across sources and with existing knowledge. Note-taking is another of those practices that can involve either paper or digital technology. The skills involved in these activities offer a great deal of overlap, but digital tools offers some unique advantages in storage, organization, and search aid retrieval. 

Chon proposes that educators make inaccurate assumptions about learning skills such as highlighting, annotation, and note preparation and use. She offers an example in which she  began asking her students if they had experience using pdf tools to highlight and annotate assigned content. She had been assigning pdfs and had begun to wonder how students processed these resources. She reports that 30% responded that they were highlighting and annotating the assigned material and many were unaware such tactics were possible. Her point was that educators (she teaches at a university) should not assume that computer experienced students have skills appropriate to making use of digital tools with such assignments. It struck me that this question should be asked for more educators assigning digital content.

After reviewing several sources proposing how educators might help students develop annotation/note-taking skills (also sometimes labelled as deep reading), I have begun interpreting the instructional tactics as a variant of reciprocal teaching. As instructional strategy, reciprocal teaching begins with the teacher modeling a specific skill accompanied by “thinking aloud”.  Individual components or subskils are then assigned to individual students and applied to a common reading assignment. Student experiences and any products produced are shared and discussed. Finally, students move on to the application of the combination of practiced skills and seek assistance when necessary.

With highlighting and note-taking, the skills are a bit different from those emphasized in the original focus on reading comprehension. However, the general process of teaching/learning is very similar. For example, with note-taking, the components might include the identification of essential information, the summarization of these key ideas, and efforts to cross-reference these ideas to existing knowledge and other inputs (ideas presented in class, other reading assignments). When learning these skills, some educators recommend the use of printed material before moving to digital content. Sharing individual student efforts perhaps as displayed on a classroom white board allows for discussion and analysis. 

The sources I provide below provide multiple examples of how this generalized strategy can be implemented. I understand that many may not want to purchase this material. I was able to find an alternate source for “Beyond the Yellow Highlighter”. Searching for this title should also reveal discussion and examples of implementation shared by other educators. 

Cohn, J. (2021). _Skim, dive, surface: Teaching digital reading_. West Virginia University Press.

McIntosh, J. (2019). Clip, Tag, Annotate: Active Reading Practices for Digital Texts. In _Digital Reading and Writing in Composition Studies_ (pp. 176-188). Routledge

Porter-O’Donnell, C. (2004). Beyond the yellow highlighter: Teaching annotation skills to improve reading comprehension. _English Journal_, 82-89.

Loading

Teaching and authoring to learn

I have long been interested in the instructional strategies of teaching and what I call authoring (e.g., writing) to learn. These are concepts that seem to resonate with educators and that have a substantial research base arguing for effectiveness in applied settings. Both teaching and authoring seem excellent culminating activities for project-based learning activities which is another approach that many educators find appealing. Some of my existing resources on these topics are available here and here.

From time to time I review the research literature to catch up on new developments related to the topics I write about. This has recently been the case with what is formally called “noninteractive teaching”. This is a variant of teaching to learn in which the student as educator writes or generates a video intended to explain something to learners (e.g. Book Creator, Explain Everything, Screencastify). As a classroom strategy this approach is argued to be more practical as the task does not require the arrangement of the interactive component of teaching. It is in some ways similar to writing to learn, but it now frequently involves the creation of multimedia content or the use of video. Students can create these products as an assignment.

Applied research in education can be very frustrating. Findings often do not replicate for many reasons – the content to be learned, the background knowledge of the learner, the outcome variable used, and so on. This is what I seemed to encounter when reviewing recent research on “noninteractive teaching”. I had hoped to write a review of research, but decided to reference a recent review instead.

Several authors (Lachner & colleageus) who have investigated non-interactive teaching generated an article to speculate about about the inconsistencies in their own and related research (see reference at conclusion of this post). They generated a review paper that first differentiates interactive teaching from non-interactive teaching and then attempts to address the inconsistencies in findings of the second category of studies. The review is thorough and I recommend it for academics interested in this topic. While urging researchers to continue to refine their understanding of this activity, the authors concluded with the observation that the following recommendations seemed to have emerged – verbal (video) rather than text-based, from memory rather than while accessing a source, and restudy after presentation rather than presentation being the final experience.

I disagree to some extent with the authors’ conclusion regarding the generation of text (text integrated with other media) as multiple studies support the learning that results from summarization and writing to learn. Why the expectation that writing to teach would generate a different outcome for the author makes no sense to me. My effort here is to report the recommendation of the group summarizing the recent research focused on the generation of content intended to teach.

Lachner, A., Hoogerheide, V., van Gog, T., & Renkl, A. (2021). Learning-by-Teaching Without Audience Presence or Interaction: When and Why Does it Work?. Educational Psychology Review, 1-33.

Loading

Teaching to learn – 1

Teaching to learn is one of the generative activities that intrigues me. This may partly be a function of personal experiences as a teacher. Being put in the position of having to stand in front of others and explain requires something different from the study of a topic. Just what is different though. I intend to offer a couple of posts based on research to answer this question. 

The first explanation is based on research claiming the preparation to teach amounts is a variant of the retrieval practice effect. Retrieval practice involves attempts to recall the content to be studied. This is best done with recall that does not involve prompts such as would be the case with multiple choice questions. Something like flash cards with general requests for recall/explanation would be an example. 

A study by Koh and colleagues (reference at the end of the post) was designed to test this hypothesis. 

The study involved four groups – a control group that studied the assigned content; the teaching group that studied the material and taped a 5 minute presentation without notes; a group that studied and then delivered a 5 minute presentation using a prepared script; a group that studied and then completed free recall questions about the material.

Consider what is required in each treatment. The teaching group not allowed to use a script had to rely on what they knew when making their presentations. The group allowed to use a script could rely on the external source of information. The free recall group used an established method of retrieval practice.

Both the teaching group and the retrieval practice group performed better on a later test than the control condition which involved traditional study behavior. The group that taught with the use of the script did not. The authors argued this pattern supports the retrieval practice explanation.

This study was designed to test the value of retrieval practice and showed the two retrieval practice treatments were most effective. For classroom application, consider what was actually involved. Making a video is only part of what one does when teaching. It demonstrates a value in the requirement of using what you know to create a representation. This sounds like writing to learn or in this case multimedia content creation to learn. 

Teaching involves other potentially important activities not testing in this research. What about interacting with students? Interaction is even more demanding than presentation as it is not totally under the control of the educator and involves more cognitive flexibility. There are other aspects of teaching that may also contribute to personal learning. 

Koh, A. W. L., Lee, S. C., & Lim, S. W. H. (2018). The learning benefits of teaching: A retrieval practice hypothesis. Applied Cognitive Psychology, 32(3), 401-410.

Loading

External tasks to influence internal tasks

My way of exploring what I know of cognitive psychology to the selection of effective learning tasks is captured in the phrase – external tasks to influence internal tasks. Learners must accomplish learners themselves, but educators can provide experiences to learn from (exposure to information) and propose activities that potentially influence how learners process these experiences. The word potential is important here because the educator is assuming that the learner will apply the external task in a way that engages important cognitive tasks and the external task does not complete more effective cognitive activity learners might have applied on their own (i.e. busy work). When applied to a group which is most commonly how assignments are made, these caveats are probably both violated for certain individual students.

This analysis may sound obvious but as is usually the case the devil is in the details. There must be some understanding of what cognitive behaviors produce learning and which of these cognitive behaviors might be engaged by the assignment of an external task. Both requirements have been addressed by great numbers of research studies.

Many cognitive psychologists use the phrase generative learning to refer to the approach I have described in my own way. To move this presentation toward application it would be useful to read a 2016 paper by Fiorella and Mayer. These authors identify general categories of activities that have shown to have effective generative capabilities. The paper references multiple studies that evaluate examples of the application of each type of task.

This specific article identified eight learning strategies that promote generative learning and provides a review of research relevant to each strategy.

Summarizing
Mapping
Drawing
Imagining
Self-Testing
Self-Explaining
Teaching
Enacting

As a way of simplifying what these generative tasks ask of learners consider the following two ways of simplifying what the tasks require.

The first four strategies (summarizing, mapping, drawing, and imagining) involve changing the input into a different form of representation.

The final four strategies (self-testing, self-explaining, teaching, and answering practice questions) require additional elaboration.

Just example of how this type of consolidation of research might be applied consider how the list might be applied to note-taking another of the topics I have been addressing lately. The Cornell note template is popular with educators. The template asks that learners use two items from this list in the full application of the Cornell method. The template includes an area for summarization and encourages the use of the column that normally appears to the left of the area for taking notes for questions.

Fiorella, L., & Mayer, R. E. (2016). Eight ways to promote generative learning. Educational Psychology Review, 28(4), 717-741.

Loading

Reality of Learning Tactics

Folks like me and many I follow get all excited about the latest learning tactics and the research that investigates if and why the tactics work. Every once in a while I think back to some observations I made while lecturing to large groups of undergraduates who took Introductory and Educational Psychology.

After an introduction to the Cornell Note-Taking system, I asked if anyone recognized what I was describing from middle-school or high school. Typically, a third-or so of the students would raise their hands. I then would ask how many were using the Cornell system to take notes on my presentation. In doing this for many years, I think I may have found one or two students who were using the system. For the occasional ed psych prof who reads my posts, give these questions a try and see what you discover.

I often ask about this experience in my grad courses seeking an explanation. Nothing much ever emerges from this request, but I would often observe that more research should be focused on the barriers to the adoption of proven study tactics. The Cornell system is simple enough. It can’t be exposure since the Cornell system is introduced in K12 and college study skill programs. Maybe the younger students were required to show that they were using the system.

The one exception I can think of to my observation regarding college student application of study tactics is the use of flash cards. At least some students in fields that require the memory for lots of specifics (I tend to think of PT and OT students) I noticed breaking out their decks of cards while waiting for my classes to begin. So there is this interesting exception to investigate. Why flash cards and note Cornell notes?

Loading

Adding a stage and an activity to the notetaking model

I came across a study on notetaking behavior that I thought might be a missing link in the other notetaking ideas I have been reading lately. Kiewra is a name that comes up repeatedly in thois area of research. I am guessing one of his students (Luo), Kiewra and Samuelson published a study in 2017 proposing that storage and review may not be an adequate way to engage in product note use. Perhaps a three stage approach would be more productive. Rather than just recording and review, effective notetaking might benefit from an intermediate stage – revision.

Luo, et al. (2016) investigate revision during pauses in a presentation or for the same amount of time immediately after the presentation. They speculate about benefits mostly from the increase in content added as notes or a type of retrieval practice explanation for later achievement gains.

I admit that this seems different from what I would describe as a generative effect. A three stage model makes some sense. For example, the Cornell notetaking system was developed to encourage a revision process and this additional activity was about more than just adding content that had been missed. Aherns book on Smart Notes proposed several types of notes generated over time:

  1. Make fleeting notes.
  2. Make literature notes.
  3. Make permanent notes

Perhaps the focus on preparing for an exam is different than the Smart Note notion for long-term storage and personalized understanding. Whether for more meaningful use at a later date or to improve personal understanding, revision might make the most sense if was more what Ahrens had in mind.

Ahrens, S. (2017). How to Take Smart Notes: One Simple Technique to Boost Writing, Learning and Thinking–for Students, Academics and Nonfiction Book Writers. Sönke Ahrens.

Luo, L., Kiewra, K. A., & Samuelson, L. (2016). Revising lecture notes: how revision, pauses, and partners affect note taking and achievement. Instructional Science, 44(1), 45-67.

Loading