Learning Aloud

Flashcards – I

Giving Flashcards a New Look

I thought you might appreciate a different example from the type of thing I tend to focus on explaining how a study technique used by students at all levels might be improved through the use of digital technology. When I taught educational psychology I included presentations on what I considered the classic recommended techniques – e.g., SQ3R, Cornell Notes. As I have explained in other things I have written on this general topic I would also ask how many of the students were aware of these strategies and then how many were using them in our class. Most were familiar with at least these two, but I can’t remember anyone ever saying they were using these approaches. Most took notes, but not Cornell notes and then they studied these notes. They tended to highlight the assigned reading content.

A topic I mentioned because I knew the strategy, but did not spent a lot of time on was flashcards. I have thought about why the use of flashcards was not a topic I discussed more since. I think at the time this was because there seemed a stigma associated with learning methods that were considered rote and not the way to develop understanding and a long term benefit. I kind of see a similar attitude surfacing from time to time now and it is often in the context of “why would you expect someone to store that in memory when they can just look it up with Google”?

I have changed my mind about flashcards. I think the students had a more realistic perspective than I did. They knew flashcards worked for them. I also think my perspective on effective learning has also changed some as I have read more about how humans learn. I have to decided to generate at least two posts on digital flashcards. I will being with what is mostly a tutorial describing a digital flashcard system I like. Eventually, I will provide additional content explaining what research has to say about study strategies that can be implemented with flashcards.

Elementary students have used flashcards to drill on math facts for decades and I commonly see university students studying for anatomy or geology examinations using nearly the same procedure. Flashcards provide individuals or individuals within small groups a way to systematically practice retrieval and evaluate understanding both of which are important and documented ways to improve performance and retention (Rohrer, & Pashler, 2010). I assume you understand how flashcards are traditionally used from observation if not from personal experience. In a typical approach a student has a stack of cards with a question on one side of each card and a response on the other side. Elementary math fact cards may have been purchased commercially, but the review material used by college students was probably generated by the learner. When a few minutes are available, a student brings out the cards and begins going through the stack. Do you know the answer to the math problem? Do you remember the origin and insertion for the muscle listed? You flip the card over to see if you were correct. Perhaps you developed a variant of this approach you thought improved the process. You may have discarded each card after initial consideration into a “I know that” pile or a “I don’t know that pile” and then spend more time with the “I don’t know that pile” instead of going through the entire stack a second time.

Researchers have considered how the use of review techniques of this type might be optimized. These techniques might involve how frequently you view cards with information you do not know versus cards with information you do know or on how many different days you review material. Some of these findings and other inherent advantages of digital media are now incorporated in free or low-cost software or services you can find and use online or download to your personal device.

So, I might begin by asking the question, How might flashcards on the computer, a smartphone, or tablet offer advantages in comparison to the standard paper variety? I don’t think this is another version of the paper versus screen debates that now seem common, but some could see it that way.

What follows is an example of a flashcard application and then a consideration of the unique opportunities this and similar applications offer. These examples makes use of Flashcards Deluxe (information about this application and a few other flashcard options appears at the end of this segment); an app for the iPad or iPhone.

Flaschards Deluxe. A classroom example I sometimes use to explain that technology is not confined to the classroom we call the Turtle River Water Quality Project. One aspect of this project involved the collection and identification of macroinvertebrates. The prevalence of different macroinvertebrates (organisms without a backbone that can be viewed without aid of a microscope) can be used to evaluate water quality. Some macroinvertebrates can tolerate poor quality water and some cannot. So the collection and identification of macroinvertebrates allows a way to evaluate the “health” of a body of water.

Before making the trip to collect specimens, a teacher might want to prepare students by familiarizing them with the set of organisms they might encounter. This task involves learning to associate the name and whether an organism is tolerant of pollution with an image of the organism. Imagine a three-sided flash card – if there was such a thing. One side might contain an image of the organism, the second side would contain the name of the organism, and the third side would indicate whether the organism was pollution tolerant. If this were possible to create as a physical, paper card, the student would look at a picture, attempt to remember the name and tolerance of the organism, and then flip the card to check the information recalled or view the correct information. The student would then move to the next card and repeat the process.

The digital equivalent, created with Flashcards Deluxe, would work like this. The student would view the image appearing on the first card trying to remember the name of the organism and whether or not the organism is an indicator of healthy water (see Figure Front View). The student would then “tap” the image to flip the card and reveal the name of the organism and would tap the card a second time to reveal whether or not the organism is associated with healthy water (see Figure Composite Feedback). Selecting the “check mark” or “X” appearing at the bottom of the card would indicate whether the student had judged performance to be adequate or not and would move the student to the next image. When the student reached the end of the deck, all cards associated with a judgment of inadequate recall would automatically be repeated.

Front view of Flashcard Deluxe Card

Back views of Flashcards Deluxe Card

Individual flashcards are created using the following template. Text, images, and sound are added to the slots on this template representing the three sides of a card.

This video provides more details on this tool.

Digital Advantage

What case can be made for the digital version of flashcards? Here are some thoughts on potential advantages. All of these characteristics are available in some products or services, but not necessarily in all so you would want to review the capabilities of a given product before investing a lot of time.

A) Multiple media – Some digital flashcards can incorporate images and sounds (e.g., our example). If the value in multimedia flashcards is not obvious, consider situations in which the study task involves identification of orchestra instruments by the sound the instruments produce, recognition of vocabulary in a foreign language class based on listening rather than reading, or the identification of molecular structures in biochemistry. These would be study situations that would be difficult or impossible to address with paper index cards. You can create multimedia resources yourself. For example, you can collect images with your phone and use these images to create study materials. Would this be useful in preparing for biology or anatomy lab practicals?

B) Existing and shareable review materials – There are massive collections of existing review materials already available for a wide variety of academic areas. There appear to be basic standards or at least common formats for exporting sets of cards so that many of the resources developed through one service can be used on the software provided by other vendors. These collections can be downloaded, combined, and edited by individual students to best address personal study goals. It will often be more efficient to take a large existing collection and delete the cards not appropriate for your class than to create a new stack from scratch. A teacher can do the same thing and then share a set of cards optimized for a specific class with students. We include links to several vendors who store and share collections at the end of this section.

C) Data and suggested review strategies – It might be best to consider what we are describing here as study systems rather than just study materials. In addition to the information to be learned, these systems can keep track of performance on individual items and data such as the last date a given card was reviewed. If the student is so inclined, some systems can present items that cause the student difficulty more frequently or even schedule which content should be reviewed on a particular day within a designated time period. This is where our previous reference to what research can tell us about optimized review and rehearsal comes into play. Computers are good at keeping track of and applying numerical performance data. So, it turns out that free or low-cost digital resources offer more than the option of thumbing through the stack of cards you carry in your backpack. If you presently make use of “flashcards” or see such resources as having a use in classrooms that are or will be your responsibility, we encourage you to explore the following resources. I have included some resources that can be used from a computer and some from a tablet.

Resources:

Sites providing repositories of flash cards:

Flashcard Exchange

Quizlet

Flashcard products:

Quizlet is an online system you can use with your browser. Study existing stacks or create your own.

Flashcards Deluxe – This application is available for the iPad and iPhone from the Apple App Store. The application was created by OrangeOrApple.

AI, Tech Tools, and the Writing Process

Student access to AI has created a situation in which educators must consider when AI should and should not be used. I think about this question by considering the difference between what skill or skills are the focus of instruction and whether AI will replace a skill to improve the efficiency of the writing task or will support a specific skill in some way. It may also be useful to differentiate learning to write from writing to learn. My assumption is that unless specific skills are used by the learner those skills will not be improved. Hence when AI is simply used to complete an assignment a learner learns little about writing, but may learn something about using AI.

Writing Process Model

The writing process model (Flower & Hayes, 1981) is widely accepted as a way to describe the various component skills that combine to enable effective writing. This model has been used to guide both writing researchers and the development of instructional tactics. For researchers, the model is often used as a way to identify and evaluate the impact of the individual processes on the quality of the final project. For example, better writers appear to spend more time planning (e.g., Bereiter & Scardamalia, 1987). For educators and instructional designers, understanding the multiple processes that contribute to effective writing and how these processes interact is useful in focusing instruction.

Here, the writing process model will be used primarily to identify the subskills to be developed as part of learning to write and writing to learn and I will offer my own brief description of this model. It is worth noting that other than composing and rewriting products, other uses of technology to improve writing and increase the frequency of writing experiences seldom receive a lot of attention (Gillespie, Graham, Kiuhara & Hebert, 2014).

The model

The model identifies three general components a) planning, b) translation, and c) reviewing.

Planning involves subskills that include setting a goal for the project, gathering information related to this goal which we will describe as research, and organizing this information so the product generated will make sense. The goal may be self-determined or the result of an assignment.

Research may involve remembering what the author knows about a topic or acquiring new information. Research should also include the identification of the characteristics of the audience. What do they already know? How should I explain things so that they will understand? Finally, the process of organization involves establishing a sequence of ideas in memory or externally to represent the intended flow of logic or ideas.

What many of us probably think of as writing is what Flower and Hayes describe as translation. Translation is the process of getting our ideas from the mind to the screen and this externalization process is typically expected to conform to conventions of expression such as spelling and grammar.

Finally, authors read what they have written and make adjustments. This review may occur at the end of a project or at the end of a sentence. In practice, authors may also call on others to offer advice rather than relying on their own review.

One additional aspect of the model that must not be overlooked is the iterative nature of writing. This is depicted in the figure presenting the model by the use of arrows. We may be tempted, even after initial examination of this model, to see writing as a mostly linear process – we think a bit and jot down a few ideas, we use these ideas to craft a draft, and we edit this draft to address grammatical problems. However, the path to a quality finished product is often more circuitous. We do more than make adjustments in spelling and grammar. As we translate our initial ideas, we may discover that we are vague on some points we thought we understood and need to do more research. We may decide that a different organizational scheme makes more sense. This reality interpreted using our tool metaphor would suggest that within a given project we seldom can be certain we have finished the use of a given tool and the opportunity to move back and forth among tools is quite valuable.

Tech tools and writing

Before I get to my focus on AI tools, it might be helpful to note that technology tools used to facilitate writing subprocesses have existed for some time. For example, spelling and grammar checkers, outline and concept mapping, note-taking and note-storage, citation managers, online writing environments allowing collaboration and commenting, and probably many other tools that improve the efficiency and effectiveness of writing and learning to write. Even the use of a computer allows advantages such as storage of digital content in a form that can easily be modified rather than the challenge of making improvements to content stored on paper. The digital alternative to paper changes how we go about the writing process. I have written about technology for maybe 20 years and one of the bextbooks offered the type of analysis I am offering here not about AI tools, but about the advantages of writing on a computer and using various digital tools.

A tool can substitute for a human process or a tool can supplement or augment a human process. This distinction is important when it comes to writing to learn and learning to write. When the process is what is to be learned, this substitution is likely to be detrimental as it allows a learner to skip needed practice. In contrast, augmentation often allows the opposite as a busy work activity or some incapability is taken care of allowing more important skills to become the focus.

Here are the types of tools I see as supporting individual writing processes.

Planning – Organization and Research

Prewriting involves developing a plan for what you want to get down on paper (or screen in this case). A writer goes about these two subprocesses in different ways. You can think or learn about a topic (research) and then organize these ideas in some way to present. Or, you can generate a structure of your ideas (organize) and then research the topics to come up with the specifics to be included in a presentation. Again, these are likely iterative processes no matter which subskill goes first.

One thing AI does very well is to propose an outline if you are able to generate a prompt describing your goals. You could then simply ask the AI service to generate something based on this outline, but this would defeat the entire purpose of learning about the topic by doing the research to translate the outline into a product or developing writing skills by expanding the outline into a narrative yourself.

Since I am writing about how AI might perform some of the subskills identified by the writing process model, I asked ChatGPT to create an outline using the following prompt.

“Write an outline for ways in which ai can be used in writing. Base this outline on the writing subprocesses of the writing process model and include examples of AI services for the recommended activity for each outline entry.”

The following shows part of the outline ChatGPT generated. I tend to trust ChatGPT when it comes to well established content and I found the outline although a little different from the graphic I provided above to be quite credible and to offer reasonable suggestions. As a guide for writing on the topic I described, it would work well.

I had read that AI services could generate concept maps which would offer a somewhat different way to identify topics that might be included in a written product. I tried this several times using a variety of prompts with ChatGPT’s DALLE. The service did generate a concept map, but despite making several follow-up requests which ChatGPT acknowledged, I could not get the map to contain intelligible concept labels. Not helpful.

Translation

Tools for improving the translation process have existed in some form for a long time. The newest versions are quite sophisticated in providing feedback beyond basic spelling and grammatical errors. I write in Google docs and make use of the Grammarly extension.

I should note that Grammarly is adding AI features that will generate text. Within the perspective I am taking here I have some concerns about these additions. Since I am suggesting that writing subskills can be replaced or supported, student access to Grammarly could allow writing subskills the educator was intending students to perform themselves to be performed to some degree by the AI.

If you have not tried Grammarly, the tool identifies different types of modifications the tool proposes different modifications the writer might consider changing (spelling, missing or incorrect punctuation, alternate wording, etc.) and will make these modifications if the writer accepts the suggestion. The different types of recommendations are color-coded (see following image).

Revision

I am differentiating changes made while translating (editing) from changes made after translating (revision). Minor changes such as spelling and grammar would seem more frequently fixed as edits by this distinction and major modifications made (addition of examples, restructuring of sections, deletion of sections, etc.) while revising. Obviously, this is a simplistic differentiation and both types of changes occur during both stages).

I don’t know if I can confidently recommend a role for AI for this stage. Pre-AI, one might recommend that a writer share their work with a colleague and ask for suggestions. The AI version of Grammarly seems to be moving toward such capabilities. Already, a writer can ask AI to do things like shorten a document or generate a different version of a document. I might explore such capabilities out of curiosity and perhaps to see how modifications differ from my original creations, but for work that is to be submitted for evaluation of writing skill would that be something an educator would recommend?

I have also asked an AI tool to provide an outline, identify main ideas or generate a summary of a document I have written just to see what it generates. Does the response to one of these requests surprise me in some way? Sometimes. I might add headings and subheadings to identify a structure I thought was not as obvious as I had thought.

Conclusion:

My general point in this post was that questions of whether learners can use AI tools when assigned writing tasks should be considered in a more complex way. Rather than the answer being yes or no, I am recommending that learning to write and writing to learn are based on subprocesses and the AI tool question should be considered in response to a consideration of whether the learner was expected to be developing proficiency in executing a subprocess. In addition, it might be important to suggest that learning how to use AI tools could be a secondary goal.

Subprocess here were identified based on the Writing Process Model and a couple of suggestions were provided to illustrate what I mean by using a tool to drastically reduce the demands of one of the subprocesses. There are plenty of tools out there not discussed and my intention was to use these examples to get you thinking about this way of developing writing skills.

References:

Bereiter, C., & Scardamalia, M. (1987). An attainable version of high literacy: Approaches to teaching higher-order skills in reading and writing. Curriculum inquiry, 17(1), 9-30.

Flower, L., & Hayes, J. R. (1981). A cognitive process theory of writing. College composition and communication, 32(4), 365-387.

Gillespie, A., Graham, S., Kiuhara, S., & Hebert, M. (2014). High school teachers use of writing to support students’ learning: A national survey. Reading and Writing, 27, 1043-1072.

Obsidian/Smart Connections Workflow

I use Obsidian plus the plugin Smart Connections to inform my blog writing activities. I write for educational practitioners and academics so I try to carefully base my content on sources that I have read and in many cases intend to cite in the content I generate. With this goal, Obsidian represents an archive I have developed over several years to store and organize notes from hundreds of books, journal articles, and websites. I explore my collection in different ways sometimes seeking notes on a specific article I want to emphasize and sometimes exploring to locate what I have read that is relevant to a topic that I might want to include but perhaps do not recall at the time.

In some cases, I want to use an AI tool to support my writing. I seldom use AI to actually generate the final version of content I post, but I may explore the possible organization of material for something I want to write or I might use an AI tool to generate an example of how I might explain something based on the notes I have made available to the AI tool.

The combination of Obsidian augmented by the Smart Connections plugin allows me to implement a workflow I have found useful and efficient. I have several specific expectations of this system:

I have already read the source material and taken some notes or generated some highlights now stored in Obsidian. I want to write based on this content.
I may not recall relevant sources I have stored in Obsidian because of the passage of time and the accumulation of a large amount of material. I want the AI system to understand my goals and locate relevant content.
I want the AI system to identify specific sources from the content I have reviewed rather than the large body used to train the LLM. I want the system to identify the specific source(s) from this material associated with specific suggestions so that I am aware of the source and can cite a source if necessary.
When a specific source has been identified I want to be able to go directly to the original document and the location within that document that is the location for the note or highlight that prompted the inclusion in the AI content so that I can reread the context for that note or highlight.

Obsidian with the Smart Connections plugin does these things and is to some extent unique because all of the material (the original content) is stored locally (actually within iCloud which functions as an external harddrive) allowing the maintenance of functioning links between the output from Smart Connections, the notes/highlights stored in Obsidian, and the original documents (pdfs of journal articles, Kindle books, web pages).

I do not know for certain that the Obsidian-based approach I describe is the only way to take the approach I take. I am guessing my approach works in part because I am not relying on an online service and online storage. I also use Mem.ai because it allows me to focus on my own content, but linking back to source documents does not work with this service. Mem.ai does include the AI capabilities as part of the subscription fee, but I don’t know when this might be an advantage. The Smart Connections plugin does require the use of an OpenAI API (ChatGPT) and there is a fee for this access.

Example:

Here is an example of what working with the Obsidian/Smart Connections setup is like. I am working on a commentary on the advantages and disadvantages of K12 students having access to AI in learning to write and writing to learn. I propose that writing involves multiple subprocesses and it is important to consider how AI might relate to each of these subprocesses. My basis for the list of subprocesses is based on the classic Flower and Hayes Writing Process Model. I had written a description of the Writing Process Model for a book I wrote and this section of content was stored within Obsidian as well as notes from multiple sources on AI advantages and disadvantages in the development of writing skills. I have not read a combination of the writing process model with ideas about the advantages and disadvantages of AI so this is the basis for what I think is an original contribution.

The following is a screenshot of Obsidian. The Smart Connection appears as a panel on the right side of the display. The left-hand panel provides a hierarchical organization of note titles and the middle panel provides access to an active note or a blank space for writing a new note.

In the bottom textbox of the Smart Connections panel, I have entered the following prompt:

Using my notes, how might AI capabilities be used to improve writer functioning in the different processes identified by the writing process model. When using information from a specific note in your response, include a link to that note.

Aside from the focus of the output, two other inclusions are important. First, there is the request to “use my notes”. This addition is recommended to ensure a RAG (retrieval augmented generation) approach. In other words, it asks the AI service use my notes rather than the general knowledge of the AI system as the basis for the output. The second supplemental inclusion is the request to include a link to that note which is intended to do just what it says – add links I can use to to see where ideas in the output came from.

The output from Smart Connections is in markdown. I copied this output into a new blank note and the links included are now active.

I purposefully selected a note that initially was part of a web page for this final display. I had originally used a tool that allowed the annotation of web pages and then the exporting of the annotated and highlighted content as a markdown file I added to Obsidian. This file included the link from the note file back to the online source. As you can see, the link from Obsidian brought up the web page and with the assistance of the activated service added as an extension to my browser displays what I had highlighted within this web page. Interesting and useful.

Conclusion:

We all have unique workflows and use digital tools in different ways because of differences in what we are trying to accomplish. What I describe in this post is an approach I have found useful and I have included related comments on why. I hope you find pieces of this you might apply yourself.

Attacks on Higher Education

My own story includes a career working in higher education so I suppose that is mostly responsible for my reactions to what now seem to be the attacks on my occupation. I always want to point out one fact. The state through the legislature and the budget controlled by the legislature fund less than a third of the cost of state institutions. Yet, legislatures can pass laws that dictate how these institutions function. Most revenue necessary to run institutions must be generated by the institutions through tuition, grants, and donations and institutions must compete for these resources.

Two of the issues that some states want to control are the curriculum (what is taught) and tenure. These two issues are interrelated in this way. Tenure protects faculty members against even their own institutions. Since tenure gives a professor job security for decades, undercutting it is one way conservatives can more easily change campus culture, experts say, as well as fight against the liberal values they claim have taken over schools. For example, In Indiana, legislation is being proposed to review tenured professors every five years based on “free inquiry, free expression, and intellectual diversity.” Tenure is intended to be based on productivity and teaching quality, but the practice is being challenged because of what is being taught rather than issues of quality performance.

The American Association of University Professors (AAUP) released data showing that in the fall of 2021, only 24 percent of faculty in U.S. colleges held full-time tenured appointments, compared to 39 percent in the fall of 1987. Rather than hire for tenure-track “lines”, more and more institutions are hiring adjunct faculty and more and more instruction is being delivered by adjunct faculty members. I saw this happen in the department for which I once served as chairperson and I thought this approach created a divisive class system. Tenure is already a probationary period with no commitment to the long term and all of us know profs who simply were let go because there was a financial challenge rather than a performance issue.

Anyway, this post is really about the motivation for eliminating tenure which despite other issues seems most basically about political control of institutions. The core complaint seems to be that college faculty are too liberal (or progressive whichever you prefer) which is proposed to be relevant because these values are being passed on (indoctrination is a common description) to unwary or cowed students. Efforts have been made to determine the accuracy of the first claim and seem to show few students actually claim they feel pressured. The second issue is more difficult and I think really breaks down into two underlying claims; a) liberal values are taught beyond what would be the appropriate information for a course, and b) students are afraid to voice opposition when issues of opinion as opposed to facts are discussed or are penalized in some way when they do challenge opinions or facts.

Party affiliation versus values:

Complaints when it comes to what is being taught seem to come primarily from one party. Why? Party labels seem far less helpful when reacting to accusations than specific values. I doubt many educators focus on party when teaching their content, but values and facts related to values could sometimes be the focus of instruction. I am not so naive to argue that party affiliation and values are not at least moderately correlated, but even knowing this I think a lot about why this is the case. Shouldn’t values be more important and what are the specific values taught that Conservatives reject?

I think people hide behind general labels. I assume that values determine party affiliation and not the other way around. Declared party affiliation could also be associated with combinations of values. Owning up to specific values and beliefs is what is really important. Owning up to the beliefs professed by your party is also at least important to consider. It seems cowardly to say there are too many Democrats on college faculties rather than to claim you object to the discussion of the causes of prejudice, inequity, and privilege or whatever other values you find objectionable.

PEW has done a lot of data collection on values and party affiliation in trying to provide data about education and other issues. They have a simple questionnaire about specific values allowing anyone to compare their specific values to political categories from extreme progressive to extreme conservative. I encourage you to take it for two reasons: a) see where you fall on the political continuum and b) consider the issues raised in assigning you to a given category and why they could be interrelated.

Pew Survey

I fell toward the end of political spectrum PEW classified as embellishment liberal (13% fall into this category).

PEW summarizes folks like me in the following way.

“They hold liberal positions on nearly all issues and support an expanded role for government and a larger social safety net. They also hold liberal attitudes on issues of racial and ethnic equality. Establishment Liberals are more likely than any other group to say that compromise is how things get done in politics. About half say they are satisfied with the way things are going in the country today, and an overwhelming majority say they approve of the job Joe Biden is doing as president.”

However, what I think is more important is what I think about specific items PEW used to arrive at these generalities.

The PEW survey presents items as a forced choice. See the following example:

Once, you have completed the survey, the system reviews your responses and compares your answer to the probability of others in your assigned group and to all respondents. So. in the following example, 61% of all those responding and 80% of the others assigned to the Establishment Liberals category agreed with my choice.

I recommend the PEW survey as it could recommend a level on which discussions could occur. I can explain why there are specific areas in which I think the country could benefit from government oversight and a safety net. You might disagree, but at least that would offer something concrete to talk about. Isn’t the level that a discussion should occur in a political science classroom.

Party affiliation among higher education educators.

Efforts to get academics to claim their party orientation do indicate that there is a general progressive orientation. Such data sources also show that political orientation varies significantly with discipline.

Perhaps the concern of the Republicans is more driven by the increasingly progressive leanings over time. This time difference is consistent by discipline (see above) and quite abrupt in recent years. Notice the shift beginning in 1997 or so. Initially, this shift seemed to be marked by fewer Conservatives toward more Liberals with a later shift from Moderate toward Liberals. The makeup of higher ed faculties changes gradually so it seems unlikely this rapid shift was the result of hiring alone.

Analysis

There is an expression sometimes applied when commenting on software applications – Is it a feature or a bug? Does the system contain an error or is the system doing what it is doing by design? Perhaps my rephrasing gets to the real issue here. What is the purpose of higher education and education in general? You have a sense of this issue when you hear some politicians complaining higher education does not focus enough on employment and has too many majors that are not directly tied to specific jobs. Why are there so many liberal arts requirements and so few STEM requirements? Do K12 and college students really need to learn about the history of slavery and continuing inequities related to race, gender, and generational differences in wealth? I happen to think so and these facts are relevant to the content I taught. Maybe those in math, physics, and coding don’t, but an important argument can be made relevant to these areas of study as well. Consider the present questions and reflection raised by the movie Oppenheimer. There is even the question of bias that keeps popping up because AI ends up statistically replicating the existing biases in the mathematics and algorithms generating the responses to AI prompts based on the corpus of content used to construct large language models (LLMs). Instead consider the old CS saying – garbage in, garbage out. Perhaps we need to recognize that it is now skewed values in, skewed values out. Values cannot be isolated by job category and why need to be concerned with the development of values in everyone.

Writing to Learn Research – Messy

Writing to learn is one of those topics that keeps drawing my attention. I have an interest in what can be done to encourage learning and approach this interest by focusing on external tasks that have the potential to manipulate the internal cognitive (thinking) behavior of learners. My background in taking this perspective is that of an educational psychologist with a cognitive perspective. I have a specific interest in areas such as study behavior trying to understand what an educator or instructional designer can do to promote experiences that will help learners be more successful. The challenge seems obvious – you cannot learn for someone else, but you may be able to create tasks that when added to exposure to sources of information encourage productive “processing” of those experiences. We can ask questions to encourage thinking. We can engage students in discussions that generate thinking through interaction. We can assign tasks that require the use of information. Writing would be an example of such an assigned task.

Writing to Learn

Writing to learn fits with this position of an external task that would seem to encourage certain internal behaviors. To be clear, external tasks cannot control internal behavior. Only the individual learner can control what they think about and how they think about something, but for learners willing to engage with an external activity that activity may change the likelihood productive mental behaviors are activated.

I found the summary of the cognitive benefits of writing to learn useful and consistent with many of my own way of thinking about other learning strategies – external tasks that encourage productive internal behaviors. Writing based on content to be learned requires that the writer generate a personalized concrete representation at the “point of utterance”. I like this expression. To me, it is a clever way of saying that when you stare at the screen or the empty sheet of paper and must fill the void you can no longer fool yourself – you either generate something or you don’t. You must use what you know and how you interpret the experiences that supposedly have changed what you know to produce an external representation.

To produce an external product, you must think about what you already know in a way that brings existing ideas into consciousness (working memory) by following the connections activated by the writing task and newly acquired information. This forces processing that may not have occurred without the external task. Connections between existing knowledge and new information are not necessarily made just because both exist in storage. Using knowledge to write or to perform other acts of application encourages making connections.

Such attempts at integration may or may not be successful. Having something external to consider offers the secondary benefit of forced metacognition. Does what I wrote really make sense? Do the ideas hang together or do I need to rethink what I have said? Does what I have proposed fit with the life experiences (episodic memory) I have had?

Writing ends up as a generative process that potentially creates understanding and feeds the product of this understanding back into storage.

Graham, Kiuhara & MacKay, M. (2020)

In carefully evaluating and combining the results of many studies of writing to learn, these researchers intended not only to determine if the impact of writing to learn had the intended general benefit but to use the variability of writing tasks and outcomes from studies to deepen our understanding of how writing to learn encouraged learning. Surely, some activities would be more beneficial than others because of the skills and existing knowledge of learners or the specifics of the assigned writing tasks. So, the meta-analysis is asking if there is a general effect (Is writing to learn effective), and secondarily are there significant moderator variables that may help potential practitioners decide when, with whom, and how to structure writing to learn activities?

The Graham and colleagues’ research focused only on K12 learners. Potential moderator variables included grade level, content area (science, social studies, mathematics), type of writing task (argumentation, informational writing, narrative), and some others. I have a specific interest in argumentation () which is relevant here as a variable differentiating the studies because it requires a deeper level of analysis than say a more basic summary of what has been learned.

Overall, the meta-analysis demonstrated a general benefit for writing to learn (Effect size = .30). This level of impact is considered on the low end of a moderate effect. Graham and colleagues point out that the various individual studies included in the study generated great variability. A number of the studies demonstrated negative outcomes meaning in those studies the control condition performed better than the group spending time on writing to learn. The authors propose that this variability is informative as it cannot be assumed that any approach with this label will be productive. The variability also suggests that the moderator variables may reveal important insights.

Unfortunately, the moderator variables did not achieve the level of impact necessary to argue for useful insights as to how writing to learn works or who is most likely to be a priority group for this type of activity. Grade level was not significant. The topic area was not significant. The type of writing task was not significant.

Part of the challenge here is having enough studies focused on a given approach with enough consistency of outcomes to allow statistical certainty in arguing for a clear conclusion. Studies that involved taking a position and supporting that position (e.g., argumentation) produced a much larger effect size, but the statistical method of meta-analysis did not reach the level at which a certain outcome could be claimed.

One interesting observation from the study caught my attention. While writing to learn is used more frequently in social studies classrooms, the number of research studies associated with each content areas was the smallest for social studies. Think about this. Why? I wonder if the preoccupation of researchers and funding organizations with STEM is responsible.

More research is needed. I know practitioners and the general public get tired of being told this, but what else can you recommend when confronted with the messiness of much educational research? When you take ideas out of carefully controlled laboratories and try to test them in applied settings the results here are fairly typical. Humans left to their own devices as implementers of procedures and reactors to interventions are all over the place. Certainly, the basic carefully controlled research and the general outcome of meta-analysis focused on writing to learn implementation are encouraging, but as the authors suggest the variability in effectiveness means something, and further exploration is warranted.

Reference

Graham, S., Kiuhara, S. A., & MacKay, M. (2020). The effects of writing on learning in science, social studies, and mathematics: A meta-analysis. Review of Educational Research, 90(2), 179-226.

A use for Obsidian Unlinked Mentions

Have you had the experience of coming across an application feature and wondering why did a software designer decide to go to the trouble of creating and then shipping that feature? Somewhere I encountered a comment on an Obsidian feature called an Unlinked Mention. It took me some time to find it and then even more time in an effort to understand why it exists. I am still not certain how it is to be used and why there wouldn’t be similar features that would be more useful. I have come up with one way I find it offers some value so I will explain what seems a hack and then hope others can find my description helpful in encouraging similar or additional uses.

Note: My description and proposed actions are based on Obsidian on a computer. Some of the actions I describe I could not get to work on my iPad.

So, I think an unlinked mention is supposed to be understood as something like a backlink. In Obsidian when you create a link among two notes (A – B), Obsidian recognizes but does not automatically display the backlink (B-A). For a given note (A), you can get Obsidian to display any backlinks to that note using the backlinks option for the right-hand panel of the Obsidian display. For the note that is active in the middle panel, the right-hand panel should indicate linked mentions and unlinked mentions. You may have to select which you want displayed and it is possible nothing will be displayed for either option. The linked mentions are the backlinks and you can select and display the backlinked notes from this display.

The unlinked mentions are other notes that contain the same exact phrase as you have used to title Note A. Who knew? Why? Maybe I never quite understood the power of a title or how my notes were supposed to be titled. I have tried to think about this and I still don’t get it.

Here is my hack and I think a way to take advantage of unlinked mentions. Start with a blank note and add a title likely to be used within other content you have stored within other notes. To make the effort, your word or phrase would have to be something you want to investigate. I used the word “metacognition” because this is an important concept in the applied cognitive psychology research I read and attempted to apply to educational uses of technology. I have notes about this concept, but the greatest value I found in this hack was taking advantage of all of the Kindle notes and highlights I had stored in Obsidian via Readwise. In my account, there are more than 200 books worth of notes and highlights and the content for each book is often several pages long. I create notes myself as I read, but there is all of this additional content that may contain things I might find useful. Certainly, several of these books would contain content, especially highlights, focused on metacognition.

Once I have my new note with the simple title “metacognition” and for this note look under unlinked mentions in the right-hand column, I now have lots of entries. At this point, my note is still blank, but I now can access many other mentions of metacognition from this list of unlinked mentions. If I select one of these mentions, a “link” button appears and if I select this button Obsidian generates a forward link in the A document and adds the A document to my blank B document as a backlink. The B note is still blank.

Here comes the hack. One of the core plugins for Obsidians is called backlink (use the gear icon from the panel on the left) and it contains a slider that will display backlinks at the bottom of a note (see following image). Now you can display backlinks on your blank note that allow access to the unlinked content you have linked. See the second image below.

The process I have described is a way to generate a collection of links on a topic that would not be available without this hack. It is the process that finds specific mentions of a concept within much larger bodies of content (the highlights from Kindle books) that I find useful. Give it a try.

AI tutoring based on retrieval generated augmentation

Here is a new phrase to add to your repertoire – retrieval generated augmentation (RAG). I think it is the term I should have been using to explain my emphasis in past posts to my emphasis on focusing AI on notes I had written or content I had selected. Aside from my own applications, the role for retrieval generated augmentation I envisioned is as an educational tutor or study buddy.

RAG works in two stages. The system first retrieves information from a designated source and then uses generative AI to take some requested action using this retrieved information. So, as I understand an important difference, you can interact with a large language model based on the massive corpus of content on which that model was trained or you can designate specific content to which the generative capabilities of that model will be applied. I don’t pretend to understand the specifics, but this description seems at least to be descriptive. Among the benefits is a reduction in the frequency of hallucinations. When I propose using AI tools in a tutoring relationship with a student, suggesting to the tool that you want to focus on specific information sources seems a reasonable approximation to some of the benefits a tutor brings.

I have tried to describe what this might look like in previous posts, but it occurred to me that I should just record a video of the experience so those with little experience might see for themselves how this works. I found trying to generate this video an interesting personal experience. It is not like other tutorials you might create in that it is not possible to carefully orchestrate what you present. What the AI tool does cannot be perfectly predicted. However, trying to capture the experience as it actually happens seems more honest.

A little background. The tool I am using in the video is Mem.ai. I have used Mem.ai for some time to collect notes on what I read so I have a large collection of content I can ask the RAG capabilities of this tool to draw on. To provide a reasonable comparison to how a student would study course content, I draw some parallels based on the use of note tags and note titles. Instead of using titles and tags in the way I do, I propose a student would likely take course notes and among the tags label notes for the next exam with something like “Psych1” to indicate a note taken during the portion of a specific course before the first exam to which that note might apply. I hope the parallels I explain make sense.