Month: March 2008

This is a very long post. Skip to the last couple of paragraphs if you are only interested in what I have to say about wiki applications.

One of the weird things about social science research is that advances are pretty much regarded as equivalent to “significant” outcomes. As an applied researcher, you can find a way to evaluate what folks who don’t do research argue is a great idea, but if the data do not support the claim, you cannot publish the results. Yes, there is all of the stuff you learn in methods classes about not being able to prove the null hypothesis. But, what about the claims made by those who do not worry about data. What information is available to counter such pronouncements.

In my life as a researcher, one of the themes I pursue is the possibility that technology offers opportunities to improve the study effectiveness of novice college students enrolled in unsupportive course environments. I translate this description as those 200 or so freshmen (and women) taking my class in Introduction to Psychology. These are students new to the college environment who often have undeveloped study skills, mediocre reading comprehension scores, and a range of motivational issues including a lack of understand as to why they are enrolled in the course in the first place. They are assigned a 400+ page book and they listen to me talk for approximately 2 hours per week. They do have an hour experience with a graduate student in a small group setting. It is not my decision to teach in this manner. I would love to interact with a dozen or so highly skilled and motivated students who have read widely and come to class prepared to discuss and debate. I actually also have this experience, but that would be my graduate class (sometimes). It is a matter of economics – the less skilled students pay their money and sacrifice so that more productive environments can be provided should the less accomplished students survive for a year or so. However, the basic economics of a state school college education are not the issue here.

My solution is to look for situations in which an economy of scale can be brought to bear on the learning process. I do not consider what happens in my classroom learning. I consider what happens there as information presentation. Yes, I know that others feel their classroom encourages a great deal of thinking and learning. I have heard the sage on the stage vs. guide on the side comparison. Cute – but not very insightful. To me, this is an issue of degree. Learning takes a good deal of time. Learning, the personal integration of new information with existing knowledge and skills, DOES NOT happen in a couple of hours a week. College works a little differently than high school. I cannot study with many of my students. I must assume they can study on their own. I must assume they can think, read, discuss, ponder, identify personal examples of key concepts, etc. outside of the room in which they for the most part simply listen to me. I only hope that I offer them something to ponder, discuss, etc.

What I may be able to do is to understand some aspects of the personal learning process as it relates to the environment my students and I share and create tools that guide (as in guide on the side) specific aspects of cognitive activity within this general environment. Again, what happens outside of the classroom may be more much more important than what happens within the classroom.

One topic that interests me concerns note taking and note reviewing. How might these processes be improved. I have studies in which I have evaluated the costs and benefits of giving students lecture outlines and lecture summaries. These resources can improve the note taking and note studying processes. Outlines identify key ideas and the structure of presentations and offers a resource that can be used to take notes. Lecture summaries, prepared by “expert students”, can serve as a more complete record of the presentation than many students take themselves and can be used to identify misinterpretations. Technology makes it easy to offer these resources and to collect data on when and if students use the resources. I have authored several publications related to studies using these resources. In general, use of the resources is related to improved performance on lecture-related test items.

If students are skilled in the use of participatory web tools and motivated by opportunities to collaborate, it made sense to me that a wiki (actually multiple wikis available to 20-30 students) might offer a cost-effective and generative alternative to “expert notes”. Students could contribute what they know and collectively fill the gaps and correct the misrepresentations of their peers.

Here is the outline of my recent study. I loaded lecture outlines into multiple wikis before the lectures were given and had an expert note taker create a complete set of notes on one wiki (the group receiving complete notes was rotated so all students were given the same level of expert help). Data indicating both viewing of and contributing to the wikis were recorded. Student use was then related to examination scores. First, stident contribution to the wikis was very limited. Second, student use was not related to examination performance and on one of three exams those who used the wiki more frequently scored lower on the lecture exam questions. We had willing students complete a questionnaire attempting to understand these outcomes and discovered little. Those who had less comfort with technology used the wiki less, but few suggested this was a problem.

I am unable to explain these data. In past experiments I have consistently been able to find significant positive correlations between use of the prepared lecture outlines, complete notes and exam scores. These results cannot differentiate differences in student motivation and the unique benefits of using quality notes as alternative explanations, but the correlations have been consistently positive.

I write about these results here because “the system” makes in useless to submit these results to an academic journal. My message is that the data are important and advocates need to base comments more on such data and on a careful analysis of student activity rather than a personal logic or descriptions of a few cases. I do not think we are yet to the point we fully understand student use of participatory technologies or what should work in given situations.

Here is an opportunity to take advantage of the interest in the tight primary races. The Delegate Counter Game from CNN allows users to work through various scenarios that could possible play out in the remaining primaries. Predict and test different outcomes for different states.

The lead article in Time magazine a couple of weeks ago concerned issues of teacher quality. What are the characteristics of quality teachers? How is teaching performance evaluated and rewarded?

The characteristics of quality teachers were as follows (my interpretation):

• An expectation that students can learn.
• Deep knowledge of the subject area one teaches.
• Experience – probably 6-7 years to reach potential.

Issues associated with these characteristics are important to recognize. For example, keeping teachers in the profession is a serious problem. So many new teachers quit. They are replaced by other new teachers, many of whom quit. Even those who take a calloused view and assume there will always be more cheap labor on the way should be concerned that no matter how enthusiastic the new faces, there is a benefit to students achieved by working against the new teacher drop out problem.

The Time article reported some interesting data I had not encountered on subject matter knowledge (not pedagogical knowledge). Student achievement shows an increasing correlation with teacher subject matter knowledge as grade level increases. I guess this makes sense and the pre-service training models do recognize this distinction in typically requiring a content area major for secondary educators (some now also require a major for elementary ed majors). However, what ever the teacher’s area of training and mastery of that area, 30% of middle and secondary school educators are teaching outside of their area of training OR a closely related area of training. This figure is significantly higher for the physical sciences. It was reported that the issue of “misassignment” is one of the significant differences between the U.S. and countries with higher student achievement.

Original and continued commitment to content area knowledge is a significant issue. So – if you are a teacher, what book are you reading this week? Does anyone besides you care?