Those of us who train undergraduate psychology major give several reasons for teaching all majors about research methodology. Most majors will never do research. We know this and argue that learning the research process is a good way to develop critical thinking skills. We suggest that in thinking carefully about the methodology used in research it is sometimes possible to offer a different interpretation of the outcome than that given by the researcher. This is one reason researches are required to carefully describe their methodology and to outline their results before offering an interpretation for what was observed.
It is so easy to get taken in by the story told by authors even when very possible alternatives are there for the taking. I use the following example in my undergraduate educational psychology class.
Imagine a researcher hypothesizes that the optimism and generally positive attitude of teachers represent an important factor in motivating students. One way to test this proposal might be to observe classroom behavior and count the frequency of smiling behavior (I call this variable smiles per hour – SPH) and then determine if variability in SPH correlates with the average class achievement level. Assume that such research demonstrates a positive relationship. Would administrators then be justified in attempting to use optimism/positiveness (personally observed or commented on in letters of references) as an important marker in hiring decisions? Should we be looking for teacher characteristics associated with higher levels of student performance?
It makes a good story and I can usually convince the class that this would make sense. I then ask one question that immediately brings to light a very different way of understanding the story. Imagine you are a teacher and for whatever reason you are working with a class of very difficult students. They have bad attitudes, are terrible to each other, and in general just do not care about learning. How much fun will you be having this year?
Those who write books as advocates for positions are basically story tellers. They martial what evidence they can to support the story they tell. Hopefully, they have thought carefully about the arguments they make and are reporting a complete description of the data available. Still, the job they take on is to tell a convincing story. It is your job as a reader to critically evaluate what you have been told.
Consider the Bauerlein story line (Dumbest Generatation) at least as I interpret it. Bauerlein essentially argues that tweens and adolescents have been taken in by the potential of the participatory web and have to some extent isolated themselves from adult influences because of the desire of adolescents to socialize with peers and explore topics of interest to adolescents. In other words, participatory technologies are both enabling and motivating, but the focus for adolescents is not on issues or skills of substance. Worse yet, such experiences encourage adolescents to be less tolerant of traditional school experiences because they seem dull in comparison. Juxtaposed with this observation of adolescent behavior the author describes international differences in academic achievement noting that despite advantages in wealth students from the U.S. do not score well. The argument assumes rejection of adult values and experiences and a self-indulgent focus on peers and peer values results in lower achievement outcomes.
It makes a convincing story. However, sometimes when interpreting nonmanipulative research or stories, it is a good exercise to consider other models of how variables might be related.
There is always it is not A causes B, but B causes A. What would this mean here – perhaps nonrewarding or nonstimulating learning experiences causes students to seek out stimulating experiences elsewhere.
There is also the third variable argument – perhaps C influences both A and B resulting in the impression that A is the cause of B. How about school time as a possible C. Differences in the time devoted to education in other countries might impact levels of achievement. In those countries of means spending less time in school settings, learners might have more time to explore online experiences they find interesting.
So, pick your own story line here. There are multiple ways to tie observations together. However, note that meaningful intervention requires that we pick the story line describing causality for the most students.
