Tag: Research

Something concrete enough to discuss

I find that future educators often glaze over when I mention research. When faced with this reaction, I often propose that researchers face a challenge many “experts” on educational topics are not required to address. Researchers must be very concrete when it comes to the topics they study. I did say concrete. The notion that academics are abstract is a representation and when true applies to explanations they offer but not the techniques they use. Unlike other experts who can offer generalizations, researchers must conduct experi ents. They are doer and not talkers. They have to define their interests in terms of specific situations and actions. I describe this as “operationalization” which may not be a familiar term. It pretty much means researchers are required to explain how an hypothesis is turned into an investigation. They cannot hide behind vague terms – motivation, engagement, creative, etc.

The skills of critical thinking and literacy make a good example. Both sound like areas in which we want to encourage achievement, but what are such skills when it comes to a specific setting? My own interest is in these skills as applied in online learning. Here is an example of a study developed by a researcher focused on these skills. I encourage educators to consider the “Methods” section.

The great thing about the specificity of research is that the methodology offers something concrete to discuss and debate.

Loading

PBL Challenge

It seems we point to the findings of science selectively. Most folks I know profess amazement that some ignore the overwhelming scientific evidence on climate change or evolution. They are concerned when the “scientific perspective” is not the basis for what students experience if these topics are considered.

Why then is the assumption that our best evidence should not guide practice applied when selecting learning activities? You will have to trust me on this (unless someone really wants to review my reference list), but direct instruction consistently results in better academic performance than project based learning, problem based learning, discovery learning, etc. How do PBL advocates rationalize this reality? I seriously want to know because I find the PBL philosophy appealing as well. I just personally struggle with ignoring what research findings suggest.

I try arguing with myself seeking answers. I know many of the proposals. Direct instruction works when the dependent variable is simple, factual knowledge. Direct instruction turns learners off because it is boring. Direct instruction results in learning that fails when it comes to application or flexibility. However, whatever the counter argument, the position is only an hypothesis unless tested. Show me the data (or the money if you prefer). I am waiting to be convinced.

I am aware of what I consider successful PBL research. Success is possible. Here is what I think until shown otherwise. I am guessing that successful PBL takes far more skill to implement with classroom groups than direct instruction. Most PBL attempts probably do not meet an acceptable standard.  I know this sounds harsh, but what is the goal here? In general, I think many students are simply lost or overwhelmed when self directed. I do not think a substantial proportion of students are any more motivated by many PBL tasks. The outcome data simply do not support the argument that common implementations of PBL are as productive as more traditional methods.

So, at this point in my career, I do no longer have the opportunity to conduct research studies. I do have great interest in this topic and continue to search the journals for interesting studies. Learning experiences should not be promoted by talk or novelty.

Loading

Web content evaluation – data for a change

I sometimes complain that pundits and keynoters receive too much blog attention and researchers too little. Since the researchers I follow seldom seem to blog, perhaps I should post in support of their activity.

So much attention has been focused on the quality of online resources and the skills necessary to critically evaluate these resources as a literacy component of 21st Century functioning that one might think this area would have generated considerable research activity. There seem to be plenty of recommendations for practice, but little formal assessment of skill or of the success of interventions.

The recent AERJ article by Wiley and colleagues (citation at end of post) describes an interesting study I feel both evaluates the value of commonly suggested practices for evaluating web sites (e.g., identify the page author and possible motive for offering the information) in terms of whether students (college students in this case) learn to apply such skills and whether the development of such skills influence how students then go about completing an online inquiry task. I thought the procedure used in the study was creative – basically offer students a fabricated Google results page based on a given search phrase and have participants evaluate the various links. Social psychologists and other researchers often employ deception in their research. The research demonstrated that more specific guidance and a more active evaluation task resulted in improved performance on a second site evaluation task AND the use of higher quality information in an inquiry task.

This study needs to be replicated with younger learners.

BTW – the methodology is similar (evaluate a set of sites addressing a given topic) to that proposed on the Beck “Good, bad and ugly” site.

Wiley, J., Goldman, S.R., Graesser, A.C., Sanchez, C.A., Ash, I.K. & Hemmerich, J.A. (2009). Source evaluation, comprehension, and learning in Internet science inquiry tasks. American Educational Research Journal, 46, 1060-1106.

Powered by ScribeFire.

Loading

New Data In From Math & Reading Software Evaluation

A major study supported by NCLB has been evaluating the impact of math software in classrooms. I appear to be on a mailing list resulting in my receipt of project summaries (I specifically asked about this research last year). The data from the first year basically showed little benefit. Data from the second year of the study are now available (pdf of executive summary).

A couple of issues were being evaluated – did experience with the software mater, did the impact on achievement vary from product to product.

Regarding experience:

For sixth grade math, product effects on student test scores were statistically significantly lower (more negative) in the second year than in the first year, and for algebra I, effects on student test scores were statistically significantly higher in the second year than in the first year.

Regarding individual products:

One product had a positive and statistically significant effect.  Nine did not have statistically significant effects on test scores.

In 2007, I attended a detailed analyses of the first round of this research at AERA. I anticipate there will be a comparable report this year.

Loading

EdTech Research Agenda

The recently authorized National Center for Research in Advanced Information and Digital Techologies comes with a stated set of priorities:

• Research, development and demonstrations of learning technologies that could include simulations, games, virtual worlds, intelligent tutors, performance-based assessments, and innovative approaches to pedagogy that these tools can implement.
• Design and testing of components needed to build prototype systems. This could include tools for answering questions, for building and evaluating the construction of simulations and virtual worlds that could include sophisticated physical and biological systems or reconstructions of ancient cities brought to life with intelligent avatars (models of humans in virtual spaces).
• Research to determine how these new systems can best be used to build interest and expertise in learners of different ages and backgrounds. This will give educators, parents, employers, and learners the information they need to make informed choices.

It will be interesting to see what research funds are eventually made available and what realistic opportunities various institutions have to compete for these resources.

Andy Carvin offers additional information on this initiative.

Loading

You Still Must Think

Perhaps too much has been made of generational difference in the way technology has been used. And, perhaps there is a confusion between comfort level and productivity.

A research study just released concludes:

The first ever virtual longitudinal study carried out by the CIBER research team at University College London claims that, although young people demonstrate an apparent ease and familiarity with computers, they rely heavily on search engines, view rather than read and do not possess the critical and analytical skills to assess the information that they find on the web.

The group responsible for this study intends to track individuals longitudinally. The group also seems to contend that bad scholarship is like a disease – the rest of us are catching it from our students (I made the part about the disease model up – the group says no such thing, but they do observe that similar patterns are emerging across generations).

(pdf explaining the research and intent of the group is available to those in which the problem has not progressed past the point of no return – read while you still can) 😉

BTW- the details in the pdf are a little sparse. This is more of an issue piece, but the issues are interesting and linked to some research.

Loading

Communicating in Email

I came across a brief Wired post exploring the inability to communicate actual intent in email. The Wired articled referenced “recent” research published in the Journal of Personality and Social Psychology by Epley and Kruger.

The Wired article refers to Epley and Kruger in stating:

The researchers took 30 pairs of undergraduate students and gave each one a list of 20 statements about topics like campus food or the weather. Assuming either a serious or sarcastic tone, one member of each pair e-mailed the statements to his or her partner. The partners then guessed the intended tone and indicated how confident they were in their answers.

Evidently, those receiving the messages understood the tone at about chance level.

I think this and the explanation of the researchers (we are egocentric and know what we want to convey and assume that the message says that) is interesting and should be part of the message when talking with teachers about email. Evidently, we often lack the metacognitive ability to differentiate the meaning in our head from the meaning on the screen until we receive the reply indicating we have been misunderstood.

I wanted to read the original work and tried to locate the authors and theme in Google Scholar. The Wired article does not provide a reference.

I found Kruger, J., Epley, N., Parker, J. & Ng, A. (2005). Egocentrism over e-mail: We communicate as well as we think? JPSP, 89, 925-935. The article does deal with egocentrism and email, but the Wired piece leads with comments from the experiments and seems to imply newer work.

Blogged with Flock

Loading