What is that expression – The wheels of justice grind slow, but they grind exceedingly fine? It was the slow part of this expression that came to mind.
I am preparing for my instructional design class and I want to talk about what research tells us about instructional software. Reading research articles and reviews of research articles is often frustrating for education graduate students. The results often seem inconclusive and marginal making it difficult for those searching for clear guidance to reach any kind of conclusion.
Then it came to me – the Department of Education would provide the information I need. From some dark and seldom accessed region of my brain, I dredged up the recollection that the U.S. Department of Education in keeping with the requirements of NCLB that educational practice be based on “rigorous scientifically based research methods to provide evidence of effectiveness” had undertaken to “carry out a national study of the conditions and practices necessary for technology to be used effectively to improve teaching and learning.” This program was mandated by Congress in 2001 and the press release identifying the software to be evaluated and the companies to conduct the reseach is dated 2004.
“Why bother with the journals I normally read?” I thought. No reason to consider the findings of those university professor types, why not rely on the research sponsored by the government?
Do you remember this topic? Companies were worried their software would not be considered (only a few commercial programs were accepted for evaluation) and were worried that only “approved” software that demonstrated achievement gains could then be purchased with federal funds (the list of software selected is included in the site I link to above).
The Feds would certainly be in a unique position to do certain types of research. Major studies are very expensive – to assure cooperation schools and teachers will likely be compensated to make certain the methodological definition of the control and treatment conditions are maintained, supervision to make certain compliance with such assignments will also be required, there will likely be the need to purchase content specific evaluation instruments, etc.
After wading through the literature and the typical criticisms, I do want to read the methodology of these well funded studies. What will serve as a dependent variables? Will there be multiple dependent variables to assure that a range of outcomes and possible benefits and liabilities be evaluated? What will be considered a “control group”? etc.
Evidently, government funded initiatives do not operate under the “publish or perish” directive. I have been looking, but I cannot find anything that would indicate what has been accomplished to this point. I can find online resports announcing that a company’s software has been selected for the research and I can find comments on the government initiative and discussions of what qualifies as quality research. What I can’t seem to locate are SRI or Mathematica (the companies selected to perform the research) reports. I will keep trying and if something surfaces I will add it here.
A related project, the What Works Clearinghouse, provides what I think is a useful tool for those interested in educational research. Among other services, the site lists various research studies and identifies which meet and which do not meet “quality” research standards. It appears that middle school math is the only content area with reported studies at this time.
