The question of exactly what conditions must be met before online participatory experiences result in increased rather than decreased insight continues to fascinate me. I am using the opportunity presented by teaching introductory topics in social psychology to explore topics in group information gathering and processing. Groupthink and selective bias were known issues long before bloggers attempted to educate themselves by reading the comments of others with similar views. Part of what I am doing is translating some of the research from the textbook into examples students may appreciate. I am not certain that I will discuss blogs and the bias in blog rolls, but I think the question of whether you prefer FoxNews or CNN and what is your political orientation is likely to make some sense. I am also thinking that using the “poll the audience” example from “Who wants to be a millionaire” will be more likely to connect than talking about wikis. Students are very familiar with wikipedia, but for them it exists as an online information rather than a participatory opportunity.
I collected data in my last class that I am preparing to present and discuss during our next meeting. James Surowieki begins the “Wisdom of Crowds” (at least the audio version) by telling a story about Sir Francis Galton. According to Surowieki’s account, Galton was somewhat of an elitist and believed that expertise should be trusted over the wishes of the group in making important decisions. However, an event at a country fair resulted in a somewhat different perspective. Galton observed an event in which a large of group offered predictions on the dressed weight of a live ox. Within the group were some (e.g., farmers) who might have unique knowledge in such matters and many others who would seem to know very little. After the event, Galton secured the data and conducted his own analysis. Galton, a pioneer in statistical techniques, discovered that the average guess of the group was remarkably close to the actual weight. The aggregate of the crowd’s knowledge, the input of experts combined with the input of the agriculturally uneducated, was superior to nearly every individual prediction (and the predictions of most of the experts).
So, I decided to see what I could do to replicate this demonstration. I was not exactly certain where I might locate an ox and to tell the truth I am not exactly sure what an ox is (we had cows back on the farm in Iowa). I decided to substitute a large box of Hot Tamales. I have no explanation for this choice and understand that the connection between the ox and a favorite candy possibly defies any known form of logic. Anyway, I offered a box of Hot Tamales to the student or students who came closest to the box I held up. Because others may not have my personal experiences, I also opened a second box to show the size of a hot tamale and passed the open box around for individuals to sample. Students submitted their estimates on signed slips of paper.
It turned out the box held 131 pieces of candy. The average estimate was 143 which was somewhere in the 60th percentile of all estimates (significantly above average). It looked like my little experiment had kind of worked, but I was hoping for something more spectacular. When examining the individual estimates I discovered one entry that predicted the box would contain 1000 pieces of candy. How could anyone seriously believe this could be true? Perhaps I had tapped into a type of learning disability – some failed form of basic numeracy. More likely, this situation may have involved a student who did not appreciate the seriousness of the task I had presented. Such things do happen in Intro Psych. Anyway, I discarded this one entry – statisticians might label it an outlier – and the results were magical. The group average was 131.7. The closest prediction was 135 (3 individuals).
I have some difficulty grasping exactly how this works. As I understand the explanation, any “estimate” consists of knowledge and error. As long as the error does not reflect bias it will be random and pretty much cancel itself out (as much error above the true value as below). I remember the concept of “true score” from my early statistical training, but I am not certain if this is the same thing. As you are reading the answers and encounter values from the 60s to 250s, it is difficult to imagine that the mean will be nearly right on.
So what might the Intro students know that would be valuable? I suppose most have useful notions of volume and numeracy. It turned out some had knowledge I did not anticipate. In passing the box around several noticed the nutritional information. This box contains 7 servings. Each serving consists of 20 pieces. Lucky for me, since I am purchasing a box for those making the best estimates, the manufacturer must only have to offer approximate information. Galton might have sided with the smug experts. Ha, no candy for you!
