Month: May 2017

Value added evaluations

The expectations of education are constantly changing. While not all parties agree, the purpose of education includes more than the development of vocational skills or at least assumes that the skills required for present employment will not be sufficient to the vocational skills needed in the future.

One popularized approach to describing such expectations is to describe them collectively as 21st-century skills. The skills included are open to opinion, but cognitive skills such as communication effectiveness, problem-solving, critical thinking, and creativity are typically included. Such goals are not new, but the emphases in moving beyond accumulated knowledge and more task-specific performance skills have increased.

One important characteristic of such skills is that responsibility for development extends beyond a single course or program. I assume this stems in part from the lack of certainty in how such skills will be applied. Flexibility in application seems a natural requirement to meet such uncertainty.

The phrase “value added” is sometimes used to describe growth in a skill over time. Some may be familiar with the concept as applied to teacher evaluation. The notion in that context suggests that instead of comparing student performance at some end point, gain scores would be a better approach to evaluating how successful a teacher has been in working with these students. When used to evaluate the skills of an educator, the method has some intuitive appeal but is not without limitations to clean interpretation. When the concept is applied to the impact of education on any of the 21st-century skills, the approach assumes researchers can measure the level of such skills and one should be able to show improved functioning with educational experiences.

I became interested in the topic after reading the book Academically Adrift. This book described a longitudinal study of college students focused on the development of critical thinking. Using an instrument developed to assess critical thinking, the book came to what seemed to be the shocking conclusion that gains in critical thinking were minimal and pretty lacking with a sizeable proportion of college students.

A similar study has now been released assessing the gains in writing skills (prepublication release from Oppenheimer, at al.). Again, this study used a longitudinal approach with repeated measures of carefully evaluated writing samples. The study found that women and those enrolled in humanities and social science majors were better writers, but did not show greater improvement. College students did reliably improve in writing skill (7%). The researchers make no further comment beyond noting that this improvement could be reliably documented.

These studies are important, but difficult to implement. The skills themselves are difficult to assess and the duration of such studies requires a commitment often to appreciated by those evaluating the significance of research. An examination of the effort required should be noted for those assuming value added demonstrations are a simple matter. I encourage a reading of the methodology from the writing study for those wanting to learn more about what effort is required.

Loading

Degree of bias vs falsehoods

Some months ago I began cross-posting some of the things I write elsewhere to Facebook. I was interested in exploring the potential of social media as an opportunity for argumentation. I suppose this word (argumentation) might be foreign to many, but you might want to translate it as debate. This is essentially the notion that discussions can explore issues through processes that involve assertions, supporting assertions with facts/data, and taking on the assertions made by others through the same processes of counter-assertations and facts/data. I have long been a promoter of social media as a way to involve the public in important issues and thought rather than write for the audience I have I should take a more open approach.
 
I must say I have not been particularly impressed with what I have observed in Facebook. Few participants actually participate. There is mostly the linking of things folks have found online with little personal reflection or commentary. There tends to be little interaction and far less commentary at a deep level. I am not certain how to change this. I understanding that actually writing takes a good deal of effort, but without some effort I think little is accomplished.
 
Another issue I have noticed is the type of content people are willing to post. Again, I come from a tradition of argumentation based on sources (mostly in the scientific literature). I understand that many political issues are not based on formal sources, but so much of the content I see comes from sources of such poor reputation. It seems that some seek extreme examples and about the only way to do this is to promote content from extreme and sketchy sources.
 
My wife and I have discussed this issue often because of our common background in educational technology. She recommended that I recommend a site for evaluating the credibility of sources. Try this site (and read about their approach if you are concerned). There is a textbox at the top of the page that returns an evaluation of news sources – both bias and factual accuracy.
 
Bias and factual focus can be different things. Bias for some might be described as spin – how one tries to explain facts. Factual focus is really whether facts are even the basis for the content. I would interpret the notion of fake news as news that is not factually based. The site I recommend will indicate if a “news” source has promoted positions that are known be false.
 
Try sites you know and assume promote a spin – CNN, MSNBC, NPR (use National Public Radio), Wall Street Journal. You will note that these sites are all “centrist” with a bias (left, right). All are typically fact based. If you can, try some of the sources for the Facebook posts that seem extreme (if you can identify the source). My experience has been that such content is described as biased to a far greater degree than the sources I list above, but also rated as far more likely to promote false narratives. This is the type of information Facebook should add to any “sourced” content.
 
Facebook talks a good game about the promotion of false narratives but seems to be overly careful about taking practical actions that would allow authors and readers to acknowledge the credibility of their sources. You can do this for yourself.
 

Loading

Layering becomes controversial

I have written a lot lately about an instructional tactic I call layering (search this blog for the layer tag). What I mean by layering is the use of a service that allows an educator or designer to add elements such as highlights, questions or annotations on top of an existing website or online video. Just to be clear because this issue bears on the following comments, the way this work is that the designer enters the address of the content to be commented to a different service that requests the content from the original provider, allows the educator/designer or student to make additions on top of the original content, and saves a way to represent this combination. There is no actual change to the original content, the original content is served each time the composite is requested, all content including ads is served, and the composite is only visible to viewers requesting the content through the layering service.

My writing has focused on the educational potential of such services. The idea is that much of the content existing on the Internet was not designed as an instructional product. Educators/designers can add elements that help learners process this content for understanding and retention. In addition, it can be beneficial for learners to make their own additions (much as you might do while reading a book) for personal use or sharing with peers.

It is important to recognize that the developers of these layering services may not have these exact goals in mind or they may see their services in a broader way. I tend to think in terms of educators and students. The developers of these services may have a perspective closer to “annotate the web”. The idea is each of us can markup public documents as a way to provide input or challenge to the positions taken by original authors.

What has surprised me is that there is a growing backlash against these layering services. I can kind of understand the position of those who are concerned because it seems similar to the anger present in those who resent viewers blocking the ads inserted in content. I can understand the position – you do not have to view the content I created so please view it as I intended.

Another way to understand this backlash is to note that many bloggers or YouTube creators may turn off comments. Some may not want to be criticized, but many have experienced attacks or negative comments that go well beyond just disagreeing with positions taken.

I am listing two sources explaining the opposition to group annotation. One from the Chronicle of Higher Education and one from blogger Audrey Watters. I always recommend that my readers use the links I provide in case my summary of the positions taken by others are simplistic or simply wrong.

There is something kind of interesting about the way we write and link in addressing this specific situation. I see the educational value in layering and in a way I am suggesting that layering should be continued. In a way, I am doing something very similar to what layering accomplishes although in a less precise way. I am taking the work of another and arguing that the position taken fails to see another perspective. I would not have been allowed to do this as an annotation or as a comment, but I am doing pretty much the same thing. I also remember in the early days of the web that some objected to others linking to their content.

Watters blog

Chronical of Higher Ed article

I hope there is a solution to this dilemma. It makes no sense to me that the connection of ideas on the web would require the consent of all involved. This is not the way the web has evolved and it is not the way those of us who have worked in academia have always done our work. We cite and connect, but we do not seek permission. Audrey Watters mentions a script she uses to block a specific annotation service (hypothes.is). If this type of thing were widely available, it might be one solution.

I do see public annotation (when connected to the same layering service) and educational layering (offering the composite to specific students) as different. Like so many tools for working online, the same tool can be used in both ways and I guess flexibility can lead to problems.

Loading

Reasonable student privacy

 

I came across this report from the EFF (Electronic Frontier Foundation) claiming that companies with an online presence are “spying” on children. Google was singled out. This caught my attention as I have been a fan of both the EFF and Google so I have been trying to understand just what is going on.

Spying is a strange choice of words, but I assume it was selected because it attracts attention. The accusation concerns the collection of user data without appropriate disclosure. This is a touchy subject for me. I have been upset that the FCC has eliminated expectations that ISPs not collect and sell user data so it was of great concern that Google be accused of doing something similar within its program for K12 schools. This should not be happening.

Close reading of the EFF announcement did not provide the detail I needed. The EFF “spying” piece reported data collected from educators, parents, and students establishing according to EFF that there was a lack of awareness or acceptance that limited possible safe guards. For example, if parents were aware of issues and wanted their children not be involved this was difficult because it would make more work for administrators and educators wanting to have all members of a class involved in the same project. These issues make some sense, but the issue is not really Google’s fault.

There were still vague references in the recent EFF statement that seemed to place blame on the companies providing services. I eventually located a better source. Several years ago EFF filed a complaint with the FTC (Federal Trade Commission) identifying the ways in which Google had violated the provisions of the Student Privacy Pledge. The privacy pledge is a set of provisions a panel established and offered to online companies as a way to document commitment to the provisions. The EFF claims this commitment establishes a legal commitment.

I have excerpted and included below the three major violations the EFF claims in the complaint to the FTC.

Google is violating the Student Privacy Pledge in three ways.

First, when students are logged in to their Google for Education accounts, student personal information in the form of data about their use of non-educational Google services is collected, maintained, and used by Google for its own benefit, unrelated to authorized educational or school purposes.

Second, the “Chrome Sync” feature of Google’s Chrome browser is turned on by default on all Google Chromebook laptops –including those sold to schools as part of Google for Education –thereby enabling Google to collect and use students’ entire browsing history and other data for its own benefit, unrelated to authorized educational or school purposes.

And third, Google for Education’s Administrative settings, which enable a school administrator to control settings for all program Chromebooks, allow administrators to choose settings that share student personal information with Google and third-party websites in violation of the Student Privacy Pledge.

My interpretation and comments
Google does not differentiate what students do as school work and what they might do for personal reasons while connected to their accounts at school or elsewhere. This means Google collects data on:
  • browsing behavior on every single Google-operated site students use,
  • stores what students have searched for on the Internet and the results they click on, and
  • the videos they search for and watch on YouTube.

Even when Google aggregates and anonymizes the student personal information it collects, as the company does for Google for Education core services, Google still uses the data for its own benefit, unrelated to authorized educational or school purposes.

If chrome-synch is enabled, Google collects a student’s entire browsing history and chrome-synch comes enabled on all chrome books or may be enabled by school personnel.

Students could also use other equipment outside of the school setting to complete course assignments and the settings associated with this equipment may allow student data to be collected.

I have tried to summarize and convey concerns accurately, but I do encourage you to read the legal complaint yourself.

If I understand the concerns I am left wondering:

  • What is reasonable to expect, and
  • What do we expect for free?

I suppose Google could add some type of permission priorities that would allow the system to use parent permissions to trump any priorities set by the school. First, there would be the issue of getting parents to input their wishes. This would not be easy. There is also the what do you expect for free issue. What should users expect of Google to safe guard against educators over-riding parental wishes should there be a purposeful or accidental contradiction in what educators and parents want. I must admit to some sympathy for educators who have a couple students whose parents do not agree with assignments for the rest of the class.

The issue of whether the chrome-synch default is set to off or on for Chromebooks could be something Google could address. I am guessing Google does not control this as devices as chromebooks are mainly now provided by other companies and these companies do not sell only to educational institutions. The chromebook providers assume certain benefits of having synch engaged and do not want to explain the advantages and disadvantages to all users many of whom are looking for a simple device and do not want to mess with settings. This would seem to imply that checking the settings would be pushed to schools and there is then the EFF concern that schools may not make the necessary adjustment  (or may even switch the synch feature on for chrome on other devices).

Finally, there is the issue of what students might do on their own at school or at home. Yes, students might do things they should not or they might fail to take precautions that have been recommended to them and their parents. Mixing work and personal interests is certainly something I do all of the time. Expecting Google to somehow control for such behavior seems unrealistic. The risk could be diminished by using only school provided equipment under school supervision, but this seems both unrealistic and contrary to hoping students will work in multiple settings. If EFF wanted to take a strict position on this issue, they would come out against BYOD programs (bring your own device) which would make the issues described here even more difficult for Google and schools.

I am a fan of the EFF. I think identifying some of these concerns is worth doing. However, expecting Google to remedy these issues is unrealistic for multiple reasons some of which would be very difficult to address issues assuming the lack of cooperation from schools, parents, and students. If EFF had discovered that Google was selling student data to third parties, I would be concerned. Google would then be using its free services in a way that clearly was contrary to what it had promised. If parents and educators are concerned about students viewing ads on sites not controlled by Goggle, I would propose that the solution is to stick with Google apps.

Loading