This post is a final thought on the Moneyball/data series of blog posts in the last two weeks.
Among the numerous changes created by the ubiquitous collection of big data is the emergence of amateur data crunchers. Some of the most thought-provoking analysis of baseball statistics comes from people who are interested enough to crunch numbers on their own time. Some of these are young people who are hoping to discover something exciting enough to attract the attention of a professional baseball team. Others are hoping to be picked up by a sports website that may pay them. Even if they are ultimately aiming to be hired based on their work, they are spending lots of time without any promise of payment.
This dynamic is also playing out in other fields. When you have time to spare, check out A day in the life of a NYC taxi—which uses publicly available data to track the movement of cabs around the city, tracking when the cab is full and empty, when it starts and ends trips, and the fares. The creator of the site, Chris Whong, considers himself a civic hacker. He says, “I’m always interested in some new approach to understanding the civic environment, understanding cities via technology. For me, that manifests itself usually in visualization and usually in playing with urban data. So I’m always looking for another juicy data set to find some nuggets of truth or some nuggets of information that are not otherwise available just by looking at the rows and columns.” Whong estimates that he spent about 50 hours of his own time on the project.
Other examples exist as well. Nate Silver’s 538 website often includes stories about similar data-crunching being done by individuals with little or no organizational backing, and no clear revenue motive. I also see examples on The Atlantic’s Citylab site. On these and other sites, and on topics ranging from sports to politics to urban design, fascinating and useful work is being done by self-described “data hackers.”
Does this same situation exist in education? I know of a couple of bloggers who do this type of work—but compared to other fields, it appears that unaffiliated education data crunchers are few.
If the answer is yes, others do exist, please let me know examples by email to (john (at) evergreenedgroup.com) or in the comments section below.
If the answer is no, then that’s another reason to make education data (scrubbed to protect student privacy) more widely available.