Where are all the education data geeks?

This post is a final thought on the Moneyball/data series of blog posts in the last two weeks.

Among the numerous changes created by the ubiquitous collection of big data is the emergence of amateur data crunchers. Some of the most thought-provoking analysis of baseball statistics comes from people who are interested enough to crunch numbers on their own time. Some of these are young people who are hoping to discover something exciting enough to attract the attention of a professional baseball team. Others are hoping to be picked up by a sports website that may pay them. Even if they are ultimately aiming to be hired based on their work, they are spending lots of time without any promise of payment.

This dynamic is also playing out in other fields. When you have time to spare, check out A day in the life of a NYC taxi—which uses publicly available data to track the movement of cabs around the city, tracking when the cab is full and empty, when it starts and ends trips, and the fares. The creator of the site, Chris Whong, considers himself a civic hacker.  He says, “I’m always interested in some new approach to understanding the civic environment, understanding cities via technology. For me, that manifests itself usually in visualization and usually in playing with urban data. So I’m always looking for another juicy data set to find some nuggets of truth or some nuggets of information that are not otherwise available just by looking at the rows and columns.” Whong estimates that he spent about 50 hours of his own time on the project.

Other examples exist as well. Nate Silver’s 538 website often includes stories about similar data-crunching being done by individuals with little or no organizational backing, and no clear revenue motive. I also see examples on The Atlantic’s Citylab site. On these and other sites, and on topics ranging from sports to politics to urban design, fascinating and useful work is being done by self-described “data hackers.”

Does this same situation exist in education? I know of a couple of bloggers who do this type of work—but compared to other fields, it appears that unaffiliated education data crunchers are few.

If the answer is yes, others do exist, please let me know examples by email to (john (at) evergreenedgroup.com) or in the comments section below.

If the answer is no, then that’s another reason to make education data (scrubbed to protect student privacy) more widely available.









NJ online snow day rejection shows that policy still restricts online learning

The opening of this article captures the story of a school in New Jersey that moved to learning online during a day the school was closed, only to find that it couldn’t count the day.

“State education officials are viewing a New Jersey school district’s “virtual school day” as an innovative idea.  However, the day will not count toward the official 180 days of school due to state laws requiring facilities be available during the school day.”

District Superintendent P. Erik Gundersen was motivated to try the new approach to a snow day because the district had already used its planned three days, and would have to schedule another school day if it lost an additional day to weather. As reported by CNN,  “Gundersen alerted teachers that he expected to cancel classes and asked them to develop lessons students could complete from home…[when school was cancelled] students logged in on school-provided laptops, they were able to ask teachers questions, work through assignments or jump into class discussions, even if they sometimes took breaks to shovel the walkways.”

At the time, the district didn’t know if the state would allow the day to count toward the required 180 days, and subsequently the New Jersey Department of Education said no, despite some signs of success: “more than 96% of Pascack Valley Regional High School District students and all of the staff logged onto their district-issued laptops.”

How much learning took place on the snow day is unknown, as logging on isn’t a good proxy for learning activity. The state could reasonably argue that the school district has to demonstrate some level of learning to meet state requirements. But the reason that the state has denied the virtual snow day counting toward state requirements appears to have nothing to do with whether learning took place, and is instead due to a state policy that assumes that facilities must be open for learning to occur.

The specifics of this case are unclear, but it shows that it is difficult to be at the cutting edge of innovation in education, because laws often don’t allow for original approaches. There are few other fields in which innovators have as many regulatory and political minefields as they find in K-12 education.






Moneyball lessons for education (Part 3)

Earlier posts explored how the Oakland A’s baseball team moved from using advanced statistics to better understand players’ performance to the use of big data to further improve their understanding of players (link), and how a similar approach is possible and useful with the advent of digital technology in education (link).

The increased use of data in education holds promise, but a potential pitfall of an overemphasis on data is the inability of current data capture and analysis to effectively reflect 21st century skills. Often called the 4 C’s—critical thinking, communication, collaboration, and creativity—21st century skills are deemed “essential skills for success in today’s world” by the Partnership for 21st Century Skills, a partnership of Apple, Ford, National Board for Professional Teaching Standards, NEA, Pearson, and other similar organizations and companies.

Much of the educational software available today will tailor questions based on the student’s ability, allowing students to work within their zone of proximal development. The software is also designed to give students and educators constant feedback (data) on where individual students are struggling and succeeding. This data gives educators a tool that didn’t exist just a generation ago to personalize education for individual students, and is particularly useful in areas where there is one correct answer, as is often the case in math.

This is an important and useful development, but with limits. I am not aware of commercially available software that is capable of assessing critical thinking and creativity. With most educational software, there is often only one “correct” answer to a given problem. But, especially outside math and grammar, students who think creatively will derive and justify unique answers to some problems. Our most creative and critical thinkers will develop their own questions about the problems on which they are working. I’ve certainly never seen a software package that can assess the quality of questions that a student is asking or follow the creative thinking process to an alternate solution.

Educators should use the data that is available to them in such a way that allows them the opportunity to focus on fostering 21st century skills in their students. Technology exists that can replace routine tasks of the traditional classroom, but it doesn’t exist to promote 21st century skills in our students. Honing these skills within the framework of an integrated curriculum is where teachers in a modern classroom should focus their expertise.

Moneyball lessons for education (Part 2)

An earlier post explored how the Oakland A’s baseball team, and their general manager Billy Beane, have been using advanced statistics – and now big data – to better understand players’ performance.

A key quote from Beane’s recent Wall Street Journal article highlights two issues that are related to education, assessment, and evaluation:

“Having advanced performance data at even the most junior levels will make it less likely that players get filtered out based on 60-yard-dash times or radar-gun readings, and more likely that they advance on the merits of practiced skills. The ability to “paint the corners” of the strike zone, to swing only at pitches within that zone, and to manage the subtle footwork required of a difficult fielding play is accessible to any player willing to commit to the “10,000 Hour Rule” (the average amount of practice Malcolm Gladwell, in his book “Outliers,” says is needed to excel in selected fields). A whole new class of players whose skill sets previously were not fully appreciated will be able to reach the highest levels thanks to a more nuanced understanding of their abilities.”

The concept of evaluating players based on constant review of game performance, instead of one-time measures such as 60-yard dash times, is especially applicable to education in the digital learning era. Although students learn throughout the school year, for school accountability purposes most students are tested once a year (or less) in just a few subject areas. More frequent assessments are used to evaluate students’ grades and advancement, but the number is still small relative to what is possible, or more importantly, relative to what is potentially helpful for student learning. These coarse student assessments based on a few items are similar to evaluations of players with radar gun readings and 60-yard dashes rather than on finer abilities that are reviewed more regularly. As learning moves online, an ever-increasing amount of students’ work can be easily tracked and assessed, constantly and in real time. If a teacher and school can “watch” how well a student is answering 1,000 math problems throughout the entire school year, the need to administer a 20-question test every so often is greatly diminished—and perhaps the test is not necessary at all.

Some of the pushback against the use of data seems to be based on the important view held by parents and teachers that students are not numbers, and that they should not be taught based on ideas or recommendations formed from data gleaned from large numbers of students. The term “big data” suggests that each student is just one among many, and that students are not being taught based on their individual characteristics. It’s not uncommon in the stories about the use of data in education to hear a teacher or parent say “students aren’t just a set of numbers.”

But the use of data, when done well, is not at all about reducing students to numbers. Personalized assessment and feedback allow unique elements of each student to show through, helping them to avoid getting lost in the crowd. Beane notes how better information helps the A’s find players whose skills are more subtle than throwing a baseball 100 miles per hour. They can find the players whose “skill sets previously were not fully appreciated,” like pitchers who have impeccable control, or players who run the bases better than others.

In education, better, more personalized information allows teachers to identify students whose talents may not be “fully appreciated” by state assessments, and perhaps help those students to achieve the levels of literacy and numeracy necessary to allow their other talents to flourish. Data can also allow schools to challenge the students who are excelling with advanced courses, deeper learning, and real-world connections, to ensure that those students don’t become bored. More information doesn’t turn students into a cog in the machine, but instead allows each student to excel in his or her own way.

Another concern about big data in education is centered on critical thinking skills. This topic will be explored in our next post.

Moneyball lessons for education (Part 1)

Moneyball was a popular book (2003), and a subsequent movie (2011), that recounted the story of how the low-budget Oakland A’s baseball team, led by general manager Billy Beane, found a way to compete with teams with far higher budgets. Beane drafted young and inexpensive players who were good enough to contend with teams from far larger markets in New York, Boston, and elsewhere. In addition to being a great story, the key ideas also have implications for K-12 education and policy, which I’ll set up here and explore in more detail in the next post.

The book didn’t just tell a scrappy underdog story, although this theme made for an excellent movie arc. The A’s success wasn’t based on the luck of finding those underdogs who could compete. Instead, Beane identified baseball talents that were being undervalued by other teams. For example, all teams would pay a premium for players with high batting averages. But Beane realized the truth in “a walk is as good as a hit” and saw that teams were not paying very much for players who walked often but had low batting averages. He was able to get these players easily by drafting or trading for them, and he paid them less than the players with high batting averages. These guys were almost as good as—and sometimes better than—the players with high batting averages, but cost far less.

At the core, Beane was building a team by finding and exploiting market inefficiencies, much as the most successful investors such as Warren Buffett have done. But unlike Buffett, who maintains some structural advantages in his investing, Beane had created a problem.  By acquiring a certain type of player and growing them into high-value assets, he was signaling to every other team what sort of player was being undervalued. Therefore, his advantage in selecting players who walked often, or were particularly good at defense, was short lived because other teams saw his strategy and invested in this method too—which then drove up the price of those players.

Because Beane then needed a new strategy, the Oakland A’s subsequently became one of the leading teams in the use of advanced statistical analyses to predict players’ outcomes. What he and others found was that many of the traditional baseball statistics were accurate in describing what had happened but poor at predicting the future. For example, a pitcher’s Earned Run Average (ERA) counts how many earned runs he gives up per nine innings—which is a pretty good description of results. But it turns out that pitchers’ ERAs vary quite a bit from season to season due in part to luck. Instead, Beane and other baseball analysts found that other statistics such as strikeouts, walks, and the rate at which the pitcher gives up home runs were far better than ERA at predicting future results. Teams that used these and other complex metrics could avoid overpaying for pitchers who were likely to regress to a lower level of performance, and to find the bargain pitchers whose results were likely to improve.

Why am I describing all this on an education blog? First, when someone as successful as Billy Beane shifts direction, it’s worth watching what he thinks is important now and in the future.  In a recent article in the Wall Street Journal, Beane describes his latest direction: an increasing reliance on big data as central to the team’s competitive advantage. The public announcement of his strategy suggests that he believes that it’s not going to be easy—that the predictive power of big data has promise, but it is hard to get right.

Second, the way that most K-12 education treats data analysis and statistics is like pre-Moneyball baseball—at best. Education policy and practice simply have not kept up with the increasing focus on data collection and analysis that is now common to many fields.

Within these ideas lie several important lessons for education. The growing, sophisticated use of data is increasingly common, and is arguably the most important developing issue in many fields from medicine to agriculture. But using data well is not easy for anyone, including schools, and many current practices and policies are impeding the move to better analytics.

The next post will further explore how these ideas apply to education.


WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera