Does online learning research have to be new to be useful?

A new report from the National Education Policy Center reviews the Harvard study of Florida Virtual School outcomes that we discussed in an earlier blog post. The press release headline summarizes its conclusion: “Online Education Report Offers Little New or Useful.

I found the headline to be as interesting as the rest of the report, because the headline reflects a flaw in the way that some researchers think about the impact of research on policy and practice.

Academic researchers base their careers on creating and publishing new knowledge. With few exceptions, study findings must be new to be valuable. Research begins with an exhaustive literature review, to ensure that new studies build on and do not replicate existing knowledge, except in the cases where a study is explicitly meant to test existing data or ideas. Indeed, research that attempts to confirm or refute existing studies tends to get much less attention than the original research, even if the new study rebuts previous findings.

Most of the personal and professional accolades that researchers strive for are based on finding new knowledge. Nobody in research-based institutions (e.g. R1 universities) builds a career on publicizing or confirming existing knowledge. Indeed, findings that may be presented in a new way or to a new audience are often dismissed as not being “new,” even if the original knowledge may have been known to only a tiny sliver of academics and few others. Hence the assumption behind the headlines: the findings aren’t new, and therefore they are not useful.

But this view is research-centric and does not reflect the way that much of policy and practice operates. Policymakers and practitioners tend to hear about new studies because of media reports (directly or via colleagues, re-posts, and similar), not because they are perusing academic journals. Media outlets, however, rarely run a story about a study that is several years old, even if the findings still hold true. And in my experience most people have a bias towards information that has been published recently. A study from 2014 is valued more highly than one from 2009, and in a rapidly changing field such as digital learning the period for which studies are considered relevant tends to be short. Therefore a study that explores the outcomes and benefits of online courses is valuable to policymakers and practitioners, whether or not it is entirely new.

The concluding sentence of the report summary states “Given the limitations of research such as this new study, researchers have moved beyond simply investigating whether one medium is better than the other and begun—and need to continue—investigating under what conditions K-12 online and blended learning can be effectively designed, delivered, and supported.” I agree with that statement—but neither the statement nor the NEPC report refute the value of the Harvard study for policy and practice.

Fast innovation versus slow ideas

The Evergreen Education Group has been studying online and blended learning for 14 years, and the Keeping Pace report that will be released in two weeks is our 11th annual edition. As we consider the national digital learning landscape as of late 2014, we believe that key questions about digital learning include:

  • How fast is digital learning spreading across K-12 education?
  • Is digital learning creating transformational change, incremental improvements, or little/no change?
  • Should we expect to see broad, scalable, measurable improvements in the near future?

These questions can be examined from at least two angles. Researchers look at these questions from a position that should be as unbiased as possible, ascertaining what is occurring regardless of what they believe would be better for students, and leaving aside questions about how implementation could be better or faster. Advocates look at these questions with their advocacy in mind. Ultimately they want to know what is happening so that they can better shape the future, and ensure that broad, scalable, measurable change will occur.

Despite these differences, both sides benefit from creating a theory of action. If digital learning is changing the educational landscape, how is it doing so? If it is not, despite some demonstrated successes, why not?

A New Yorker article from 2013, Slow Ideas: Some innovations spread fast. How do you speed the ones that don’t? uses changes in medical practice to explore some of these questions in ways that hold lessons for education. It looks at two innovations from the late 1800s: anesthesia and new methods of protecting against infection. The use of anesthesia spread very quickly, while the use of anti-infection procedures did not. Why? The author, Atul Gawande, suggests reasons that have implications for the spread of digital learning.

In summary, he believes that anesthesia spread because its use was relatively simple, and its positive impact immediate (surgery went from requiring the restraining of patients screaming in pain to operating on motionless, apparently comfortable patients). Anti-infection efforts, in contrast, required multiple steps (disinfecting tools, changing clothes, and more) that involved numerous people who had to change their behaviors, and the results were not immediately obvious, because the infection would show up later. In addition, infection rates would only be known when the anti-infection efforts could be studied and compared to traditional practices.

Observers of digital learning can debate which types of digital learning resemble anesthesia, and which resemble anti-infection efforts. In my view, much of online learning (fully online courses and schools) is similar to anesthesia, while much of blended learning (especially as applied in existing schools) resembles anti-infection efforts.

Gawande doesn’t just describe these differences, however. He also explores implications for those who are trying to spread innovations. Ultimately, he argues that the solutions that are considered “scalable” often do not work.

“In the era of the iPhone, Facebook, and Twitter, we’ve become enamored of ideas that spread as effortlessly as ether. We want frictionless, “turnkey” solutions to the major difficulties of the world—hunger, disease, poverty. We prefer instructional videos to teachers, drones to troops, incentives to institutions. People and institutions can feel messy and anachronistic. They introduce, as the engineers put it, uncontrolled variability.

But technology and incentive programs are not enough. “Diffusion is essentially a social process through which people talking to people spread an innovation,” wrote Everett Rogers, the great scholar of how new ideas are communicated and spread. Mass media can introduce a new idea to people. But, Rogers showed, people follow the lead of other people they know and trust when they decide whether to take it up. Every change requires effort, and the decision to make that effort is a social process.”

Although Gawande is not describing education and the spread of digital learning, his words describe the challenges of spreading educational innovations. This is particularly true as innovations spread from the early adopting teachers, who are willing to find digital content and tools on their own time, to the large majority of teachers and schools. A few articles and training videos, and even “train the trainer” methods, may not be enough to create the level of change that advocates seek. In Gawande’s opinion, methods are required that are less scalable, and more expensive—but ultimately will create the change that is sought.

Parallels between jury duty and high school

Last week I read a blog post that resonated with me and many others, and then unexpectedly had an experience that brought home the author’s point in a memorable way.

The blog post, titled “A veteran teacher turned coach shadows 2 students for 2 days – a sobering lesson learned,” describes how an experienced teacher at a private high school shadowed two students for a day each, to better understand the high school experience from a student perspective. Her key findings include:

1. “Students sit all day, and sitting is exhausting… students move almost never. And never is exhausting. In every class for four long blocks, the expectation was for us to come in, take our seats, and sit down for the duration of the time. By the end of the day, I could not stop yawning and I was desperate to move or stretch.”

2. “High School students are sitting passively and listening during approximately 90% of their classes…It was not just the sitting that was draining but that so much of the day was spent absorbing information but not often grappling with it.”

When I read the post I thought—yes, that sounds like my high school experience. I suppose not much has changed in some schools in the decades since I was a high school student. I also compared the experience being described to some of the blended schools that I have visited, in which students are much more engaged because they are actively learning in a variety of ways that limit the amount of time in which the teacher is lecturing to the entire class.

The next day I began a three-day experience that brought home just how stultifying a day of simply listening can be, and how much more engaging the alternatives can be, when I was called for jury duty.

The first morning of jury duty was somewhat absorbing, as the judge and attorneys asked jurors to reflect on their life experiences (so that they could determine who would be struck from the jury, with or without cause). We weren’t just listening, because we had to be ready to answer a question to the entire jury, or to answer a possible cold-call question.

Then we moved into two days of opening statements, witness testimony, and closing arguments—and all we did was listen. We were barred from having any discussion of the case with anyone on or off the jury. We were not allowed to look up any information about the case, press reports, or legal terms. Our time was controlled by the judge, who told us when to arrive, when we would take breaks, and when we would be done for the day.

The case was compelling. It included felony charges related to a domestic violence incident that involved three Mexican nationals, so in addition to the seriousness of the charges for the defendant, it included implications for US citizenship and/or deportation for at least three people.

And yet…by the second half of the first afternoon, all of us on the jury found ourselves struggling to concentrate, and to track all that was being described, charged, and defended. It wasn’t for lack of trying, as we all recognized the seriousness of the case. But our ability to ask questions was severely limited, and we didn’t have the time or the option to grapple with the information. It was a tidal wave of testimony that overwhelmed us, and left us exhausted at the end of the days.

The closing of the testimony, and beginning of deliberations, immediately changed the demeanor of the jurors. Now we were in (partial) control, and everyone on the jury dug into the discussion. We wrestled with legal definitions, assessed credibility of witnesses providing different accounts, and weighed contradictory evidence. It was hard work, and on that third day we kept at it for 13 hours, ending near 9:30pm with a verdict.

Part of the reason for the dedication, I’m sure, is that everyone recognized the ramifications of the charges for all involved. But I was also struck by the change in the group as we went from at first being forced into passive listening, and then subsequently into actively working with the details of the case. We examined evidence, shared notes, traded ideas, and argued the various points. Nobody ended the deliberations with the same set of views that they had started with.

The portion of the trial that resembled the blogger’s description of her high school from a student’s perspective was tiring, deadening, and stultifying. The portion of the trial that looked like a good blended school was hard, lively, and engaging.

Examples of success in blended learning: survey deadline extended

We have received a strong response to our call for examples of blended learning success. We have also heard from quite a few schools requesting more time, so we have extended the deadline to the end of the day (midnight PT) on October 26.

The survey is available at Most surveys are taking between 15 and 20 minutes to complete.

As we explained in our previous blog post:

We believe that proof points will help practitioners who need to demonstrate to a variety of stakeholders that blended learning can be successful in a setting that the stakeholders are familiar with. We intend therefore to publish case studies that collectively will cover a variety of elements including different geographic areas, school/district sizes, and urban/suburban/rural characteristics.

Specifically, we are seeking:

  • Examples from regular traditional public schools and districts, not including specialized schools or charter schools.
  • Blended learning implementations that can demonstrate improvements in outcomes based on student achievement as determined by assessments, course grades, or other measures.
  • A range of implementation types, geographic areas, student populations, grade levels, and subject areas.

We will review all submissions, and through evaluations of the survey responses and follow-up interviews, determine the best examples that represent an assortment of blended learning successes. Case studies will be developed based on these examples. We will invite the schools that are selected to be featured in the case studies to co-present with us at the November 2015 iNACOL Blended and Online Learning Symposium. Evergreen and Christensen will pay a portion of expenses to defer travel costs.

KP 2014: Growth in students in fully online schools appears to be slowing

We have completed writing and editing of Keeping Pace 2014, and will be releasing the report at the iNACOL Blended and Online Learning Symposium in Palm Springs during the first week of November. In the next few weeks we will highlight some key findings here on the blog.

Every year in Keeping Pace, we look at the status of online schools that enroll students from across multiple districts in a state, and may attract students from across the entire state. We count the number of states that allow these types of schools, and the number of students attending these schools.

Our count for KP 2014 shows that thirty states have fully online schools enrolling students from across the entire state*. For school year (SY) 2013–14, we estimate that about 316,000 students attended these statewide fully online schools, a year over year increase of 6%.

Many of these schools are charter schools, often authorized by a state-level authorizer. Some of the online schools are run by districts that attract students from other districts across the state. Pennsylvania, Ohio, and Arizona are the states with the most fully online schools and students in absolute numbers. No state has more than about 3% of its students attending fully online schools, and states with relatively high percentages of students in such schools (between 1.5% and 3%), but lower absolute numbers, include Colorado, Nevada, and Idaho.

Why is growth slowing? Several factors appear to be in play. First, fewer states are allowing these schools to open for the first time, or often are doing so with restrictions (e.g., limits on the number of online schools that may open, or the total number of students who may attend them). We counted 30 states allowing these types of schools in 2011, and while some states are now allowing fully online schools that didn’t three years ago (e.g. Maine and New Mexico), other states that used to have a small number of fully online schools no longer do (e.g. Missouri and Virginia).

Second, in most states that have had online schools for a decade or more, and have the largest online student populations, growth in the number of students attending these schools is slowing.

It’s possible that some of the growth is shifting from statewide online schools, for which enrollment numbers are found relatively easily, to single district online schools, which are very difficult to track. In addition, students may be moving to single district blended schools that have some required onsite component, instead of attending a school that is entirely online.

* Technically, California does not allow any statewide online schools, because of its requirement that schools enroll students only from within contiguous counties. However, for simplicity we count California as among the states with online schools statewide, because at least one fully online school is available to all students in the state.

WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera