A shift toward efficacy

I recently had time to peruse Pearson’s report The Incomplete Guide to Delivering Learning Outcomes, which “outlines Pearson’s own efficacy programme and shares the company’s strategy and initiatives in its first phase.”

Although the report was published in 2013 and has received some attention, the depth at which Pearson explores efficacy and examines how the concept applies to its products is notable both for the company, and for its applications to education and policy more broadly. For example, the title of the introduction—From Inputs to Outcomes—is almost exactly the same as the title of a report that we did with iNACOL a couple of years ago (Measuring Quality From Inputs to Outcomes), suggesting that these are issues that apply to schools, policymakers, and many stakeholders in addition to companies.

The report has depth and makes many points that are worth consideration, starting with its definition of efficacy in the education context as requiring “a measurable impact on improving people’s lives through learning.” As the dictionary definition of efficacy is “the power to produce an effect,” Pearson’s addition (“improving people’s lives”) is not a trivial distinction. The company is going further than the usual definition and saying the effect must be a measurable improvement on a learner’s life. The fact that this definition of efficacy requires a measurement of an outcome, and that the outcome must be related to the learner’s life and not simply be any grade or test score, is critical. Some test scores, such as SATs, are likely to have an impact on the student’s life, while others, such as a state assessment, probably will not.

The report goes on to say that Pearson is beginning to evaluate all of its products and potential acquisitions with efficacy as a chief measure. If the largest education company in the world does this successfully, it may have an effect throughout many of the areas in education (e.g. content, technology systems) that are provided by vendors. And if that happens, perhaps the same standard could be applied in other sectors as well. For example, a key tenet of charter school proponents has been that successful charter schools should grow, and unsuccessful ones should be shuttered. But it has proven difficult to close unsuccessful charter schools, so a key component of the promise of charters has not been realized.

It may be easy to overlook just how profound an impact a true focus on efficacy across educational organizations could be. In a previous career I spent many years as an environmental consultant, working with several national non-profit organizations based in Washington, DC. One was a well-funded think tank that published influential reports on a variety of topics, convened meetings and working groups with policymakers and companies, and generally worked within the system to create changes to improve environmental performance. After I had worked with this organization for a couple of years, the board and executive management decreed that all future projects would be evaluated based on the change that they would produce in the world. The organization explicitly defined change as a measurable outcome, and stated that a report or a meeting was not a “change in the world.” The result over time was a more effective organization focused on real-world outcomes. Reports were still published, and meetings convened, but in all cases a clear connection to a further outcome was required.

Based on that experience years ago, and others since, I appreciate the move that Pearson is making to focus on efficacy. I also believe that every school that is moving to digital learning should be asking a similar set of questions: what change is this shift going to produce, how will we measure it, and how will we know if our desired outcome has been achieved? If a school can’t answer those questions, it likely should delay its expansion of digital learning until it can.

Toronto library redesign for modern learning and collaboration

Late last week I was in a dingy hotel room in Toronto, and in search of a better place to work I decided to try the main branch of the Toronto public library. I had no idea that I was about to experience a modern example of creative use of space that builds on digital technology, and is an example for every school thinking about how to foster creative learning and collaboration environments.

The library is well into a $34 million revitalization to transform it “into a library of the future – one that will meet the diverse needs of today’s and tomorrow’s learners.” The new design includes the following features:

  • Small group discussion “pods”—these are small, circular, glass-encased areas with a table in the middle—that allow for two or three people to converse while not disturbing others nearby, and also, because they are glass, maintain the wide open feel of the overall open floor plan.


  • Tables with different configurations, so that a single person or a group of many different sizes could easily find a spot to work. Many of the options for people working alone were next to large windows allowing in abundant natural light.

Toronto library work area

  • Small rooms to the side of the main room that were bigger than the pods, and had whiteboards.
  • A bank of televisions set to different channels, with circular seats in front of all of them collectively. Above each seat was a cone that directed the audio from one of the TV programs down to the seat below. If you were standing anywhere other than directly below the cone you couldn’t hear the audio, so it didn’t disturb anyone working nearby.

Toronto library tvs

  • Wifi was available for those with their own computers, and desktop computers for others to use.
  • Power outlets were distributed throughout the building.
  • An excellent coffee shop connected to the library, which allowed visitors to bring coffee in.

On this sunny Sunday afternoon the Toronto library and coffee shop were both crowded with people of all ages, in seemingly endless configurations, alone or in groups, using various computing devices and/or paper materials.

It’s going to be a long process to get to the point where most learning environments in middle and high schools are creatively geared toward multiple types of learning and collaboration, given the development time and capital costs involved. It’s notable that many students will start their education in an elementary school classroom that encourages different types of learning and collaboration, perhaps end their formal education in a college or university with similar spaces, but spend six of their most formative in-between years largely in classrooms with rows of desks in front of a teacher in which active learning and collaboration have to overcome the physical space instead of being nurtured by it.

See the Toronto library web page for details and more pictures.

Data privacy: implications for digital learning, and beyond

Among the policy issues that we are researching for Keeping Pace 2014 is student data privacy. Twenty states have enacted a total of 28 bills related to data privacy this year, after all states combined enacted just one in 2013. (This and other summary information comes from the Data Quality Campaign, which has been tracking data privacy bills and generously sharing information.) Although the majority of these bills do not explicitly prohibit the use of student data for instructional purposes, several have provisions that are challenging for educators. We will discuss these restrictions, and implications of poorly considered data privacy policies on digital learning, in some detail in the Keeping Pace annual report that will be released at the iNACOL Symposium that begins November 4. We stress the “poorly considered” aspect of these laws, because there is no question that student data privacy is a legitimate issue. Student data must be protected, but it can be protected without creating undue burdens for educators.

A recent article from Pew, States Collaborate to Keep Track of Students, reminds us that the expanded use of data is not an issue related only to digital learning. The article describes how states attempting to follow student outcomes, particularly of students once they graduate from high school or drop out, are often limited to researching students who remain in the state. Four states—WA, OR, HI, and ID—are using grant funding to collaborate to follow students who leave their home state and move to one of the other three states. In one example, in a data set of about 40,000 students, the states were initially able to understand outcomes for 62% of students, which increased to 69% when they included data from the other three states. That in itself isn’t a huge increase, but it suggests that if more than four states were sharing information then researchers would have a much better sense for student outcomes.

As education policy increasingly moves from a focus on inputs-based quality assurance to an equal (or greater) focus on student outcomes, the increased use of data become ever more important. Understanding student outcomes shouldn’t stop when a student leaves high school for at least two reasons. First, students who drop out of high school may eventually go back—perhaps in another state—or earn a GED. Second, as more and more jobs require some post-secondary education, understanding how many students are completing a post-high school degree is important to determine education policy goals.

Privacy advocates may counter that while those goals are laudable, they don’t override individuals’ interest in privacy. For the most part, however, that argument is a red herring. Data can be shared and used without revealing personally identifiable private information, and the best data privacy policies aim to strike this appropriate balance. Data usage is rapidly increasing in many other fields while addressing most privacy concerns. See, for example, the ways in which medical data are being used to determine drug problems and harmful interactions, or the ways that people are willingly sharing personal information with companies in order to get discounts or free services.

The use of data is increasing across most sectors, in ways that carry some risk but ultimately benefit providers and users. The same trend is occurring in education, and its growth will help educators and students.

What does a blended classroom look like?

We received a question on the blog this week about how to design classroom spaces for blended learning that we thought warranted a full post here. If a school is using blended learning in a transformative way, it is likely that classrooms will look different from a traditional set-up with rows of desks facing the teacher. The space must allow for independent work, small group instruction, and occasional full-group instruction, as well as the need for flexibility on any given day.

Decisions about physical space are driven first by the academic goals of the blended learning implementation, and then by the instructional approach being used by the program. For example, building an after school credit recovery program would suggest a different physical space than a blended middle school class. The 2013 Planning for Quality guide (also available inside Keeping Pace 2013) assists program leaders with the decision-making process, helping to refine academic goals and begin to give shape to the program.

Once the academic goals and instructional approach are determined, it is time to focus on other elements, including the physical classroom and/or school building space. Established charter schools using blended learning, and particularly some of the larger charter management organizations (CMO) that have launched multiple blended schools in recent years, can serve as useful examples. Case studies and articles that have been published about Carpe DiemRocketship, USC Hybrid High, Aspire, Summit (and here), Nexus Academies, and Flex Academies provide ideas about what might work in your school. Also consider the lessons being learned by some of the more mature CMOs, including Rocketship, which has talked a great deal in the last year or so about its shift from a lab-based rotation model to a classroom-based model, and how that influences classroom design.

Merit Prep published a brief about its classroom design process as part of its work for its Next Generation Learning grant. Next Gen has created an active community of blended learning thought leaders, making many resources available on its website. Often these schools and organizations have received significant grant funding, and are required to write up their reasons for designing their schools the way they do, and share their thoughts about what is / isn’t working as the model is implemented.

Finally, the Partnership for 21st Century Skills released a paper on 21st Century Learning Environments that looks at the theory behind classroom and school design, and how learning environments must shift to accommodate the interconnected, dynamic, personalized learning that is happening in schools today.

Most students need personal attention from teachers—and most digital learning advocates know that

I’m often reminded that digital learning advocates should never tire of repeating that digital learning does not render teachers obsolete, that online content is usually used by teachers who are either online or in the physical classroom, and that the personal connection between students and mentors—whether teachers, or coaches, or other adults—is critical to the success of students.

A report by Maine Public Radio on Portland Public Schools opening an online program this fall doesn’t delve too deeply into the supposed impersonal nature of online learning, although one quote—and several of the questions that the reporter asked me—touch on this issue. In response to the reporter’s questions I explained the extent to which online courses use teachers, and how many online teachers say they know their online students better than the students they used to have in physical classrooms, because of the amount of and depth of online communications. I was reminded that in Maine online schools are new, and digital learning isn’t widely understood, so misunderstandings abound.

It’s not just in Maine, however, where people hold misconceptions about digital learning. A recent New York Times op-ed, Teaching Is Not a Business, suggests that education technology is inherently impersonal. Combining online learning advocates with education reformers who believe in competition, the author writes “Both camps share the belief that the solution resides in the impersonal, whether it’s the invisible hand of the market or the transformative power of technology.”

Of course technology can be impersonal, but it doesn’t have to be. Students today are communicating ever more via Facebook, texting, and other digital means—and they don’t consider those methods impersonal. What matters is how technology is implemented in education. Technology can foster personal relationships that are strengthened by digital communication, but technology can also be isolating if it is used without a teacher or mentor.

How advocates talk about digital learning also matters. While the New York Times article builds on an inaccurate premise, a recent Slate article is more reasoned. Bill Gates Is an Autodidact. You’re Probably Not discusses the extent to which many creators of education technology are self-taught, and the degree to which being self-taught is unusual.

“The experiences of ed tech creators and promoters are notably influential—and notably unusual. Most people are not autodidacts. In order to learn effectively, they need guidance provided by teachers. They need support provided by peers. And they need structure provided by institutions. Amid all the effusions about how ed tech will “change the way we learn,” however, these needs rarely merit a mention. Instead we hear about the individual and his app, the person and her platform, as if teachers, classmates and schools were unnecessary and unwelcome encumbrances.”

This statement strikes me as accurate, although more for ed tech “creators” than “promoters.” Plenty of digital learning advocates realize that students need teachers, classmates, and institutions, including the large majority of educators who are implementing technology within schools. But the media attention on the creators, and sometimes the creators and promoters themselves, lose sight of these facts. That is why digital learning advocates should never tire of repeating that digital learning does not make personal connections obsolete.

WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera