SRI Khan Academy study: “the teacher’s role is still central”

Previous blog posts (post one, post two, and post three) have discussed the recently released SRI study of Khan Academy. Here we conclude with a final post that summarizes the findings of the report, because it is well worth repeating verbatim:

Our learnings in this study made it clear that the teacher’s role is still central even in the wake of the adoption of new technologies. The achievable classroom benefits of using new technologies can include building stronger connections with students, and developing a clearer and deeper understanding of what students actually know. At their best, the new technology tools can enable teachers to do what they find most fulfilling: interacting with students to have a positive impact on their learning experience.

With this in mind, our study shows that teachers still need support in integrating online instructional resources into the curriculum; they need digital content that is curated and aligned with grade level standards, and models of use that demonstrate the resource’s value with students like theirs. As we move toward greater classroom use of self-paced instructional resources, students will also need additional support to navigate this transition, which may vary depending on the fit between the individual student and the online environment. To understand the supports needed so that all students can excel in self-directed online learning environments, research should be pursued to understand the roles that non-cognitive student characteristics—motivation, persistence, resourcefulness—play in student success in these environments. Finally, experimental studies of the impacts of different Khan Academy implementation models and other digital learning tools like Khan are needed to determine effects not only on math achievement but also on students’ attitudes toward mathematics and their capacity for self-directed learning.

To repeat: “the teacher’s role is still central even in the wake of the adoption of new technologies.”

There is a fear in K-12 education that technology will replace teachers. While technology certainly exists that could replace some teaching roles, it is not sophisticated enough to replace the interpersonal relationship between the teacher and student that is seen as a hallmark of high performing classrooms. As we note in the Teaching portion of the Planning for Quality section that has been included in Keeping Pace each of the last three years (here is the 2013 section), teachers are a critical component of a high-quality online learning program. While there is certainly a percentage of students who will thrive in a self-paced, student-driven environment, most students are not prepared for an educational change that radical.

SRI Khan Academy study provides important implementation examples

As discussed previously (post one and post two), the SRI study of Khan Academy implementation in schools did not result in a simple answer to the question “does it work?” But the report does provide valuable insights into how several school sites were implemented, and some of their lessons learned during the two year study. In doing so it raises four key points:

  • The implementations are very different from one another,
  • The use of data generated by the Khan platform is as important as the videos and practice problems,
  • The addition of Khan Academy is often part of a larger instructional approach, and the success of Khan cannot be separated from the success of the overall methods, and
  • Most of the schools are experimenting, and changed their approaches during the time when the study was taking place.

Key elements of the study sites include:

Site #2:

An innovative 9th and 10th grade math program in…two small charter high schools that opened in fall 2011. The schools are co-located in a neighborhood where 45% of students qualified for free or reduced-price meals. These trailblazing schools emphasize a self-paced learning model, in which students take on significantly more personal responsibility for directing and managing their own learning than is typical at a traditional school. Technology plays a significant role in facilitating this approach… This site was by far the most distinctive of the nine sites participating in the study because of its emphasis on preparing students for college using a self-paced, self-directed learning model. The site’s mission focused on helping students learn how to set learning goals, hold themselves responsible for meeting those goals, and evaluate their own progress continuously along the way. More broadly, these schools encourage students to learn how to advocate for themselves as learners. Its math program was designed to support students’ simultaneous development of content knowledge, academic skills, and critical non-cognitive skills.

During the first year of the study, when use of Khan Academy was mandated for all students, the schools had the highest use levels among study sites, with students spending about 22% of their instructional time on Khan Academy activities, compared to less than 10% at all of the other sites. In the second year, this site’s students used Khan Academy for fewer hours than the first year, because the schools changed their math instruction model to an entirely self-paced, self-directed approach, and Khan Academy use was now entirely at the discretion of students. While students used it for shorter periods of time during the second year of the study, they used it in more independent and innovative ways. This approach was more tightly aligned with the site’s mission, and may be helpful to schools investigating how to use technology to personalize learning and make it more student-directed.

Site #8:

Khan Academy was used on a daily basis by two educators: one teaching an algebra readiness class for ninth graders and a learning lab class, and the other teaching a mixed 9th and 10th grade Algebra 1 class, a mixed 9th and 10th grade geometry class, and a 10th grade Algebra 2 class…The school primarily used Khan Academy to support teacher-directed, whole-class instruction. During the daily math class, all students focused on the same practice problem sets at the same time. Rather than using traditional worksheets, however, they worked online using Khan Academy exercises. This approach may be helpful to schools exploring how to use technology to give students more opportunities to practice their math skills in a way that is integrated into the existing curriculum and also reinforces their teacher-led lessons. The two teachers at this site primarily employed Khan Academy as a resource to help students devote more structured, productive time to practice activities designed to help them fill in gaps in their math knowledge and skills and reinforce skills covered by the teacher in their daily lesson.

Site #9

This middle school used a “rotation” model for its 6th graders, largely because it did not own enough computers for each student to have one. There were only about a dozen netbooks—inexpensive laptop computers designed for Internet access and wireless communication—available for a classroom of about 25 to 30 students. The school devoted a 2-hour daily block of time for 6th grade math instruction, which typically began with a 20-minute whole-class warm-up period consisting of announcements and “mental” math exercises (problem solving without calculator or pen-and-paper). The class then divided into three groups of 8 to 9 students for the rotation stations. Each group spent about 30 minutes at each of the three stations, one of which was Khan Academy. At a second station, students worked in a small group with a teacher who gave a mini-lesson, and at the third students practiced independently on math worksheets or took an assessment. One day of class each week was reserved for whole-group instruction, activities, or testing.

The question is often raised in simple form: does using Khan Academy work? The variety in the above implementation descriptions demonstrates why the question cannot be answered simply. Even if one implementation was shown to work (and none were), that finding would not apply to other implementations.

Khan Academy Study by SRI Reviews Implementation and Not Outcomes

This is post two of four about the SRI study about Khan Academy; post one can be found here.

We are often asked: do online and blended learning work? As we have explained several times (e.g. in Keeping Pace 2011, pp 40-49), the question is more difficult than it might appear for at least two reasons.

First, defining “what works” is not easy. If “working” is defined as providing new opportunities to students, then the answer is undoubtedly yes. If it is defined as improving student outcomes in some cases, then again the answer is yes, as with students taking online Advanced Placement courses and in some programs exceeding national averages for AP exam scores. But if “working” is defined as raising student achievement across large scales, based on randomized control trials, the answer is that the research is limited and inconclusive.

The second reason that answering the “does it work” question is hard is because so many types of online and blended learning exist. The result is that comprehensive research is difficult and expensive, and applies to only a very small percentage of overall online and blended learning implementations.

The recent report from SRI looking at Khan Academy use in schools supports and amplifies the above points:

“SRI conducted an implementation study rather than an evaluation of Khan Academy’s impact on learning, given that implementation of Khan Academy varied across sites, Khan Academy tools and resources evolved over time, and schools, Khan Academy, and researchers collaborated regularly. Ongoing communications and tight relationships between the participating teachers, students, and school leaders with Khan Academy and SRI International resulted in both a more useful tool for teachers and students, and a better understanding of the various ways Khan Academy can be used in school settings to promote student learning.

…teachers and students were using Khan Academy tools and resources in considerably different ways across the nine study sites, and some of the sites also changed the ways they used it during the course of the two-year study. For these reasons, it was methodologically unsound to conduct a rigorous evaluation of Khan Academy’s impact on learning during the study period, including any use of randomized control trials, which would have required Khan Academy tools and resources to remain unchanged during the study and for teachers and students to use Khan Academy the same way. Moreover, at all but one of the sites, Khan Academy was principally used as a supplementary tool—not as the core primary curriculum—so the effects of Khan Academy cannot be separated from those contributed by other elements of the math curriculum.” (emphasis added)

While the above text may sound like the study did not yield useful results because it did not come to any conclusion about Khan’s impact on learning outcomes, the study does in fact contain practical information. We will review these findings in the next blog post.


Khan Academy and Implications of Teacher Control

This is post one of four about the recent SRI Study about Khan Academy; post two can be found here.

Among the challenges in implementing blended learning into existing schools is that, in most cases, teachers view their classrooms as their domain. Keeping Pace research suggests that successful blended learning implementations are almost always at the school level (or higher, e.g. district level or consortium level). The school-level approach to blended learning, however, collides with the view that teachers own what occurs in their classrooms, because school-wide blended learning implementations often select content, technology platforms, end-user devices, instructional models, and other elements of instruction that teachers are often accustomed to controlling within the classrooms.

Larry Cuban’s recent blog post “Business as Usual in Corporations and Schools” makes a similar point, within a larger (and well worth reading) discussion of power, status, and hierarchy in companies and in schools. Cuban is not an uncritical advocate for technology in education, and often questions people who suggest that online and blended learning are already having a large-scale impact on teaching and learning. He is among the realists who see things as they are, and not as they wish them to be. Speaking of power and hierarchies in schools, he says:

“With all of the rules and hierarchical levels from classroom through the state superintendent of education, teachers have one thing in their power to do: close the classroom door. They are (and have been) gatekeepers for student content, skills, and attitudes…”

Although the end of the quote above may be a slight overstatement—because content is selected and skills determined by schools, districts, and states—it captures the view that teachers do expect a high level of control over their classrooms.

After a brief discussion about “the talk of moving to project-based learning and shifting the teacher’s role from the sage-on-the-state to the guide-on-the-side,” he concludes with this:

“On occasion, some of the reforms have stuck in some schools where teachers weathered criticism and supported one another through cascades of hype and criticism. In these scattered instances, teachers kept their doors open and built a stable school culture supporting…instructional reforms…[including] rich student-centered activities. But not most teachers who returned time and again to practices that worked better for them than the “new” reform simply because they could close their classroom doors.

These are (and have been) abiding features of public schools and companies that no amount of talk and hype about doing business differently has changed.”

Cuban’s comments came to mind when I read these sentences in the SRI report on Khan Academy implementation:

“Even though Khan Academy is primarily known for its video library and has been associated with the “flipped” classroom model (i.e., teachers assign students videos about new concepts to watch as homework, and use class time to extend the video lectures with discussion and interactive activities), teachers participating in the research were more focused on exploring how online, personalized practice opportunities for students could be incorporated into their existing instructional activities.” (emphasis added)

Khan Academy is widely used (10 million unique users per month as of February 2014, with about 65% of users from the United States, according to SRI), and it is undoubtedly helping many of those students. But whether any of the school-level implementations of Khan are game-changing has not yet been determined, as we will discuss in forthcoming posts. To the extent that usage in schools continues to be built around existing instructional activities, there is reason to doubt that outcomes will be significantly improved.

Michigan study looks at effectiveness of online learning

The Virtual Learning Research Institute, a division of the Michigan Virtual University, has released Michigan’s K-12 virtual learning effectiveness report. With this report Michigan joins the ranks of the few states (others include California and Colorado) that have released comprehensive reports about online and/or blended learning activity.

The key findings “include an apparent growth in the number of students and schools participating in virtual courses, with the majority of virtual enrollments coming in the core subject areas. Students taking virtual courses in a supplemental capacity appear to be more successful when they take only a few virtual enrollments a year. Developing practices to better support students who take higher amounts of virtual enrollments should be a priority.”

An important caveat is included that reflects the state of digital learning data: “Statistics shared in the report must be interpreted with care due to concerns about the accuracy of data reported to the state about virtual enrollments during the first three years of its collection.” Specifically, “MVU believes that the accuracy of the virtual delivery flag tied to student enrollments through the Michigan Student Data System Teacher-Student Data Link is less than optimal.”

Despite the data concerns, the information is valuable. MVU finds that 55,271 students took an online course in school year 2012-13. That number reflects an increase of 52% over the past two years, but with almost all of that increase taking place between school year 2010-11 and school year 2011-12. It is unclear if the greatly reduced growth in the past year is real or is a result of inaccurate data. In school year 2012-13 students took a total of 185,053 course enrollments, which more than doubled the number from two years prior. This number represents about 1% of all course enrollments in the state.

About “90% of the virtual enrollments each year came from students in grades 9-12,” and core subjects accounted for most of the enrollments. “Mathematics consistently accounted for the largest percentage of virtual enrollments with approximately 20% of virtual enrollments each of the past three years. Other core subjects like English Language and Literature, Social Sciences and History, and Life and Physical Sciences…equaled or exceeded 10% of the virtual enrollments for a school year.

The discussion of completion rates and impacts is too complex to easily summarize, but the conclusion is worth quoting:

“On the one hand, there is evidence within this report that may lead some to claim that K-12 virtual learning simply is not working in Michigan. Detractors could cite lower completion rates for virtual enrollments, or they could focus on the finding that the results for students in poverty are not on par with students who are not. On the other hand, there is evidence that virtual learning is clearly working. Proponents could cite that about 40% of Michigan schools had an 80% or higher completion rate for their virtual enrollments. Or that students who take one or two virtual courses a year have a completion rate of almost 70%. Both statistics seem like even more significant accomplishments given that the data indicate schools tend to limit virtual learning options for students, seeing it more as a credit recovery option than as an initial credit solution.

Regardless, the findings presented here are not intended to further polarize along the lines of virtual learning either working or not working, but rather to aid in understanding under what conditions virtual learning can work and in doing so, with an understanding of the current educational climate and educational demands of the 21st century, change the collective mindset from “if” to “how.” Existing research indicates, and the research team knows from experience, that virtual learning is far more likely to yield the desired results when the course content is high-quality, the virtual instructor is skilled at teaching online, and the student has wrap-around support including active local mentors and parents.”



WordPress SEO fine-tune by Meta SEO Pack from Poradnik Webmastera