The Chronicle of Higher Education and Campus Technology recently covered a campus bookstore-sponsored study on Washington state's landmark Open Course Library project. Since the articles left out some important facts, we're devoting this post to set the record straight on how far the project has come and what the data actually show.
The Open Course Library (OCL) is one of the first and highest profile system-wide efforts to leverage open educational resources (OER) for textbook affordability. Created by the Washington State Board for Community and Technical Colleges (SBCTC) in 2009, OCL worked with teams of faculty, librarians and instructional designers to outfit the system's 81 highest enrollment courses with materials that cost $30 or less per student per course. Wherever possible, the teams created or repurposed OER to meet course needs, but also used other free and low-cost materials from other sources, including traditional publishers.
OCL's stated goals are to reduce the cost of textbooks for students and provide high-quality, low-cost course materials for faculty members' optional use. By all these measures, OCL has been a success.
- According to an analysis conducted by the Student PIRGs, OCL saved students more than $5.5 million as of the end of the 2012-2013 academic year. Extrapolating the same methodology, the cumulative savings will rise to $8.3 million by the end of this academic year.* Compare that to the $1.8 million cost of the project (split between the state legislature and Gates Foundation), the return in student savings is quadruple the original investment.
- SBCTC's own research suggests that OCL has successfully increased awareness of OER among the system's faculty. According to a 2013 statewide survey, 67% of faculty had searched for OER in their discipline and 60% of those faculty had used OER in their teaching practices. "Open Course Library" was listed among the more frequently used sources of OER content.
- While OCL is not an OER publishing project, it contributed to the development of a number of high-quality resources that are widely used both in Washington and elsewhere. For example, OCL helped fund the software behind WAMAP, a popular mathematics assessment platform that 12,000 Washington students used this fall alone. Commercial alternatives cost $50-100. Another great example is the open textbook developed for OCL's Precalculus I & II course, which has saved students more than $1 million to date.
So, what do campus bookstores have to say about all this? The National Association of College Stores (NACS) recently released the results of a study on OCL's impact conducted by 25 of SBCTC's campus bookstores. NACS has been a supporter and ally to the OER movement for many years, so the study's negative tone and extreme conclusions came as a surprise and disappointment to many of us who support OCL's work.
The study's findings, in themselves, are neither unfavorable nor unexpected. They simply show that according to campus bookstore data, a relatively small portion of SBCTC faculty members reported assigning free and low-cost materials identified in OCL's first 42 courses, and that some of the materials were not reported as being used at all. There are two important points to put these findings in proper context.
- First, bookstore data paint a piece of the picture, but not the complete picture. One of the challenges with evaluating any free online resource is the difficulty tracking exactly who uses it. A substantial portion of OCL's materials are either open or free online, so bookstores inevitably only capture a subset of the materials' use — this is true both within and outside of SBCTC. Also, NACS knows better than anyone that federal price disclosure requirements have limits and focus on textbook-related materials, so a portion of OCL's content would not clearly qualify for reporting. So, the study provides a useful measure of OCL's usage, but not a comprehensive one.
- Second, and more to the point, adoption of specific materials was not the goal of OCL in the first place. The project recognized from the start that teaching is not one size fits all, and therefore set its goals much more broadly on promoting textbook affordability regardless of where the content comes from. Naturally OCL hoped that other faculty would benefit from the course developers' work — and both NACS and SBCTC's data show that many have — but the rate is not the project's measure of success. The overall use of affordable course materials is.
Unfortunately, the NACS study misunderstood both of these points and arrived at conclusions that mischaracterize OCL's impact and aims. The study assumes that bookstore data show the complete picture of OCL materials' use and that the goal of the project was uniform use of OCL materials by the majority of faculty in the system. It concludes that the materials were "slow to catch on" and achieved "no significant savings," when the study's data are insufficient to support either claim. The Chronicle and Campus Technology amplified this misunderstanding in their articles by not including alternate viewpoints or noting the study's clear limitations.
It is a shame that the study is clouded by these misconceptions, because the underlying results are valuable. The study adds to our understanding of OCL's impact in the SBCTC system and underscores the challenge of tracking the adoption of online resources through conventional systems. It asks important questions for future research and makes the key point that creating low-cost materials alone does not equate to widespread adoption. And, once NACS fulfills its promise to release the full dataset and methodology, further analysis could produce invaluable insight relating to specific courses and schools. We appreciate the work NACS put into the study and wish these positive elements could have taken center stage.
We still believe that NACS has its heart in the right place, and we echo the call for collaboration on future research that SBCTC's Executive Director extended in his letter to The Chronicle. Bookstores have an important role in the future of course materials, and in many ways stand to benefit from new models like OER. We all share an interest in studying the impact of projects like OCL to be more successful moving forward, and it can be done more constructively and accurately by working together.
* Calculated based on the April 2013 Student PIRGs report "Affordable Textbooks For Washington Students: An Updated Cost Analysis of the Open Course Library." OCL's cumulative savings at the end of the 2012-2013 academic year was $5,480,173 including $2,799,757 saved during 2012-2013. Assuming the savings for 2013-2014 will be similar to 2012-2013, we added $2,799,757 to $5,480,173 to arrive at $8,279,930 or $8.3 million cumulative savings by the end of 2013-2014.