Announcing the Open Access Spectrum (OAS) Evaluation Tool

Twitter icon
Facebook icon
Google icon
LinkedIn icon
e-mail icon
October 19, 2015

Announcing the Open Access Spectrum (OAS) Evaluation Tool
New Service Will Quantitatively Document Journals' Degrees of Openness

Overview
SPARC is pleased to announce the launch of a new program that builds upon this valuable resource. The Open Access Spectrum (OAS) Evaluation Tool provides a concrete, quantifiable mechanism to independently analyze publications' policies. It offers unprecedented insight and transparency into scholarly journals’ degree of openness. 

Developed in 2013 and revised in 2014, the HowOpenIsIt?® guide lays out, in descriptive fashion, the array of polices a journal can have in the continuum between “Open” and “Closed” across six critical dimensions – reader rights, reuse rights, copyrights, author posting rights, automatic posting, and machine readability. The organizations behind the HowOpenIsIt? Guide – SPARC and PLOS– have been joined by BioMed Central, Copernicus Publications, eLife, Frontiers, Open Access Scholarly Publishers Association (OASPA), and Research Libraries UK (RLUK) in the development of the Open Access Spectrum (OAS) Evaluation Tool. This new service, in effect, converts a qualitative resource – the HowOpenIsIt? Guide – into a quantitative tool. For more on the mechanics of how the certification program works, see below.

The OAS Evaluation Tool generates an “Openness” score that is straightforward, easy to understand, and free. The program provides critical information to authors, libraries, research funders, government agencies, and other interested parties. It can be used to help determine compliance with funder policies, institutional mandates, and researchers’ individual values.  It also offers a unique opportunity for publishers to independently validate their journals’ degree of openness and compliance with funder/campus policies.

Selection Criteria
An initial batch of 500 journals has been included at launch, with another 500 to follow in the next few months. These journals encompass a range of disciplines, countries of origin, and business models. The set of 1,000 journals was created from the freely available Scimago dataset, which was divided into open access and non-open access journals on the basis of a lookup of their ISSN in Directory of Open Access Journals (DOAJ). The top 600 journals from the non-OA set and the top 200 from the OA set based on ranking by Scimago Journal Rank were selected. Additionally, 200 journals were selected from across the Scielo, Redalyc, Bioline, AJOL, and DOAJ databases to ensure geographic and subject diversity. 

Rationale
Why is the OAS Evaluation Tool necessary? While the rapid growth of OA has seen an expansion in the availability of scholarly articles, it has also generated no small degree of confusion within the research community. Many journals claim to be “open” while actually placing moderate or severe restrictions on what an author or reader can do with an article, for example.  It has become clear that not all “open” is created equal. The OAS Evaluation Tool will provide independent, expert evaluation of journal OA policies beyond just “is this article free to read?”

Scoring
The OAS Evaluation Tool uses the HowOpenIsIt? Guide as the basis for a 100-point scale. Journal policies are mapped against the scale to generate a certified score that is straightforward, easy to understand, open, free, and:

  • Provides critical information to authors, libraries, research funders, government agencies, and other interested parties.
  • Includes distinct evaluations for a journal’s policies regarding reader rights, reuse rights, copyrights, author posting rights, automatic posting, and machine readability.
  • Can be used to help determine compliance with funder policies, institutional mandates, and researchers’ individual values.
  • Offers a unique opportunity for publishers to independently validate their journals’ degree of openness and compliance with funder/campus policies.
  • Pulls together— in one single, accurate web resource— information that is otherwise buried across scores of publisher websites.

Overview of Evaluation Process
OAS evaluations are performed by a team of scholarly communication, library, and publishing experts. Collectively, they have decades of experience in the field.  At least three team members participate in each evaluation. The team creates a provisional score for a journal. This provisional score is shared with the journal’s publishers and/or editors, who will have the opportunity to provide any clarifying documentation. The score can be revised based on this additional information. Each journal’s certified score is time stamped to indicate precisely when the evaluation was performed. Should either the publisher or anyone else have reason to believe that the journal’s policies have changed since that time, they are encouraged to contact the OAS Evaluation Tool team to request an update.

Dataset
Organizations are encouraged to build service on top of the OAS Evaluation Tool, or to integrate it into existing web sites and applications. A full CSV of the Open Access Spectrum Evaluation Tool dataset and a REST API for search and access to scores are available here.