University research is generally funded from the public purse. The results, however, are published in peer-reviewed academic journals, many of which charge subscription fees.

I had to use freedom of information laws to determine how much universities in New Zealand spend on journal subscriptions to give researchers and students access to the latest research - and I found they paid almost US$15 million last year to just four publishers.

There are additional costs, too. Paywalls on research hold up scientific progress and limit the public’s access to the latest information.

Read more: Not just available, but also useful: we must keep pushing to improve open access to research

The project took more than three years because universities originally refused to release the information. I had to make a complaint to the Ombudsman, the government official charged with determining whether information from the state sector should be publicly disclosed.

The Ombudsman’s final opinion ruled unambiguously that the public’s right to know outweighs any commercial interests of the publishers and universities.

The cost of knowledge

The following points stand out in a preliminary analysis of spending by New Zealand universities on subscriptions to journals from Elsevier, Springer, Wiley and Taylor & Francis between 2013 and 2016.

The total amount spent on the four publishers is substantial, around US$14.9 million in 2016 (the total spending on all publishers is likely at least 2-3 times that). The University of Auckland, with 33000 students and 2200 academic and research staff, spent US$3.8 million, including US$1.6 million on Elsevier.

The mean expenditure per academic/research staff member in 2016 was around US$1800.

The University of Canterbury is getting a much worse deal than the others, 35% above the mean.

The rate of increase of subscription costs (17%) over the period clearly exceeds the Consumer Price Index inflation rate over the period (2-3% in New Zealand, USA and Europe).

The publisher with the highest percentage increase over the period was Taylor & Francis (33%).

Obtaining the information

Many journal subscription prices are high (for example, the prominent biology journal Cell is over US$2000 per year), especially given that the funding for the research typically comes from public sources.

With the advent of the internet, many people predicted a major drop in expenditure on journals, but the opposite has occurred. One reason is that the main commercial publishers use anti-competitive practices such as bundling of unrelated journals (so-called “Big Deals”) and confidentiality clauses in contracts.

Price secrecy allows sellers to use differential pricing and weaken the negotiating situation of buyers, leading to market inefficiency. The fact that each journal has a monopoly on its specific content means that journals cannot be easily substituted by others.

In 2014, Timothy Gowers and others used freedom of information laws to extract the relevant price information from universities in the UK. In 2009, less extensive work in the USA had also been done by Ted Bergstrom and colleagues. Data from Finland and Netherlands has recently been made public.

I requested data from seven of New Zealand’s eight universities, which collectively have around 8400 academic/research staff and 130000 students. The process was long and required persistence. Following the Ombudsman’s ruling, the universities complied, supplying me with data on spending on journals from Elsevier, Springer, Wiley and Taylor & Francis.

There are some subtleties, such as assumptions about exchange rate conversions and exactly which products from the listed publishers the money is spent on. Interested readers can consult the raw data.

Is open access the answer?

The restricted access inherent in the subscription model makes it hard for journalists, politicians and the general public to use scholarship for better evidence-based decision making.

Recently, open access journals have emerged. They place no barriers on readers but still have production costs. The “Gold Open Access” model, in which authors or funders typically pay for each article, has become popular with traditional publishers. They often set the article processing charge level at around US$2000 to US$3000.

The analysis above implies that wholesale conversion to such article processing charges will not save money for universities. Several independent estimates put a reasonable article processing charge at no more than US$500 (less in some disciplines).

The key problem is not the particular model of payment for journal article production and distribution, but the dysfunctional market in publishing services. Although they are a large part of the problem, commercial publishers are not entirely to blame. For example, the research community uses historical journal reputation to evaluate researchers, making it harder for new, better run journals to enter the market.

The right kind of open access

Even with the best will in the world, there is an inevitable time lag for new journals to become established. To make faster progress, it is necessary to decouple the ownership of current journal titles from the provision of editorial and publication services, so that competition among publishers helps to control prices. This reclaiming of community control is the most fundamental of the recently formalised Fair Open Access Principles.

New organisations such as MathOA, PsyOA, LingOA and the Fair Open Access Alliance have been set up to facilitate large-scale conversion of subscription journals to an open access model, with community control of journals and no direct author payments. This of course involves mass defections by editorial boards.

We expect that global savings of at least 75% of current payments to journals can be made by using modern publishing providers such as Scholastica and Ubiquity, and by reallocating subscription payments toward article processing charges. What is the research community waiting for?