Data on the availability of scientific literature are often obtained from single-nation studies ( Nisonger, 2011 ; Malapela & De Jager, 2017 ), and to the best of our knowledge, worldwide comparisons have not been carried out. Consequently, it is still not well known how paywalls may affect researchers worldwide. The primary objective of this study was to evaluate the accessibility of recent full text articles in the field of ophthalmology in established institutions distributed worldwide using “institutional and Hinari access,” then “alternative ways” (PubMed Central, ResearchGate (RG), Google Scholar and Online Reprint Request (ORR)), and finally Sci-Hub. The secondary objective was to calculate extra costs institutions or researchers must bear to recover all full texts, buying them on publishers’ websites.

Research studies are based on bibliographic research work to achieve “the state of the art.” This stage not only makes it possible to carry out research studies based on existing and well established scientific foundations, but also prevents carrying out studies already conducted by other researchers in the world. Thus, access to scientific articles for researchers around the world is crucial for assessing high-quality research. In the current revenue-models of scientific journals, access to scientific papers is often restricted by paywalls. This implies that full text articles are only available upon subscription or pay-per-view on publisher websites. One is forced to note that paywalls still limit access to approximately 75% of scholarly documents in all disciplines ( Bosman & Kramer, 2018 ; Piwowar et al., 2018 ), including biology and medicine ( Boudry & Durand-Barthez, 2017 ; Bosman & Kramer, 2018 ). Moreover, “paywalled articles” are known to disadvantage the lowest-income countries in terms of access to articles ( Aronson, 2004 ; Himmelstein et al., 2018 ).

When researchers do not have access to full text “paywalled articles” through the different means of access at their disposal, they can buy them individually on publisher’s websites (pay-per-view). However, this extra cost presents the disadvantage of being borne directly by the institution or laboratory, and sometimes by the researchers themselves. Extra costs of unavailable full text “paywalled articles” were calculated both in US dollars and as the percentage of gross domestic product (GDP) per capita at purchasing power parity (PPP). For each “paywalled article,” the cost of the pay-per-view of each article was recovered from the publisher’s website in US dollars. When the price was in euros, it was converted to US dollars using the exchange rate in use on April 5th, 2018. The cost of unavailable full text articles in each institution was calculated in US dollars, and was also expressed as the percentage of the GDP per capita at PPP (current international $), using the World Development Indicators from the online databases of the World Bank ( World Bank, 2018 ). The GDP is the market value of all officially recognized final goods and services produced within a country in a given period. Using the percentage of GDP per capita at PPP allows us to assess the real financial burden of providing full text articles in relation to the standard of living of each country.

Countries of researchers who agreed to collaborate were classified by continents according to the United Nations classification ( United Nations, 2018 ) and classified by income level according to the World Bank classification ( World Bank, 2018 ). Each participant had to search for “paywalled articles” in order to recover their full texts, using their institutional access and Hinari resources if available in their institutions, following a normalized protocol (see Supplemental Informations 1 and 2 ). The Hinari offer is available to two groups of countries: group A (free access for 69 countries) and group B (1,500 US dollars per calendar year for 50 countries) ( World Health Organization (WHO), 2018 ).

The recruitment of the participants in the different institutions included in this study was done by searching recent articles (published in 2017 or 2018) in the field of ophthalmology. The researchers were contacted by email using corresponding authors’ email addresses, or by contacting researchers already known by CB and FM directly. Only researchers working in capital cities or belonging to the most populated cities in their countries were chosen. Emails were sent to correspondents/colleagues to ask them to participate in this study, with the goal of reaching at least 25 countries located worldwide and spread over all continents.

The search for articles to be included in this study was carried out on March 29th, 2018 using the PubMed database ( http://www.ncbi.nlm.nih.gov/pubmed ), developed by the National Center for Biotechnology Information at the National Library of Medicine. The search strategy was: eye diseases [MH] AND 2017:2018 [DP] AND English [LA] AND Journal Article [PT] where MH stands for “Medical Subject Headings,” DP “Date of Publication,” and PT “Publication Type.” “Journal Article” includes the following publication types: journal articles, introductory journal articles, and reviews as previously described in the field of ophthalmology ( Boudry et al., 2016 ). Data were downloaded from PubMed in Comma-Separate values and were imported to Microsoft Excel 2013 (Microsoft, Redmond, WA, USA) for data processing. This publication search resulted in a total of 11,103 articles recently published in the field of ophthalmology from which 200 ( Tenopir et al., 2010 ) were randomly selected. The margin of error has been computed at 95% confidence level.

Using institutional/Hinari access, the average extra cost (in US dollars) of unavailable full text “paywalled articles” would be $2,790.8. However, this extra cost varies greatly from one institution to another: from $258 (Leiden University Medical Center) to $5,240.9 (State University of Milagro and Pakistan Institute of Community Ophthalmology). Using alternative ways in addition to institutional/Hinari access, extra costs would be reduced by an average of $1,918.6. Interestingly, complementary use of Sci-Hub reduces the average extra cost of unavailable full texts even more, dividing this average extra cost of unavailable full texts by 14 ($198.4 instead of $2,790.8 using institutional/Hinari access). Likewise, the average extra cost expressed as the percentage of GDP per capita at PPP is reduced more than 10-fold (2.4% instead of 25.4% using institutional/Hinari access and “alternative ways”) for the 27 institutions studied. This extra cost is thus substantial in some institutions, for example, in Togo, where it is at 18.8%.

Used alone, Sci-hub allows the recovery of 108 full texts (93.9%) of 115. Despite its illegal nature, researchers may be tempted to use this website, to try to find the unavailable full texts using institutional/Hinari access and “alternative ways.” Interestingly, using Sci-Hub as a complement to institutional/Hinari access and “alternative ways” allowed the recovery of an average of 109.8 (95.5%) full texts (SD = 1.8). Thus, the range of the number of available full texts varied very slightly from 108 (93.9%) to 113 (98.3%) of the 115 “paywalled articles.” Moreover, for Pakistan Institute of Community Ophthalmology and State University of Milagro, complementary use of Sci-Hub allowed them to recover 108 full texts of 115 instead of 37. As shown in Fig. 2 , when considering access by continent ( Fig. 2A ) or by income level ( Fig. 2B ), using complementary “alternative ways” tightens the gaps between continents and between countries of low and high income level. Using Sci-Hub as a complement to institutional/Hinari access and “alternative ways” totally eliminates the inequalities of access to “paywalled articles” by continent or income level. Note that the two institutions (University of Dakar and Lomé University) located in low income countries have particularly bad access despite using Hinari access. These findings suggest that Hinari program may not succeed in providing widespread access to less privileged countries.

Regarding “alternative ways,” 82 e-mails of corresponding authors were found via PubMed or via the Publisher website, and ORRs were sent to them. A total of 31 responses were received (43.9% success rate), allowing us to obtain 26.96% of “paywalled articles” using ORRs. No full texts of paywalled articles were found on the open archive PMC (paywalled articles included in this study were very recent and still under publisher embargoes, prohibiting self-archiving in OA repositories), and only 13 (11.30%) were found on social academic networks and internet via Google Scholar.

Discussion

In order to describe the difficulty in accessing scientific literature, a number of studies only focusing on one country have been conducted, mostly in the USA, to assess the availability of journals or books in university libraries (Nisonger, 2011). Other studies have been conducted to analyze global access and use of digital resources in research, mainly in African universities (Harle, 2009; Malapela & De Jager, 2017; Bruijns et al., 2017). To the best of our knowledge, only one study in 2011 previously evaluated the accessing of recent full text “paywalled articles,” and included seven institutions located in Africa, North and South America, Asia, and Europe (Voronin, Myrzahmetov & Bernstein, 2011). Furthermore, the problem of accessing medical information in low-income countries is still being openly discussed (Aronson, 2004; Godlee et al., 2005; Goehl, 2007; Himmelstein et al., 2018). We believe that our study provides the most complete comparative study examining worldwide access to recent full text articles. In addition, it is the first to study the financial implications of limited access to scientific literature and the first to assess Sci-Hub’s performance as an alternative to legal institutional access. The field of ophthalmology was chosen because, for several years, our research team has been focusing its research on bibliometrics in the field of ophthalmology.

Our study nevertheless has a number of limitations. The number of countries studied is relatively low, particularly Hinari eligible countries, which can be explained by the difficulty in finding researchers in the field of ophthalmology in most of the Hinari group A and B eligible countries. Indeed, when we looked for collaborators to participate in this study, we found very few researchers in the field of ophthalmology located in these countries (personal data). Moreover, institutions studied in each country are inevitably not representative of the overall situation in the country, and requesting participation from all the institutions in a country is impossible. Nevertheless, to minimize the differences between institutions, only those located in capital cities or belonging to the most populated cities in their countries were chosen. Finally, it was not possible to obtain an average value of access to full text “paywalled articles” by country and to calculate correlations between the number of accessible full texts and socio-economic parameters (e.g., GDP per capita). For the same reasons, it was impossible to assess whether the use of Hinari resources significantly changed the number of available full text “paywalled articles.” Furthermore, we wanted to compare the results obtained with the three ways of accessing full text “paywalled articles.” Our data included the average number of full texts available and the average extra costs, in US dollars and expressed as the percentage of GDP per capita at PPP. We found that using institutional/Hinari access; using institutional/Hinari access and “alternative ways;” and using institutional/Hinari access, “alternative ways,” and Sci-Hub were significantly different at p < 0.05 (p-value < 0.001). However, as the ways of accessing full text “paywalled articles” were not independent (because of the cumulative effect that binds them), these statistical results were not taken into account. We deliberately limited the search to articles published in 2017–2018 to include only very recent articles. This allowed us to access the latest findings which are thus presumed to be more informative for clinicians and researchers than older ones. Nevertheless, due to embargo periods, we must note that including articles published over a wider time period could have led to more frequent inclusion of articles freely available on publishers’ websites (“publisher OA” articles) and articles using “alternative ways.”

Our study nonetheless provides a global vision of the difficulty involved in accessing scientific literature around the world, highlighting the shortcomings of institutional access, which can have significant financial consequences for researchers seeking to overcome them. Our study also shows that the use of the website Sci-Hub helps to overcome these shortcomings and financial consequences.

The OA movement has been progressively making more articles openly available. However, articles behind paywalls (“paywalled articles”) are still more numerous than “publisher OA” articles in our study (57.5% and 42.5%, respectively). These results are in agreement with the most recent studies, which estimate that, in biology and medicine, 39.1% to 50% of articles published are “publisher OA” articles (Boudry & Durand-Barthez, 2017; Bosman & Kramer, 2018). We found that researchers in ophthalmology working in the 27 institutions included in this study can access only an average of 46.8% (with very large disparities, varying from 0% to 94.8%) of full text “paywalled articles” using their institutional and Hinari access. Furthermore, half of the 27 institutions studied can offer access to just over 50% of full text “paywalled articles,” a value very close (56%) to that determined in 2011 for seven institutions located in Africa, North and South America, Asia, and Europe (Voronin, Myrzahmetov & Bernstein, 2011) showing that the situation has not improved in recent years. As previously pointed out by Machin-Mastromatteo, Uribe-Tirado & Romero-Ortiz (2016), this shows that, “many universities are unable to acquire subscriptions for years, because they are seriously hindered by budget limitations and the lack of interest and policies from the state for supporting research and access to scientific resources.” Researchers in ophthalmology working in the less privileged institutions are thus in very poor situations. Indeed, although they use “alternative ways” (i.e., PubMeD Central, RG, Google Scholar and ORR) to recover full texts, they are forced to use Sci-Hub if they want to access a sufficient number of full text articles necessary to conduct their research satisfactorily. Researchers cannot afford these articles because their yearly budget for accessing articles without using Sci-Hub is too high, with sums of money up to $3,597 (institutions in Pakistan or Ecuador), or more than 200% of the GDP per capita at PPP (202.3% in Ecuador).

Results of the present study showed that “alternative ways” might help researchers access full text “paywalled articles.” We would like to draw particular attention to the usefulness of ORR. Although ORR does not immediately provide access and is a laborious procedure which depends on many factors that vary between individuals, ORR are quite efficient for obtaining full text “paywalled articles” (Kanthraj, 2010). Applied to our set of “paywalled articles” they allow free access to the full texts of 31 (26.96%) of 115, which is not negligible. An effort should be made to make them more widely known, particularly in less privileged countries, where they are the least used (Burrows, 1996). Another way to promote the use of ORR would be to have publishers and Pubmed make an extra effort to communicate, free of charge, corresponding authors’ email addresses (only 37 emails were found on the PubMed website, and 81 were accessible free of charge on publisher websites). Indeed, some publishers disclose corresponding authors’ email addresses only on the full text of articles attainable only through subscription, making it impossible to use ORR. Widely used globally (Bohannon, 2016), Sci-Hub raises many questions about the future of scientific publication (Russell & Sanchez, 2016; McNutt, 2016; Machin-Mastromatteo, Uribe-Tirado & Romero-Ortiz, 2016; Strielkowski, 2017). Our results showed that using Sci-Hub makes it possible to drastically reduce the inequalities of access to “paywalled articles” in the institutions studied, in terms of number of accessible full texts and the money spent to buy them on publishers’ websites. As already pointed out by Bendezú-Quispe et al. (2016) and Machin-Mastromatteo, Uribe-Tirado & Romero-Ortiz (2016), our data show that Sci-Hub can help clinicians in ophthalmology working in less privileged institutions or countries worldwide by allowing them to obtain essential information and respond appropriately to patient care needs. Without this option, they would not be able to cope with the demands of their profession. Nonetheless, as shown by Bohannon (2016) and Machin-Mastromatteo, Uribe-Tirado & Romero-Ortiz (2016), Sci-Hub is not only used in less privileged countries, and a correlation has been shown between the number of downloads per 1,000 inhabitants on Sci-Hub and the GDP per inhabitant (Greshake, 2016). As examples, the United States is the fifth largest downloader after Russia, and a quarter of the Sci-Hub requests came from the 34 members of the Organization for Economic Co-operation and Development, the wealthiest nations with, as shown in this study, the best journal access. The use of Sci-Hub by these countries can be defined as a use “by convenience” rather than necessity (Bohannon, 2016; Hoy, 2017; Lawson, 2017), and could easily decrease by improving library interfaces (Faust, 2016). In the long term, Sci-Hub might disrupt the whole system of academic publishing because it harms publishers due to the lost profits generated by its use (Strielkowski, 2017). In order to limit such losses, publishers could be tempted to increase their subscription rates (Russell & Sanchez, 2016), which have been steadily increasing in recent years (Hoy, 2017; Himmelstein et al., 2018), which may lead to cancellations or reduction of subscriptions by institutions worldwide (Schiermeier & Mega, 2017). This would have the effect of further accentuating inequalities. Publishers could also be tempted to further generalize “gold OA” (authors pay article processing charges (APC) to publish their articles) as the default publication model (Novo & Onishi, 2017; Strielkowski, 2017). This model is not beyond criticism because it favors the existence of predatory publishers (Beall, 2012). It also generates inequalities between authors who have funds to pay APC (whose cost is often prohibitive) and those who do not (Danda, 2014), despite many publishers providing discounts at the request of the authors or according to their geographical location. Some publishers have reacted by bringing lawsuits against Sci-Hub to shut down the site (Kemsley, 2017; Greco, 2017). So far, these attempts have been unsuccessful, and it is likely that future attempts will lead to the same outcome (Hoy, 2017). It now seems that Sci Hub will cease to operate only if and when the conditions that make it essential disappear (Lawson, 2017), i.e., that access to the articles through legal channels will not be as unequal as we have shown in this study. To achieve this, in addition to the suggestion mentioned above, several paths can be taken: including more countries in the Hinari program (particularly upper-middle income countries) (Bendezú-Quispe et al., 2016); promoting Green OA (self-archiving in OA repositories) by setting up institutional or national OA policies (Kirsop & Chan, 2005; Machin-Mastromatteo, Uribe-Tirado & Romero-Ortiz, 2016), (e.g., as it has been done recently in France) (Boudry & Durand-Barthez, 2017); and implementing subscription–based access for a reasonable price. This has already been done in other areas, particularly in the field of music, “offering individual subscription-based access to all articles from all imaginable databases for a price that most scientists in any corner of the world could afford” (Strielkowski, 2017).