In this paper we present the first comprehensive bibliometric analysis of eleven open-access mega-journals (OAMJs). OAMJs are a relatively recent phenomenon, and have been characterised as having four key characteristics: large size; broad disciplinary scope; a Gold-OA business model; and a peer-review policy that seeks to determine only the scientific soundness of the research rather than evaluate the novelty or significance of the work. Our investigation focuses on four key modes of analysis: journal outputs (the number of articles published and changes in output over time); OAMJ author characteristics (nationalities and institutional affiliations); subject areas (the disciplinary scope of OAMJs, and variations in sub-disciplinary output); and citation profiles (the citation distributions of each OAMJ, and the impact of citing journals). We found that while the total output of the eleven mega-journals grew by 14.9% between 2014 and 2015, this growth is largely attributable to the increased output of Scientific Reports and Medicine. We also found substantial variation in the geographical distribution of authors. Several journals have a relatively high proportion of Chinese authors, and we suggest this may be linked to these journals’ high Journal Impact Factors (JIFs). The mega-journals were also found to vary in subject scope, with several journals publishing disproportionately high numbers of articles in certain sub-disciplines. Our citation analsysis offers support for Björk & Catani’s suggestion that OAMJs’s citation distributions can be similar to those of traditional journals, while noting considerable variation in citation rates across the eleven titles. We conclude that while the OAMJ term is useful as a means of grouping journals which share a set of key characteristics, there is no such thing as a “typical” mega-journal, and we suggest several areas for additional research that might help us better understand the current and future role of OAMJs in scholarly communication.

Data Availability: The data are owned by and have been obtained from Elsevier’s Scopus database. Any researcher with access to the Scopus database can obtain the data using the methods described in the paper. Readers that do not have access to the Scopus database should contact Elsevier to obtain a license. As per the Elsevier terms of use, the authors may also be able to provide limited access to data, subject to Elsevier’s agreement.

The next section of the paper describes the criteria that we have adopted to select OAMJs for analysis, and the data collection processes. We then present results for a single mega-journal–BMJ Open–primarily as a means of introducing the modes of analysis employed in this study. In the following parts we structure our analysis of mega-journals in four sections, each relating to an area of investigation–output, author characteristics, subject areas, and citation analysis.

This paper seeks to establish the bibliometric profiles of eleven leading OAMJs as of early 2016. It is the first such overview of the field and thus establishes a baseline for future studies as these, and other, OAMJs evolve over the coming years. The purpose of the paper is to explore systematically the following four characteristics of mega-journals, all of which are commonly used in bibliometric analyses [ 2 ]:

The newness of OAMJs has meant that quantitative studies of them are far less common than of conventional journals, and we have been able to identify only four such publications to date. The first of these, by Fein [ 4 ], appeared in 2013 and was based on the analysis of a set of 28,252 documents (comprising not just articles, but also reviews and editorial matter etc.) that had been published in PLOS ONE in the period 2007–2011. This set was evaluated in terms of five different types of criterion, viz journal output, journal content, journal perception, journal citations, and journal management. Each of these criteria was considered using one or more metrics, with results presented for the numbers of articles per month, authors’ countries, tag clouds based on the words comprising article titles, the citing article’s author’s country and document type, citation rates broken down by document type and year, the time from submission to publication, and the composition of the editorial board. Burns [ 5 ] studied a small sample of 49 articles published in the first few months after the launch of PeerJ in 2012, focusing on the journal’s peer review procedures, author demographics, usage data (as estimated by altmetrics such as article downloads and social media references) and citations to the articles (where the wide range of journals citing the sample articles suggested that the latter are highly varied in the subjects that they discuss). The study by Solomon [ 6 ] involved a Web-based survey of 2,128 authors who had published in BMJ Open, PeerJ, PLOS ONE or Sage Open. The survey focused on the characteristics of mega-journal authors and of the papers they had submitted for publication, the reasons for their choice of journal, and their approach to the payment of APCs. Finally, Björk and Catani [ 7 ] compared the distributions of citations to articles published in PLOS ONE and Scientific Reports with the corresponding distributions for articles from several conventional journals (where the review process requires consideration of novelty and significance when deciding which articles should be accepted for publication). Little difference was observed in the two sets of distributions, leading the authors to wonder whether “simple, soundness-only” refereeing might be more widely adopted.

The radical nature of these differences has the potential to bring about substantial changes in the journal publishing industry and in scholarly communication more generally, and there is already a small but growing body of literature associated with various aspects of the OAMJ phenomenon [ 1 ]. In this article, we present a quantitative analysis of eleven OAMJs using established bibliometric methods that have been developed to describe the publication and citation profiles of conventional journals. These methods are discussed by Anyi et al. [ 2 ], who note that a bibliometric journal study provides a quantitative portrait of that journal in terms of characteristics such as growth of the journal over its lifetime, the geographic distribution of contributions to it, the most prolific authors and institutions, the most heavily cited articles (and why these articles had attracted so much interest) and the extent to which the journal is cited by journals outside its own particular subject domain inter alia. Journal analyses have been reported across a very wide range of disciplines: the reviews of Tiew [ 3 ] and of Anyi et al. [ 2 ] summarised nearly 200 such studies that had been published in the period 1969 to 2008.

Following the advent in 2006 of PLOS ONE, the very first open-access mega-journal (OAMJ), the last few years have seen the arrival of many others, such as AIP Advances, the Open Library of the Humanities, SAGE Open and Scientific Reports. An OAMJ is an open-access, online-only, peer-reviewed publication that is normally funded through pre-publication article processing charges (APCs). This is, of course, no different from many other open-access (OA) journals, but OAMJs have two further characteristics that differentiate them from conventional OA publications. Firstly, they cover very broadly defined subject domains, as against the highly specific focus that characterises a typical academic journal, whether OA or non-OA. Related to this is the scale of their operations, with some (but by no means all) OAMJs publishing far greater numbers of articles each year than would appear in a conventional journal. Secondly, they adopt a different approach to peer review, eschewing traditional approaches to selection on grounds of novelty and/or significance and instead publishing all submissions that the reviewers agree are technically sound. In this respect, an OAMJ can be regarded as a distribution mechanism rather than having the gatekeeper role associated with top-tier journals.

Initial descriptive statistics were calculated in MS Excel ® , with subsequent statistical analysis conducted using IBM SPSS Statistics v22 ® . Formal statistical tests (the independent-samples t-test, Welch analysis of variance (ANOVA), and the post-hoc Games-Howell test) were used when appropriate. Graphical representations of cumulative citation distributions were created in MS Excel ® , and show curves smoothed with standard Excel smoothing (based on a Catmull-Rom spline).

With the exception of some figures relating to subject area coverage, which were obtained directly from the relevant OAMJ websites, all data for this paper were obtained through Scopus.com. As previously discussed, Scopus was found to have slightly greater coverage of OAMJ titles than Web of Science. While research has shown that Scopus covers a higher proportion of journals in general than Web of Science, it is important to note the variations in coverage for different subject areas [ 15 ]. In particular, the coverage of social science and arts and humanities journals has been found to be significantly lower than for titles in the natural sciences and biomedical fields. Four forms of Scopus data were collected for the study in March 2016: the bibliographic records for articles published in the eleven mega-journals; aggregations of institutional affiliations and associated nationalities for each OAMJ, available through the “Analyze search results” function of Scopus; “Citation overviews” for each OAMJ, which includes the number of citations by year for each article published in the journal; and aggregations of Journal Name and Subject Area for citing articles, again available through the “Analyze search results” function. A limitation of Scopus is that a number of functions, including the downloading of citation data, are only available up to a maximum of 20,000 lines of data. A series of filters were therefore used to segment the outputs for high volume journals (e.g. PLOS ONE and Scientific Reports), and the resulting downloads subsequently merged. One important point to note is that Scopus data relating to author nationality is based on the location of the author’s affiliated institution. Thus results and discussion relating to “author nationality” in this paper should be understood in this context.

An initial list of 63 potential OAMJs for the present study was obtained from literature sources, personal knowledge, and publisher information. Of these, 20 were identified that satisfied all of the four primary criteria listed above. However, in the context of the bibliometric analyses to be conducted here, there is a further, essential, criterion: the OAMJ must be indexed by Web of Science and/or Scopus, the two principal sources of curated publication and citation data. Furthermore, in order to be able to conduct meaningful citation analysis, we required these data to be available from 2013 or earlier (thereby excluding mega-journals launched in 2014 or later). Of the two databases, Scopus was found to provide better coverage, with data available for 11 of the 20 OAMJs, viz AIP Advances, BMC Research Notes, BMJ Open, F1000, FEBS Open Bio, Medicine, PeerJ, PLOS ONE, SAGE Open, Scientific Reports and SpringerPlus. The principal characteristics of these journals are detailed in Table 1 (with much greater detail on them relating to all of the Björk criteria being provided as Supporting Information S1 Appendix ). Of these, three journals perhaps merit special mention. Although founded in 1922, the journal Medicine was until mid-2014 a traditional, highly selective subscription journal publishing between 30 and 50 articles per year. It then transitioned to a mega-journal model, with a “soundness only” review policy and a gold open-access economic model. It therefore offers a unique opportunity to evaluate how the shift to a mega-journal model has affected its bibliometric profile. F1000 Research operates a post-publication peer-review model, whereby all submissions that pass initial in-house checks are published, with formal peer-reviews added later by members of the F1000 community. Only articles that have received two “Approved” or one “Approved” plus two “Approved with Reservations” reviews are submitted for indexing in databases such as PubMed, Web of Science and Scopus. While some commentators suggest this model aids the publication of poor quality science [ 14 ], only articles indexed in Scopus, and therefore that have received two positive peer-reviews, are included in our analysis. Finally SpringerPlus is included in our analysis, despite Springer announcing in June 2016 that the journal was to close, with no new submissions being accepted. It should also be noted that while the eleven journals mentioned above are the focus of this paper, data relating to a number of non-mega-journals are included in the analyses for comparison purposes.

We have summarised the characteristics of OAMJs in the previous section but it is less easy to provide a precise definition, since different authors describe them in different ways [ 8 – 11 ], and some that have been described as mega-journals have occasioned concerns as to the nature of the peer review processes that were used [ 12 , 13 ]. The criteria put forward by Björk [ 10 ] provide arguably the most comprehensive way of describing an OAMJ, and we have used a modified form of these in the selection of the journals that are discussed in the main body of this paper. Björk presents two sets of criteria to characterise OAMJs. The four primary criteria are as follows: big publishing volume (or at least aiming to achieve this); peer review that takes account of scientific soundness only, without consideration of originality or significance; a broad subject area; and full open access with publication funded by APCs. Björk suggests that a journal must satisfy all of these primary criteria if it is to be regarded as an OAMJ, and it should additionally satisfy at least some of seven secondary criteria: a moderate level of APCs (for which a figure of $1500 or less is suggested); a reputable, high-prestige academic publisher; editorial control by academic, rather than publisher, editors; use of Creative Commons licenses so that materials (graphics, data etc.) can be re-used by readers without the need to obtain formal permission from the author(s); the use of altmetrics to provide post-publication evaluation of significance; provision of facilities to enable readers to input comments on the article; the inclusion of reviews–sometimes referred to as “portable reviews”—from other journals that had rejected the article prior to its publication in the OAMJ; and rapid publication (for which a figure of less than six months from submission is suggested).

Consideration was also given to the quality of citing journals. This was estimated by calculating the weighted mean SNIP value for the top 50 citing journals. As shown in Table 2 , in the case of BMJ Open, this was found to be 1.303, compared with 1.823 for PLOS Medicine, 1.323 for BMC Public Health, and 0.757 for Experimental and Therapeutic Medicine. Since the data failed the Levene test for equality of variances, a Welch analysis of variance (ANOVA) was employed, and showed significant variation (F(3, 3052) = 430.223, p < .001). The Games-Howell test was used for post hoc testing, as recommended by Ruxton et al [ 18 ], and showed that BMJ Open was significantly different to PLOS Medicine and Experimental and Therapeutic Medicine (both p < .001), but not significantly different to BMC Public Health (p = .896).

Taken in conjunction with Fig 1 , which shows the cumulative citation frequency for each journal, we observe that BMJ Open presents a citation pattern very similar to that of BMC Public Health, a mid-ranking, relatively high-volume journal that operates with a traditional peer review policy. These two journals, together with Experimental and Therapeutic Medicine, are notably different from the longer established PLOS Medicine (which started publishing in 2004).

As of mid-January 2016, the point at which all citation analyses were conducted, BMJ Open articles had been cited a total of 16,111 times. However in order to make meaningful comparisons between the citation data for all the mega-journals under review, citation analyses were conducted only for articles published in 2013, that year being the first for which all journals under review (with the exception of Medicine) had articles indexed by Scopus. The 894 articles published in BMJ Open in 2013 have been cited 5,867 times, giving a mean number of 6.6 citations per article. Following Björk & Catani [ 1 ], we also calculated the proportion of articles with 0, 1 and 2 citations, these figures being 10.0%, 12.5% and 13.6% respectively. The total proportion of articles with two or fewer citations is therefore 36%. In order to make sense of these data, similar calculations were made for three other comparable medical journals. There were three principal criteria for selecting comparison journals–subject area (broad-scope medicine), output (they should be similar in size to BMJ Open), and journal impact (used as a proxy for quality, the comparison journals should include a top, medium and low level journal). Given the broad scope of mega-journals, and the range of disciplines covered, it was decided to use Source Normalized Impact per Paper (SNIP) rather than Journal Impact Factor (JIF) as a measure of impact. SNIP seeks to measure a given journal’s contextual citation impact, and offers a means of comparing journals across different fields [ 16 , 17 ]. In the case of BMJ Open (2014 SNIP = 1.043), the journals selected for comparison were PLOS Medicine (3.207), BMC Public Health (1.229) and Experimental and Therapeutic Medicine (0.535). Table 2 shows some general information about these journals’ output and citations, along with the proportion of articles with 0, 1 and 2 citations.

BMJ Open accepts submissions in a range of medical fields, including (but not limited to) “clinical medicine, public health and epidemiology … health services research, health economics, surgery, qualitative research, research methods, medical education, [and] medical publishing” (BMJ, 2016). Scopus does not allow for easy analysis of article subject by sub-discipline as it indexes all articles published in the journal under the “Medicine” subject area (one of 27 such broad subject areas), and the BMJ Open platform, unlike some other mega-journals such as PLOS ONE and PeerJ, does not offer sub-disciplinary classification.

Analysis of author nationality and affiliation highlights a number of key characteristics. There are 118 different author nationalities represented in the journal, with 34.0% of articles having at least one author from the UK. The USA and Australia are the next most common author nationalities, with 17.0% and 12.5% of articles respectively. No other nationality is responsible for authorship of more than 10% of articles, and analysis of author nationality by year reveals little variation over time. The most common author affiliations generally reflect the nationality findings, with the four most common (based on the proportion of articles with at least one author from that institution) all being to UK institutions (UCL = 4.2%, University of Oxford = 4.2%, King's College London = 3.0%, London School of Hygiene & Tropical Medicine = 2.8%). The University of Sydney (2.6%) is the fifth most common affiliation, with one Canadian (University of Toronto), one Swedish (Karolinska Institutet) and three further UK institutions (University of Bristol, University of Birmingham, and Imperial College London) completing the top 10. A further four UK and four Australian institutions rank between 11 and 20, but it is striking that despite being the second most common author nationality, the first USA institution (Harvard School of Public Health) ranks only 31 st . This is perhaps a consequence of the much greater number of US institutions, so that submissions are spread across a broader number of research centres.

We begin by presenting an analysis of one prominent mega-journal–BMJ Open. This is intended primarily to introduce the methods employed in our comparison of the eleven selected mega-journals, while also offering a profile of a journal that by two basic measures—article output and age as listed in Table 1 –is broadly representative of the majority of the selected journals.

Results and Discussion

Mega-Journal Output The total output to the end of 2015 of all eleven mega-journals is 178,075 articles, of which 44,820 were published in 2015. This represents an increase of 15% on the 2014 figure (38,995). Scopus has indexed a total of 1,826,143 articles published in 2015, meaning that the combined output of the mega-journals under review accounts for 2.5% of all articles. Fig 2 plots the annual output of articles for each of the eleven OAMJs using a logarithmic scale. It should be noted here that since the journals were launched at different times of the year, output data for the first year of publication does not represent a full 12 month period, and is therefore best excluded from any analysis of growth. An initial review of growth suggests that for a number of journals an initial period of relatively fast growth is followed by a levelling off or slight decline in publishing output. PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Fig 2. Mega-journal output by year (base 10 logarithmic scale used for y-axis). https://doi.org/10.1371/journal.pone.0165359.g002 Table 3 presents the articles published in each OAMJ in 2015, and the percentage change since 2014. PLOS ONE’s output, which since 2013 has been around 30,000 articles per year, clearly dwarfs all other titles, with the recent exception of Nature’s Scientific Reports, which published 10,600 articles in 2015. While PLOS ONE’s output has declined by around 4,000 articles since its high of 31,404 in 2013, the concurrent growth of other mega-journals–most notably Scientific Reports and Medicine—has ensured that total OAMJ output has grown each year since 2006. PPT PowerPoint slide

PowerPoint slide PNG larger image

larger image TIFF original image Download: Table 3. OAMJ Output in 2015, and % change from 2014. https://doi.org/10.1371/journal.pone.0165359.t003 Table 3 suggests it might be reasonable to separate the eleven OAMJs into three groups: 1) those publishing over 10,000 articles in 2015 (PLOS ONE and Scientific Reports); 2) those publishing between 777 and 1694 articles in 2015 per year (Medicine, BMJ Open, BMC Research Notes, AIP Advances, SpringerPlus, and PeerJ), and 3) those publishing 215 or fewer (SAGE Open, F1000 Research, and FEBS Open Bio). It is perhaps instructive to compare the output of OAMJs with other journals. We note that of the 21,862 journals indexed in Scopus, only 3 (PLOS ONE, RSC Advances, and Scientific Reports) published more than 10,000 articles in 2015, while 134 journals (0.6%) published more than 1,000 articles and 420 (1.9%) more than 500 articles. Only 174 (0.8%) published more than the smallest group-2 journal—PeerJ’s 777 articles. We also note that one of the group-3 journals, SAGE Open, while substantially smaller than most of the other OAMJs listed here, is nonetheless the 16th largest journal assigned to the “Social Science” subject area in Scopus. It is also instructive to consider the range of growth and contraction figures presented in Table 3. To put these figures in context, the total number of articles in Scopus is 0.9% lower in 2015 than 2014, a function perhaps of delays between article publication and their indexing in Scopus. While overall OAMJ output is rising, five of the eleven titles produced fewer articles in 2015 than 2014, including each of the three smallest OAMJs. The most striking increase in output belongs to Medicine, and is a function of the journal’s transition midway through 2014 from a small selective journal to a mega-journal model. The 1,694 articles it published in 2015 were more than its total output for the previous 50 years, and the journal now ranks fifth largest (by output) of all journals in Scopus assigned to the “Medicine” subject area. The only other journal to more than double in size is Scientific Reports, and it is useful to note that this growth was achieved without impacting the output of Nature’s two more prestigious journals, Nature and Nature Communications (both of which show slight increases in Scopus indexed output over the same period), i.e., the OAMJ has greatly increased the article output of the Nature stable of journals. Discussion. Several issues emerge from this analysis of mega-journal output. Björk’s criterion relating to OAMJ size says that a mega-journal should have a “big publishing volume (or aiming for it)”. This leads inevitably to the question of how large a journal needs to be before it can be considered “big” (or indeed “mega”). Two mega-journals–PLOS ONE and Scientific Reports—clearly dwarf all others, and it is difficult not to view their competitors as comparatively small. The question is further complicated by disciplinary differences in journal size. Humanities and social science (HSS) journals, for example, are typically smaller than titles in the STM disciplines, again affecting notions of “bigness”. Perhaps the key question here is whether “big publishing volume” should be interpreted within the current scholarly communications context (in which case all but three of the eleven mega-journals are undeniably “big”) or instead whether the mega-journal phenomenon is intended to herald some new paradigm of PLOS ONE-size titles, in which case the current size of group 2 and 3 titles may come to be seen as nowhere near large enough to signify such a paradigm. Whether any of these titles, or indeed others not included in this analysis (for example Royal Society Open Science or Heliyon), will grow to group-1 size is open to question. Certainly it is notable that overall OAMJ output rose significantly (15%) between 2014 and 2015, despite a fall of 9% in PLOS ONE’s output. This rise is however largely attributable to just two journals–Scientific Reports and Medicine–and it is important to note that these journals have by far the largest JIFs of any of the OAMJs (5.578 and 5.723 respectively). While a purely bibliometric analysis is insufficient to prove the fact, it does appear likely that the comination of high impact factor and “objective” peer review policies make these journals particularly attractive publication venues for researchers. For this reason it will be interesting to observe future publication volumes for Medicine, should its impact factor decline (as seems likely). It remains to be seen which if any of the other OAMJs, which have in general shown relatively slow growth in recent years, are able to grow to group-1 size.