Background: In general, journals can be divided in three categories: subscription-model, open-access, and hybrid (that is, open-access by choice). One measure of an article's impact is the number of citations it receives after publication. Open-access publishing may make articles more widely available because there is no financial barrier to a reader seeing the full-text version. As a result, we wondered whether articles published in fully open-access journals would be more likely to be cited than articles in other kinds of journals.

Questions/purposes: We assessed the yearly number and proportion of poorly cited articles published in orthopaedic journals and compared the proportion of poorly cited articles that were published in subscription-model journals with the proportion of poorly cited articles that were published in open-access and hybrid journals.

Methods: We identified all original articles (n = 135,029) published in orthopaedic peer-reviewed journals (n = 204) that were active from 2002 to 2012 and indexed in the Scopus® citation database. For each journal, we recorded the type of access (subscription-model, open-access, or hybrid journal), their most-recent CiteScore, the number of well- and poorly cited articles per year (more than five versus five or fewer citations after publication) calculated from the date of publication until December 31, 2017 (data collection April 2018), and then calculated if the journals themselves were poorly cited per year (defined as journals that published 75% or more articles ranked as poorly cited per year). We compared the proportion of poorly cited articles in subscription-model journals with the proportion of poorly cited articles in open-access journals. Additionally, we compared these with hybrid journals.

Results: In total, 48,133 (36%) articles were classified as poorly cited. The total number and proportion of poorly cited articles increased over the years, from 2121 of 7860 (27%) in 2002 to 6927 of 16,282 (43%) in 2012. The proportion of poorly cited articles in subscription-model journals increased from 226 of 395 (57%) in 2002 to 411 of 578 (71%) in 2012. The proportion of poorly cited articles in open-access journals decreased from 264 of 434 (61%) in 2002 to 296 of 801 (37%) in 2006, and then increased again to 1387 of 2259 (61%) in 2012. When we compared yearly proportions of poorly cited articles in subscription-model versus open-access journals using Mann-Whitney U tests, we only found a difference in 2012, with a higher proportion of poorly cited articles in subscription-model journals that year (median [IQR] of poorly cited article proportions for open-access, 0.61; IQR, 0.38-0.96 and subscription-model journals, 0.92; IQR, 0.54-1.0; p = 0.049). Comparisons of poorly cited articles for all three types of access showed lower proportions of poorly cited articles in hybrid journals for each year, with the lowest proportion found in 2002 (0.20; IQR, 0.09-0.67; p = 0.003).

Conclusions: We found no difference in the likelihood that an article would be cited based on whether the article appeared was published in a subscription-model journal or an open-access journal. A future study might compare open-access and paywall articles on similar topics published in the same journal or investigate the characteristics of poorly cited articles, so that researchers and editorial staffs might understand which topics are more impactful and to determine if any important work is less-well appreciated. Additionally, an article-by-article analysis will provide more insight in citation rates for articles published within hybrid journals.

Level of evidence: Level III, therapeutic study.