Introduction

Publishing is an essential part of scientific research. However, printing a paper is not sufficient - it is also necessary that the work be read by the broadest possible audience, and that it be cited by others. Because of that, most journals work hard to get indexed in international databases, based on the common sense conviction that indexing will naturally be followed firstly by worldwide visibility, and thereafter by increased citation. (1-3)

However, evidence of this is lacking in the literature. In fact, a thorough search returned only editorials rejoicing over a given journal having been indexed in a particular database, or commenting on the difficult path toward achieving such goal and on the prospects considering this high-valued accomplishment. (4-7) We retrieved a single cursory follow up evaluation which in effect showed that the number of citations of articles published in five Brazilian journals more than doubled (as gauged by the Institute for Scientific Information's impact factor) after their inclusion in the Scientific Electronic Library Online (SciELO). (1)

Therefore, we set out to evaluate the impact of the Jornal de Pediatria's inclusion in SciELO in 2002 (8) and in MEDLINE in 2003, (9) by taking the rate of article submission as a proxy indicator of visibility and attributed quality.

Methods

We analyzed total article submission to Jornal de Pediatria from 2000 through 2004. Secondary outcomes were submission of articles from foreign countries and acceptance figures for the same period.

Since there were no changes in either the editorial board or the methods of manuscript submission throughout the study period, we considered the potential impact of three events on submission rates: the first was the launch of our bilingual (Portuguese/English) free-access website, in March 2001; the second was the indexing in SciELO, in July 2002; and the third was the indexing in MEDLINE, in August 2003. Thus, four stages were considered: stage I, which we called "pre-website" (15 months), stage II, "website" (16 months), stage III, "SciELO" (13 months), and stage IV, "MEDLINE" (16 months).

Statistical analysis comprised simple regression for trend analysis, one-way ANOVA on rank transformed data with Duncan's posthoc test to compare the number of submissions in each period, and Fisher's Exact test with Finner-Bonferroni p-value adjustment to compare foreign submissions in the four periods.

Results

There was a significant trend toward linear increase in the number of submissions to Jornal de Pediatria along the study period (p = 0.009) (Figure 1). Table 1 shows the number of articles submitted in each category in stages I through IV. The number of submissions was not statistically different in stages I and II (p = 0.148), but it was statistically higher in stage III (p < 0.001 vs. stage I and p = 0.006 vs. stage II) and stage IV (p < 0.001 vs. stages I and II, and p < 0.05 vs. stage III). The upward variation in the submission of letters to the editor was the most striking among the article categories, followed by original papers.

Figure 1 -
Jornal de Pediatria: number of articles submitted and accepted per month, 2000 and 2004

Table 1 -
Jornal de Pediatria: Submissions and acceptance rates, per stage, 2000 through 2004

There was an erratic and pronounced fluctuation in monthly submissions in stages III and IV (Table 2).

Table 2 -
Jornal de Pediatria: Mean monthly submissions per stage, 2000 through 2004

Since the absolute number of accepted articles per issue remained stable throughout the study period, the rate of acceptance naturally decreased. This was most pronounced for case reports.

The number of foreign articles submitted in stages I through IV was 1, 2, zero and 17, respectively, with p < 0.001 for the comparison of stage IV with previous stages.

Discussion

We observed an unequivocal association between SciELO indexing and a rise in manuscript submissions from Brazil to Jornal de Pediatria, while MEDLINE indexing clearly led to an increase in both Brazilian and foreign submissions. These findings substantiate the empirically held views that authors submit more papers to journals that are indexed in international databases. (1,10-16)

Authors prefer indexed journals because of their visibility, and also because research funding agencies attribute special value to those publications. (10,17) In Brazil, this is one of the major criteria employed by the Federal Coordinating Agency for the Improvement of Higher Education (CAPES) to rank graduate programs. CAPES classifies journals by means of a system known as Qualis, according to which top journals are indexed in MEDLINE and have a high impact factor, as measured by Thomson Scientific's Journal Citation Reports (JCR), (18,19) next come MEDLINE-indexed journals with a low impact factor; and third are journals indexed in MEDLINE but not covered by the JCR.

The fact that research funding agencies in Brazil, and perhaps in other countries as well, value above all coverage by the JCR deserves further remarks. First of all, many have pointed out that counting citations is a flawed way to appraise academic worth, among other reasons because it does not measure citations to any single article. (20-25) Second, if an article is cited frequently as an example of poor research, it will nevertheless raise the journal's impact factor. In addition, journals with a print version in a language other than English will hardly get coverage from organizations like Thomson-ISI, allegedly because their impact factor is too low - which is probably the case, since international readers will most likely not cite articles published in a language that they cannot read. It should be noted that the number of MEDLINE citations to English language articles has increased from about half the total records in that database before the 1970s to nearly 90% today. (26)

Moreover, whenever MEDLINE cites a non-English article (meaning an article whose original print version is in any language other than English, irrespective of an English version being provided), its title is listed between square brackets. So, although Jornal de Pediatria - like many journals officially indexed as "non-English" - does provide a link to free English full text, it is fair to presume that those brackets alone cause too many readers to shy away. Two interesting questions that stem from this scenario are: how many investigators searching MEDLINE use language filters, or how many will access and cite articles whose titles appear in square brackets? And - would ISI coverage raise a journal's the impact factor, just as MEDLINE indexing raises submissions?

We are certain that Jornal de Pediatria has been publishing higher-quality articles in preparation for and because of indexing. This partially explains the striking decrease in the rate of acceptance, which has halved along the study period. The number of original articles published has been stable since the 2001 March/April issue, when Jornal de Pediatria reached its printed page limit. This, together with the increase in submissions, led to stricter judgment criteria, and the rate of article acceptance naturally decreased. This effect was most pronounced for case reports, indicating a change in Jornal de Pediatria's characteristics, with higher priority being placed on original research papers.

In conclusion, we have shown that the indexing of Jornal de Pediatria in SciELO and MEDLINE increased its worldwide visibility, as well as an overall perception of attributed quality, thus stimulating researchers to submit their manuscripts for publication.