Fareed H. Abdulahad 1, M.B.Ch.B., M.Sc.
Nazar P. Shabila 2, M.B.Ch.B., M.Sc.

1. Department of Anatomy, College of Medicine, Hawler Medical University, Erbil, Iraq - fareed.hanna@hawlermu.org
   
2. Department of Community Medicine, College of Medicine, Hawler Medical University, Erbil, Iraq - nazar.shabila@hawlermu.org

Correspondance
Nazar P. Shabila
Department of Community Medicine, College of Medicine, Hawler Medical University, Erbil, Iraq
Email: nazar.shabila@hawlermu.org
Tel.: 00964- 750- 4450611

The quality of a journal can be assessed from different perspectives and through a variety of ways depending on the purpose of the evaluation. The well-known and commonly used evaluation measure is the journal impact factor¹. This evaluation measure is based on average indicators and is obtained through measuring the average number of citations to an article published during the previous two years. The impact factor is actually based on objective and output-related concepts such as the volume and intensity of citations or the yearly number of published articles. It is often used by universities, institutions and research organizations to assess researchers, projects and proposals^2-4.

The other method of journal evaluation is assessing the perceptions of the researchers and academics about the quality of a journal. Researchers and academics have specific preferences with respect to the journals they would like to publish in, the journal they would like to regularly read and the representative of the journal to their specific fields of interest. In survey-based methodology a journal can be evaluated by asking researchers and academic for their views on the journals' quality^2,5.

Thus the main applied approaches to evaluating journals can be broadly divided between citation-based studies or perception-based analyses. The medical literature is rich with studies that use citation-based methods to rank journals. However, cases that use survey-based methodologies to study perceptions are in scarcer supply^2,4,6.

There are three locally published medical journals in Iraqi Kurdistan region, one per each medical school from the three governorates in the region. Zanco Journal of Medical Sciences (Zanco J Med Sci) is the oldest of these journals and is issued by Hawler Medical University in Erbil governorate. It includes research produced from the colleges of medicine, dentistry, pharmacy and nursing. According to our best knowledge, no formal evaluation has been conducted to the locally published medical journals in Iraqi Kurdistan region including Zanco J Med Sci. For this reason, we have asked a number of medical doctors and academics in the region about how they appreciate and evaluate the quality and the impact of this journal, its impact on their fields of activities, and how the journal can be improved. The survey's results can improve our understanding about the role of locally published medical journals and can be used to improve the impact of these journals.

A questionnaire was administered to collect data on perception of the quality of Zanco J Med Sci. The questionnaire was developed by reviewing relevant literature, conducting personal interviews and through expert consultation. The questionnaire was self-administered and consisted of two parts: 1) independent variables including personal, educational and employment factors; and 2) dependent variables including information about their subscription, reading and contribution to the journal as well as their evaluation of the quality of the journal and suggestions to improve this quality.

Some 315 copies of the questionnaire were dispatched to all medical schools and departments of health in the three governorates of Iraqi Kurdistan region through mail survey with a request to dispatch these copies to medical doctors and academic staff working there through a convenient method. It was not possible to select a random sample of the study population due to lack of lists and expected difficulties in reaching individual persons through mail survey. The questionnaire was dispatched on November 12th 2009. The participants were advised to complete the questionnaire and send it back to the authors through the same mail. We waited until May 11th, 2009 to receive responses.

Stata version 9.1 was used for statistical analyses. Statistical methods used included frequency and percentage for univariate analysis. Pearson's chi-squared test and Fisher's exact test were used for bivariate analysis according to their applicability to examine the relationships between independent and dependent variables. A p-value of 0.05 and below was considered statistically significant result.

For analytical purposes, the respondents were categorised to specialties of basic medical sciences and clinical medical sciences as well as to university employees and department of health (DoH) employees.

A total of 144 (45.7%) of 315 eligible participants responded to the survey (Table 1). The basic characteristics of the respondents included; 75 (52.1%) respondents were from Erbil governorate versus 69 (47.9%) respondents from the other governorates, 31 (21.5%) respondents with educational and working background in the fields of basic medical sciences versus 113 (78.5%) from the fields of clinical medical sciences, and 127 (88.2%) respondents working for universities versus 17 (11.8%) working for the DoH. Only 53 (36.8%) of respondents were holding administrative or policy-making positions including 49 (92.5%) in the university and 4 (7.5%) in the DoH.

The results showed that 43 (29.9%) respondents subscribed to the journal and 71 (49.3%) were receiving the journal of which 34 (47.9%) were receiving it regularly and 37 (52.1%) were receiving it irregularly. A statistically significant higher proportion of respondents from Erbil governorate, those working for the university, those with background in basic sciences and those holding administrative positions were subscribed to and receiving the journal than those from Sulaymaniyah governorate, those working for the DoH, those with background in clinical sciences and those not holding administrative positions respectively (Table 2).

Seventy seven (53.5%) of respondents used to read the journal, of which 67 (87.0%) read it in print format, 3 (3.9%) read it in electronic format while 7 (9.1%) read it in both formats. A statistically significant higher proportion of respondents from Erbil governorate, those working for the university, those working in the fields of the clinical sciences and those holding administrative positions were reading the journal than those from Sulaymaniyah governorate, those working for the DoH, those working in the fields of basic sciences and those not holding administrative positions respectively (Table 2)

Out of 77 respondents who read the journal, 9 (11.7%) read the whole journal, 33 (42.8%) review the contents page of the journal and read selected articles, 24 (31.2%) read the abstracts and read selected articles and 11 (14.3%) look for interesting illustrations and read those articles.

Out of 36 respondents that hold administrative or policy making positions and read the journal, 19 (52.8%) find useful research results in the journal that one can use for health administration or health policy decision making. Only 2 (5.6%) respondents out of 36 of those who hold administrative or policy making positions and read the journal, had made decisions based on research findings from the journal. There were no statistically significant differences between different characteristics of the respondents and finding and using useful research results (Table 3).

Sixty (77.9%) respondents from those who read the journal wish to see review articles, 16 (20.8%) wish to see correspondence, 29 (37.7%) wish to see short reports and 15 (19.5%) wish to see editorials in the journal.

In terms of content quality, 5 (6.5%) respondents rate it as very good, 65 (84.4%) respondents rate it as satisfactory and 7 (9.1%) respondents rate it as unsatisfactory. In terms of review quality, 11 (14.3%) respondents rated it as very good, 59 (76.6%) respondents rated it as satisfactory and 7 (9.1%) respondents rated it as unsatisfactory. Those who do not hold administrative positions were more positive about the review quality than those who hold such positions (97.6% versus 83.3%, p=0.03). Only 18 (23.4%) respondents had contributed to the journal in past while 56 (72.7%) respondents consider contributing in the future. The detailed associations between the respondents' perception of the journal quality and their contribution with their different characteristics are shown in Table 4.

Periodic evaluation of a journal's quality is necessary to identify its shortcomings and this can help in improving the journal quality. Journal evaluation based on surveys of academics' perceptions of journals is a recognized and well-documented method. However, it has been criticized because of the difficulty in eliciting clear and consistent information on preferences from a group of individuals and aggregating these into representative measures of preference⁷. The perceived quality of a journal will be undertaken by individuals using different conceptions of what constitutes a good journal. This is in accordance with the nature of quality as a poorly defined construct⁸.

The relatively low response rate in this survey is attributed to using mail-based survey. The mailing system in the region does not function properly as it involves long delays. Similarly, there is very limited experience with mail surveys in the region and the country. It is well-documented that mail surveys are generally associated with low response rates^8,9. The considerably higher response within Hawler Medical University might be due to the fact that the survey only used the university mailing system without the need to use the governmental system. The involvement of the governmental mailing system is an important factor in having severe delays and a lower response rate from the other governorates. The other reason for low response from DoH and the other governorates might be due to lack of effect of the journal in and/or not dispatching sufficient numbers of the journal to the DoH and the universities from the other two governorates.

The respondents who claimed that they are subscribed to the journal were around 30%. These were mainly from the issuing university faculty members as most of them are automatically subscribed to the journal through the university regulations. However, a number of these faculty members stated that they are not subscribed to the journal indicating that they are not aware of their automatic subscription.

Generally, a relatively low proportion of respondents receive the journal. However, such rate was relatively high in Erbil governorate and that may be attributed to their university subscription. The relatively high proportion that claimed that they receive it irregularly might be attributed to their unawareness that the journal is issued only twice yearly and the fact that the journal used to face delays in its publishing. The significant difference between university employees and DoH employees in terms of receiving the journal indicate that the journal is more widely distributed in the university due to higher subscription rate. The higher proportion of those with basic sciences background being receiving the journal is related to their more regular presence in the colleges and subsequently better access to the journal whereas those in the fields of clinical sciences are usually away from the college working in teaching hospitals. This may indicate also poor distribution of the journal in teaching hospitals.

Having around 70% of respondents from Erbil governorate reading the journal is relatively low as the journal is the only local medical journal in the governorate. Reading the journal is mainly in print format with very low proportion of respondents reading it in electronic format. This can be due to unavailability of a specific website for the journal and having each complete volume of the journal available on the university website as one large sized document. The higher proportion of readers from the university staff and those with basic sciences background in comparison to DoH staff and those with clinical sciences respectively is again related to the reported differences in subscription and receipt of journal.

As the journal covers different fields of basic and clinical sciences of the different medical sciences, it is an expected finding that the highest proportion of readers review contents page and read selected articles followed by those who read abstracts and read selected articles.

While a considerable proportion of respondents who hold administrative positions and read the journal find useful research results that can be used for health administration or health policy decision making, these were all from university employees. Even with this considerable proportion, only two respondents claimed that they had made decisions based on research findings from the journal. This may indicate non-representation of published research to the needs of the community or poor use of research by managers and policy makers.

A very high proportion of respondents wish to see review articles in the journal and a good proportion wish also to see correspondence, short reports and editorials. The journal is in fact very much focused on publishing original studies, which is mainly due to that fact that authors prefer to contribute with original studies that are required for their scientific promotion. However, other types of papers are also interesting to read.

Both the content and the review qualities were generally rated as satisfactory with no significant rating differences by those with basic and clinical sciences backgrounds, or university and DoH employees. Rating the journal as very good or unsatisfactory was limited to a small number of respondents with having higher rating as very good for review quality than contents quality. This may indicate that while the journal is currently in a satisfactory shape, there might be important opportunities for improvement.

A relatively low proportion of respondents have contributed to the journal even if most respondents were university teaching staff. This may be due to low research production in the region. No significant differences were reported between those with basic and clinical sciences backgrounds or the university and DoH employees. The higher contribution among DoH employees can be attributed to the very small sample size of those contributed from DoH (n.=2).

An important limitation to this study is the low response rate especially from those not working for universities. This will introduce bias and may reduce the significance of results obtained. Another important limitation is that the inability to select a random sample of the study population for inclusion in the study to be representative, limits generalizability of the findings.

While the journal's contents and review quality were generally rated as satisfactory with presence of important opportunity to further improve them, the rate of reading the journal and journal's impact is below a satisfactory level. The accessibility of the journal to academics and medical professions needs improvement in both the print and online formats.

1. Garfield E. Citation analysis as a tool in journal evaluation. Science1972 Nov 3;178:471-9.
   
2. Rousseau S. Journal evaluation by environmental and resource economist: a survey. ETE Working Paper Series 2007 Nov;5:1-16.
   
3. Saha S., Sanjay S. Christakis DA. Impact factor: a valid measure of journal quality? J Med Libr Assoc 2003 Jan;91(1):42-6.
   
4. Garfield E. The history and meaning of the journal impact factor. JAMA 2006 Jan; 295(1): 90-3.
   
5. Rousseau R. Journal evaluation: Technical and practical issues. Library Trends, 2002 Winter; 50(3): 418-39.
   
6. DeLisa JA. Evaluating journal quality - The role of citation data and the impact factor. Journal of Spinal Cord Medicine 2004; 27(1): 1-1.
   
7. Milne MJ. Toward the end of academic freedom, diversity, judgment and accountability: A critique of Cassar and Holmes (1999) Journal yardsticks. Accounting, Accountability and Performance 2000; 6(1): 99-116.
   
8. Lowe A, Locke J. Perceptions of journal quality and research paradigm: results of a web-based survey of British accounting academics. Organizations and Society 2005 Jan; 30(1): 81-98
   
9. Green J and Browne J. Principles of social research. Open University Press; UK: 2006.