Figure 1. Strategy for searching databases. CINAHL = Cumulative Index of Nursing and Allied Health Literature, CJHP = Canadian Journal of Hospital Pharmacy.
Michelle Hung , Mark DuffettABSTRACT
Background:
Research projects are a key component of pharmacy residents’ education. Projects represent both a large investment of effort for each resident (up to 10 weeks over the residency year) and a large body of research (given that there are currently over 150 residency positions in Canada annually). Publication of results is a vital part of the dissemination of information gleaned from these projects.
Objectives:
To determine the publication rate for research projects performed under the auspices of accredited English-language hospital pharmacy residency programs in Canada and to describe the study characteristics of residency projects performed in Ontario from 1999/2000 to 2008/2009.
Methods:
Lists of residents and project titles for the period of interest were obtained from residency coordinators. PubMed, CINAHL, the Canadian Journal of Hospital Pharmacy , and Google were searched for evidence of publication of each project identified, as an abstract or presentation at a meeting, a letter to the editor, or a full-text manuscript. The library holdings of the University of Toronto were reviewed to determine study characteristics of the Ontario residency projects.
Results:
For the objective of this study relating to publication rate, 518 projects were included. The overall publication rate was 32.2% (60 [35.9%] as abstracts and 107 [64.1%] as full-text manuscripts). Publication in pharmacy-specific journals (66 [61.7%] of 107 full-text manuscripts) was more frequent than publication in non-pharmacy-specific journals. The publication rate of projects as full-text manuscripts remained stable over time. Of the 202 Ontario residency projects archived in the University of Toronto’s library, most were cohort studies (83 [41.1%]), and the most common topic was efficacy and/or safety of a medication (46 [22.8%]).
Conclusions:
Most hospital pharmacy residents’ projects were unpublished, and the publication rate of projects as full-text manuscripts has not increased over time. Most projects were observational studies. Increasing publication rates and creating a central database or repository of residency projects would increase the dissemination and accessibility of residents’ research.
KEYWORDS: publication rate , hospital residency , research , pharmacy education , study characteristics
RÉSUMÉ
Contexte :
Les projets de recherche sont un élément clé de la formation des résidents en pharmacie. Ils représentent à la fois un investissement important en temps pour chaque résident (jusqu’à 10 semaines au cours de l’année de résidence) et une imposante masse de recherche (compte tenu qu’il y a actuellement plus de 150 postes de résidence au Canada chaque année). La publication des résultats est une partie essentielle de la diffusion de l’information issue de ces projets.
Objectifs :
Déterminer le taux de publication des projets de recherche menés sous l’égide des programmes anglophones de résidence en pharmacie d’hôpital au Canada et décrire les caractéristiques expérimentales des projets menés dans le cadre de la résidence en Ontario entre 1999–2000 et 2008–2009.
Méthodes :
Les listes des résidents et des titres de projets de recherche pour la période en question ont été obtenues des coordonnateurs des programmes de résidence. Des recherches ont été effectuées dans PubMed, CINAHL, le Journal canadien de la pharmacie hospitalière et Google pour corroborer la publication de chaque projet recensé, sous forme de résumé ou de présentation lors d’un congrès, d’une lettre à la rédaction ou d’un manuscrit complet. Les archives de la bibliothèque de l’Université de Toronto ont été examinées pour déterminer les caractéristiques expérimentales des projets de résidence menés en Ontario.
Résultats :
En ce qui a trait à l’objectif de cette étude relatif au taux de publication, 518 projets ont été recensés. Le taux de publication global était de 32,2 % (60 [35,9 %] sous forme de résumés et 107 [64,1 %] sous forme de manuscrits complets). La publication dans des revues spécialisées en pharmacie (66 [61,7 %] des 107 manuscrits complets) était plus fréquente que dans des revues non spécialisées en pharmacie. Le taux de publication des projets sous forme de manuscrits complets est demeuré stable dans le temps. Des 202 projets de résidence archivés dans la bibliothèque de l’Université de Toronto, la plupart étaient des études de cohorte (83 [41,1 %]) et le sujet le plus courant était l’efficacité ou l’innocuité d’un médicament (46 [22,8 %]).
Conclusions :
La plupart des projets de recherche des résidents en pharmacie hospitalière n’ont pas été publiés et le taux de publication des projets sous forme de manuscrits complets n’a pas augmenté dans le temps. La plupart des projets étaient des études observationnelles. L’augmentation des taux de publication et la création d’une base de données centrale ou d’un dépôt pour les projets de résidence permettrait d’accroître la diffusion de la recherche effectuée par les résidents et son accessibilité.
MOTS CLÉS: taux de publication , résidence hospitalière , recherche , enseignement de la pharmacie , caractéristiques expérimentales
[Traduction par l’éditeur]
Research is an important part of pharmacy practice and education. The discipline of clinical pharmacy involves the generation of new knowledge to improve patients’ health and quality of life.1 Pharmacists are called upon to further the profession and, more important, to improve patient care through research.2 A critical opportunity for training and mentorship in research occurs during pharmacy practice residencies. Pharmacy residents in accredited programs in Canada must complete a research project during their residency year, with the end product being a manuscript suitable for publication.3 Projects represent a large investment in terms of both time (with a maximum of 10 weeks allocated for project work3) and effort for each resident. Residents generate a large body of research, given that there are over 150 residency positions in Canada each year.4
Publication is important to disseminate the results of research. When research is published, others may review the methods, apply the findings, or adapt an algorithm or process tree developed in the course of the study. In addition, smaller studies may be aggregated for systematic reviews or meta-analyses. Beyond the clinical and scientific value of the project, publication has benefits for the resident and his or her coauthors, the residency program, and the institution.5 Publication helps build the reputation of residency programs and associated institutions,5 and the involvement of preceptors during this process increases their experience in publishing and may facilitate future publications.
The primary objective of the study reported here was to determine the publication rate for research projects performed under the auspices of accredited English-language hospital pharmacy practice residency programs in Canada, between 2000 and 2009. A secondary objective was to describe the study characteristics of projects completed in the province of Ontario over the same period.
Hospital pharmacy practice residency projects completed in Canada over the period 2000–2009 were included in this study. For the primary objective of this review (the evaluation of publication rates), all accredited English-language programs were included. For the secondary objective (the examination of study characteristics), only residency programs in Ontario were included.
Projects completed after 2009 were excluded from the analysis to permit sufficient time after completion for publication or archiving in the university library. The French-language programs in Canada (accounting for about 70 residency positions per year) were excluded because the researcher (M.H.) would not have been able to accurately discern the publication status of any projects from these programs. For the secondary objective, a smaller sampling frame was used, consisting of hospital pharmacy practice residency programs in Ontario, because these projects are archived in a central location at the University of Toronto (the Gerstein Science Information Centre). Publication was defined as the existence of a full-text article, letter to the editor, or abstract or poster presented at a conference. For projects published as both an abstract and a full-text manuscript, only the manuscript was included in the analysis.
Canada has no single archive of all hospital pharmacy practice residency projects. Therefore, multiple strategies were used to identify eligible projects from Canadian residency programs. Potentially eligible residency programs were identified from the list of programs accredited by the Canadian Hospital Pharmacy Residency Board (CHPRB) (www.cshp.ca/programs/residencytraining/accreditedprograms/index_e.asp). A list of residents and projects was then compiled by performing the following steps:
searching websites of individual residency programs
searching the annual brochure for pharmacy practice residency programs in Ontario and any available invitations to residency nights in other provinces
hand-searching the library holdings of the University of Toronto
contacting residency coordinators, as described below.
An email message was sent twice to each coordinator, with a follow-up telephone call if there was no response to the email messages. Coordinators were re-contacted, if necessary, to clarify any uncertainty or discrepancies in their replies. In British Columbia, where several residency programs were consolidated in 2007 into a larger, more centralized program, the individual coordinators of the component programs were contacted.
To determine the publication status of residents’ projects, a standard search strategy was adapted (as shown in Figure 1).6,7 Several databases were searched in the following sequence: PubMed (1999 to February 28, 2011), all online issues of the Canadian Journal of Hospital Pharmacy ( CJHP ) (2000 to February 28, 2011), Cumulative Index of Nursing and Allied Health Literature (CINAHL) (1999 to February 28, 2011), and Google (no time frame specified). The searches were performed between February 28, 2011, and April 25, 2011.
|
|
||
|
Figure 1. Strategy for searching databases. CINAHL = Cumulative Index of Nursing and Allied Health Literature, CJHP = Canadian Journal of Hospital Pharmacy. |
||
PubMed, CJHP , and CINAHL were searched using a combination of each resident’s last name, first name, and/or initials. Alternate names (such as maiden and married names), when available, were used for independent searches to identify any publications under another name. A different approach was used for searching Google. Combinations of the resident’s name and key words from the title of the project were searched repeatedly to narrow results until either a match or only an unrelated item resulted. Exact matches or very close matches between a project title and a publication title were considered to indicate publication, and abstracts and/or full-text articles were examined if necessary. One project had unknown publication status as its title was missing; as a conservative assumption, this project was deemed not to have been published.
In Ontario, residency coordinators are expected to submit completed projects to the University of Toronto for archiving. Thus, for the secondary objective, 2 strategies were employed. First, the theses section of the Gerstein Science Information Centre at the University of Toronto, where holdings are sorted alphabetically by surname, was hand-searched for likely projects (i.e., projects on a pharmacy-related topic, as indicated by title and contents, that were archived in 1999 or later and that had been conducted in association with a hospital). The contents of each such project identified were examined, and authors’ names were later cross-checked with project lists from residency coordinators to confirm status as a residency project. Additional directed searching was conducted to specifically look for projects on the residency coordinators’ lists that were not identified by the initial hand-search. A librarian conducted a second independent search for any known projects that were not identified in the initial search.
For all residency projects identified, the publication status was recorded, and for all published projects, the type of publication, year published, journal name (if applicable), and database in which the publication was found were recorded. Appendix 1 contains definitions and explanations of the data extracted.
For projects fulfilling the inclusion criteria for the secondary objective, the following data were extracted: number of coauthors or co-investigators, disciplines of the coauthors or co-investigators, number of centres involved, time frame examined in the study, trial type, study design, intervention type (for interventional studies), and study topic. Appendix 2 contains definitions and explanations of the data extracted.
Microsoft Office Excel 2007 (Microsoft, Redmond, Washington) was used to manage and record the data collected. SPSS version 19 (IBM, Armonk, New York) was used to conduct the statistical analyses. The publication rate and study characteristics were reported as medians (and interquartile ranges), frequencies, counts, and proportions, as appropriate. The Mantel–Haenszel test for trend was used to assess changes over time, and a p value less than 0.05 was considered statistically significant. To assess changes in the study characteristics of randomized controlled trials and prospective studies, Fisher’s exact test was used to compare the period 2000–2004 with the period 2005–2009.
Approval from a research ethics board was not required for this study. Conference abstracts and published manuscripts are publicly accessible, and past residents’ names are posted online with other CHPRB information (www.cshp.ca/programs/residencyTraining/accreditedPrograms/index_e.asp) and are also publicly available.
Data were available for 26 of the 28 eligible residency programs. One centre did not reply to the request for information, and one was unable to gather the necessary information in time for inclusion in this study. Several centres were missing some data, as the residents’ names and project titles for some years had not been recorded or had been lost. A total of 518 projects were included, representing approximately 84% of the projects completed in the study period. The missing data for both the primary and secondary objectives are summarized in Table 1. For the primary objective, missing data refer to both residents’ names and project titles for the time frame indicated, and missing data are grouped by province. For the secondary objective, missing data represented projects that were unavailable for examination in the university library, and missing data are grouped by residency year.
Table 1.
Missing Data for Primary and Secondary Objectives
The overall publication rate was 32.2% (167 projects; 95% confidence interval [CI] 28.3%–36.4%), with 60 abstracts (35.9%, 95% CI 29.0%–43.5%) and 107 full-text publications (64.1%, 95% CI 56.6%–71.0%). There were no letters to the editor. Publication rates for each province and residency year are presented in Table 2. No other study calculating time to publication after completion of a residency was found. For projects included in the current analysis, the median time to publication was 1 year after completion of the residency (interquartile range: 1, 2), and 75% of published abstracts and manuscripts appeared within the first 2 years after completion of the residency. Yearly publication rates ranged from 24% to 38% (Table 2), but there was no significant trend over time for overall publication ( p = 0.40) or for publication of abstracts ( p = 0.13) or full-text manuscripts ( p = 0.92). The publication rate for the residency year ending in 2009 was lower than for other years; this may be explained by lack of time for publication between completion of the residency and the time of analysis. However, excluding the year 2009 from the analysis did not produce a statistically different publication rate.
Table 2.
Publication Rates and Type of Publication for Projects Undertaken by Canadian Hospital Pharmacy Practice Residents, by Province and Year of Completion
The 107 full-text manuscripts identified in the current study were published in a total of 45 different journals. Most of these projects (66 [62%]) appeared in pharmacy-specific journals, most commonly the CJHP (41 [62% of pharmacy-specific journals]) (see Table 3). The remaining 41 projects were published in a total of 36 non-pharmacy-specific journals, most commonly The Canadian Journal of Infectious Diseases and Medical Microbiology , which accounted for 3 articles (7% of publications in non-pharmacy-specific journals). Appendix 3 describes the sources in which published projects were located.
Table 3.
Publication of Residency Projects in Pharmacy-Specific Journals
Data were available for all 13 of the Ontario residency programs that met the inclusion and exclusion criteria. Of the 261 projects from the residency years 1999/2000 to 2008/2009, only 202 (77.4%) were found in the university’s library. Only projects that have been completed, submitted by the residency coordinator, bound, and processed by the university will appear in the library. Table 1 lists the number of projects missing by year.
Study characteristics for the residency projects are summarized in Tables 4 and 5. Most of the projects involved a single centre (180 [89.1%]) and most were prospective (109 [54.0%]) and observational (134 [66.3%]). The projects were about evenly divided among those with no coauthors or co-investigators (62 [30.7%]), those with 1 to 3 collaborators (74 [36.6%]), and those with 4 or more collaborators (66 [32.7%]). Collaboration with other health professionals was common, most often physicians and nurses.
Table 4.
Characteristics of Studies Undertaken by Hospital Pharmacy Practice Residents, 1999/2000 to 2008/2009
Table 5.
Other Characteristics of Studies Undertaken by Hospital Pharmacy Practice Residents, 1999/2000 to 2008/2009
No significant changes over time were found in the proportion of projects that were prospective ( p = 0.07) or randomized controlled trials ( p = 0.42). Furthermore, no significant trend over time was identified when the data were dichotomized into the periods 2000–2004 and 2005–2009 (prospective, p = 0.16; randomized controlled trials, p = 0.63).
To our knowledge, this is the first study to examine publication rates of Canadian hospital pharmacy residents’ projects and the study characteristics of such projects. In previous studies, the focus has been on the publication of full-length papers following the publication of abstracts or presentations at a meeting. For example, Ensom and Walker8 found that 25% of abstracts published in the CJHP (only some of which arose from residency projects) were subsequently published as full-length papers (in any journal). The current study examined post-residency publication of abstracts, letters to the editor, and manuscripts. The manuscript publication rate here was 20.6%, which appears generally consistent with rates in a US study on the publication of pharmacy residents’ projects.9 In the US study, publication rates after presentation at an annual residency conference declined over time: 20% in 1981, 15.7% in 1991, and 12% in 2001.9 The rates in both the current study and the previous US study were lower than those estimated for medical residents’ projects (24.8% and 40%).10,11 The study most similar to the current analysis examined the design of emergency medicine residents’ studies and their presentation at national conferences and publication in peer-reviewed journals.11 Publication rates for pharmacy residency projects as determined in the current study were lower than those for the emergency medicine residency projects11: 32.2% versus 58.9% for abstracts and full-text manuscripts combined and 20.6% versus 40% for full-text manuscripts alone. One potential explanation for this difference is the longer duration of the emergency medicine residency program (3 years), which gives the residents more time to conduct and complete research.
As stated above, most previous studies have focused on publication following presentation at a meeting. A Cochrane review on the full publication of biomedical research studies initially presented as abstracts or in summary form found that the median time to publication was 1.5 years and that the highest rates of publication occurred 3 years after presentation at a meeting.12 In the current study, the median time to publication (1 year) was within the time frame reported in the Cochrane review. By the end of their residencies, residents should have a manuscript ready for publication, which may put them further ahead, in terms of the publication process, than researchers in regular practice.
Of the 45 journals in which full-text manuscripts of residents’ projects were published, the majority (36 [80%]) were not specific to pharmacy. This indicates high diversity in potential readership. It also demonstrates that residents’ research may benefit specialty areas other than pharmacy.
The proportion of prospective studies in the current study was similar to that in a previous study of pharmacy residency projects in the United States9 (54% versus 47%, respectively). The breakdown by study type was also similar to the previous US study9: most projects were observational, about one-third were interventional, and most studies occurred in a single centre. This similarity in choice of methods likely reflects the constraints of time, regulatory and ethical approval, and funding required for prospective studies, in particular randomized controlled trials; these constraints make such studies impractical for pharmacy practice residencies.
As noted previously, 30.7% of the Ontario projects listed no coauthors or co-investigators. Although project preceptors may not have been listed as such, preceptors and members of the residency advisory council typically play an integral part in the research process and were often listed in the acknowledgements section, if such a section was present. However, the content of the acknowledgements section was not specifically examined and recorded in this study.
This study had several strengths. Information from multiple sources, along with a rigorous and systematic search strategy, was used to develop a comprehensive list of residents and projects. A uniform search strategy and standardized data collection tools were used to reduce the risk of bias or missing data.
The most important limitation in this study was missing data for several centres and years. Multiple avenues were used in attempts to collect complete information, and data for approximately 80% of known residency projects were obtained. The amount of missing data was estimated from the approximate number of residents usually registered in each program (at the time of this analysis) and the number of years for which data were unavailable. Given that programs are likely to expand over time and gain more residents, the estimated amount of missing data may be inflated. However, unless the publication rate is grossly different in centres with missing data than in programs elsewhere in the country, the overall publication rate should not be significantly affected. The province that accounted for the majority of missing data (British Columbia; 68 [69.4%] of 98 missing projects) had an overall publication rate of 34% (see Table 2), similar to the national publication rate (32.2%). As mentioned in the Methods section, there was a change in the oversight of this province’s residency programs in 2007. Before amalgamation, the individual programs had been running for 40 years, and although project records may exist somewhere, they could not be retrieved. Interestingly, there appears to have been an increase in the number of published projects from this province since 2006 (see Table 2), possibly coinciding with the creation of a central residency coordinator position with program consolidation. However, given that much of the information for years preceding 2005 could not be obtained, the publication rate for those earlier years, and hence the appearance of an increase, may be inaccurate.
In Ontario, it was expected that the university’s library would contain reports for all residency projects completed during the study period; however, this was not the case, probably because of delays in processing the large number of reports (there being over 30 residency positions in this province each year). Most of the missing projects were from a few institutions, with the majority of data missing from 2007 to 2009 (35 [59%] of 59 missing projects). These 35 projects accounted for 40% of the 88 projects completed in that period. The missing data may have significantly affected results for the final 3 years of the study period.
The results presented here may not be generalizable to French-language residency programs in Canada. Although the province used for analysis of the secondary objective (Ontario) represents approximately 42% of residents in the English-language programs, the study characteristics determined through this analysis may not be applicable to residency programs in other provinces.
Overall, 77.8% of projects performed in the study period remained unpublished at the time of the analysis. Although the current study did not specifically examine barriers to publication, potential barriers have been identified in previous studies,6,12–14 including lack of time, ongoing status of the study, lack of interest on the resident’s part, lack of interest from faculty members, responsibility for writing the manuscript resting with a coauthor (rather than the resident), difficulty with a coauthor, and low likelihood of acceptance for publication because of the methodological limitations presented by the study time.
The dissemination of results is a critical aspect of research and of research training. Increasing publication of residents’ projects also increases the number of pharmacists with experience in the research process and promotes pharmacists not only as clinicians, but also as researchers. Access to previous projects would allow new residents, as well as other health care professionals, to build on previous work and to learn from the experiences of previous residents. Although some studies may be perceived as specific to the study institution, their results may still be of use to other institutions, residents, and researchers and should be accessible.
Most Canadian hospital pharmacy residents’ projects included in this study remained unpublished at the time of analysis. The overall publication rate from 2000 to 2009 was 32.2%, and the annual rate did not change significantly over time. Most studies were prospective and observational, assessing the efficacy and/or safety of a medication. Increasing publication and other methods of dissemination and increasing the accessibility of reports (e.g., through a central database or repository) would increase the impact of residents’ research.
1. American College of Clinical Pharmacy. The definition of clinical pharmacy. Pharmacotherapy. 2008;28(6):816–7.
2. Task Force on a Blueprint for Pharmacy. Blueprint for pharmacy: the vision for pharmacy. Ottawa (ON): Canadian Pharmacists Association; 2008.
3. Canadian Hospital Pharmacy Residency Board. Accreditation standards 2010. Ottawa (ON): Canadian Society of Hospital Pharmacists; 2009 [cited 2011 Jun 13]. Available from: www.cshp.ca/programs/residency-Training/CHPRB_Standards_2010_-_FINAL.pdf
4. Residency training: CHPRB accredited programs. Ottawa (ON): Canadian Society of Hospital Pharmacists; 2010 [cited 2010 Sep 28] Available from: www.cshp.ca/programs/residencytraining/accreditedPrograms/index_e.asp
5. Baker DE. The last step of the residency projects: publication [editorial]. Hosp Pharm. 2010;45(9):672–3.
6. Bhandari M, Devereaux PJ, Guyatt GH, Cook DJ, Swiontkowski MF, Sprague S, et al. An observational study of orthopaedic abstracts and subsequent full-text publications. J Bone Joint Surg Am. 2002; 84-A(4):615–21.
7. Ahlers-Schmidt CR, Chesser A, Maher J, Vernon S. Dissemination of student and resident research: publication rates after poster presentation [letter]. Med Teach. 2009;31(3):315.
8. Ensom MHH, Walker SE. From abstract to publication: What makes the grade? Can J Hosp Pharm. 1998;51(6):280–4.
9. McKelvey RP, Hatton RC, Kimberlin CA. Pharmacy resident project publication rates and study designs from 1981, 1991, and 2001. Am J Health Syst Pharm. 2010;67(10):830–6.
10. Tintara H, Lumbiganon P, Preutthipan S, Tannirandorn Y; Research Subcommittee of the Royal Thai College of Obstetricians and Gynecologists. Research publication by the Royal Thai College of Obstetricians and Gynecologists residency training program, 1994–2003. J Med Assoc Thai. 2007;90(5):870–5.
11. Holmes JF, Sokolove PE, Panacek EA. Ten-year experience with an emergency medicine resident research project requirement. Acad Emerg Med. 2006;13(5):575–9.
12. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007 Apr 18;(2): MR000005.
13. Gill S, Levin A, Djurdjev O, Yoshida EM. Obstacles to residents’ conducting research and predictors of publication. Acad Med. 2001; 76(5):477.
14. Sprague S, Bhandari M, Devereaux PJ, Swiontkowski MF, Tornetta P 3rd, Cook DJ, et al. Barriers to full-text publication following presentation of abstracts at annual orthopaedic meetings. J Bone Joint Surg Am. 2003; 85-A(1):158–63.
Competing interests: None declared.
See pages 94 and 95 for appendices for this article.
Canadian Journal of Hospital Pharmacy , VOLUME 66 , NUMBER 2 , March-April 2013
