Monica Lee, Jenny Chiu
The COVID-19 pandemic brought significant disruptions to pharmacy experiential education. To ensure the safety of students and staff, university and rotation site educators needed to make changes rapidly to adapt to the dynamic environment.
To explore the impact of the COVID-19 pandemic on pharmacy students and their preceptors during experiential rotations and to identify any barriers to learning that arose and opportunities for improvement.
Two online questionnaires were developed to explore the perceptions of pharmacy students and preceptors during experiential rotations. The following topics were examined: support for rotations by the hospital and the university, perceived safety, accessibility of resources, interpersonal interactions, professional development, assessment and evaluation, and overall impressions. All Advanced Pharmacy Practice Experience students from the University of Toronto who completed 1 or more rotations at North York General Hospital during the 2020/21 academic year and their preceptors were invited to participate.
Sixteen and 25 questionnaires were completed by students and preceptors, respectively. Both groups agreed that they were adequately prepared for the rotations and felt safe. There was a decrease in interpersonal interactions, while the use of virtual communication tools increased. Lessons learned included the need for timely communications and access to resources for learners and preceptors, contingency plans for staff shortages and outbreaks, and workspace assessments.
During the COVID-19 pandemic, implementation of experiential rotations was associated with many challenges, but pharmacy learners and preceptors believed the overall experience was not significantly affected.
KEYWORDS: experiential education, COVID-19, pharmacy education, pharmacy preceptor, pandemic, pharmacy students
La pandémie de COVID-19 s’est accompagnée de perturbations importantes dans le domaine de la formation pratique en pharmacie. Les éducateurs de l’université et du lieu de stage ont dû rapidement apporter des changements pour s’adapter à l’environnement dynamique et assurer la sécurité des étudiants et du personnel.
Étudier les effets de la pandémie de COVID-19 sur les étudiants en pharmacie et leurs précepteurs pendant les stages pratiques et identifier les obstacles qui se sont présentés ainsi que les améliorations possibles.
Deux questionnaires en ligne ont été préparés pour étudier les perceptions des étudiants en pharmacie et des précepteurs pendant les stages pratiques. Les sujets suivants ont été examinés: le soutien de l’hôpital et de l’université pour les stages, la perception de la sécurité, l’accessibilité des ressources, les interactions interpersonnelles, le perfectionnement professionnel, l’évaluation et les impressions générales. Tous les étudiants du programme Advanced Pharmacy Practice Experience de l’Université de Toronto qui ont effectué un ou plusieurs stages à l’Hôpital général de North York au cours de l’année universitaire 2020–2021 et leurs précepteurs ont été invités à participer.
Les étudiants et les précepteurs ont répondu à 16 et 25 questionnaires, respectivement. Les deux groupes ont convenu qu’ils étaient bien préparés aux stages et qu’ils se sentaient en sécurité. On a observé une diminution des interactions interpersonnelles, tandis que l’utilisation d’outils de communication virtuels a augmenté. Les leçons tirées comprennent: la nécessité de communiquer en temps opportun et l’accès aux ressources pour les apprenants et les précepteurs; les plans d’urgence en cas de pénurie de personnel et d’épidémies; et les évaluations de l’espace de travail.
Pendant la pandémie de COVID-19, la mise en œuvre des stages pratiques a été associée à de nombreux défis, mais les apprenants en pharmacie et les précepteurs ont estimé que l’expérience globale n’a pas été touchée de manière significative.
MOTS CLÉS: formation pratique, COVID-19, formation en pharmacie, précepteur en pharmacie, pandémie, étudiants en pharmacie
The World Health Organization (WHO) declared the COVID-19 outbreak a global pandemic on March 11, 2020.1 Since then, this disease has transformed many aspects of our lives. To limit transmission of SARS-CoV-2, various measures were taken, including the closure of nonessential services, the implementation of restrictions on entry into Canada, and the practice of social or physical distancing.2 Regional decisions to implement “lockdowns” resulted in the closure of in-person classes, including those at the university level.3 Although pharmacy learners could gain valuable experience during a global health crisis, numerous issues made it challenging for institutions to continue clinical placements. First, hospital pharmacies diverted their resources to managing drug shortages,4 exploring alternative methods of medication administration,5 and redesigning staffing models to accommodate potential staff shortages. Second, the methods to ensure the safety and well-being of staff and learners were uncertain while information about the coronavirus continued to evolve. Third, conservation of personal protective equipment (PPE) was thought to be crucial, because inventory was scarce and supply chains were unpredictable.6 Pharmacy educators and hospital pharmacy administrators were suddenly faced with the challenges of implementing experiential rotations during this unprecedented time.
The entry-to-practice Doctor of Pharmacy and postbaccalaureate PharmD programs in the Leslie Dan Faculty of Pharmacy (LDFP) at the University of Toronto (U of T) require candidates to complete Advanced Pharmacy Practice Experience (APPE) rotations in various settings.7 With a strong commitment to teaching and learning, the Pharmacy Department at North York General Hospital (NYGH) has historically offered an average of 40 APPE placements per academic year. Following the WHO’s declaration of the COVID-19 pandemic, all nonpaid, prelicensure rotations (including those for pharmacy) at NYGH were halted on March 16, 2020, and U of T cancelled all experiential rotations for the remainder of the academic year. At NYGH, all pharmacy rotations resumed on July 13, 2020. When on-site placements were permitted, U of T and NYGH made plans for the safe reintegration of learners during a period when there were still many unknowns.
The NYGH Centre for Education provided the hospital’s education leads with planning tools to assess the feasibility of resuming in-person rotations. Considerations included assessment of the number of learners on site, space limitations, and COVID-19 resources. Supports implemented by the hospital included training for donning and doffing of PPE, making the medical library available for use exclusively by learners, and resources outlining what to expect when on placement during the pandemic, as well as new communication platforms such as Microsoft Teams (Microsoft Corporation). The university developed student guidelines in the event of exposure to COVID-19, as well as procedures outlining provision of care for patients with COVID-19. The university also developed a COVID-19 curriculum, which learners in all health disciplines had to complete before their return to on-site rotations.
The goals of this study were to explore how these modifications affected pharmacy students during their experiential rotations and to identify barriers to learning and opportunities for improvement. The results of this study may help to inform pharmacy educators and on-site education coordinators about modifications needed to improve future rotations when faced with similar circumstances.
A literature search was conducted to determine whether a validated survey existed to help address the research questions. No prior studies on this topic were found, so the authors developed 2 questionnaires through consensus. The questionnaire for students (available by request to the corresponding author) consisted of 29 items, divided into a section for demographic characteristics and the following 7 domains to evaluate the impact of the pandemic on the students’ experience during their rotations:
support for rotation by the hospital and university (e.g., COVID-19 protocols)
perceived safety (e.g., infection control protocols and PPE)
accessibility of resources (e.g., physical workspace; hardware and software; remote access to electronic health records, drug information resources, and hospital intranet)
interpersonal interactions (e.g., with patients, families, health care providers)
professional development (e.g., rounds, education sessions)
assessment and evaluation
These 7 domains reflect the components and key requirements for learner preparedness for a hospital-based rotation.8
The questionnaire for pharmacist preceptors (also available by request to the corresponding author) was developed from the student questionnaire, with modifications to elicit pharmacists’ perceptions of how the students felt regarding some domains. Two former pharmacy students and 2 pharmacists tested the preliminary questionnaires to evaluate readability and clarity of the questions and to estimate the time required to complete the survey. Feedback was incorporated to improve the readability of the questions. The final versions of the questionnaires were created using SurveyMonkey (Momentive).
All LDFP Doctor of Pharmacy students who completed an APPE placement at NYGH between July 13, 2020, and April 30, 2021, as well as their pharmacist preceptors, were invited, on the last day of their respective rotations, to participate in the study. Preceptors were invited to provide consent and complete a questionnaire for each student on their rotation. Some preceptors taught more than 1 rotation block and provided consent for each block. The pharmacy education coordinator identified eligible participants from the placement schedule provided by the LDFP. For students who had multiple rotations at NYGH, the invitation to participate was extended at the end of their last NYGH rotation. Students who did not complete their APPE rotations and Doctor of Pharmacy students from institutions other than U of T, as well as their preceptors, were not eligible to participate.
For each potential participant who provided written informed consent, one of the study investigators sent the survey link by email (through SurveyMonkey) after completion of the rotation, with 2 weekly follow-up emails sent to increase response rates. Data were collected anonymously.
The data from each response were extracted into a spreadsheet (Excel 2016, Microsoft Corporation) by a single investigator (J.C.). The second investigator (M.L.) verified the data entry for accuracy. Quantitative data were analyzed using descriptive statistics. Qualitative data were reviewed and common themes extracted under each domain upon agreement between the 2 authors.
The research protocol was reviewed and approved by the NYGH Research Ethics Board in July 2020.
There were 29 rotation blocks at NYGH during the 2020/21 academic year: 26 involving direct patient care and 3 involving non–direct patient care. Of the 18 eligible APPE students, 17 (94%) consented to participate in the study, but only 16 completed the questionnaire. Five (31%) of these 16 student participants had completed an early practice experience rotation in a hospital setting before their APPE rotations at NYGH. Eleven (69%) of the student participants completed more than 1 rotation at NYGH. Twenty-one pharmacist preceptors and copreceptors were eligible to participate in the study. Given that some preceptors had multiple students, the total number of potential consents (and hence the potential number of preceptor questionnaires) was 30. However, only 25 questionnaire responses were available for analysis, because invitations to 3 preceptors were missed, and 2 preceptors did not submit any response. Of the 25 preceptor responses submitted, 1 was incomplete; any data available from this questionnaire were included in the analysis. With respect to teaching experience, the pharmacist preceptors had supervised a median of 4 students (range 1 to 7) per preceptor over the past 2 academic years. The areas where rotations were held are listed in Table 1.
TABLE 1 Characteristics of Student Participants and Rotation Types at North York General Hospital
Fourteen (88%) of the 16 student respondents agreed that they were adequately prepared to start the rotation with the resources provided by NYGH, whereas 12 (75%) felt this way about the resources provided by U of T (Figure 1). Five student participants believed that the university could have done more to prepare them for their rotations during the pandemic. From the pharmacist preceptors’ point of view, 20 (80%) and 23 (92%) of 25 responses indicated that the university and the hospital, respectively, had adequately prepared students to start the rotation with COVID-19 resources and support. Similarly, 21 (84%) and 24 (96%) of the 25 preceptor responses indicated a belief that the preceptors themselves were adequately prepared for the rotations with the resources provided by the university and the hospital, respectively. Ideas from students and preceptors on how to improve their preparedness are summarized in Table 2.
FIGURE 1 Perceptions of students and preceptors regarding preparation for the rotations. LDFP = Leslie Dan Faculty of Pharmacy (University of Toronto), NYGH = North York General Hospital.
TABLE 2 Additional Ideas Suggested by Participants to Help Prepare for the Rotations
All but 1 of the student participants felt safe working in the hospital environment during the pandemic. Twenty-three (92%) of 25 preceptor responses indicated a belief that their students felt safe. The reasons for students feeling safe or unsafe are listed in Box 1.BOX 1 Factors that Made Students Feel Safe or Unsafe during Rotationsa
Participants were asked to indicate the amount of time they were able to access computer terminals on site, applications such as Microsoft Teams, and online resources by remote access. The results are illustrated in Figure 2. With respect to workspace and computer terminal access in the hospital, preceptors consistently perceived these resources to be less accessible than did students. All student respondents indicated that they were able to reliably access Microsoft Teams, the NYGH email system, and virtual care resources (where applicable) the majority of the time. A few participants reported problems with remote access. One person was able to use remote access only some of the time, whereas 2 indicated they had access about half the time. Specific problems with electronic tools were delays in registration or initial set-up, a delay in resolving a particular problem, inconsistency in remote access, and connectivity problems when using a particular tool. Positive feedback and suggested improvements from students and preceptors are summarized in Table 3.
FIGURE 2 Perceptions of students and preceptors regarding accessibility of resources for students. NYGH = North York General Hospital.
TABLE 3 Positive Feedback and Suggested Improvements Regarding Resources and Their Accessibility
Perceptions of how interactions with patients and families, other health care providers and interprofessional collaborators, preceptors and other pharmacy personnel, and peers differed during the pandemic were elicited in this part of the questionnaire. Table 4 summarizes the themes identified and their respective frequencies, based on comments from student and preceptor respondents.
TABLE 4 Themes Related to Students’ Interpersonal Interactions during the COVID-19 Pandemic
Although some of the students and preceptors reported no differences, others felt that there were fewer face-to-face interactions, which resulted in increased use of telephones to interact with patients and families and increased use of Microsoft Teams to interact with health care providers. There was also less engagement with patients’ families because of visitor restrictions.
The need for physical distancing, the use of PPE, and the lack of face-to-face opportunities all contributed to challenges with communication. One student expressed concerns about a higher risk of medication errors as a result of these barriers.
Compared with experiential rotations in previous academic years, 6 students believed there were fewer opportunities for professional development at NYGH during the pandemic, whereas 4 students expressed the opposite perception. Most students commented on having to learn to use virtual platforms to communicate, attend rounds, and participate in presentations. They were able to find workarounds when they encountered barriers (e.g., performing medication histories over the phone rather than in person, finding ways to accommodate language barriers). Working remotely was a new experience for some. These students learned to create a stringent work-from-home routine to ensure work–life balance. One student suggested that alternative activities be arranged so that they could continue to have learning opportunities outside their assigned clinical areas, when shadowing of other pharmacists was not feasible.
Among preceptor respondents, the top themes were recognition of students’ development of virtual presentation skills and their provision of virtual care. Other themes mentioned were skills in working and learning remotely, which increased students’ independence, and students’ roles and responsibilities in infection control and prevention.
Most students did not think that they were assessed or evaluated differently by their preceptors during the pandemic. Similarly, most students did not think there were any challenges in performing self-assessment during the pandemic. However, some respondents pointed out that patient interaction was evaluated on the basis of a smaller sample size because of restrictions in place during the pandemic. In addition, it was more difficult to find interprofessional team members to complete the interprofessional competency assessment, given that the interaction between students and these team members was reduced. Informal assessments were performed less often, with the focus on the midpoint and final evaluations. Some preceptors found it more challenging to evaluate their students because speaker phones were not always available and therefore they could not listen to the other side of students’ telephone conversations (e.g., with patients or colleagues). Students were less frequently present on the unit to interact with patients, families, and other health care professionals, which resulted in fewer opportunities for assessments. Space limitations made it more challenging to find private areas for conducting evaluations and feedback.
Most participants felt that the COVID-19 resources and supports provided by U of T and NYGH adequately prepared them for their rotations and made them feel safe. However, 1 participant commented that a discussion of workflow changes and expectations during orientation would have been helpful. Some preceptors were unaware of the COVID-19–specific resources that were available to students. Given the ever-changing information and protocols, timely development, implementation, and dissemination of information was a challenge. Ideally, a central internet site for both NYGH and U of T, dedicated to COVID-19–related questions and resources, might have been beneficial, especially at the start of learner reintegration. However, during the latter part of the academic year, both the university and the hospital had developed separate learner-dedicated websites with COVID-19–related rotation information.
With respect to the challenges in communication, providing students with tips on how to communicate effectively while wearing PPE may help them to build better rapport with the individuals with whom they interact. With the increased use of telephones, it was soon realized that conferencing telephones were not readily accessible on the units. Consequently, a conferencing telephone was added in the pharmacy, which also enabled preceptors to assess interactions between students and families. The use of other technologies, such as tablets for videoconferencing, could also be considered. Using a platform such as Microsoft Teams helped with communications when staff were working off-unit, since this platform offered secure, real-time communication and videoconferencing.
Students and preceptors spent less time together in person, which led to missed opportunities for spontaneous teaching. Remote teaching and virtual care posed their own set of problems, such as a reduced variety of patients, problems with online access, and the need to develop online communication skills. Although team rounds took place virtually in many units, these did not universally include students. Ensuring that participants have access to meeting links or quick status updates for patients would have been beneficial. Workspace issues could have been alleviated by providing laptops to students, which would have allowed for more flexibility in where work could be done.
Assessments and evaluations were not overly affected. However, consideration should be given to modifying evaluation criteria to accommodate limitations, for example, prolonged absences (e.g., because of quarantine) or reduced interprofessional collaboration or patient interactions (e.g., use of role playing).
Despite the challenges, the pandemic did provide an opportunity for new and innovative experiential practices, such as leveraging the use of technology for secure remote access to electronic health records and provision of virtual care. These technologies enabled students to work from home, which helped foster learner independence. Other organizations have adopted similar strategies.9–12 Some of the changes implemented in our setting will continue in postpandemic rotations, such as permanently using Microsoft Teams to replace pagers as a communication tool. Working from home will also remain an option if a student is unable to come on site. The use of virtual platforms will likely remain as an option for assessing patients and providing clinical services in the future.
Overall, participants in our study thought that students had a positive learning experience at NYGH. For example, they developed several new skills that would not have been likely if not for the pandemic. With continued evolution of the pandemic and variability in each wave of infection, it would be useful to develop a “playbook” for future pandemics or disaster preparedness, incorporating lessons learned in this study.13,14 As well, teaching hospitals and pharmacy schools could work in partnership to develop a collaborative curriculum that would include patient cases, specific topic discussions,15 or alternative experiential opportunities such as simulation or virtual computer-based cases.11–13,16,17 These approaches could also be useful when there is a significant staff shortage. Communication among learners, preceptors, the university, and the hospital could be optimized by providing regular updates and actively seeking feedback from preceptors and learners to alleviate anxieties associated with constant changes.14
Our study had a few limitations. First, it was a single-site study limited to students from 1 teaching institution, which limits the generalizability of the results. Preparation of students to cope with COVID-19, as well as the resources available, might have differed from those at other sites or institutions. Second, NYGH is a technologically advanced organization, with availability of electronic health records and the ability to provide students with remote access to various resources. Although other facilities may not have similar technologies, some of the themes identified in this study could help to generate ideas for other sites to consider. Third, because the study was designed as an online, self-administered questionnaire, respondents could not be questioned further to elaborate on their responses, which limited the utility of some of the feedback received. Focus group sessions could have enabled the collection of more detailed information. Fourth, because of competing priorities during the pandemic, the researchers missed inviting 3 preceptors to participate in the study; this oversight might not have affected the results significantly, given that themes reached saturation. Fifth, given the perpetual changes occurring throughout the pandemic and the constant pivoting of practices and resources, some themes identified in this study may no longer be pertinent. Lastly, our study did not explore the emotional impacts of COVID-19 on students and preceptors or how they felt, which were found to be important lessons learned in other studies.9,18,19
The pandemic introduced challenges in experiential rotations that have never been encountered before. The ongoing evolution in safety practices and availability of resources demanded adaptability and flexibility from both pharmacy learners and preceptors. Despite these problems, learners considered their overall learning experience to be well supported and safe. Although conventional learning activities might have been limited, innovative ways to meet the rotation objectives resulted in the development of new skills. For future pandemic planning, key points to consider are timely communication, provision of adequate pandemic-related resources to learners and preceptors, development of contingency plans in the event of staff shortages and outbreaks, and assessment and arrangement of alternate workspaces.
1 WHO Director-General’s opening remarks at the media briefing on COVID-19 – 11 March 2020. World Health Organization; 2020 [cited 2023 Jan 17]. Available from: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19---11-march-2020
2 Review of Canada’s initial response to the COVID-19 pandemic. Canadian Public Health Association; 2021 Feb 16 [cited 2021 Dec 19]. Available from: https://www.cpha.ca/review-canadas-initial-response-covid-19-pandemic
3 U of T cancels in-person classes, switches to online and other means, due to COVID-19 outbreak [news release]. 2020 Mar 13 [updated 2020 Mar 17; cited 2021 Dec 19]. Available from: https://www.utoronto.ca/news/u-t-cancels-person-classes-all-three-campuses-due-covid-19-outbreak
4 Elbeddini A, Hooda N, Yang L. Role of Canadian pharmacists in managing drug shortage concerns amid the COVID-19 pandemic. Can Pharm J. 2020;153(4):198–203.
5 Clinical experiences keeping infusion pumps outside the room for COVID-19 patients. Institute for Safe Medication Practices (US); 2020 Apr 3 [cited 2021 Dec 19]. Available from: https://www.ismp.org/resources/clinical-experiences-keeping-infusion-pumps-outside-room-covid-19-patients
6 Personal protective equipment (PPE) use during the COVID-19 pandemic. Ontario Health; 2020 Mar 25 [updated 2020 Mar 30; cited 2021 Dec 19]. Available from: https://www.oha.com/Bulletins/Personal%20Protective%20Equipment%20use-COVID-19_Released_March_30_2020.pdf
7 Faculty of Pharmacy. PharmD experiential rotations. University of Toronto; [cited 2021 Dec 19]. Available from: https://www.pharmacy.utoronto.ca/current-students/pharmd/oee/pharmd-experiential-rotations
8 Beavers L, Christofilos V, Duclos C, McMillen K, Sheehan J, Tomat L, et al. Perceptions of preparedness: how hospital-based orientation can enhance the transition from academic to clinical learning. Can Med Educ J. 2020:11(4):e62–e69.
9 Sasser CW, Wolcott MD, Morbitzer KA, Eckel SF. Lessons learned from pharmacy learner and educator experiences during early stages of COVID-19 pandemic. Am J Health Syst Pharm. 2021;78(10):872–8.
Crossref PubMed PMC
10 Louiselle K, Elson EC, Oschman A, Duehlmeyer S. Impact of COVID-19 pandemic on pharmacy learners and preceptors [letter]. Am J Health Syst Pharm. 2020;77(14):1097–9.
Crossref PubMed PMC
11 Moreau CM, Maravent S, Hale G, Joseph T. Strategies for managing pharmacy experiential education during COVID-19 [commentary]. J Pharm Pract. 2021;24(1):7–10.
12 Fuller KA, Heldenbrand SD, Smith MD, Malcom DR. A paradigm shift in US experiential pharmacy education accelerated by the COVID-19 pandemic. Am J Pharm Educ. 2020;84(6):ajpe8149.
Crossref PubMed PMC
13 Law E, Scott MC, Moon YSK, Lee AJ, Bandy VT, Haydon A, et al. Adapting pharmacy experiential education during COVID-19: innovating remote preceptor resources, tools, and patient care delivery beyond virtual meetings. Am J Health Syst Pharm. 2021;78(18):1732–8.
Crossref PubMed PMC
14 Hall AK, Nousiainen MT, Campisi P, Dagnone D, Frank JR, Kroeker KI, et al. Training disrupted: practical tips for supporting competency-based medical education during the COVID-19 pandemic. Med Teach. 2020;42(7):756–61.
15 Montepara CA, Schoen RR, Guarascio AJ, McConaha JL, Horn PJ. Health-system implementation of a collaborative curriculum for advanced pharmacy experiential education during the COVID-19 pandemic. Am J Health Syst Pharm. 2021;78(10):890–5.
Crossref PubMed PMC
16 Almohammed OA, Alotaibi LH, Ibn Malik SA. Student and educator perspectives on virtual institutional introductory pharmacy practice experience (IPPE). BMC Med Educ. 2021;21:257.
Crossref PubMed PMC
17 Lucas C, Williams K, Bajorek B. Virtual pharmacy programs to prepare pharmacy students for community and hospital placements. Am J Pharm Educ. 2019;83(10):2085–91.
18 Cernasev A, Desai M, Jonkman LJ, Connor SE, Ware N, Chandra Sekar M, et al. Student pharmacists during the pandemic: development of a COVID-19 knowledge, attitudes, and practices (COVKAP) survey. Pharmacy (Basel). 2021;9(4):159.
Crossref PubMed PMC
19 Nagy DK, Hall JJ, Charrois TL. The impact of the COVID-19 pandemic on pharmacy students’ personal and professional learning. Curr Pharm Teach Learn. 2021;13(10):1312–8.
Crossref PubMed PMC
Competing interests: Jenny Chiu is a member of the University of Toronto Leslie Dan Faculty of Pharmacy and Toronto Academic Health Science Network Education Coordinators committee (representing North York General Hospital). No other competing interests were declared. ( Return to Text )
(Return to Top)
Funding: The North York General Hospital Pharmacy Department paid for the subscription to the Advantage version of the Survey Monkey application. No other funding was received. ( Return to Text )
The authors sincerely thank Daniel Chan, Lily Zhang, Emma Koivu, and Bonnie Lam for their contributions in reviewing the draft questionnaires.
Canadian Journal of Hospital Pharmacy, VOLUME 76, NUMBER 2, Spring 2023