In Ontario, there are 34 PHUs that deliver public health programs and services in accordance with the Ontario Public Health Standards (OPHS) [36]. We conducted a cross-sectional survey containing close-ended and open-ended questions with PHUs in Ontario from 1 June to 22 December 2022.

To develop the survey, we first referred to several frameworks used to evaluate faith-based engagement in the context of immunization program delivery for ethnoracial communities made structurally vulnerable [4, 37, 38]. We chose the Consolidated Framework for Implementation Research (CFIR), a well-accepted and frequently used framework in implementation studies [39]. Survey questions were designed based on five conceptual domains to collect data on the interventions (domain 1), inner (domain 2) and outer setting (domain 3), roles and characteristics of interventions and respondents (domain 4) and implementation processes (domain 5) used by PHUs to partner with FBOs and other CBOs to improve vaccine confidence and uptake among ethnoracial communities made structurally vulnerable [39].

To capture with whom, how, and by what means are PHUs working with FBOs, and what evaluation methods were involved, we developed a 4-Part 24 question electronic survey (See “Survey Questions” in Supplementary Material S1). The survey was developed by MYS and validated and tested by the core research team (DBH, EDR, SA, AA, SAF). Respondents could also directly upload documents which included resources and materials created with FBO partnerships. Finally, PHUs were invited to send relevant documents and multimedia resources via email to further demonstrate how they worked with FBOs to encourage vaccine uptake among priority ethnoracial groups. Our survey covered all PHUs involved in the High Priority Communities Strategy. The survey captured PHU assessment of their vaccine programs’ incorporation of antiracism principles and gender considerations; the degree of involvement of FBOs in the design, implementation, and evaluation of the vaccine program; and description of formal and informal methods that PHUs use to evaluate the success of faith-based interventions for reaching priority ethnoracial groups. Anti-racism principles include: (i.) broad activities that account for the inherent power inequities in vaccine delivery programs (e.g., weighting staffing needs to distribute vaccinations based on race-based infection data); (ii.) dedicating resources to the design and delivery of vaccine programs that address unique race-and-gender based needs; (iii.) meaningfully involving ethnoracial communities made structurally vulnerable in the decision-making process; (iv.) taking positive action to incrementally redress racial inequities (e.g., introducing Public Health Standards that introduce racially concordant community-led vaccine promotion specialists in communities made vulnerable); and (v.) measuring progress of vaccine interventions in advancing racial equity [40].

To increase survey response rate, a Tailored Design Method (TDM) was applied [41] to ensure easily comprehensible language and concise instructions and clarifications were used throughout, such as graphs to support the understanding of the question. We showed respondents their progress in percentages when answering questionnaires and applied conditional formatting for numeric or text responses where needed to reduce response errors. Participants were given the option to leave their survey and return later to reduce incomplete survey responses.

We acquired a list of contacts responsible for overseeing COVID-19 immunization delivery in 34 respective PHUs from the Ontario Ministry of Health in May 2022. Using purposeful sampling, we distributed the survey to a total of 84 people, including 18 Medical Officers of Health, 43 immunization program specialists, 12 emergency response managers, and 11 relevant program delivery personnel within PHUs. We applied a hierarchical approach, first sending the survey to immunization program specialists and emergency response managers listed as first lines of contact.

We used REDCap Survey Distribution Tools to distribute the questionnaire via REDCap (Research Electronic Data Capture) and to keep track of response status. We sent out 3 to 5 reminder emails, 2 weeks apart, to remind recipients to complete the survey or forward the survey to relevant, qualified personnels within the PHU based on their experiences during the COVID-19 pandemic and vaccine rollout. Reminder emails were terminated once the respondent completed or declined to complete the survey. A final follow-up was sent to respondents who declined to participate to solicit reasons for their decision. Survey collection was implemented between 1 June 2022 to 22 December 2022.

Study data were collected and managed using REDCap electronic data tools hosted at the University of Toronto [42, 43]. Descriptive analysis of responses was performed in Microsoft Excel.

Of the 28 respondents representing 28 PHUs, participants on average had 3 years of experience (range 1–30 years). See Table 1 for respondent role within their PHUs.

PHUs identified 13 main ethnoracial communities that were prioritized for COVID-19 vaccine promotion in their area (see Figure 1). The majority of PHUs identified Indigenous and South Asian populations as priority communities (23/28, 82.1% and 15/28, 53.6% respectively); four (4/28, 14.3%) PHUs also identified Caucasian as a priority group, specifically those belonging to Anabaptist and Low German Speaking Mennonite communities as well as Ukrainian refugees.

PHU respondents listed several categories of partner organizations that supported vaccine rollout to ethnoracial communities (see Figure 2). Majority of PHUs partnered with CBOs (25/28, 89.3%), FBOs (23/28, 82.1%), health systems services and facilities (e.g., hospitals; 23/28, 82.1%), and government sector (e.g., provincial/municipal/regional services; 22/28, 78.6%). PHUs also partnered with workplaces (e.g., large factories); public and private recreational facilities (e.g., community centres); non-for-profit organizations (e.g., immigration services); for-profit organizations (e.g., pharmaceutical companies and businesses); and transit services (e.g., local busing systems). Four (4/28, 14.3%) PHUs indicated partnering with “other” organizations, which respondents identified as local First Nations Groups and local ethnocultural organizations (e.g., South Asian community groups, Latinx community groups, Black community organizations).

PHU respondents identified several methods to engage FBOs to support the vaccination program, with email and phone contact with FBOs as the starting point for most PHUs (60.7% email and 50.0% phone contact, respectively). Other examples of partnership initiation included word-of-mouth (14/28, 50.0%), such as through direct personal relationships with PHU staff. PHUs also sought to reach FBOs through print media (7/28, 25.0%), webpages hosting vaccine information for faith communities (6/28,21.4%), as well as tailored messaging on social media for faith-based communities (4/28, 14.3%). PHUs also invited faith leaders to vaccine forums and virtual townhalls to encourage their participation in the vaccine campaign, or where needed, provided in-person message delivery by public health nurses (e.g., to Mennonite communities and the Amish). In some instances, FBOs had initiated first contact with PHUs when, for example, offering their places of worship as vaccine clinics.

Three of the 6 PHUs involved in the High Priority Communities Strategy partnered extensively with FBOs to advance the strategy’s goals. They relied on churches (e.g., branches of Christian churches that deliver sermons in non-official Canadian languages to ethnic populations that primarily speak other languages), mosques, and faith-based community services to reach marginalized communities that often overlap with low-income, high material deprivation, lower socioeconomic status, and contain higher percentage of immigrants, newcomers, and seasonal workers whose mother tongue is non-English. Three PHUs reported partnering minimally with FBOs; one PHU indicated working with Indigenous communities to improve vaccine uptake by increasing accessibility and reducing barriers by addressing questions and concerns.

On a 5-point Likert Scale, 10 (10/28, 35.7%) PHUs considered faith-based vaccine delivery programs to have “extremely incorporated” gender considerations, while 8 (8/28, 28.6%) considered them not to have incorporated these considerations (See Figure 3). More than half (15/28, 53.5% and 16/28, 57.1%) incorporated some degree of anti-racism principles and ethnoracial sensitivities, respectively.

Majority of PHUs (23/28, 82.1%) considered FBO partnerships as effective in increasing vaccine uptake among hard-to-reach populations and providing equitable access (See Figure 3). These PHUs scored an average of 4.16 (Effective) out of 5 (Very Effective): Middlesex London, Toronto Public Health, Ottawa Public Health, City of Hamilton Health Services, Peel Public Health.

Five PHUs (5/28, 17.9%) considered FBO partnerships to be ineffective in reaching hard-to-reach populations and providing equitable access. These PHUs scored an average of 1 (Not Effective) out of 5 (Very Effective): Niagara Region Public Health, Lambton Public Health, Algoma Public Health, Kingston, Frontenac and Lennox & Addington Public Health and Sudbury & District Health Unit. Three indicated that they did not engage with FBOs in their vaccine rollout plan.

Reasons for an ineffective rating included not having sizeable hard-to-reach populations within their jurisdiction or receiving lukewarm responses from FBOs and therefore not pursuing a partnership. Other PHUs expressed that other community outreach and equity-based interventions were more effective than working directly with FBOs. For instance, some PHUs relied on CBOs with strong connections to ethnic communities with faith affiliations to promote vaccines, and found faith-based programs to be less effective at reaching out to diverse ethnoracial populations.

Twelve (12/28, 42.9%) PHUs reported use of formal and informal methods to evaluate vaccine intervention success of faith-based and community partnerships (See Figure 4). Four (4/28, 14.3%) PHUs reported using formal evaluation methods such as surveys with CBO/FBOs, interviews with key informants in the community, and focus groups with target populations and faith-based partners. Eight (8/28, 28.6%) PHUs reported using informal evaluation, such as verbal and written feedback from faith leaders and community members, post-event/community outreach/vaccine clinic debriefs with attendees, and ad hoc surveys/questionnaires with community leaders and FBO/CBO partners.

Twelve (12/28, 42.9%) PHUs reported not to have relied on any formal or informal evaluation methods. Among these, 4 (4/28, 14.3%) PHUs reportedly relied on monitoring changes in vaccine uptake in areas with high COVID-19 infection and low vaccine rates. The data were combined with local knowledge on ethnoracial populations within PHUs through existing public health programs (e.g., “School Immunization Program”, “Healthy Babies Healthy Children” program, “EarlyON”), and that of partners (e.g., other public sectors, local community health centres) to determine intervention success.

Our study has several strengths. The survey addressed an understudied topic in Canada of PHU-FBO partnerships and used robust survey development methods that captured partnership processes and interventions involving PHUs and FBOs during a global health emergency. The study also had a strong response rate which covered a wide range of PHUs in terms of geographical region, size of jurisdiction and ethnoracial diversity of populations served.

We note some limitations in our study. The limited sample size precludes any statistical tests. Given the cross-sectional design, we were not able to assess the quality of partnerships or the growth in partnership over time. In our data collection, we also noted differences in interpretation of what was considered faith based. For example, some respondents did not consider engagement with Indigenous groups as faith-based, whereas some interpret spiritual engagement as faith-based. This was addressed in the interpretation of our results where possible.

FBOs can be a valuable community partner as PHUs work to increase reach of health messages and uptake of public health interventions. Future work should focus on data availability particularly timely, nimble, and ethical data collection and disaggregating data on the basis of race, gender and other social stratifiers to inform more nuanced analyses. PHUs with sizeable ethnoracial diversity could consider facilitating knowledge exchange about promising implementation guidelines amongst PHUs with contextual and demographic similarities. Finally, there should be consideration for strengthening the capacity of PHUs to engage with CBOs/FBOs and measure implementation processes and outcome evaluations. Ongoing monitoring and evaluation of the impact of FBO engagement on vaccine confidence and uptake among ethnoracial communities is needed to inform future community engaged vaccine programming in Ontario and other jurisdictions.