Evaluating indoor residual spraying in Côte d’Ivoire: the role of routine malaria data quality

Abstract

Objectives:

Indoor residual spraying (IRS) was introduced in Côte d’Ivoire in 2020. We seek to identify potential factors that could compromise the quality of routine malaria incidence and determine how impact quantification of IRS on malaria incidence depends on data quality by emphasizing robust counterfactuals with contemporaneous covariates.

Methods:

Using mixed-effect models and correlation analysis we compare data from two sources (routine surveillance and re-digitized registries). To quantify decrease in incidence post intervention, we apply Bayesian structural time series analysis with posterior feature selection to determine impact confounders.

Results:

The presence of IRS, test positivity and reporting rates are significant factors impacting data quality. For high-quality data, we estimate a 12% resp. 19% decrease in incidence rate 1 year after the intervention, while for low-quality data estimates are 30% resp. 42%. Important covariates for counterfactual modeling are control district incidence, reporting and test positivity rates.

Conclusion:

Improvements in data quality by scrutinizing test positivity rates and health facility reporting can help avoid overly optimistic impact estimations for IRS. Incidence control districts remain inevitable in order to factor out impact confounders.

Introduction

After 2 decades of steadily declining malaria mortality from 2000 to 2019, global progress in the fight against malaria has recently stalled and, in some regions, reversed [1]. In Côte d’Ivoire, Malaria remains a leading cause of morbidity and mortality, accounting for around 33% of outpatient visits at health facilities [2]. Vector control interventions including indoor residual spraying (IRS) and insecticide-treated nets (ITNs) remain important cornerstones of malaria prevention [3]. Côte d’Ivoire included IRS in its National Malaria Strategic Plans 2016–2020 and 2021–2025 in order to reduce malaria burden in high transmission districts [2]. IRS was introduced in two districts in 2020 and carried out annually until 2022. After 2022, IRS was stopped due to loss of funding through the US President’s Malaria Initiative (PMI), despite the evidence of its impact on clinically relevant outcomes in ‘real-world’ settings [4]. Compared to randomized control trials (RCTs), real-world evidence from routine surveillance systems is less costly to produce, specific to the local context and of practical use to decision-makers. On the other hand, such analyses do not typically use any control groups or counterfactuals which is crucial to all retrospective impact evaluations [5–7]. Counterfactuals should represent the temporal evolution of the outcome measure in the hypothetical absence of the intervention. Clinical impact of vector control can be estimated using confirmed malaria cases. Every month, such data are reported to a centralized database based upon facility-level paper registries using the District Health Information Software (DHIS2) system. The National Malaria Control Program (NMCP) of Côte d’Ivoire has implemented the DHIS2 database as early as 2015 with only recently improved consistency and validity [8]. In a project aiming at assessing the impact of IRS, PMI in collaboration with the NMCP re-digitized key malaria indicators from health facility consultation registers during the years 2019–2022 of four districts: Nassian and Sakassou with IRS campaigns conducted in 2020&2021, and Dabakala and Beoumi as control districts. Based on this data (henceforth referred to as PMI data), a comprehensive impact evaluation estimated a significant reduction in clinical incidence following the IRS campaigns [9].

Complementary to the impact quantification undertaken in [9], our analysis has two main objectives. First, we seek to identify potential factors contributing to the differences between data reported into DHIS2 and the data re-digitized by PMI, assuming that DHIS2 data are generally of lower quality and compromised at the level of data entry and compilation. Second, we utilize a model-based approach to interrogate how differences in data quality alter intervention impact quantification. We emphasize building counterfactuals incorporating a large number of contemporaneous covariates relying on Bayesian structural time series analysis. This technique has been suggested as a viable alternative to interrupted time series analysis for health impact assessment [10, 11] with clinical, routine and remote-sensed data [12–14] and probabilistically combines covariates such as rainfall, vegetation cover, temperature, but also routine health surveillance quality and most importantly reported malaria incidence from control districts to factor out possible confounders of IRS impact.

The results of this analysis should help promote critical use of routine malaria surveillance data for decision-making. Intensified data validation and audits combined with analytics that allow identifying potential causal confounders should further add to the existing body of evidence on how IRS affects malaria incidence in the ‘real’ world. In light of increasingly limited resources for malaria surveillance, treatment and prevention, the importance of high quality and appropriately contextualized data for decision making to National Malaria Control Programs cannot be overestimated.

Methods

Data sources and quality assessment

We extracted and post-processed routine surveillance data for malaria from the DHIS2 database in collaboration with the NMCP of Côte d’Ivoire. In the country, the reporting follows the hierarchy of referral in the sanitary pyramid, i.e., health facilities in the periphery with recently added community health workers (CHW) and local general hospitals (Hôpital Général, HG) refer to regional hospitals (Centre Hospitalier Régional, CHR) who refer to university hospitals (Centre Hospitalier Universitaire, CHU). The nested spatial organization consists of 113 health districts within 26 health regions that report to the Ministry of Public Hygiene and Universal Health Coverage at the national level. We considered clinical malaria cases of all ages reported at the level of health districts. These monthly data were entered into DHIS2 from paper registries in health facilities, HG, and CHR by local health workers and validated by the NMCP. As for CHW and CHU, inconsistency in reporting across districts and time, prohibited further consideration for this analysis. Since presumed cases (i.e., without confirmed tests) have been reported into the DHIS2 database only since 2021, we considered the difference between suspected and tested cases as presumed cases whenever the latter were not directly available in the database.

In preparation of the impact evaluation for IRS in the districts Nassian and Sakassou with control districts Dabakala and Beoumi, PMI decided to collect data directly from health facility registries in the four study districts due to insufficient data quality [15]. Patient data were abstracted from consultation registries (“registres de consultations curatives”) for the period September 2018 to April 2022, we refer to details of inclusion criteria to recently published study results [9].

In order to control for factors unrelated to the intervention, we also included environmental, entomological and surveillance covariates into the causal impact analysis (see Supplementary Table S1). Remote-sensed data (rainfall, average daily temperature, leaf cover, average daily humidity) was extracted from the ERA5 database [16], aggregated monthly at the health-district level. Entomological covariates (monthly indoor and outdoor biting rates and monthly entomological inoculation rates) were extracted from the PMI VectorLink annual entomological reports for the years 2019–2021 [17–19] for the four districts of interest. To take into account changes in the health surveillance we also derived from the DHIS2 database the following district-level monthly quantities: health facility reporting rate (short: hf_reporting), monthly testing rate (short: testing), test positivity rate (short: tpr_total), the number of active health facilities (short: N_hf_active) and coverage of intermittent preventative therapy for pregnant women for the first trimester (IPT1) (short: ipt1_coverage). Since the 2021 mass ITN campaign was rolled out in the four districts from May until June 2021, we assume that residual effects of the 2017 ITN campaign are negligible during the IRS evaluation period from July 2020 to July 2021. Although IPT1 concerns only a small proportion of the population at risk, we consider it as a general proxy for access to healthcare. We normalized and scaled all covariates prior to further analysis.

Incidence rate calculations with adjustments

To adjust incidence (to both DHIS2 and PMI crude data), we used the following epidemiological indicators. First, we extracted monthly confirmed malaria cases reported and validated at the level of health districts in the DHIS2 system or by PMI respectively. Second, we used presumed cases (short: pr) from the DHIS2 database which are defined as cases with malaria-like symptoms without having been tested. Third, we calculated the test positivity rate (short: tpr) as the ratio of confirmed over tested cases. Here, we assumed that cases were tested with either blood spot or rapid diagnostic test (RDT). For the reporting rate, we considered for each health district the ratio of months within a year, where health facilities reported into the DHIS2 system. Finally, the healthcare seeking rate (short: s) was extracted from the 2016 Malaria Indicator Survey [20] and the 2021 Demographic and Health Survey (DHS) report [21] estimates of healthcare seeking for children with fever made at the regional level. Sakassou, Beoumi and Dabakala are located in the Center-North region and Nassian in the North East region. We interpolated these values linearly between the years. We then calculated adjustments to incidence rates for both data sources DHIS2 and PMI using the same quantities as detailed in Supplementary Table S2. District-level population data were extracted from the 2021 census data with 3% annual growth rate extrapolation. We calculated incidence rate as the ratio of adjusted incidence over district population.

Multiple imputation of missing data

In the period of interest, we have detected 6.7% of missing data among all contemporaneous covariates across the four health districts (see Supplementary Figure S1). We used the R package Amelia [22] to perform multiple imputations of missing data for a subset of contemporaneous variables (biting_rate_hcl_outdoor, biting_rate_hcl_indoor, EIR_ibpn), with health district as cross-sectional variable and the month during the year as temporal variable to account for seasonality. The variable EIR_ibpn refers to the entomological inoculation rate (EIR), i.e., the number of infectious bites received by a host within a year. Based on the rule-of-thumb we performed 7 imputations for 7% of missing data. We averaged imputed data points and used them to replace missing data in the original data set. In addition, we also imputed two incidence points for the Beoumi district, which had suspicious outliers.

Mixed effect models to analyze difference between data sources

In order to identify associations between difference in incidence rates recorded in the DHIS2 and PMI data sets, we performed a mixed-effect linear model analysis [23] on the difference of log-transformed incidence data with a random intercept for the health district. We used a backward stepwise model selection approach [24] to eliminate sequentially factors from two models (see Supplementary Tables S3, S4): model1 had only health system covariates (IRS intervention, hf_reporting_rate, N_hf_active, tpr_total, ipt1_coverage), model2 had in addition also environmental variables (rainfall, temperature, leaf, humidity, biting rates, EIR) without and with 1 month lag obtained from cross-correlation analyses. For the resulting parsimonious model, we determined the coefficient estimators and two-sided p-values.

Correlation analysis between incidence rates and contemporaneous variables

We performed a Pearson correlation analysis between monthly incidence rates and contemporaneous variables for lags of up to 1 month in the past. This analysis was performed across health districts and per incidence data source.

Bayesian structural time series analysis for impact quantification

Bayesian structural times series analysis [25, 26] is an extension of classical time series analysis techniques such as auto-regression moving average models. The method integrates elements of state-space models (i.e., observation and hidden state parameters are estimated concurrently) and Bayesian feature selection (e.g., through spike-and-slab priors, acting like a probabilistic “on-off switch” for every variable in a regression model) and provides forecasting of time series with high-dimensional time-dependent covariates. We utilize this method to construct counterfactuals for intervention impact quantification. For technical details we refer to the Supplementary Figure S2. For monthly outcome measures (malaria incidence rates or EIR) we implement local linear and seasonal effects, and use contemporaneous regressors for lagged time series that are independent from the incidence rates (e.g., rainfall, temperature). In order to train the pre-intervention model we use either the incidence data in the IRS districts (control: “none”), incidence data in both IRS and non-IRS districts (control: “incidence), or all contemporaneous covariates including incidence from all districts (control: “all covariates) but excluding entomological covariates, since those are supposed to be statistically dependent on incidence. Similarly, for the impact analysis with EIR, we use EIR data in IRS districts only (control: “none”), EIR from both IRS and non-IRS districts (control: “EIR”), or all contemporaneous covariates except incidence covariates, since those are supposed to be statistically dependent on EIR.

From the Bayesian feature selection for models with contemporaneous covariates, we extract marginalized posterior distribution of regression parameters to quantify how often a regressor has been included in the model, but also its magnitude and whether 95% of the probability mass was either positive or negative (“consistent regressors”). Causal impact is determined by the difference of predicted incidence in the absence of the intervention (i.e., from the counterfactual) and the actually recorded data in presence of the intervention, averaged over one-year after the intervention deployment. We consider causality as established if in less than 5% of 10,000 simulated counterfactuals we observe incidence inferior to the observed incidence with IRS.

Results

Factors influencing the difference in incidence between DHIS2 and PMI data

After adjusting crude incidence of malaria cases from DHIS2 and PMI data sources (Supplementary Table 2, adjustment 2), we observed a range between 15 and 100 monthly incidence points per 1000 population between September 2018 and April 2022 with peaks occurring generally between July and October (Figure 1). Two consecutive months for the control district of Sakassou had zero incidence recorded in the DHIS2 data. We considered these as missing data and imputed them using multiple imputation algorithms (see Methods). PMI incidence rates were generally higher than those based on DHIS2 data, and Sakassou followed more consistently the rainfall seasonality trends than Nassian. The impact analysis focused on incidence data until August 2021, date of the start of the second deployment of IRS. Analyzing the difference between incidence rates from the two data sources including model selection for a comprehensive set of contemporaneous and categorical variables (see Supplementary Tables S2, S3) suggested that there were several statistically significant factors explaining the difference. We identified the presence of the first IRS intervention, increased IPT1 coverage and test positivity rates to be statistically associated with a decrease in difference (P < 0.05) Table 1.

FIGURE 1

Correlations between incidence and contemporaneous variables

For the correlation analysis between monthly incidence rates and contemporaneous variables, we considered both data from the same month and with a one-month lag across all four districts before and after the intervention (Figure 2). We noticed that entomological variables were highly correlated amongst each other (e.g., 0.57 and 0.53 Pearson correlation coefficient between EIR and indoor or outdoor biting rates). Biting rates also positively correlated with malaria incidence for the PMI data (0.53 and 0.57 correlation coefficient). Strikingly, none of the entomological variables had significant correlations with malaria incidence data from DHIS2. On the other hand, DHIS2 incidence data significantly correlated with rainfall and humidity, albeit with smaller coefficients than PMI data. A similar pattern was observed for correlations between incidence and health surveillance yield and quality covariates (number of health facilities active, health facility reporting rates) which had significant correlations with PMI incidence data (0.27, and −0.32 resp.), whereas these same covariates did not correlate with DHIS2 incidence data. Only test positivity rates positively correlated with incidence data from both sources.

FIGURE 2

Quantification of causal impact of IRS towards malaria incidence

Based on our findings from the correlation analysis, we included all except the entomological covariates into our Bayesian structural time series analysis (see Methods) as these were causally dependent on the intervention. We trained a model on incidence data prior to the intervention including contemporaneous variables and utilized the resulting model estimates to predict a counterfactual for the Nassian and Sakassou district during the intervention period. Finally, to measure impact, we compared the counterfactual incidence with actually recorded incidence data (see Supplementary Figure S3). Using the incidence in the IRS districts alone during the model training (Figure 3, control = none), we obtained an estimated average decrease in incidence during the first year post intervention of 30% (95%CI: 11%–43%) for Nassian and 36% (95%CI: 20%–48%) for Sakassou using DHIS2 data. Incidence decrease according to PMI data was not significant. Including incidence data from the control districts (Dabakala for Nassian and Beoumi for Sakassou) to the pre-intervention model training resulted in significant impact of 12% (95%CI: 3%–19%) resp. 19% (95%CI: 10%–27%) for PMI data and 30% (95%CI: 19%–38%) resp. 43% (95%CI: 34%–50%) for DHIS2 data. Considering all covariates besides those related to entomology yielded an estimated impact for PMI data of 9% (not significant) for Nassian and 23% (95%CI: 12%–31%) for Sakassou. No significant impact for DHIS2 data was determined in that case. The large error estimates from the credence intervals suggest a high variability in posterior variable selection.

FIGURE 3

Contemporaneous covariates influencing the counterfactual

Utilizing the Bayesian structural time series model, we obtained regression coefficients for each of the contemporaneous covariates. Posterior distributions from the Bayesian updating stem from spike and slab priors (see Methods), such that not all coefficients were necessarily selected in each model run. We focused on the marginalized regression coefficients that consistently either had negative or positive values for 95% of the model runs in order to quantify covariates influencing the counterfactual, and consequently the evaluated impact. E.g., a consistently positive regression coefficients would mean increased incidence for the counterfactual for an increase in the covariates, such that the overall intervention impact would be have been overestimated without the covariate, and vice versa for negative regression coefficients.

Considering empirical distributions of posterior regression coefficients (Figure 4), we observed that incidence from the control district had positive estimates across datasets and health districts with high probability of inclusion for the model selection process. Health facility reporting rates were consistently associated with negative coefficients for PMI data, suggesting that for periods with increased health facility reporting during the intervention, one would have estimated a decrease in the counterfactual and consequently a decreased impact of IRS. For DHIS2 on the other hand, test positivity rate, number of health facilities active and rainfall had positive coefficients, such that an increase in these factors might be the main driver of the overestimated impact of IRS (Figure 3). We stress that the posterior distribution of covariate regression coefficients used for the impact quantification are high-dimensional and not independent from each other such that the underlying causality might rather be a combination of many factors than just those mentioned above.

FIGURE 4

Quantification of causal impact of IRS towards entomological inoculation rates

Utilizing the same approach as for the causal impact analysis towards incidence, we also investigated causal impact towards EIR. This time we excluded malaria incidence as covariates, as these were not independent from EIR rates. We observed a strong and statistically significant causal impact of IRS on the decrease of EIR at the order of 70% averaged during 1 year post intervention (Figure 5). Among consistent regressors, only EIR values from control districts were chosen with high inclusion probability (Figure not shown).

FIGURE 5

Discussion

With improved quality and availability of routine surveillance data for malaria, retrospective impact analysis becomes an important tool to bridge the gap from efficacy in clinical trials to effectiveness at operationally relevant scales such as health districts [15]. For NMCPs, retrospective impact quantification is highly relevant as it provides context-specific information to evaluate program efficiency and to plan future investments. In the present study, we interrogate clinical incidence data extracted for the first time in its full breadth from the DHIS2 routine surveillance system in Côte d’Ivoire [8]. Established since 2015, this system has since seen major improvements (e.g., including new indicators, capturing community health workers and private sector) but is still underused to provide analytics for decision support. Côte d’Ivoire has rolled out a pilot of indoor residual spraying in 2020 and 2021 in two health districts thanks to support from PMI. Concurrently with this intervention, PMI has also collaborated with the NMCP to re-digitize health facility data from paper registries to quantify the impact of IRS. Our study specifically focuses on quantifying how data quality impedes impact quantification and how a Bayesian framework could inform causality between covariates and impact indicators. We summarize and contextualize our four main findings.

First, correlation analysis with remote-sensed temporal covariates such as rainfall or biting rates revealed that DHIS2 data is not following expected patterns of seasonality whereas PMI data does. Likewise, we observed highly significant correlation of PMI incidence with indicators of surveillance quality such as reporting rates and number of active health facilities, whereas DHIS2 does not. We conclude that DHIS2 data very likely does not faithfully represent incidence dynamics at monthly scales and that coarser temporal aggregation would be better suited for analysis with such data. IRS deployment and test positivity rates are the factors most strongly associated with decreasing gaps between PMI and DHIS2 incidence data across several statistical models. We suggest that viability and sensitivity of tests and their reporting should be further improved in line with observations from key-informant interviews [8].

Second, although using a completely different method, our results of 12% resp. 19% incidence decrease 1 year post intervention for PMI data are in good agreement with a recently published analysis [9], which reports 16% resp. 15.8% reduction in Nassian resp. Sakassou. Using DHIS2 data, our statistical models estimated much higher impact of 30% reps. 42%, which is close to 44% and 47% DHIS2-based estimates for IRS in Ghana and in Uganda [27, 28]. This further highlights the importance of the data sources, while acknowledging the statistical uncertainty inherent to our estimates.

Third, following the Bayesian model averaging approach, test positivity rates, incidence from control districts and reporting rates of health facilities were identified as consistent regressors. Increased test positivity and incidence in control districts during the intervention period would increase counterfactual incidence and thus not taking these covariates into account would overestimate the impact of IRS. Likewise, increased facility reporting rates would decrease counterfactual incidence and not considering these would lead to underestimated impact.

Fourth, a separate analysis focused on EIR data, which to our knowledge has only been used to choose insecticides [29] but not for IRS impact quantification before. Our result of a strong causal impact of IRS of 70% decrease corroborates estimations of more than 80% vector mortality on walls published in end-of-spray reports [30, 31]. Transmission model studies [32] suggest that a 90% decrease in EIR in high endemic areas such as Nassian and Sakassou would yield a 38% decrease in clinical incidence under active case detection. Although causality between EIR and incidence remains elusive, our results indicate that entomological surveillance data could be used as surrogates for intervention impact. Nevertheless, the limited spatial and temporal scope of the entomological studies should warrant any generalization of an EIR decrease as metric of IRS impact.

This study has several limitations. The biggest hurdle to generalize our findings is data quality. We have extracted routine surveillance data from the DHIS2 database with the help of the monitoring and evaluation division at the NMCP. Despite upfront database validation work by the NMCP and data post-processing, our correlation analysis showed that temporal inconsistencies in the malaria incidence data remain and hamper fine-tuned time series analysis. The improvement in the PMI database concerned mainly electronic data entry, and our comparative analysis clearly points towards improved data quality, e.g., in terms of expected patterns of seasonality. Our findings may not directly extrapolate to health systems characterized by stronger, active surveillance networks or near-universal health-seeking coverage, where baseline case detection sensitivity is higher and less susceptible to missing facility reports or testing stockouts. Our conclusions regarding IRS impact and counterfactual estimation are most relevant under the following operational and methodological criteria: First, routine health facility reporting systems (e.g., DHIS2) where case counts are subject to fluctuations in reporting completeness and diagnostic testing consistency. Second, analytical workflows that explicitly screen and adjust for reporting completeness and test positivity rates to prevent overestimating intervention efficacy. Finally, evaluative frameworks that incorporate contemporaneous non-IRS control districts to factor out secular trends.

Although our Bayesian analysis considered many contemporaneous covariates, the underlying causality graph remains elementary. We focused on the relationship between each covariate and the incidence outcome and did not model the putative causal relationships between covariates.Incidence time series from the control district were consistently identified as key to achieve high accuracy for the counterfactual model training in the intervention district. This finding suggests that incidence measurements from control districts are vital for counterfactual model parameter learning as they encode unobserved variables and processes. Relying solely on surrogate data that is independent from the intervention (e.g., rainfall, vegetation index) but does not contain incidence measurements would be insufficient to construct counterfactuals that de-convolute causal impact on malaria transmission.

From the study limitations arise also several opportunities to improve the operability of routine data for both evaluation and strategic planning. Routine correlation analysis with remote-sensed data and outlier analysis could help single out spurious signals and guide the deployment of data quality improvement measures akin to those described in PMI’s re-digitization protocol [9]. Overestimating the impact of particular interventions comes to additional expenditures, which could be invested in strengthening the routine surveillance system at the level of data entry. Climate-related changes in patterns of vector dynamics and clinical malaria incidence call for systematic use of remote-sensed data and summary statistics of extreme events as contemporaneous variables to more faithfully estimate intervention impact. The main open question that arises from our study is the problem of causality within the complex dynamic process of disease transmission of malaria. Ideally, a causal diagram would single out known mechanism, such as the impact of rainfall on the infectious vector population, which should be causally related to disease incidence in the following month. Learning causal diagrams from high-quality data and using more complex observational models (e.g., over-dispersion and zero-inflation) could further improve the Bayesian inference.

In conclusion, we highlight that investments to improve data quality of routine surveillance by focusing on indicators such as test positivity and health facility reporting are key to avoiding overly optimistic estimations of the impact of IRS on clinical incidence in areas with high malaria prevalence such as Sakassou and Nassian. The careful choice of control districts for such analyses is crucial in order to factor out potential intervention confounders.

References

• 1.

  Global Malaria Program WHO Geneva. World Malaria Report 2023. Geneva: World Health Organization.

  • Google Scholar
  • View reference in article

• 2.

  Programme National de Lutte contre le Paludisme - PNLP. Plan Stratégique National De Plaidoyer En Matière De La Lutte Contre Le Paludisme En Côte D’Ivoire 2018-2023. Baltimore, United States: Johns Hopkins University (2019).

  • Google Scholar
  • View reference in article

• 3.

  BhattSWeissDJCameronEBisanzioDMappinBDalrympleUet alThe effect of malaria control on Plasmodium falciparum in Africa between 2000 and 2015. Nature (2015) 526(7572):207–11. 10.1038/nature15535

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 4.

  PryceJMedleyNChoiL. Indoor residual spraying for preventing malaria in communities using insecticide‐treated nets - pryce, J - 2022 | cochrane library (2024). Available online at: https://www.cochranelibrary.com/cdsr/doi/10.1002/14651858.CD012688.pub3/full (Accessed September 9, 2024).

  • Google Scholar
  • View reference in article

• 5.

  HiltonERRabeherisoaSRamandimbiarijaonaHRajaratnamJBelemvireAKapesaLet alUsing routine health data to evaluate the impact of indoor residual spraying on malaria transmission in Madagascar. BMJ Glob Health (2023) 8(7):e010818. 10.1136/bmjgh-2022-010818

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 6.

  ChandaEColemanMKleinschmidtIHemingwayJHamainzaBMasaningaFet alImpact assessment of malaria vector control using routine surveillance data in Zambia: implications for monitoring and evaluation. Malar J (2012) 11(1):437. 10.1186/1475-2875-11-437

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 7.

  EpsteinANamugangaJFNabendeIKamyaEVKamyaMRDorseyGet alMapping malaria incidence using routine health facility surveillance data in Uganda. BMJ Glob Health (2023) 8(5):e011137. 10.1136/bmjgh-2022-011137

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 8.

  SelingerCAngoaG. Analyse Rétrospective 2012-2022 Des Activités De Lutte Contre Le Paludisme En Côte D’Ivoire (2023).

  • Google Scholar
  • View reference in article

• 9.

  HiltonERGning-CisseNAssiAEyakouMKoffiJGnakouBet alReduction of malaria case incidence following the introduction of clothianidin-based indoor residual spraying in previously unsprayed districts: an observational analysis using health facility register data from Côte d’Ivoire, 2018–2022. BMJ Glob Health (2024) 9(3):e013324. 10.1136/bmjgh-2023-013324

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 10.

  BernalJLCumminsSGasparriniA. Interrupted time series regression for the evaluation of public health interventions: a tutorial. Int J Epidemiol (2017) 46(1):348–55. 10.1093/ije/dyw098

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 11.

  TurnerSLForbesABKarahaliosATaljaardMMcKenzieJE. Evaluation of statistical methods used in the analysis of interrupted time series studies: a simulation study. BMC Med Res Methodol (2021) 21(1):181. 10.1186/s12874-021-01364-0

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 12.

  GianacasCLiuBKirkMTannaGLDBelcherJBloggSet alBayesian structural time series, an alternative to interrupted time series in the right circumstances. J Clin Epidemiol (2023) 163:102–10. 10.1016/j.jclinepi.2023.10.003

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 13.

  LiuJSpakowiczDJAshGIHoydRAhluwaliaRZhangAet alBayesian structural time series for biomedical sensor data: a flexible modeling framework for evaluating interventions. PLoS Comput Biol (2021) 17(8):e1009303. 10.1371/journal.pcbi.1009303

  • Pubmed Abstract
  • CrossRef
  • Google Scholar

• 14.

  MideletABaillySBorelJCHyRLSchaefferMCBaillieulSet alBayesian structural time series with synthetic controls for evaluating the impact of mask changes in residual apnea-hypopnea index telemonitoring data. IEEE J Biomed Health Inform (2022) 26(10):5213–22. 10.1109/JBHI.2022.3194207

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 15.

  BurnettSMDavisKMAssefaGGogueCHinnehLDLittrellMet alProcess and methodological considerations for observational analyses of vector control interventions in Sub-Saharan Africa using routine malaria data (2025). Available online at: https://www.ajtmh.org/view/journals/tpmd/112/1_Suppl/article-p17.xml (Accessed March 4, 2025).

  • Google Scholar
  • View reference in article

• 16.

  HersbachHBellBBerrisfordPHiraharaSHorányiAMuñoz‐SabaterJet alThe ERA5 global reanalysis. Quart J R Meteoro Soc (2020) 146(730):1999–2049. 10.1002/qj.3803

  • CrossRef
  • Google Scholar
  • View reference in article

• 17.

  The PMI VectorLink Project. The PMI Vectorlink Côte d’ivoire 2019 Annual Entomological Report (2020).

  • Google Scholar
  • View reference in article

• 18.

  The PMI VectorLink Project. The PMI Vectorlink Côte d’ivoire 2020 Annual Entomological Report, January 2020–December 2020. Rockville, MD: The PMI VectorLink Project, Abt Associates. (2021).

  • Google Scholar

• 19.

  The PMI VectorLink Project. The PMI Vectorlink Côte d’ivoire 2021 Annual Entomological Report, January 2021–December 2021. Rockville, MD: The PMI VectorLink Project, Abt Associates. (2022).

  • Google Scholar

• 20.

  Institut National de la Statistique - INS, Ministère du Plan et du Développement - MPD. Programme National De Lutte Contre Le Paludisme - PNLP, Ministère De La Santé Et De L’Hygiène Publique - MSHP, ICF. République De Côte D’Ivoire Enquête De Prevalence Parasitaire Du Paludisme Et De L’Anémie (EPPA-CI) 2016. Abidjan, Côte d’Ivoire: INS/MPD/PNLP/MSHP and ICF (2016). Available online at: http://dhsprogram.com/pubs/pdf/FR330/FR330.pdf.

  • Google Scholar
  • View reference in article

• 21.

  Institut National de la Statistique, ICF. Côte D’Ivoire Enquête Démographique Et De Santé 2021 Rapport Final. Rockville, Maryland, USA et la Côte d’Ivoire: INS et ICF (2023). Available online at: https://www.dhsprogram.com/pubs/pdf/FR385/FR385.pdf.

  • Google Scholar
  • View reference in article

• 22.

  HonakerJKingGBlackwellM. Amelia II: a program for missing data. J Stat Soft (2011) 45(7):1–47. 10.18637/jss.v045.i07

  • CrossRef
  • Google Scholar
  • View reference in article

• 23.

  BatesDMächlerMBolkerBWalkerS. Fitting linear mixed-effects models using lme4. J Stat Soft (2015) 67(1):1–48. 10.18637/jss.v067.i01

  • CrossRef
  • Google Scholar
  • View reference in article

• 24.

  KuznetsovaABrockhoffPBChristensenRHB. lmerTest package: tests in linear mixed effects models. J Stat Soft (2017) 82(13):1–26. 10.18637/jss.v082.i13

  • CrossRef
  • Google Scholar
  • View reference in article

• 25.

  ScottSLVarianHR. 4. Bayesian variable selection for nowcasting economic time series. In: GoldfarbAGreensteinSMTuckerCE, editors. Economic Analysis of the Digital Economy. Chicago: University of Chicago Press (2015). p. 119–36. 10.7208/9780226206981-007

  • CrossRef
  • Google Scholar
  • View reference in article

• 26.

  BrodersenKHGallusserFKoehlerJRemyNScottSL. Inferring causal impact using bayesian structural time-series models. The Ann Appl Stat (2015) 9(1):247–74. 10.1214/14-aoas788

  • CrossRef
  • Google Scholar
  • View reference in article

• 27.

  GogueCWagmanJTynuvKSaibuAYihdegoYMalmKet alAn observational analysis of the impact of indoor residual spraying in Northern, Upper East, and Upper West Regions of Ghana: 2014 through 2017. Malar J (2020) 19(1):242. 10.1186/s12936-020-03318-1

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 28.

  RonaldMHumphreyWAdokeYJean-PierreVG. Impact of population based indoor residual spraying in combination with mass drug administration on malaria incidence and test positivity in a high transmission setting in north eastern Uganda. Malar J (2023) 22(1):378. 10.1186/s12936-023-04799-6

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 29.

  KouassiBLEdiCOuattaraAFEkraAKBellaiLGGouaménéJet alEntomological monitoring data driving decision-making for appropriate and sustainable malaria vector control in Côte d’ivoire. Malar J (2023) 22(1):14. 10.1186/s12936-023-04439-z

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article

• 30.

  The PMI VectorLink Project. The PMI Vectorlink Côte d’ivoire 2020 end-of-spray Report: August 10–September 12, 2020. Rockville, MD. the PMI Vectorlink Project. Rockville, MD: Abt Associates Inc. (2020).

  • Google Scholar
  • View reference in article

• 31.

  The PMI VectorLink Project. The PMI Vectorlink Côte d’ivoire 2021 end-of-spray Report August 2–September 4, 2021. Rockville, MD: The PMI VectorLink Project, Abt Associates (2020).

  • Google Scholar
  • View reference in article

• 32.

  CameronEBattleKEBhattSWeissDJBisanzioDMappinBet alDefining the relationship between infection prevalence and clinical incidence of Plasmodium falciparum malaria. Nat Commun (2015) 6(1):8170. 10.1038/ncomms9170

  • Pubmed Abstract
  • CrossRef
  • Google Scholar
  • View reference in article