Audit and Forecast DHIS2 Immunisation Data with Python

What this project does

This project provides an end-to-end workflow for extracting, auditing, completing, and forecasting monthly immunisation data from DHIS2. It works with Measles, BCG, and OPV1 doses reported by 13 districts over 60 completed months, from September 2021 through August 2026.

The included dataset comes from the public DHIS2 demo instance running version 2.43.1. It is sample data and contains no personal or workplace information. Therefore, the results demonstrate analytical methods and data-quality problems; they must not be interpreted as findings about real health trends.

The workflow has four main stages:

1. Retrieve aggregate monthly data through the DHIS2 analytics API or load the saved CSV snapshot.
2. Audit reporting completeness, year-on-year repetition, large changes, and possible adjacent-month swaps.
3. Estimate missing values for model training while preserving flags that identify estimates.
4. Test whether forecasting models outperform a seasonal baseline that repeats the previous year's value.

Key features

• DHIS2 API extraction: Functions in pull_dhis2_data.py find and retrieve the required indicators.
• Offline reproducibility: The notebook reads dhis2_raw_data.csv by default, so it can run without an internet connection or login.
• Completeness auditing: The analysis checks whether every indicator has one value for each district and month.
• Repeat detection: It measures how often a district's value exactly matches the same month one year earlier.
• Swap detection: It flags adjacent months whose values appear in the reverse order from the previous year.
• Structured gap filling: Missing values are estimated from the same calendar month in other years or from district level and cross-district seasonality.
• Leakage-resistant forecasting: Training and testing are separated chronologically, and training gaps are filled using training data only.
• Model comparison: Seasonal naive, SARIMA, and gradient boosting are evaluated with MAE and WAPE.

Understand the project files

• DHIS2_Immunisation_Audit_and_Forecast.ipynb: the complete analysis, including charts, audit logic, gap filling, and forecasts.
• pull_dhis2_data.py: functions for locating indicators and pulling data from DHIS2.
• dhis2_raw_data.csv: a saved snapshot used for the default offline run.
• requirements.txt: the Python package dependencies.
• README.md: project documentation and summarized findings.

Installation and setup

1. Obtain the repository

Clone or download the repository, then open a terminal in its root directory. Using a virtual environment is recommended so the project's packages remain isolated from other Python projects.

python -m venv .venv

Activate the environment with the command appropriate for your operating system, then install the required packages:

pip install -r requirements.txt

2. Start Jupyter Notebook

jupyter notebook DHIS2_Immunisation_Audit_and_Forecast.ipynb

Run the notebook cells from top to bottom. By default, the analysis loads dhis2_raw_data.csv. This is the simplest and most reproducible way to explore the workflow because it requires neither a network connection nor DHIS2 credentials.

3. Optionally configure live DHIS2 access

To retrieve fresh data, set REFRESH = True in the notebook's first code cell. The server address and login can be supplied through the DHIS2_URL, DHIS2_USER, and DHIS2_PASSWORD environment variables.

DHIS2_URL=https://your-dhis2-server.example
DHIS2_USER=your_username
DHIS2_PASSWORD=your_password

The exact syntax for setting environment variables depends on your shell and operating system. Keeping credentials in environment variables prevents them from being embedded in the notebook or committed to source control.

Only connect to a live DHIS2 instance with the data owner's approval. Use de-identified aggregate data and follow the organization's security and governance requirements.

Run the basic workflow

Step 1: Load or retrieve the monthly data

Each row represents one indicator, one district, and one month. The full analysis uses 13 districts and 60 months for each immunisation indicator. Measles is included in both the audit and forecasting stages, while BCG and OPV1 are audited. ANC indicators are excluded because only 20 months are available, which is too short for studying yearly patterns in this workflow.

With REFRESH = False, the notebook reads the saved snapshot. With REFRESH = True, it uses the API helper functions to pull current results from the configured server.

Step 2: Check reporting completeness

For each indicator, the expected number of observations is calculated as follows:

13 districts × 60 months = 780 district-months

The sample contains 725 reported values per immunisation indicator, leaving 55 missing values, or 7.1%. The missingness is highly structured rather than random:

• All gaps occur in October, November, or December.
• Eleven values are missing in each year from 2021 through 2025.
• The affected districts are Pujehun, Bonthe, Koinadugu, Kono, Bombali, and Port Loko.
• Pujehun has no October-to-December reports in any available year.
• No values are missing in 2026.

This matters because summing only the districts that reported makes the national total look artificially low. Across the 15 affected months, the uncorrected total is understated by 14.3% on average and by as much as 17.1%.

Step 3: Detect exact year-on-year repeats

The notebook compares each district-month with the corresponding month 12 months earlier. Exact matches can happen naturally, but widespread repetition is unusual in real monthly health counts.

In the demo data, 65.5% of Measles district-months in 2024, 92.4% in 2025, and 71.2% from January through August 2026 exactly match the prior year. BCG and OPV1 display nearly the same pattern.

This pattern is important for interpretation. It resembles sample data generated or copied from a template, and it gives a forecasting baseline based on last year's values an unusually strong advantage.

Step 4: Review large changes and possible month swaps

Large year-on-year changes are useful warning signals, but they are not automatically errors. The notebook also checks whether two adjacent values have exchanged positions relative to the prior year.

Potential swaps appear in March and April 2024, October and November 2024, and March and April 2026. For example, Port Loko contains the values 535 and 313 in March and April, with their order changing between years. Such a swap creates two large monthly errors even though the combined total for the pair remains unchanged.

The workflow flags possible swaps but does not correct them. The data alone cannot establish which order is correct, so the original reports should be checked before making a change.

Step 5: Fill missing training values

The project fills each gap using a two-stage method:

1. If the district reported the same calendar month in other years, use the average of those available values.
2. If the district never reported that calendar month, estimate it by multiplying the district's average level by the typical seasonal pattern for that month across all districts.

The second method is necessary for cases such as Pujehun, where October through December are absent in every available year.

To check the method, the analysis hides 15% of known October-to-December values, fills them, and compares the estimates with the actual observations. Across 21 hidden values, the average error is 8.2% and the median error is 7.9%. This is only a small validation sample, so these figures should be treated as a rough guide rather than definitive accuracy estimates.

Estimated values are not treated as observed values. They are used only to construct model training data and lagged features. Missing test observations are skipped during scoring rather than filled.

Forecast Measles doses

Use a chronological train-test split

The forecasting question is whether a model can predict the next 12 months of Measles doses for each district more accurately than repeating last year's values.

• Training period: September 2021 through August 2025, covering 48 months.
• Test period: September 2025 through August 2026, covering 12 months.
• Scored observations: 145 reported test values.

A random split would allow future observations to influence predictions of earlier months. The time-based split prevents that leakage. Training gaps are filled using training-period information only, while unreported test values from October through December 2025 are excluded from evaluation.

Compare the three models

• Seasonal naive: predicts the value observed in the same month one year earlier.
• SARIMA: fits SARIMA(0,1,1)(0,1,1,12) separately for each district. This configuration was selected before viewing test results and was not tuned.
• Gradient boosting: trains one shared model for all districts using values from 12 and 24 months earlier, calendar month, and district. Values are scaled by each district's average so districts of different sizes can share a model.

The gradient-boosting features use only observations at least 12 months old. This reflects the information that would be available when producing a full 12-month-ahead forecast.

Interpret MAE and WAPE

The notebook reports two evaluation metrics:

• MAE: the average absolute error in doses per district-month.
• WAPE: total absolute error divided by total actual doses. For example, 0.036 represents a 3.6% error.

The seasonal naive model achieves an MAE of 16.1 and WAPE of 3.6%. SARIMA records an MAE of 30.0 and WAPE of 6.6%, while gradient boosting records an MAE of 32.9 and WAPE of 7.3%.

When March and April 2026 are excluded, the seasonal naive WAPE falls to 0.8%. SARIMA reaches 5.4%, and gradient boosting reaches 5.7%. Most of the baseline's error therefore comes from the apparent March-April swap.

Why the simple baseline wins

The seasonal naive approach performs best because most values in the test period are exact copies of the previous year. A model that repeats last year's observation is nearly perfect under those conditions.

SARIMA and gradient boosting instead estimate broader patterns from only four yearly cycles. They smooth the series and cannot reproduce exact copied values as reliably. They also cannot predict an adjacent-month swap from earlier observations.

This result does not establish that seasonal naive forecasting is generally superior. It reflects the unusual structure of this demo dataset. On real data with genuine year-to-year variation, more sophisticated models would have a more meaningful opportunity to improve on the baseline.

Advanced tips for adapting the workflow

Keep observed and estimated values separate

Add or retain an explicit flag for every filled observation. Use estimates where a complete training series is technically required, but do not silently present them as reported values. Scoring should remain based on observations that were actually reported.

Investigate anomalies instead of automatically fixing them

Missing reports, repeated values, large changes, and possible swaps require different responses. A missing value may be estimated for modeling, but an apparent month swap should be verified against source documents. Automated correction can replace a visible warning with a hidden error.

Use rolling-origin evaluation

The repository uses one 12-month test window. For a more robust assessment, repeat the evaluation across several chronological windows. Each run should train only on data available before its forecast period. This shows whether model performance is stable over time rather than dependent on one unusual year.

Always benchmark complex models

Retain the seasonal naive forecast even when adding tuned SARIMA models, new features, or other machine-learning algorithms. A complex method is useful only if it consistently beats a simple and operationally credible baseline on unseen periods.

Add uncertainty when working with real data

The suggested next steps include adding prediction intervals. Point forecasts alone do not communicate uncertainty from normal variation, limited historical data, or estimated training gaps.

Automate recurring audits

The same checks can be scheduled to run after each reporting cycle. A recurring process can report newly missing district-months, unexpected year-on-year repetitions, large changes, and suspected swapped months for review.

Be cautious when extending to other indicators

Before forecasting another measure, verify that it has enough history for its seasonal cycle. The available ANC indicators contain only 20 months beginning in January 2025, so this project does not use them to study annual patterns.

Important limitations

• The source is a demo server, so the analysis says nothing about real immunisation trends.
• The forecasting comparison uses a single 12-month test window.
• Only Measles is forecast.
• The SARIMA configuration is fixed rather than tuned.
• The gap-fill validation contains only 21 hidden observations.
• Pujehun's October-to-December estimates depend on seasonal patterns from other districts.
• Suspected month swaps are flagged but not corrected.

Conclusion

This project demonstrates that data-quality auditing should come before forecasting. Its checks reveal structured missingness, extensive year-on-year repetition, and probable adjacent-month swaps that strongly affect totals and model evaluation.

Start by running the notebook against the saved snapshot, inspect each audit stage, and then enable API refresh only when you are ready to use an approved DHIS2 source. When adapting the workflow, preserve chronological evaluation, label estimated values, verify anomalies against source reports, and compare every model with a straightforward seasonal baseline.