Afghanistan's Health Care Sector: USAID's Use of Unreliable Data

Special Inspector General for

Afghanistan Reconstruction

SIGAR 17-22 Audit Report

Afghanistan’s Health Care Sector:

USAID’s Use of Unreliable Data Presents

Challenges in Assessing Program

Performance and the Extent of Progress

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care

Sector

SIGAR

JANUARY

2017

For more information contact: SIGAR Public Affairs at (703) 545-5974 or [email protected]

WHAT SIGAR REVIEWED

Since 2002, the U.S. Agency for

International Development (USAID) has

obligated almost $1.5 billion to rebuild

Afghanistan’s health care sector. As of

September 2016, the agency had 18 active

health care projects totaling $589.1 million,

2 projects were expected to begin after

2016 with estimated costs of $10.5 million,

and 23 projects, costing $642 million, have

been completed since January 1, 2011.

These projects include efforts to build the

capacity of the Afghan Ministry of Public

Health (MOPH); improve the health of

women, mothers, and children; conduct

health care surveys; and procuring and

improving the quality of medical supplies.

In 2010, USAID helped develop the U.S.

Mission in Afghanistan’s Post Performance

Management Plan (PMP) for 2011 through

2015 to help U.S. Embassy Kabul set

objectives, plan, and manage its assistance

efforts, and assess progress in meeting

those objectives. The objectives for the

health care sector are to reduce (1) the total

fertility rate, (2) the maternal mortality rate,

(3) the infant mortality rate, (4) the mortality

rate for children under 5 years of age, and

(5) the percentage of chronically

malnourished children.

The objectives of this audit were to (1)

determine the extent to which USAID

collected, assessed, and disclosed the

quality of data used to report progress in the

health care sector; (2) evaluate the extent to

which the agency assessed the impact its

projects had on health care; and (3) identify

the challenges to developing the health care

sector in Afghanistan.

WHAT SIGAR FOUND

USAID has obligated nearly $1.5 billion in assistance to develop

Afghanistan’s health care sector and publicly cites numerous achievements

made in life expectancy, child and infant mortality, and maternal mortality.

However, USAID did not disclose data quality limitations. This lack of

disclosure calls into question the extent of the achievements claimed. Given

the difficulties in collecting data, USAID’s Automated Directive System

allows USAID missions to choose the best available evidence. However,

missions are required to be transparent and communicate “any limitations

in data quality so that achievements can be honestly assessed.” In all cases

SIGAR reviewed, USAID did not disclose data limitations.

For example, for life expectancy, USAID publicly reported a 22-year increase

from 2002 to 2010. USAID did not disclose that the baseline data came

from a World Health Organization report and due to limited information in

countries like Afghanistan, adult mortality rates were estimated. In addition,

a later World Health Organization report only shows a 6-year increase for

males and an 8-year increase for females in life expectancy between 2002

and 2010. For maternal mortality, USAID’s public documents cite a

decrease from 1,600 to 327 deaths per 100,000 live births between 2002

and 2010. However, upon reviewing USAID’s data, we found that the 2002

information was based on a survey conducted in only 4 of Afghanistan’s

then-360 districts. USAID’s own internal documentation acknowledged the

limitations.

USAID has also relied on data from the MOPH’s Health Management

Information System (HMIS), which contains information entered by Afghans

working at clinics and hospitals throughout the country. This includes

information on the number of patients seen and number of births that

occurred at each facility. However, according to the director general of the

MOPH department that oversees the system, “The data in HMIS [are] not

100 percent complete.” Furthermore, in 2014, the World Bank found that

although HMIS officials in Kabul require provincial officers to verify the

accuracy of reports collected in their provinces by visiting the health

facilities themselves, the officials indicated that “they rarely travelled

outside the provincial capital and rarely verified the reports.”

We found that USAID’s project evaluations and performance reports were

not linked to the broader health care assistance objectives included in the

PMP for Afghanistan, and the agency’s performance monitoring effort

lacked the information needed to prove that its efforts helped achieve its

objectives. For example, USAID provided us with final performance reports

for 8 of the 20 completed projects. Based on our review of these eight

reports, we determined that there was not a direct link between these

SIGAR

Special Inspector General for

Afghanistan Reconstruction

January 2017

Afghanistan’s Health Care Sector: USAID’s Use of Unreliable Data

Presents Challenges in Assessing Program Performance and the

Extent of Progress

SIGAR 17-22 AUDIT REPORT

For more information contact: SIGAR Public Affairs at (703) 545-5974 or [email protected]

reports and the five health assistance objectives listed in the PMP. For example, the final performance report for

the Basic Support for Institutionalizing Child Survival-III project discussed child malnutrition, one of the five health

care assistance objectives, but calculated child malnutrition differently than the metric used in the PMP.

USAID guidance only requires at least one external evaluation but does not specify when that evaluation is to be

conducted. Not having an independent final evaluation forces USAID to rely on reports from implementing partners

that may have a potential conflict of interest because the implementing partner also performed the project. These

reports could be biased, increasing the risk that USAID is using inaccurate information to influence decisions about

future health care projects. For example, in August 2012, a final report written by the implementing partner for the

$100.5 million Tech-Serve project claimed that it strengthened the MOPH Grants and Contract Management Unit’s

capacity to handle donor funds. However, only 4 months later, USAID’s own assessment directly contradicted the

Tech-Serve implementing partner’s final report, and USAID concluded that it cannot rely on the MOPH’s systems

and internal controls to manage donor funds.

Additionally, USAID did not contract for an external evaluation of the $259.6 million Partnership Contracts for

Health (PCH) project, the agency’s largest health care project in Afghanistan. According to USAID, the PCH project

did not need an external or final evaluation because both the USAID Office of Inspector General and SIGAR had

reviewed the project. USAID justified waiving its own requirement for an external evaluation of PCH on the basis of

these prior reviews. However, the USAID Office of Inspector General’s and SIGAR’s reviews did not examine the

project’s overall effectiveness or how it related to the health objectives in the PMP.

Finally, Afghanistan faces several challenges to developing a strong, sustainable health care sector. The Afghan

government lacks funds to operate and sustain its health care facilities; hospitals are unable to provide adequate

care; health care facilities lack qualified staff; and corruption throughout the system remains a concern. Because of

these challenges, many Afghans seek health care services abroad. According to a 2014 Medecins Sans Frontieres

report, Afghans have limited faith in the quality of their health care system. The report states that four out of five

Afghans bypassed their closest public clinic primarily because they believed there were problems with the

availability or quality of staff, services, or treatments there. As a result, according to MOPH, USAID, and World

Health Organization officials, Afghans spend approximately $285 million annually on health care services in other

countries, depriving the health care sector of a vital source of revenue and further weakening the government’s

ability to sustain the facilities that donors are now funding. Furthermore, according to one MOPH survey, 99 percent

of respondents said the medical care they received abroad was better than the care they received at home.

WHAT SIGAR RECOMMENDS

SIGAR is making three recommendations to USAID to ensure that government decision makers and the general public

have an accurate understanding of progress in the Afghan health care sector, to determine how USAID’s efforts have

directly contributed to reported gains in Afghanistan’s health care system, and to ensure that USAID has more insight

into the accuracy and reliability of implementing partners’ final performance reports. Specifically, we recommend that

the USAID Mission Director for Afghanistan (1) acknowledge in external reporting the limitations associated with

surveys and data the agency uses to demonstrate its achievements in the health care sector in Afghanistan, (2)

amend mission guidelines for conducting project reviews in Afghanistan to require an explicit discussion of the

applicable PMP objectives, and (3) take action to validate the accuracy of final health care project reports submitted

by implementing partners in Afghanistan. USAID concurred with all three of our recommendations.

January 19, 2017

The Honorable Gayle E. Smith

Administrator, U.S. Agency for International Development

Mr. Hebert B. Smith

USAID Mission Director for Afghanistan

This report discusses the result of SIGAR’s audit of the U.S. Agency for International Development’s (USAID)

efforts to support Afghanistan’s health care sector since January 2011. We (1) determined the extent to which

USAID collected, assessed, and disclosed the quality of the data it used to report progress in the health care

sector; (2) evaluated the extent to which the agency assessed the impact its projects had on health care; and

(3) identified the challenges to developing the health care sector in Afghanistan.

We are making three recommendations to USAID to ensure that government decision makers and the general

public have an accurate understanding of progress in the Afghan health care sector, to determine how USAID’s

efforts have directly contributed to reported gains in Afghanistan’s health care system, and to ensure that

USAID has more insight into the accuracy and reliability of implementing partners’ final performance reports.

Specifically, we recommend that the USAID Mission Director for Afghanistan (1) acknowledge in external

reporting the limitations associated with the surveys and data the agency uses to demonstrate its

achievements in the health care sector, (2) amend mission guidelines for conducting project reviews in

Afghanistan to require an explicit discussion of the applicable Post Performance Management Plan objectives,

and (3) take action to validate the accuracy of final health care project reports submitted by implementing

partners in Afghanistan.

We received written comments of a draft of this report from USAID, which are reproduced in appendix III. USAID

concurred with all three of our recommendations and outlined steps it was taking to address them.

SIGAR conducted this work under the authority of Public Law No. 110-181, as amended, and the Inspector

General Act of 1978, as amended, and in accordance with generally accepted government auditing standards.

John F. Sopko

Special Inspector General

for Afghanistan Reconstruction

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page iv

TABLE OF CONTENTS

Background .................................................................................................................................................................. 2

USAID Did Not Disclose Limitations with the Data It Used to Report on Health Care Sector Progress .................. 5

Project Reports for USAID Projects Did Not Align with Broader Health Assistance Objectives, and the Agency

Primarily Relied on Implementing Partners for Evaluations...................................................................................... 9

Afghanistan’s Health Care Sector Remains Inadequate and Faces Many Challenges ......................................... 12

Conclusion .................................................................................................................................................................. 15

Recommendations .................................................................................................................................................... 16

Agency Comments ..................................................................................................................................................... 16

Appendix I - Scope and Methodology ....................................................................................................................... 17

Appendix II - USAID’s Active, Completed, and Planned Health Care Projects in Afghanistan since January 1,

2011 ........................................................................................................................................................................... 19

Appendix III - Comments from the U.S. Agency for International Development ..................................................... 24

Appendix IV - Acknowledgments ............................................................................................................................... 28

TABLES

Table 1 - USAID’s Health Care Projects Active or Planned after January 1, 2011, by Focus Area ......................... 4

Table 2 - Active USAID Health Care Projects, as of September 30, 2016 ............................................................. 19

Table 3 - USAID Health Care Projects Completed since January 1, 2011.............................................................. 21

Table 4 - Planned USAID Health Care Projects, as of February 2016 .................................................................... 23

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page v

ABBREVIATIONS & ACRONYMS

ADS

Automated Directive System

BPHS

Basic Package of Health Services

EPHS

Essential Package of Hospital Services

HMIS

Health Management Information System

MOPH

Ministry of Public Health

NGO

nongovernmental organization

PCH

Partnership Contracts for Health

PMP

Post Performance Management Plan

SEHAT

System Enhancement for Health Action in Transition

UNICEF

United Nations Children’s Fund

USAID

U.S. Agency for International Development

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 1

According to the World Bank, in 2003, Afghanistan’s health sector was one of the poorest in the world. Overall,

the country lacked health coverage, prenatal care, vaccine coverage, and health facilities. At the time, most

rural infrastructure was either destroyed or dilapidated. Years of conflict disrupted the Afghan government’s

ability to provide health services, leaving the population to rely on nongovernmental organizations (NGO) to

fund and provide those services.

Since 2002, the U.S. Agency for International Development (USAID) has obligated almost $1.5 billion to rebuild

Afghanistan’s health care sector. According to a USAID fact sheet, the international efforts have led to

significant progress in providing assistance to Afghanistan’s health care sector.

For example, this fact sheet

reports that 57 percent of Afghans now live within an hour’s walk of a health care facility, up from only 9

percent in 2002, making it easier for them to seek medical attention, consult trained health care staff, and get

medicine.

USAID currently is providing funding to the World Bank as part of a donor-coordinated effort to provide

essential health services throughout Afghanistan. According to USAID, in 2014, an average of nearly one

million people were treated each month at USAID-supported facilities; of these, 76 percent were women and

children under 5 years old.

However, despite USAID and other donors’ efforts and funding, Afghanistan’s health care sector remains weak

and faces significant challenges in providing the necessary services for the Afghan people, operating and

maintaining facilities, and creating a self-sustaining system.

The objectives of this audit were to (1) determine the extent to which USAID collected, assessed, and disclosed

the quality of data used to report progress in the health care sector; (2) evaluate the extent to which the

agency assessed the impact its projects had on health care; and (3) identify the challenges to developing the

health care sector in Afghanistan.

To accomplish our objectives, we reviewed USAID regulations found in the Automated Directives System

(ADS), as well as USAID’s Evaluation Policy. We reviewed the Strategic Objective Grant Agreement between the

U.S and Afghan governments in support of a better educated and healthier population, and the Post

Performance Management Plan (PMP) for the U.S. mission in Afghanistan.

We also collected and reviewed all

national health surveys conducted between 1997 and 2015. We reviewed all externally conducted

performance evaluations and all final project evaluations conducted from January 2011 through December

2015 for their use of metrics and objectives found in the PMP.

We interviewed officials from USAID, the Afghan Ministry of Public Health (MOPH), the World Bank, and the

World Health Organization. We conducted our work in Washington, D.C., and Kabul, Afghanistan, from April

2015 to January 2017, in accordance with generally accepted government auditing standards. Appendix I has

a more detailed discussion of the scope and methodology used for this audit.

World Bank, Better Health Outcomes for Women, Children, and the Poor, April 18, 2013.

USAID Mission for Afghanistan, Health Sector Fact Sheet, June 14, 2014.

We reviewed ADS chapters 202, Achieving; 203, Assessing and Learning; and 579, USAID Development Data.

Department of State and USAID, U.S. Foreign Assistance for Afghanistan: Post Performance Management Plan 2011-

2015, September 2, 2010.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 2

BACKGROUND

Afghan Government’s Health Care Policies and Guidance

In 2002, Afghanistan’s interim government made improving the country’s health care sector a national priority

and established the Interim Health Policy and Strategy.

According to the 2005 Afghanistan National Health

Policy, the Interim Health Policy and Strategy laid the foundation for the health care sector by providing a

strategic plan and short-term actions that could be taken with the Afghan government’s limited resources.

Additionally, the Interim Health Policy and Strategy described how the MOPH would operate under the new

administration. The Afghanistan National Health Policy established the plans needed to implement health care

services, reduce the country’s morbidity and mortality rates, and support institutional development within the

MOPH.

It also highlighted the need for a strategic approach to health care delivery, designed to address how

citizens could receive care in areas that did not have health care services, were underserved, or had lost

access due to an emergency.

In 2008, the Afghan government released the Afghan National Development Strategy, which established its

strategy, policies, programs, and projects to be implemented over the next 5 years.

The strategy’s health care

objective was to improve the health and nutrition of the Afghan people by providing quality health care and

promoting healthy lifestyles. The MOPH planned to enhance coordination with partner organizations, donors,

and the private sector to coordinate delivering health care by setting and distributing policies, standards, and

guidelines.

In 2011, with help from USAID and international donors, the MOPH developed its Strategic Plan.

The plan set

target results to be achieved by 2015, such as increased access to primary health care; reduced mortality

rates for mothers, infants, and children under 5 years of age; and full immunization coverage across

Afghanistan. The targets also include more specific goals of providing primary health care to 75 percent of the

population, as well as reducing the mortality rate for children under 5 years of age from 257 to 117 deaths per

1,000 live births and the mortality rate for mothers from 1,600 to 960 deaths per 100,000 live births.

Structure of the Afghan Health Care Sector

In March 2003, the MOPH and the World Health Organization, in conjunction with USAID, the United Nations

Children’s Fund (UNICEF), and other partners, established the Basic Package of Health Services (BPHS) for

Afghanistan. BPHS is intended to provide a standardized package of basic health care services through small,

local clinics and district hospitals with the goal of improving health and nutrition, with a focus on women and

children. BPHS is the foundation of the Afghan health care sector and encompasses health clinics, basic health

centers, comprehensive health centers, and district hospitals, and specifies the staff, equipment, diagnostic

services, and medications each type of health care facility should have to provide services.

Government of Afghanistan, Interim Health Policy and Strategy for 2002-2004, February 2003.

Government of Afghanistan, Afghanistan National Health Policy 2005-2009, September 2005.

Government of Afghanistan, Afghan National Development Strategy for 2008-2013, n.d.

Government of Afghanistan, Strategic Plan for the MOPH (2011-2015), May 2011.

According to the Islamic Republic of Afghanistan: Millennium Development Goals Report 2012 on Afghanistan’s

progress in achieving the Millennium Development Goals, set by the United Nations in September 2000, the target for

maternal mortality was achieved as early as 2010, and the target for 2015 was set at 315 deaths per 100,000 live births.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 3

In 2005, the MOPH started the Essential Package of Hospital Services (EPHS) as a supplement to BPHS. EPHS

focuses on provincial and regional hospitals, and seeks to (1) identify a standardized package of hospital

services at each type of hospital; (2) provide the MOPH, private sector, NGOs, and donors, such as USAID,

guidance on how hospitals should be staffed, equipped, and supplied with materials and drugs; and (3)

promote a health referral system that integrates BPHS facilities, specifically the district hospitals, with

provincial and regional hospitals. District hospitals also serve as the link between the BPHS and EPHS.

Afghanistan’s health care sector consists of small clinics located in rural areas, as well as district, provincial,

and regional hospitals. Clinics are designed to bring a standardized package of core health services to all areas

in Afghanistan. District hospitals are intended to bring professional inpatient and emergency services closer to

rural areas to reduce the maternal mortality rate, the infant mortality rate, and the mortality rate for children

under 5 years old. Provincial hospitals receive patients referred from the local clinics and district hospitals, and

accept emergency patient visits. Provincial hospitals are more advanced than district hospitals in that the

provincial hospitals offer additional services, such as the ability to treat heart failure, trauma, and most

gynecological conditions. Regional hospitals primarily treat patients referred from lower-level hospitals with

surgical and other equipment, and have the expertise needed for assessing, diagnosing, stabilizing, and

treating patients, or referring those patients back to lower levels of care.

USAID’s Health Care Strategy and Projects in Afghanistan

Since 2002, USAID has helped the MOPH contract with NGOs that provide health care services. Additionally,

the agency has provided technical support to the health care sector and has funded projects to improve the

health of all Afghans.

In 2009, USAID helped develop the PMP for 2011 through 2015 to help the U.S. Embassy in Kabul set

objectives, plan, and manage its assistance efforts, and assess progress in meeting those objectives. The

objectives for the health care sector are to reduce (1) the total fertility rate, (2) the maternal mortality rate, (3)

the infant mortality rate, (4) the mortality rate for children under 5 years of age, and (5) the percentage of

chronically malnourished children.

The PMP also lists many intermediate results indicators, such as

increasing the number of antenatal care visits conducted by skilled providers from U.S.-assisted facilities,

increasing the number of children under 5 years of age who received vitamin A from U.S.-supported programs,

and reducing the average number of days the MOPH’s Grant Contracts Management Unit spends processing

payments to NGOs.

According to USAID, as of September 2016, the agency had 18 active health care projects totaling $589.1

million, 2 projects expected to begin after 2016 with estimated costs of $10.5 million, and 23 projects, costing

$642 million, that have been completed since January 1, 2011.

Table 1 summarizes USAID’s projects that

were active or planned after January 1, 2011, by focus area. Appendix II has a complete list of health care

projects that were active as of September 30, 2016; completed between January 1, 2011, and September 30,

2016; and planned, as of February 2016.

USAID’s PMP states that the percentage of chronically malnourished children is “calculated as the percentage of children

younger than 24 months whose weight is more than two standard deviations below the median weight for children in that

age group.”

USAID uses the term “antenatal care” to describe what is commonly referred to as “prenatal care.”

According to USAID, some of these projects are categorized technically as activities, rather than projects. According to

the ADS glossary of terms, an activity is a “subcomponent of a project.” However, USAID did not provide documentation

demonstrating what subcomponent of a project these activities fall under in order to meet the definition of activity;

therefore, we will continue to refer to all activities as projects.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 4

USAID’s largest health care projects have focused on funding the Afghan government’s health care facilities.

From July 2008 through June 2015, USAID spent $259.6 million on public health care facilities in 13 provinces

under its Partnership Contracts for Health (PCH) project.

In 2013, the agency started funding the World

Bank’s System Enhancement for Health Action in Transition (SEHAT) project, which seeks to support BPHS and

EPHS nationwide. USAID funds SEHAT through the Afghanistan Reconstruction Trust Fund, administered by the

Bank, which in turn gives the MOPH funds to contract with NGOs for health services.

When PCH ended in

June 2015, SEHAT took over funding the PCH facilities. USAID reported that its estimated contributions to

SEHAT would total $227.7 million from June 2013 through June 2018.

ADS Chapter 203, Assessing and Learning, which was applicable during the scope of our audit, lists USAID’s

requirements for project monitoring, evaluations, and data quality assurance. It requires each of the agency’s

missions to create a mission-wide PMP.

Similarly, the missions’ program offices are required to develop a

monitoring plan that aligns with the PMP for that mission for each project, including health care projects. To

carry out this monitoring plan, ADS 202, Achieving, which was also applicable during the scope of our audit,

allows the agency to use alternative methods, such as third-party monitors, for it missions operating in

designated high-threat environments.

For project evaluations, USAID must contract for an external evaluation

of each large project, and implementing partners must submit a final report for each project, regardless of its

size.

The 13 provinces that received PCH funding were Badakhshan, Baghlan, Bamyan, Faryab, Ghazni, Herat, Jowzjan, Kabul,

Kandahar, Khowst, Paktika, Paktiya, and Takhar.

The Afghanistan Reconstruction Trust Fund is a source of on-budget funding for the Afghan government provided by

international donors, including the United States, and managed by the World Bank. We are currently conducting an audit of

the World Bank’s efforts to monitor, manage, and account for U.S. contributions into the fund.

USAID removed Chapters 202 and 203 from the ADS in September 2016

Third-party monitors are outside entities contracted with by USAID to monitor actives and verify results.

ADS 203.3.1.3 classifies a large project as one that receives more than the average amount of funding for all projects in

a sector, such as the health care sector.

Table 1 - USAID’s Health Care Projects Active or Planned after January 1, 2011, by Focus Area

Focus Area

Number of

Projects

Estimated Total

Amount

Mothers, women, and children

$74.0 million

Nutrition

$111.1 million

Communicable disease prevention

$70.2 million

Procuring and improving the quality of medical supplies

$117.4 million

Development, capacity building, and direct assistance to the MOPH

$856.1 million

Health care surveys

$12.7 million

Total

$1.24 billion

Source: SIGAR analysis of USAID data

The focus areas reflect our categorizations of USAID’s projects.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 5

USAID DID NOT DISCLOSE LIMITATIONS WITH THE DATA IT USED TO REPORT

ON HEALTH CARE SECTOR PROGRESS

USAID often uses data from surveys and studies collected by third parties and other governments to report on

health care statistics in Afghanistan due to the security environment and other factors in Afghanistan. These

surveys and studies often have limitations and caveats as to the quality of the data contained in them.

However, USAID did not disclose these limitations when reporting progress in the Afghan health care sector.

ADS 202 authorized USAID missions operating in designated high-threat environments, like Afghanistan, to use

alternative monitoring methods, such as third-party monitors. ADS 203.3.2.1 allowed USAID to collect data

itself, from its implementers, or from other sources, and it instructs the agency to “work with implementers to

resolve any problems with data collection.” To assess progress in the health care sector, USAID uses health

care data that the Afghan government and international organizations collect.

The Afghan government collects data from two main sources: (1) various national health surveys conducted by

the Central Statistics Organization or the MOPH; and (2) information from the MOPH’s Health Management

Information System (HMIS). As acknowledged in the PMP, some health care-related data can be obtained only

through health surveys. When conducting surveys, the Afghan government works with international

organizations, such as UNICEF; aid organizations, such as the United Kingdom’s Department for International

Development; and other technical advisors. The government uses these surveys to determine various metrics,

such as infant, child, and maternal mortality. According to the Director of MOPH’s HMIS department, the HMIS

database contains data that Afghan clinics and hospitals, including those previously funded under the PCH

project and now under SEHAT, enter directly into the database. Examples of the types of data in the database

are the number of patients and number of births for each facility. According to USAID, the MOPH’s contracts

with the NGOs operating the health care facilities require that the NGOs enter accurate data into HMIS in a

timely manner.

In addition to these sources, USAID uses various international organizations’ estimates to report health care

data in Afghanistan. These organizations, such as the World Health Organization and UNICEF, obtain their

estimates using a methodology developed for all World Health Organization member states in 2003. The

methodology starts with a systematic review of all available data, such as surveys and censuses, to identify

trends in children under 5 years of age and adult mortality rates.

USAID Did Not Establish Timely Baselines or Disclose Limitations in the Data the

Agency Used to Report Progress

We found that USAID publicly reported numerous improvements in Afghanistan’s health care sector without

including disclaimers or caveats about the limitations of the data it relied on. These omissions call into

question the validity of the achievements USAID has suggested are related to its expenditure of nearly $1.5

billion in assistance provided to develop the Afghan health care sector. For some of the claims of progress

made by USAID, the baseline and progress data are either selected from sources that based their data on

unreliable information or from surveys that did not represent the entire country. In one instance, the selective

use of data could have potentially overstated the gains made toward increasing life expectancy in Afghanistan.

Since 2003, USAID’s ADS has required establishing baseline values at the beginning of a strategy or project.

For example, ADS 203, Assessing and Learning states, “The Operating Unit should include performance

baselines and set performance targets that can optimistically but realistically be achieved within the stated

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 6

timeframe and with the available resources.”

According to ADS 203, “Every performance indicator…must

have a baseline value at the beginning of a strategy or project.”

The regulation states that baselines are

required “to learn from and be accountable for the change that occurred during the project/activity with the

resources allocated to that project/activity.”

The PMP suggests that baseline studies should be conducted in

the first year or two of the PMP’s implementation, including “surveys on outcome and impact indicators be

conducted in Quarter 1 or Quarter 2 of [fiscal year] 2011 (baseline survey) and follow-on surveys a year or two

later.”

USAID does not directly collect health care data, but instead relies on data the Afghan government or

international organizations collect. Agency officials said, “The Afghanistan Mission used the best baseline

number established at that time by the Afghan Government and international [organizations].”

According to

USAID, ADS 203 regulations gives the mission flexibility, stating that “USAID must rely on the best available

evidence to rigorously and credibly make hard choices, learn more systematically, and document program

effectiveness.”

According to USAID officials we interviewed, the mission in Afghanistan used data that was as

complete and consistent as any available and as security permitted.

Although the ADS provides USAID flexibility in assessing data quality, ADS 203.3.2.2 states that USAID

missions should be transparent and

share information widely and report candidly. Transparency involves: (1) Clearly and accurately

conveying the problems that impede progress and the steps that are being taken to address them; (2)

Communicating any limitations in data quality so that achievements can be honestly assessed; and (3)

Clearly communicating when results are achieved jointly with the host country or other development

partners.

USAID has often cited progress in the health care sector since 2002. However, before 2010, USAID did not

attempt to establish a comprehensive set of baseline health care indicators, such as life expectancy, child and

infant mortality, or maternal mortality. As a result, the agency had to rely on weak, non-representative baseline

data used by the Afghan government or international organizations to show progress.

For Afghanistan, key baseline and subsequent survey data used to show gains in life expectancy and mortality

rates for infants, mothers, and children under 5 years of age have quality limitations.

To calculate progress made in the health care sector, USAID compared the infant, child, and maternal mortality

rates, as well as life expectancy baseline numbers to the 2010 Afghan Mortality Survey but did not disclose the

limitations in the quality of the data used. For example, one specific limitation was that the survey excluded,

due to security concerns, the rural areas of Helmand, Kandahar, and Zabul provinces, which constitute nearly

10 percent of the Afghan population. Moreover, the British & Irish Agencies Afghanistan Group reported that

the personnel involved in the collection of data for the 2010 Afghan Mortality Survey said they were not

USAID, ADS 203.3.4.5, Assessing and Learning, effective January 31, 2003.

USAID, ADS 203.3.9, Assessing and Learning, effective November 2, 2012.

USAID, ADS 203.3.9, Assessing and Learning, effective November 2, 2012. The prior version of ADS 203, section

203.4.5 enacted in September 2008, also called for establishing baseline values.

Department of State and USAID, U.S. Foreign Assistance for Afghanistan: Post Performance Management Plan 2011-

2015, September 2, 2010, p. 9.

USAID response to SIGAR request for information, February 12, 2016.

USAID, ADS 203.1, Assessing and Learning, effective November 2, 2012.

USAID, ADS 203.3.2.2 Assessing and Learning, effective November 2, 2012.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 7

confident in the data collected in rural areas.

The group stated that during the data collection process,

trained surveyors were replaced by “untrained and largely illiterate local villagers” in insecure areas.

For mortality rates for infants and children under 5 years of age, USAID used baseline rates of 165 deaths per

1,000 live births and 257 deaths per 1,000 live births, respectively. Using these baseline rates to compare

against the 2010 Afghan Mortality Survey, USAID determined that infant mortality decreased by 53 percent

while mortality for children under 5 years of age decreased by 62 percent between 2009 and 2010.

Baseline

data came from UNICEF’s 2009 State of the World’s Children. However, previous State of the World’s Children

reports show that these mortality rates were not updated between 1991 and 2009.

Using a 2002 survey conducted by the U.S. Centers for Disease Control and Prevention and UNICEF that

reported 1,600 maternal deaths per 100,000 live births as its baseline, USAID reported that maternal

mortality achieved a dramatic decrease to 327 deaths per 100,000 live births from 2002 to 2010.

However,

the baseline survey was conducted in only 4 of the 360 districts that existed in Afghanistan in 2002.

Furthermore, according to the author of the report, ultimately only data from 3 of the 4 districts were used in

the survey’s estimate because one district, Ragh, reported 6,500 maternal deaths per 100,000 live births and

was deemed an outlier. USAID’s own internal documentation acknowledges that the report covered less than 4

percent of Afghanistan’s population, but still was used to produce national estimates because no other data

were available. However, the agency did not mention these limitations in its external reporting on progress

made in Afghanistan’s health care sector.

For life expectancy, USAID reported a 22-year increase in life expectancy from 2002 to 2010.

To reach this

conclusion, USAID used data from the World Health Organization’s World Health Report 2005 of 41 years for

men and 42 years for women as a baseline and compared this data to the 2010 Afghan Mortality Survey,

which reported life expectancy for men and women to be 63 and 64 years, respectively. The limitations in the

2010 Afghan Mortality Survey, which are discussed above, and additional limitations in the World Health

Report 2005 were not conveyed by USAID. For example, the

World Health Report 2005

indicates that due to

limited information in countries like Afghanistan, adult mortality rates were estimated.

Furthermore, if USAID had compared the 2005 and 2012 World Health Organization reports, instead of using

the 2010 Afghan Mortality Survey, life expectancy calculations would have changed from the reported 22-year

increase for males and females to 6 years for males and 8 years for females. This is because the World Health

Report 2012 report noted life expectancy in Afghanistan as 47 years for males and 50 years for females in

2009, versus the 63 years for males and 64 years for females reported in the Afghan Mortality Survey. We sent

a formal request for information to USAID asking why the agency did not use these sources of health care data

British & Irish Agencies Afghanistan Group is a nonprofit advocacy and networking agency that aims to supports

humanitarian and development programs in Afghanistan.

British & Irish Agencies Afghanistan Group, Maternal Mortality Reported Trends in Afghanistan: Too Good to Be True?,

September 2015.

The change in infant mortality rate is calculated by comparing the baseline figure of 165 deaths per 1,000 live births

reported in the 2009 UNICEF State of the World’s Children report to the 77 deaths per 1,000 figure in the 2010

Afghanistan Mortality Survey, resulting in a 53 percent reduction. Similarly, the change in the under-5 mortality rate is

calculated by comparing the 257 figure from the UNICEF report, to the 97 figure from the Afghanistan Mortality Survey

report, resulting in a decrease of 62.2 percent.

MOPH, U.S. Centers for Disease Control and Prevention, and UNICEF, Maternal Mortality in Afghanistan: Magnitude,

Causes, Risk Factors and Preventability, November 6, 2002.

The World Health Organization’s The World Health Report 2005 reports life expectancy for men and women were 41 and

42 years, respectively, for 2002. The 2010 Afghan Mortality Survey, conducted by the MOPH, UNICEF, World Health

Organization, and others, found life expectancy for men and women to be 63 and 64 years, respectively.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 8

and others. The agency responded that because the MOPH used the 2010 Afghan Mortality Survey data,

USAID also used these figures. USAID also stated in its response that the Afghan Mortality Survey was

identified as the best source of data because:

 No other survey measured life expectancy and maternal mortality in Afghanistan;

 Other small-scale surveys, such as the World Bank-supported Afghanistan Health Survey in 2006, only

reported mortality rates for infants and children under 5 years of age

 Compared to the Afghan Mortality Survey, the methodology and sample size of other surveys were not

very robust; and

 Other surveys, such as the Multiple Indicator Cluster Survey and National Risk and Vulnerability

Assessment, were too broad in scope, and included data on economics, agriculture, education, and

other areas.

In 2010, USAID awarded the Johns Hopkins Bloomberg School of Public Health’s Institute for International

Programs a $1.9 million cooperative agreement to replicate a maternal mortality survey in four districts that

were surveyed by the U.S. Centers for Disease Control and Prevention and UNICEF in 2002.

For the new

Reproductive Age Mortality Survey II, the team collected data only in two of the districts because of security

concerns.

According to USAID, the agency, in consultation with the MOPH, decided not to release the survey

because it was not representative of the country. However, the survey’s author said it still has some value

because it documented progress in the two districts and should be released.

USAID also uses data from HMIS, which MOPH officials and independent assessments have raised concerns

about. According to the director general of the department that oversees the system, the data in HMIS are not

100 percent complete.

Similarly, a recent report by the Independent Joint Anti-Corruption Monitoring and

Evaluation Committee stated that the

HMIS is viewed as a valuable asset (and potentially) a reliable source of support for

management coherence across the whole of the Public Health system. However, reliable

HMIS monitoring of management functions, administrative processes, and services delivery

have been compromised system-wide. All types of stakeholders expressed a generally low

level of confidence in the quality and integrity of monitoring and the subsequent evaluations

of what has been observed, inspected, and/or formally audited in the management,

administration, and provision of care in the health sector.

Concerns about monitoring also surfaced in a 2014 World Bank report. The Bank found that, although HMIS

officials in Kabul require provincial officers to verify the accuracy of reports gathered throughout their

provinces by visiting the health facilities themselves, the provincial officers “indicated they rarely travelled

outside the provincial capital and rarely verified the reports.”

The report also stated that officials from

MOPH, U.S. Centers for Disease Control and Prevention, and UNICEF, Maternal Mortality in Afghanistan: Magnitude,

Causes, Risk Factors and Preventability, November 6, 2002. The four districts surveyed were Kabul City, Kabul province;

Alisheng, Laghman province; Maiwand, Kandahar province; and Ragh, Badakhshan province.

USAID and Johns Hopkins Bloomberg School of Public Health, Reproductive Age Mortality Survey II: Maternal Mortality in

Afghanistan, n.d.

SIGAR interview with the Director General of Health Management Information System Department, June 7, 2015.

Independent Joint Anti-Corruption Monitoring and Evaluation Committee, Vulnerability to Corruption Assessment in the

Afghan Ministry of Public Health, June 4, 2016. The Afghan government established the Independent Joint Anti-Corruption

Monitoring and Evaluation Committee in 2010 to monitor and evaluate national and international efforts to fight corruption

in Afghanistan. It reports to the Afghan President, Parliament, the public, donor nations, and international organizations.

World Bank, Critical Administrative Constraints to Service Delivery: Improving Public Services in Afghanistan’s

Transformational Decade, May 2014.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 9

implementing NGOs and the MOPH said HMIS data from several provinces were not accurate because the

ministry does not check them.

In an attempt to gather more reliable data, USAID is funding the Afghanistan Demographic Health Survey, led

by the MOPH and the Afghanistan Central Statistics Organization with international assistance. According to

both USAID and MOPH officials, this survey is more comprehensive, and the same methodology is used in 90

other countries. The findings of this survey could establish more accurate mortality rates. The entire survey and

its findings are scheduled to be released by January 2017.

In May 2016, Afghanistan’s Central Statistics Organization and the MOPH released the preliminary results of

the survey. The preliminary results include data on mortality rates for infants and children under 5 years of

age. The survey showed a pattern of decreasing mortality rates for both age groups during the 15 years prior to

the survey. While the survey indicates a decline in those rates, the baseline data that the Demographic Health

Survey used for the period from 2001 to 2005 is significantly lower than the data USAID used as baselines to

show the progress for the mortality rates for infants and children under 5 years of age. According to the

preliminary report, “A detailed assessment [of the declining pattern] will be carried out in the main report.”

PROJECT REPORTS FOR USAID PROJECTS DID NOT ALIGN WITH BROADER

HEALTH ASSISTANCE OBJECTIVES, AND THE AGENCY PRIMARILY RELIED ON

IMPLEMENTING PARTNERS FOR EVALUATIONS

Final Performance Reports and External Evaluations Did Not Discuss How Projects

Aligned with the Five Health Assistance Objectives in the 2011 through 2015 PMP

We found that USAID’s final performance reports and external evaluations were not linked to the broader

health care assistance objectives included in the PMP for Afghanistan, and the agency’s performance

monitoring effort lacked the information needed to prove that its efforts helped achieve its objectives.

According to ADS 203.3.3, every USAID mission has to prepare a PMP with assistance objectives and

indicators. The regulation states that monitoring and conducting evaluations of projects and activities helps

missions determine whether they are making any progress toward achieving the objectives. To that end, ADS

203.3.4.4 and the agency’s Evaluation Policy specify the types of evaluations that can be used to assess a

project’s success or failure. The policy also requires each “large project” to have an external performance

evaluation but does not state when it should be performed.

In the case of the USAID mission in Afghanistan,

we found that the mission completed the evaluations and produced final performance reports for several of its

projects, but these evaluations and reports did not discuss how the projects linked to the five health sector

assistance objectives in the 2011 through 2015 PMP.

We attempted to review final performance reports and external evaluations for 20 USAID projects that were

completed after January 1, 2011, when the 2011 through 2015 PMP went into effect. Five of the 20

completed projects in our scope met ADS criteria to be classified as “large projects” that required an external

As previously described, the ADS classifies a large project as one that receives more than the average amount of funding

for all projects in a sector.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 10

evaluation.

USAID contracted with independent firms to conduct external evaluations for four of its five large

health care projects in Afghanistan and provided us with three reports, with one report in the process of being

completed as of the date of this report.

USAID also provided external evaluations of two additional projects

that did not require such evaluations. We reviewed all five of the external evaluation reports provided and

found that four did not discuss any of the PMP indicators. The one report that did was completed before the

PMP existed and referenced performance indicators USAID developed in 2005.

However, for the fifth large project in our scope, PCH, USAID did not require an external evaluation or have a

final performance report completed. According to USAID, the $259.6 million PCH project did not need a final

performance report or external evaluation because the USAID Office of Inspector General and SIGAR had

reviewed the project.

Therefore, USAID waived its own requirement to have PCH evaluated. However, none of

these reviews assessed the project’s overall effectiveness or how it related to the five health care objectives in

the PMP. The USAID Office of Inspector General reported that measuring “the magnitude of USAID’s

contribution to the national objectives could be made only indirectly using proxy indicators because no current

demographic information or health statistics were available to measure health outcomes directly.”

Because

PCH ended in June 2015, it is now too late for USAID to conduct a final performance report or external

evaluation for the project, meaning the agency missed an opportunity to independently assess the project’s

performance.

Additionally, USAID only provided us with final performance reports for 8 of the 20 completed projects.

Based

on our review of those eight reports, we determined that there was not a direct link between the reports and

the five health assistance objectives listed in the PMP. For example, we found that:

 Three final performance reports—for Tech-Serve, Communication for Behavior Change: Expanding

Access to Private Sector Health Products and Services in Afghanistan, and the Basic Support of

Institutionalizing Child Survival-III—contained information related to the five health care assistance

objectives in the PMP. However, the Tech-Serve and the Communication for Behavior Change:

Expanding Access to Private Sector Health Products and Services in Afghanistan final reports did not

show how the project contributed to achieving those objectives, and the Basic Support of

Institutionalizing Child Survival-III final report cited a statistic similar but not identical to the PMP

assistance objective for child malnutrition.

The five projects requiring an external evaluation were: (1) Construction of Health and Education Facilities, (2)

Communication for Behavior Change: Expanding Access to Private Sector Health Products and Services in Afghanistan, (3)

Health Services Support Project, (4) Technical Support to the Central and Provincial Ministry of Public Health, and (5) PCH.

The three required external reports we received were conducted by Checchi and Company Consulting, Inc. for USAID.

The reports USAID is referring to as having reviewed PCH are: USAID Office of Inspector General, Audit of

USAID/Afghanistan’s On-Budget Assistance to the Ministry of Public Health in Support of the Partnership Contracts for

Health Services Program, F-306-11-004-P, September 29, 2011; USAID Office of Inspector General, Review of

USAID/Afghanistan’s Monitoring and Evaluation System, F-306-12-002-S, September 26, 2012; SIGAR, Health Services in

Afghanistan: Two New USAID-Funded Hospitals May Not Be Sustainable and Existing Hospitals Are Facing Shortages in

Some Key Medical Positions, SIGAR Audit 13-9, April 29, 2013; and SIGAR, Health Services in Afghanistan: USAID

Continues Providing Millions of Dollars to the Ministry of Public Health despite the Risk of Misuse of Funds, SIGAR Audit 13-

17, September 5, 2013.

USAID Office of Inspector General, Audit of USAID/Afghanistan’s On-Budget Assistance to the Ministry of Public Health in

Support of the Partnership Contracts for Health Services Program, F-306-11-004-P, September 29, 2011.

We received final reports for the following projects: (1) the Tuberculosis Collaboration and Coordination, Access to

Tuberculosis Service, Responsible and Responsive Management Practices, Evidence-Based Project; (2) Basic Support for

Institutionalizing Child Survival-III; (3) Communication for Behavior Change: Expanding Access to Private Sector Health

Products and Services in Afghanistan; (4) Health Service Support Project; (5) the Higher Education Project: Kabul Medical

University; (6) Technical Support to the Central and Provincial Ministry of Public Health; (7) Tuberculosis Control Assistance

Program; and (8) UNICEF Salt Iodization in Afghanistan.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 11

 Two final performance reports for the Higher Education Project: Kabul Medical University and the

UNICEF Salt Iodization in Afghanistan did not mention any of the five health care assistance

objectives.

 Three final performance reports for the Tuberculosis Collaboration and Coordination, Access to

Tuberculosis Services, Responsible and Responsive Management Practices, Evidence-Based project;

the Health Service Support Project; and the Tuberculosis Control Assistance Program contained data

on outputs related to the PMP objectives. However, the reports did not discuss how those outputs

contributed to the PMP’s overall objectives.

We were unable to review final performance reports for 12 of the 20 projects, including PCH, for the following

reasons.

 One project did not have a final report because it was transferred to the World Health Organization,

which continued the project.

 As noted above, PCH did not have a final report because USAID officials did not think one was

necessary due to prior SIGAR and USAID Office of Inspector General reviews of the project.

 Five projects were completed in 2015, and the implementing partners had not yet submitted their

final reports to USAID.

 Despite multiple requests, USAID did not provide us with the remaining five reports.

According to USAID officials, the agency measures its progress in achieving the PMP objectives through

national health care surveys that are conducted every few years, instead of using project performance

information. However, by not consistently assessing project performance, USAID may not know whether

projects achieved their intended effects. Additionally, there is no evidence to demonstrate how specific

projects impact the PMP’s health care indicators and advance PMP objectives.

USAID Relied on Implementing Partners’ Final Reports to Determine Project

Success

As discussed above, USAID contracted for external reviews for three of the five large health care projects in

Afghanistan.

However, these external evaluations were conducted while the projects were still active and not

after completion. Instead, USAID relied on the final performance reports produced by the implementing

partners. This is because ADS 203 only requires at least one external evaluation but does not specify when

that evaluation is to be conducted.

Not having an external evaluation after project completion forces USAID

to rely on reports from implementing partners that may have a potential conflict of interest because the

implementing partner also runs the project. These reports could be biased, increasing the risk that USAID is

using inaccurate information to influence decisions about future health care projects.

For example, in August 2012, Management Sciences for Health, the implementing partner of the $100.5

million Tech-Serve project, submitted its final performance report to USAID assessing the project’s

performance in increasing the capacity of MOPH. According to the report, the project:

Strengthened the capacity of the [MOPH’s Grants and Contract Management Unit] so that is a

stand-alone entity, now on-budget, needing virtually no outside technical assistance for its

chief functions of procurement, monitoring, management of [non-governmental organization]

As stated previously, USAID did not conduct a review of PCH.

USAID, ADS 203.3.1.3 Assessing and Learning, effective January 17, 2012.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 12

awarded contracts at a rate of over $40 million annually, and doing this while maintaining

financial accountability and [U.S. government] certification.

However, only 4 months later, USAID stated in its Stage II Public Financial Management Risk Assessment

Framework report that the “MOPH’s procurement management units did not have sufficient systems and

management capacity to implement activities and manage donors’ funds,” and the MOPH’s procurement

processes “[are] not transparent and is susceptible to manipulations.”

The assessment concluded that

USAID cannot rely on the MOPH’s systems and internal controls to manage donors’ funds. This subsequent

report from USAID calls into question whether the Tech-Serve implementing partner accurately reported the

impact that the project had on MOPH capacity.

Furthermore, in June 2016, the Independent Joint Anti-Corruption Monitoring and Evaluation Committee

conducted an assessment of the MOPH to evaluate its vulnerability to corruption. This assessment report

suggests that the MOPH still does not have the capacity to manage donor funds. For example, the report

describes the Grants and Contract Management Unit’s processes for funding BPHS and EPHS as “suspect,

compromised, corrupted, and inconsistent.”

The report states that “[f]ailures to strengthen these processes

have led to disappointment, frustration, suspicion, and weakened trust in the MOPH.” The report

recommended that the Afghan government conduct an independent investigation of the unit’s systems and

organizational capacity, and suggested that it needs to significantly improve its capabilities.

AFGHANISTAN’S HEALTH CARE SECTOR REMAINS INADEQUATE AND FACES

MANY CHALLENGES

Afghan Government Lacks Funds to Repair and Operate Facilities

One of the biggest challenges for the Afghan government is operating and sustaining health care facilities. In

April 2013, we reported that two new hospitals in Gardez in Paktiya province and Khair Khot in Paktika

province, built under USAID’s Construction of Health and Education Facilities project, were at risk because their

estimated operating costs were much higher than the facilities they were replacing.

We found that the annual

costs for the new Gardez hospital were estimated to increase between 180 and 524 percent, while those for

Khair Khot were expected to be six times higher. In a letter dated November 29, 2014, the MOPH informed

USAID that once the hospitals were officially turned over to the ministry, it would face challenges operating and

Management Sciences for Health, Technical Support to the Central and Provincial Ministry of Public Health Project

(Tech-Serve) Final Report July 2006 – August 2012, August 2012.

USAID completed its stage II assessment to determine the risks associated with providing the MOPH with on-budget

assistance. The MOPH was one of seven ministries assessed. USAID, Government of the Islamic Republic of Afghanistan

Ministry of Public Health: Stage II Risk Assessment Report, December 15, 2012. We previously conducted an audit of

these assessments. See SIGAR, Direct Assistance: USAID Has Taken Positive Action to Assess Afghan Ministries’ Ability to

Manage Donor Funds, but Concerns Remain, SIGAR 14-32-AR, January 30, 2014.

Independent Joint Anti-Corruption Monitoring and Evaluation Committee, Vulnerability to Corruption Assessment in the

Afghan Ministry of Public Health, June 4, 2016.

SIGAR, Health Services in Afghanistan: Two New USAID-Funded Hospitals May Not Be Sustainable and Existing Hospitals

Are Facing Shortages in Some Key Medical Positions, SIGAR Audit 13-9, April 29, 2013.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 13

maintaining the hospitals due to insufficient funding. In our August 2016 follow-up report, we noted these

funding issues still persist.

Additionally, in January 2014, we found numerous structural deficiencies within the Salang Hospital in Parwan

province.

The support columns had structural issues, which are especially problematic because the hospital

is in a major earthquake zone. We also found the hospital was missing safety items, such as smoke detectors

and emergency lighting. In addition, it did not have a functioning surgical unit; vaccines could not be

refrigerated due to lack of electricity; the X-ray machine was missing parts and did not have enough electricity;

only three operating light fixtures were found in the entire facility; and untreated river water was the only

source of water for the hospital because the required well and water purification systems were never installed.

In our October 2016 follow-up report, we noted that none of these deficiencies had been addressed by the

Afghan government.

In late 2015 and early 2016, we reported on the conditions at 55 local health facilities in Herat and Kabul

provinces.

Three facilities in Kabul appeared to not have access to electricity, while eight others did not have

an adequate or consistent supply. In addition, 5 of the 32 inspected facilities in Kabul province did not have

running water, and most of the buildings for all of the facilities had structural defects.

Afghanistan Lacks Hospitals Capable of Providing Skilled Care

Another major challenge is the inadequacy of hospital-level health care services. In a February 2014 report,

Medecins Sans Frontieres found that Afghanistan’s lack of an effective referral system prevents patients from

being transferred from clinics to facilities that provide more skilled care.

However, Afghanistan continues to

struggle to provide higher-level care beyond the basic functions provided by local health clinics. Similarly,

according to the World Health Organization, “an insufficient budget is allocated to secondary and tertiary

hospitals that are run by the [MOPH].”

USAID also noted shortcomings in the hospital system. In September

2014, the agency stated that the “poor quality of care provided at national hospitals has become a growing

political concern and one of the top priorities of the MOPH.”

Afghanistan’s Health Care Facilities Lack Qualified Staff

According to Medecins Sans Frontieres officials, Afghanistan does not have enough trained doctors and health

care workers. Additionally, health care workers who have been trained are often not assigned to positions that

use the skills they have been trained in. Medecins Sans Frontieres officials also stated that the lack of

SIGAR, Gardez Hospital: $14.6 Million and Over 5 Years to Complete, Yet Construction Deficiencies Still Need to be

Addressed, SIGAR 16-56-IP, August 29, 2016.

SIGAR, Salang Hospital: Lack of Water and Power Severely Limits Hospital Services, and Major Construction Deficiencies

Raise Safety Concerns, SIGAR 14-31-IP, January 29, 2014.

SIGAR, Salang Hospital: Unaddressed Construction Deficiencies, Along with Staffing and Equipment Shortages, Continue

to Limit Patient Services, SIGAR 17-09-IP, October 26, 2016.

SIGAR, Review Letter: USAID-Supported Health Facilities in Kabul, SIGAR 16-09-SP, January 5, 2016; and SIGAR, Alert

Letter: USAID-Supported Health Facilities in Herat, SIGAR 16-1-SP, October 20, 2015.

Medecins Sans Frontieres, Between Rhetoric and Reality: The Ongoing Struggles to Access Healthcare in Afghanistan,

February 2014.

World Health Organization, Humanitarian Health Action: Afghanistan, accessed September 2015.

USAID, Health Care Sector Resiliency Project USAID/Afghanistan, SOL-306-14-000086, September 30, 2014.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 14

qualified, trained doctors has led to the overuse of different kinds of antibiotics without proper diagnosis,

which could create a resistance to the prescribed drugs.

USAID has expressed similar concerns about the “human resource crisis” in Afghanistan’s health care sector,

stating that people working in the sector face “poor working conditions, including minimal financial

compensation, inadequate staffing, lack of career development opportunities or other incentives and

worsening security and [this is] further exacerbated by chronic inadequacies in both public infrastructure and

lack of training capacities.”

In addition, we have identified problems with a lack of qualified, trained health care workers. In April 2013, we

reported that some provincial hospitals supported by USAID’s PCH project did not have specialists such as

anesthesiologists, obstetricians, gynecologists, and pediatricians.

Corruption Remains a Concern in Afghanistan’s Health Care Sector

According to the Independent Joint Anti-Corruption Monitoring and Evaluation Committee’s June 2016 report,

Afghanistan’s public health sector suffers from “deep and endemic corruption problems.”

The committee

found that the ability of the MOPH’s HMIS to record management functions, administrative process, and

service delivery are susceptible to corruption, and described inconsistencies at all levels of the health care

sector’s quality assurance, describing it as “unreliable and uncoordinated.” The report further states that

“[f]raud, falsification, fakes, and forgeries have become a routine aspect of documentation in the Public Health

sector.” All of this affects the reliability and integrity of each of main elements of the health care system,

including HMIS.

These problems have contributed to the Afghan population’s limited faith in the quality of their health care

system. According to the Medecins Sans Frontieres’ report, “Four in five people had bypassed their closest

public clinic during a previous illness in the preceding three months, mostly because they believed there were

problems with the availability or quality of staff, services, or treatments found there.”

As a result, MOPH, USAID, and World Health Organization officials have said Afghans spend approximately

$285 million annually on medical tourism, depriving the health care sector of a vital source of revenue and

further weakening the government’s ability to sustain the facilities that donors are now funding. For example,

the World Bank plans to spend $408 million over 64 months on the SEHAT project supporting health clinics,

which is about $6.38 million each month. In comparison, Afghan citizens spend roughly $23.75 million each

month in search of better health care abroad.

In response to one of our requests for information, USAID officials stated that, “Afghanistan generally [does]

not provide state-of-the-art treatment for complex disorders and the Afghans who can afford it prefer [to travel

abroad] to buy the best care.”

However, two surveys conducted by the MOPH have found that the majority of

individuals traveling abroad for care were rural farmers or unemployed. Additionally, one of the surveys stated

USAID, Health Care Sector Resiliency Project USAID/Afghanistan, SOL-306-14-000086, September 30, 2014.

SIGAR, Health Services in Afghanistan: Two New USAID-Funded Hospitals May Not Be Sustainable and Existing Hospitals

Are Facing Shortages in Some Key Medical Positions, SIGAR Audit 13-9, April, 2011.

Independent Joint Anti-Corruption Monitoring and Evaluation Committee, Vulnerability to Corruption Assessment in the

Afghan Ministry of Public Health, June 4, 2016.

Medecins Sans Frontieres, Between Rhetoric and Reality: The Ongoing Struggles to Access Healthcare in Afghanistan,

February 2014.

USAID, response to SIGAR request for information, July 27, 2015.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 15

that 72.5 percent of citizens who traveled abroad for medical care had tried first to receive treatment in

Afghanistan. According to one of the MOPH surveys, 99 percent of respondents stated that the medical care

they received abroad was better than the care given in Afghanistan.

To address some of these challenges associated with revenue generation and limited ministerial capacity,

USAID started the $38 million Health Sector Resiliency project in 2015. This projects aims to foster a

strengthened, reformed, and increasingly self-reliant Afghan health care sector in preparation for the

decreased donor support anticipated over the coming decade by building capacity in the MOPH.

CONCLUSION

Since 2002, developing Afghanistan’s health care sector has been a priority for USAID, leading the agency to

obligate almost $1.5 billion to support the sector since 2002. However, due to the lack of reliable data and

potentially inaccurate or incomplete final assessments of its health care projects, it is difficult to determine the

effectiveness of USAID’s health care projects in Afghanistan. It is particularly concerning that USAID has been

using flawed and inconsistent data when reporting progress in reducing infant and child mortality rates.

Compounding that problem, the agency has also neglected to identify the limitations of that data and other

data it has relied on when claiming improvements in the health care situation in Afghanistan. By basing its

external reporting on unreliable data and by neglecting to fully explain the limitations of that data, USAID may

have distorted the results of its health care projects in Afghanistan, despite agency guidance stating that

transparent reporting is imperative. Additionally, USAID has not taken reasonable steps to improve the

reliability of the information it reports, making it even more difficult to accurately assess progress in the sector.

In addition to relying on flawed and inconsistent data when reporting progress in the Afghan health care sector,

USAID has also neglected to establish whether there is a causal connection between its projects and

improvements in Afghan mortality rates and life expectancy. Although the PMP includes health care-related

objectives, USAID does not tie its project reporting to show direct progress from USAID investments.

Furthermore, USAID’s third-party external evaluations and final reviews completed by the agency’s

implementing partners do not link project performance to health assistance objectives included in the PMP. It

is imperative that the agency’s projects demonstrate how each project advanced the agency’s overall goals for

the health care sector. Without a concerted effort to assess the extent to which projects are actually

contributing to the achievement of agency objectives, USAID will be unable to determine whether and how its

obligation of $1.5 billion has directly contributed to claimed improvements in Afghan mortality rates and life

expectancy.

We could not verify these figures in the MOPH reports.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 16

RECOMMENDATIONS

To ensure that government decision makers and the general public have an accurate understanding of

progress in the Afghan health care sector, we recommend that the USAID Mission Director for Afghanistan:

1. Acknowledge in external reporting the limitations associated with surveys and data the agency uses to

demonstrate its achievements in the health care sector in Afghanistan.

To determine how USAID’s efforts have directly contributed to reported gains in Afghanistan’s health care

system, we recommend that the USAID Mission Director for Afghanistan:

2. Amend mission guidelines for conducting project reviews in Afghanistan to require an explicit

discussion of the applicable PMP objectives.

To ensure that USAID has more insight into the accuracy and reliability of implementing partners’ final

performance reports, we recommend that the USAID Mission Director for Afghanistan:

3. Take action to validate the accuracy of final health care project reports submitted by implementing

partners in Afghanistan.

AGENCY COMMENTS

We provided a draft of this report to USAID for review and comment. These comments are reproduced in

appendix III. USAID also provided technical comments, which we have incorporated into the report, as

appropriate.

USAID concurred with all three of our recommendations and set a target date for their implementation of July

31, 2017. Regarding our first recommendation, the agency stated that it will revise its performance monitoring

mission order to state that all externally reported data should be accompanied with a statement that identifies

any known data quality limitations. In response to our second recommendation, USAID stated that it will amend

the mission order on evaluation to consider, as appropriate, questions that examine the agency’s contributions

to the most important activity and project-level results. With respect to our third recommendation, USAID

stated that it will ensure that contracting officer’s representatives and agreement officer’s representatives are

responsible for validating implementing partner data. In addition, the agency will develop training to give the

representatives further guidance and assistance on how to review and validate data.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 17

APPENDIX I - SCOPE AND METHODOLOGY

This report provides the results of SIGAR’s audit of the U.S. Agency for International Development’s (USAID)

efforts to improve Afghanistan’s health care sector. The objectives of this audit were to (1) determine the

extent to which USAID collected, assessed, and disclosed the quality of data used to report progress in the

health care sector; (2) evaluate the extent to which USAID assessed the impact its projects had on health care;

and (3) identify the challenges to developing the health care sector in Afghanistan. We reviewed health care

data for Afghanistan, related plans and agency regulations, and other documentation dated from 1997

through 2016.

To determine the extent to which USAID collected, assessed, and disclosed the quality of data used to report

progress in the health care sector, we reviewed USAID’s Automated Directive System (ADS) 203, which

describes USAID’s requirements for data collection, validation, and accuracy requirements; USAID’s data

quality assessment checklist; and ADS Chapter 579 regarding data collection to identify agency requirements

regarding data collection and assessing data quality.

Additionally, we reviewed the Department of State and

USAID’s strategic plan for fiscal years 2007 to 2012 and the U.S. Mission in Afghanistan’s Post Performance

Management Plan (PMP) for 2011 through 2015.

We examined the Health Management Information System

(HMIS) reporting template that health care providers use to report health care data to the Ministry of Public

Health (MOPH) and Afghan government strategy documents.

In addition, we reviewed USAID fact sheets from

the USAID website regarding the health care sector in Afghanistan. We also interviewed USAID and MOPH

officials as well as with nongovernmental organizations that enter data into HMIS to better understand the

HMIS reporting process and the accuracy of that process.

We also analyzed the following health care surveys that have been conducted in Afghanistan, organized by

organization:

 United Nations Children’s Fund (UNICEF) Multiple Indicator Baseline completed in 1997, and the

Multiple Indicator Cluster Survey completed in 2003 and 2011,

 Ministry of Rural Rehabilitation and Development, and the Central Statistics Organization’s The

National Risk and Vulnerability Assessment, conducted in 2005, 2007–2008, and 2011–2012

 The MOPH, the U.S. Centers for Disease Control and Prevention, and UNICEF’s 2002 Maternal

Mortality in Afghanistan: Magnitude, Causes, Risk Factors and Preventability,

 The MOPH and the Central Statistics Organization’s Afghanistan Mortality Survey 2010,

 The MOPH and the Central Statistics Organization’s preliminary results from the 2015 Afghanistan

Demographic and Health Survey,

 The Central Statistics Organization’s preliminary results from the Afghanistan Living Conditions Survey

2014,

 USAID’s 2010 Reproductive Age Mortality Survey II, and

We reviewed USAID’s ADS Chapters 202 Achieving, effective January 25, 2012; ADS 203 Assessing and Learning,

effective November 2, 2012, and a prior version effective March 19, 2004; and ADS Chapter 579 USAID Development

Data, effective March 13, 2015.

Department of State and USAID, U.S. Department of State and USAID’s Strategic Plan for Fiscal Years 2007–2012, May

7, 2007; and Department of State and Department of State and USAID, U.S. Foreign Assistance for Afghanistan: Post

Performance Management Plan 2011-2015, September 2, 2010.

Government of Afghanistan, Afghanistan National Health Policy 2005-2009, September 2005; and Government of

Afghanistan, Afghanistan National Development Strategy 2008-2013, n.d.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 18

 Government of Afghanistan’s Millennium Development Goal reports from 2005, 2008, 2010, and

2012.

We also reviewed a report by the British & Irish Agencies Afghanistan Group report that critiqued the MOPH

and the Central Statistic Organization’s Afghanistan Mortality Survey 2010, and a World Health Organization

and a World Bank report critiquing the accuracy of HMIS data.

To evaluate the extent to which USAID assessed the impact of its projects on improving the health care sector

in Afghanistan, we reviewed all external and final performance evaluation reports that USAID and third parties

completed for all of the agency’s health care projects that were active between January 1, 2011, and February

1, 2016. USAID provided external evaluations for three projects, as well as external evaluations for two

additional projects that did not require such evaluations. USAID also provided final performance reports for 8

out of 20 final evaluations. At the time of our review, one project did not have a final report because it was

transferred to the World Health Organization, which continued the project. The Partnership Contracts for Health

project did not have a final report. Five projects were completed in 2015, and the implementing partners had

not submitted their final reports to USAID. Finally, USAID did not provide us with the remaining five reports.

Additionally, we reviewed the PMP and ADS 203. We also reviewed USAID’s stage II assessment of the MOPH

and compared the results of that assessment to the results noted in the implementing partner’s final report on

the Tech-Serve project.

To identify the challenges to developing the health care sector in Afghanistan, we reviewed prior reports by

SIGAR and Medecins Sans Frontieres that identified problems within the sectors.

We also reviewed two

MOPH reports about Afghans who left Afghanistan to receive health care services abroad.

We also

interviewed officials from USAID, the World Bank, the MOPH, the World Health Organization, and international

nongovernmental organizations operating in Afghanistan’s health care sector.

We assessed internal controls to determine the extent to which the USAID and the MOPH had systems in place

to track and report on their efforts to support the health care sector in Afghanistan. The results of our

assessment are included in the body of this report. We did not use computer-processed data in this audit.

We conducted our work in Washington, D.C., and Kabul, Afghanistan, from April 2015 to January 2017. Our

work was conducted in accordance with generally accepted government auditing standards. Those standards

require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable

basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained

provides a reasonable basis for our findings and conclusions based on our audit objectives. This audit was

performed by SIGAR under the authority of Public Law 110-181, as amended, and the Inspector General Act of

1978, as amended.

British & Irish Agencies Afghanistan Group, Maternal Mortality Reported Trends in Afghanistan: Too Good to Be True?,

September 2015; K.M.I. Saeed, R. Bano, and R.J. Asghar, World Health Organization, “Evaluation of the National

Tuberculosis Surveillance System in Afghanistan” Eastern Mediterranean Health Journal, vol. 19, no. 2 (February 21,

2013); and World Bank, Critical Administrative Constraints to Service Delivery: Improving Public Services in Afghanistan’s

Transformational Decade, May 2014.

SIGAR, Health Services in Afghanistan: Two New USAID-Funded Hospitals May Not Be Sustainable and Existing Hospitals

Are Facing Shortages in Some Key Medical Positions, SIGAR Audit 13-9, April 29, 2013; SIGAR, Health Services in

Afghanistan: USAID Continues Providing Millions of Dollars to the Ministry of Public Health despite the Risk of Misuse of

Funds, SIGAR Audit 13-17, September 5, 2013; SIGAR, Salang Hospital: Lack of Water and Power Severely Limits Hospital

Services, and Major Construction Deficiencies Raise Safety Concerns, SIGAR 14-31-IP, January 29, 2014; SIGAR, Gardez

Hospital: $14.6 Million and Over 5 Years to Complete, Yet Construction Deficiencies Still Need to be Addressed, SIGAR 16-

56-IP, August 29, 2016; SIGAR, Salang Hospital: Unaddressed Construction Deficiencies, Along with Staffing and

Equipment Shortages, Continue to Limit Patient Services, SIGAR 17-09-IP, October 26, 2016; and Medecins Sans

Frontieres, Between Rhetoric and Reality: The Ongoing Struggle to Access Health care in Afghanistan, February 2014.

MOPH, Patients Acquiring Medical Treatment from India, n.d.; and MOPH, Study of Patients Acquiring Medical Care

Outside Afghanistan, n.d.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 19

APPENDIX II - USAID’S ACTIVE, COMPLETED, AND PLANNED HEALTH CARE

PROJECTS IN AFGHANISTAN SINCE JANUARY 1, 2011

Tables 2 and 3 below list the U.S. Agency for International Development’s (USAID) active and completed health

care programs in Afghanistan, respectively, as of September 30, 2016. Table 4 lists USAID’s planned health

care projects as of February 2016.

Table 2 - Active USAID Health Care Projects, as of September 30, 2016

Project

Timeframe

Total

Estimated

Cost

Project Description

Afghanistan

Demographic Health

Survey

September 2013–

September 2018

$6.7 million

Funds the implementation and completion of a

national health survey

Central Contraceptive

Procurement

March 2011–

September 2022

$25 million

Simplifies mechanism for contraceptive procurement

Challenge TB

[Tuberculosis]

January 2015–

September 2019

$15 million

Expand application of directly observed therapy and

short-course tuberculosis treatment procedures

Delegated

Cooperation on

Nutrition

December 2014–

December 2016

$5 million

Cooperative agreement with the Canadian

Department of Foreign Affairs, Trade, and

Development with the goal of improving nutritional

capacity of the Afghan health care system

Family Planning

Needs Assessment

and Behavioral Study

May 2015–

December 2016

$0.6 million

Fund two studies conducted by the United Nations

World Food Programme

Multi-Input Area

Development

March 2013–

March 2018

$30.5 million

Global Development Alliance with Aga Khan

Foundation to promote enterprise-driving

development projects in Badakhshan province

Disease Early

Warning System Plus

January 2015–

June 2017

$32.7 million

Funding given to the World Health Organization to

implement Afghanistan’s National Emergency Action

Plan for polio and operation of the Disease Early

Warning System tracking 15 communicable diseases

Weekly Iron Folic Acid

Supplementation

November 2014–

December 2017

$5.6 million

A contribution grant to the United Nations Children’s

Fund (UNICEF) to provide weekly iron

supplementation to adolescent women to reduce

long-term effects of anemia

System Enhancement

for Health Action in

Transition (SEHAT)

June 2013–

June 2018

$227.7 million

On-budget support for health clinics in Afghanistan,

previously funded under the Partnership Contracts for

Health project

Helping Mother and

Children Thrive

January 2015–

January 2020

$60 million

Project seeks to strengthen and enhance primary

care while improving the Essential Package of

Hospital Services referral system with a focus on child

and maternal health

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 20

Gender Based

Violence Program

Contribution to the

World Health

Organization

July 2015–

June 2020

$1.7 million

Provides program support for the implementation of a

gender-based violence treatment protocol for health

care providers

Health Sector

Resiliency

September 2015–

September 2020

$37.9 million

Fosters a strengthened, reformed, and increasingly

self-reliant Afghan health system by helping the

Afghan government prepare for a decreased donor

support environment

Strengthening

Pharmaceutical

Systems

August 2011–

July 2017

$34.4 million

Strengthen the Ministry of Public Health’s (MOPH)

ability to regulate and assure quality of

pharmaceutical products entering Afghanistan and

ensure essential medicines are available in public

clinics

Enhance Community

Access, Use of Zinc,

Oral Rehydration

Salts for

Management of

Childhood Diarrhea

July 2015–

July 2020

$15 million

Increases access to zinc/oral rehydration salts to

prevent and treat childhood diarrhea

Coordinating

Comprehensive Care

for Children

September 2014–

September 2019

$0.02 million

Identify and promote appropriate, gender-aware

practices in child welfare and protection, and

institutional strengthening

Initiative for Hygiene,

Sanitation, and

Nutrition

May 2016–

May 2021

$75.5 million

Improve nutritional status of women of reproductive

and children less than 5 years old

Sustaining Health

Outcomes through

the Private Sector

Plus

June 2016–

June 2018

$6 million

Seeks to harness the full potential of the private

sector and catalyze public-private engagement to

improve health outcomes in family planning,

HIV/AIDS, maternal and child health, and other areas

Regional Fortification

in the Central Asian

Republics and

Afghanistan

September 2014–

September 2019

$9.7 million

Improve processes, regulations, and monitoring of

wheat flour and edible oil fortification to address

micronutrient deficiencies

Total Cost

$589.1 million

Source: SIGAR analysis of USAID data

The Aga Khan Foundation will also contribute $30.7 million.

This figure does not include the World Bank or the European Commission’s contributions to the SEHAT project.

An additional $3.2 million in funding will come from non-health care-related USAID funding.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 21

Table 3 - USAID Health Care Projects Completed since January 1, 2011

Project

Timeframe

Total Cost

Project Description

Construction of Health and

Education Facilities

January 2008–

July 2015

$34 million

Build health care facilities

Strengthening Immunization in

Afghanistan/Routine Immunization

in Afghanistan

September 2013–

August 2015

$1.2 million

Support the MOPH’s efforts to improve

expanded program of immunizations

operations

Health Policy Project

May 2012–

August 2015

$29.8 million

Build the MOPH’s ability to regulate the

health sector and improve financial

management inside the ministry

Partnership for Supply Chain

Management

June 2009–

September 2015

$1.5 million

A global USAID mechanism for

procurement of essential medicine

commodities

Tuberculosis (TB) Collaboration

and Coordination, Access to TB

Services, Responsible and

Responsive Management

Practices, Evidenced-Based

Project

July 2011–

December 2014

$5.6 million

Strengthen the managerial capacity of the

National Tuberculosis Program and

expands access to tuberculosis treatment

training for health workers

Basic Support for Institutionalizing

Child Survival-III

March 2008–

September 2011

$4.0 million

Intended to address deficiencies related

to children’s health care at the policy,

community, health facility, and hospital

levels

Child Survival Support Grant:

Better Health for Afghan Mothers

and Children Project

September 2008–

September 2012

$2.4 million

Targeted five districts in Herat province to

support sustained improvements in

maternal, newborn, and child health

outcomes

Disease Early Warning System

October 2008–

June 2014

$8.5 million

Designed to collect accurate and timely

outbreak and seasonal trend data that

result in quick action to mitigate disease

outbreaks and prevent epidemics

Communication for Behavior

Change: Expanding Access to

Private Sector Health Products and

Services in Afghanistan

February 2006–

May 2012

$38.9 million

Aimed to use social marketing and

behavior change communications to

increase demand for, access to, and use

of quality health products available

through the private sector

Field Epidemiology and Laboratory

Training Program

October 2008–

September 2011

$0.5 million

Supported the MOPH to build public

health capacity via a regional field

epidemiology training program

Health Care Improvement Project

October 2009–

September 2013

$14.0 million

Aimed to improve health services by

working with the MOPH and the private

sector to increase the quality of health

services by developing health capacity

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 22

and infrastructure at the national and

provincial levels, with a focus on maternal

and newborn care

Health Research Challenge for

Impact: Reproductive Age Mortality

Survey II

July 2010–

May 2012

$1.9 million

Funded a health care survey that

repeated the original maternal mortality

survey conducted in 2002 across four

districts for comparison purposes and

document the changes in those districts

Health Service Support Project

July 2006–

October 2012

$60.5 million

Provided technical assistance and

capacity-building support to non-

governmental organizations contracted by

the MOPH to improve service delivery and

the quality of basic health services in

Afghanistan

Health Systems 20/20

October 2008–

September 2012

$5.1 million

Supported the MOPH to identify and

address financing, governance,

operational, and capacity constraints in

the health system

Higher Education Project: Kabul

Medical University

January 2007–

January 2011

$11.7 million

The Medical Education Component

designed to improve pre-service medical

training offered in Afghan public

universities to better meet health

workforce needs

Measure DHS: Afghanistan

Mortality Study

May 2009–

December 2011

$3.5 million

Funded a national health survey with

additional support from UNICEF, the

United Nations Population Fund, and the

World Health Organization

Technical Support to the Central

and Provincial Ministry of Public

Health

July 2006–

August 2012

$100.5 million

Aimed to strengthen the MOPH’s health

system stewardship at all levels, leading

to improvements in overall population

health

Tuberculosis Control Assistance

Program

October 2008–

June 2011

$6.2 million

Aimed to strengthen the managerial

capacity of Afghanistan’s National

Tuberculosis Program and expanded

access to the Directly Observed

Treatment Short-course

UNICEF Health and Immunization

Response Support

September 2003–

September 2011

$1.0 million

Aimed to increase community acceptance

of vaccinations in order to increase

immunization coverage in each

successive round, with a focus on

Helmand and Kandahar provinces

UNICEF Salt Iodization in

Afghanistan

October 2011–

September 2012

$0.3 million

Funded a public-private partnership with

assistance from other donors with the

goal of ensuring that 90 percent of the

households in Afghanistan have access to

and consume iodized salt

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 23

Table 4 - Planned USAID Health Care Projects, as of February 2016

Project

Timeframe

(anticipated)

Funding

Amount

Project Description

Health Private

Sector Flagship

January 2016–

December 2017

$6 million

Social marketing activity to ensure effective and

sustainable delivery of family planning material and

child survival products

Promoting Quality of

Medicines

March 2016–

late 2017

$4.5 million

Strengthen medicine quality assurance and quality

control programs

Total Estimated Funding Amount

$10.5 million

Source: SIGAR analysis of USAID data

Partnership Contracts for Health

July 2008–

June 2015

$259.6 million

USAID’s on-budget project to fund 14

nongovernmental organizations that

operate health care facilities in 13

provinces

DELIVER Project

September 2010–

September 2015

$13.0 million

Procurement mechanism for commodities

ordered through the contraceptive project

as well as ordered by the public health

clinics under PCH and SEHAT

Leadership, Management, and

Governance

September 2012–

December 2015

$38.5 million

Intended to build in-country knowledge of

health care system capacity by increasing

leadership, management, and

governance of health care providers and

managers

Total Cost

$642 million

Source: SIGAR analysis of USAID data

An additional $23 million went to education facilities.

An additional $35 million went to the education sector.

USAID refers to this project as Measure DHS: Afghanistan Mortality Study and does not expand the acronym.

An additional $7.8 million in funding supported the Ministry of Education.

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 24

APPENDIX III - COMMENTS FROM THE U.S. AGENCY FOR INTERNATIONAL

DEVELOPMENT

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 25

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 26

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 27

SIGAR Response to USAID Comments

1. We reviewed USAID’s comments and have revised the title to now read “Afghanistan’s Health Care

Sector: USAID’s Use of Unreliable Data Presents Challenges in Assessing Program Performance and

the Extent of Progress.”

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 28

APPENDIX IV - ACKNOWLEDGMENTS

Jeff Brown, Senior Audit Manager

Zubair Hakimzada, Analyst-in-Charge

Matt Miller, Senior Program Analyst

SIGAR 17-22-AR/USAID Support for Afghanistan’s Health Care Sector Page 29

This performance audit was conducted under project

code SIGAR-105A.

SIGAR’s Mission

Obtaining Copies of SIGAR

Reports and Testimonies

To Report Fraud, Waste, and

Abuse in Afghanistan

Reconstruction Programs

Public Affairs

The mission of the Special Inspector General for Afghanistan

Reconstruction (SIGAR) is to enhance oversight of programs for the

reconstruction of Afghanistan by conducting independent and

objective audits, inspections, and investigations on the use of

taxpayer dollars and related funds. SIGAR works to provide accurate

and balanced information, evaluations, analysis, and

recommendations to help the U.S. Congress, U.S. agencies, and

other decision-makers to make informed oversight, policy, and

funding decisions to:

improve effectiveness of the overall reconstruction

strategy and its component programs;

improve management and accountability over funds

administered by U.S. and Afghan agencies and their

contractors;

improve contracting and contract management

processes;

prevent fraud, waste, and abuse; and

advance U.S. interests in reconstructing Afghanistan.

To obtain copies of SIGAR documents at no cost, go to SIGAR’s Web

site (www.sigar.mil). SIGAR posts all publically released reports,

testimonies, and correspondence on its Web site.

To help prevent fraud, waste, and abuse by reporting allegations of

fraud, waste, abuse, mismanagement, and reprisal, contact SIGAR’s

hotline:

Web: www.sigar.mil/fraud

Email: sigar.pentagon.inv.mb[email protected]il

Phone Afghanistan: +93 (0) 700-10-7300

Phone DSN Afghanistan 318-237-3912 ext. 7303

Phone International: +1-866-329-8893

Phone DSN International: 312-664-0378

U.S. fax: +1-703-601-4065

Public Affairs Officer

Phone: 703-545-5974

Email: sigar.pentagon.ccr.mbx.public[email protected]

Mail: SIGAR Public Affairs

2530 Crystal Drive

Arlington, VA 22202