Detailed Information on the
CDC: Global Immunizations Assessment

Explicitly tie budget requests to the accomplishment of annual and long-term goals, and present resource needs in a complete and transparent manner.

Review opportunities to conduct an evaluation of management of Global Immunization's measles activities at domestic headquarters in Atlanta, Georgia.

Measure: Number of countries in the world with endemic wild polio virus.

Explanation:Global polio eradication depends on the engagement of leaders in the four priority countries with areas that have never interrupted indigenous wild poliovirus transmission (Nigeria, India, Pakistan, and Afghanistan) and those of the international community particularly of the G8 countries to ensure follow through on their commitments of resources. The targets remain ambitious in light of the major risk posed by the shortfall in global resources. The U.S. is one source of funding for this effort but generally polio eradication efforts are funded by a number of other governments as well. CDC is advised by global partners that if all G8 and EC countries do not meet their pledges by November, some essential activities may be postponed or cancelled.

Measure: Number of global measles-related deaths.

Explanation:The objective of this measure is to reduce global measles mortality. By 2010 CDC and global immunization partners aim to reduce the global measles-related mortality by 90 percent compared with this estimate from 2000. CDC's contributions to the achievements in the African Region were recognized with a special award at the African Region Task Force on Immunization (TFI) meeting in December 2007. The ambitious targets come from a model used to generate the preceding year coverage that is based on routine and campaign related performance data that is captured by a joint WHO/UNICEF reporting form. WHO and UNICEF convene a panel committee to review this data annually and reach consensus on estimates of disease burden.

Measure: The portion of the annual budget that directly supports the program purpose in the field.

Explanation:Field activities include grants and cooperative agreements, foreign travel, consulting services abroad (e.g., contracts for overseas consultants, Embassy ICAAS charges) and salary/benefits of the program staff that are assigned abroad (including temporary assignments). Less than 10% will support salaries and benefits for program staff in headquarters, administrative expenses, overhead costs and other expenses that are not provided to or expended in the field.This measure demonstrates that the majority of the Global Immunization Program's funding is used to support mission-critical activities directly through CDC's global partners, the WHO, UNICEF, PAHO and UNF. Specifically, these funds are used to purchase measles and polio vaccine and/or to provide technical or operational support through these agencies. CDC will maintain this efficiency and support for these activities in order to continue to meet global health goals. Field activities include grants and cooperative agreements, foreign travel, consulting services abroad (e.g., contracts for overseas consultants, Embassy ICAAS charges) and salary/benefits of the program staff that are assigned abroad (including temporary assignments). Less than 10% will support salaries and benefits for program staff in headquarters, administrative expenses, overhead costs and other expenses that are not provided to or expended in the field.

Measure: Number of non-import related measles cases in all 47 countries of the Americas as a measure of maintaining elimination of endemic measles transmission.

Explanation:This performance measure corresponds with the goal adopted by the PAHO for Latin America and the Caribbean. According to available surveillance information, measles transmission has been interrupted in all countries of the Western Hemisphere since November 2002. However, imported measles cases, with limited secondary spread, continue to occur in several countries, including the U.S. Deaths from measles complications in the Americas have virtually disappeared. Globally, measles caused an estimated 242,000 deaths in 2006 and was the leading cause of death among children under five years of age from a vaccine-preventable disease.

Measure: Number of countries in the world with endemic wild polio virus.

Explanation:Measured by the number of countries.

Is the program purpose clear?

Explanation: The purpose of the Global Immunization program at the Centers for Disease Control and Prevention (CDC) is to eliminate or reduce vaccine-preventable diseases overseas. These efforts protect American children from disease imported to the United States or acquired abroad in order to protect against the medical costs of morbidity and mortality associated with vaccine-preventable diseases and for humanitarian reasons. The program is focused on eradicating polio globally, reducing the global burden of measles and eliminating measles in four of six World Health Organization (WHO) regions. The polio eradication activity was created in 1993 in CDC's National Immunization Program following an appropriation for polio eradication in FY 1991. The program broadened to become the Vaccine Preventable Disease Eradication Division in 1997 with the addition of global measles activities and finally the Global Immunizations Division in 2002.

Evidence: Evidence includes the appropriations history, the program mission statement, Congressional budget justification documents, the National Immunization Program 2005 annual report, the global polio eradication initiative strategic plan, World Health Assembly resolutions, and the Public Health Service Act authorization for international cooperation activities (Sec. 307). Also PHS Act Section 301 and the Foreign Assistance Act of 1961.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses a specific and existing problem of polio and measles disease transmission in multiple countries. Polio eradication efforts have made considerable progress, but the disease remains endemic in six countries and six additional countries have reestablished transmission of polio virus imported from Nigeria. Measles remains one of the leading causes of death for children worldwide and is the leading cause of preventable blindness. It can be transmitted through the air and is one of the most contagious diseases in the world.

Evidence: In 2003 and 2004, wild poliovirus spread from Nigeria, where immunization efforts had been halted, to other countries in west and central Africa. Polio transmission was reestablished in six countries and imported polio cases occurred in seven others. An outbreak of polio in Sudan affected 125 children in 2004. In 2000, there were approximately 30 million measles cases and 777,000 deaths worldwide. More than half of the deaths occurred in Africa. As of June 7, 2005, the global case count of polio was 448 cases, including 220 cases in Yemen where a larger polio outbreak is ongoing. Endemic countries include Nigeria, India, Pakistan, Niger, and Afghanistan. Measles is no longer endemic in the United States, but imported cases occur and pose a threat for outbreaks. Sources of data include from the polio partnership (www.polioeradication.org), measles/rubella bulletins from the Pan American Health Organization (PAHO), and from CDC (e.g., www.cdc.gov/mmwr/pdf/wk/mm5408.pdf), and the WHO Monthly Bulletin.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While there are numerous parties involved in global polio and measles efforts, the program has a unique role in the amount of financial assistance and type of technical assistance it provides for polio eradication and measles reduction and elimination efforts. The program provides financial resources to grantees that deliver vaccines to children who would otherwise not receive them. The program's funding is primarily for surveillance and vaccine purchase. The program also provides laboratory support in the field and at headquarters, in collaboration with CDC's National Center for Infectious Diseases. By assigning staff in partner organizations, the program works to limit duplication and ensure resources are distributed optimally and consistent with CDC strategies. These activities involve multiple international organizations and local and national governments. USAID supports surveillance and communication efforts through WHO and in SE Asia and Africa. The State Department provides diplomatic leadership and support in the US and through local embassies abroad.

Evidence: The US has made the largest financial contribution to the polio eradication effort, through this program, at nearly $1 billion to date. A demand for funds remains with the WHO estimating a $50 million funding gap for 2005 in May and the program's contributions do not constitute an unintended subsidy. Other major financers of polio eradication include Rotary International, World Bank, Japan, United Kingdom, the Netherlands, Germany, Canada, the European Commission and the Gates Foundation. For partners, UNICEF is the only organization in the world with a global vaccine procurement and distribution network. WHO can coordinate surveillance and other public health activities globally and is recognized by governments around the world. PAHO is the WHO regional office for the Americas and has an important role for measles activities there. The program extends its impact by leveraging resources with partners, including members of the G-8 and other nations and private foundations and organizations, especially for vaccine purchase.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence of major design flaws that limit the program's efficiency or effectiveness. Measles and polio vaccines are cost-effective and efficient interventions. The program's key strategies include to support the purchase of vaccine, to develop and promote strong disease surveillance systems, to provide expert laboratory support, to strengthen routine immunization programs, and to promote safe injection practices. The program carries out these efforts by providing grants to WHO, the Pan American Health Organization, the United Nations Children's Fund (UNICEF) and other international partners for vaccine procurement and technical, laboratory and programmatic support.

Evidence: In addition to supporting procurement of vaccine, the program provides short- and long-term consultation and technical assistance to WHO, PAHO, UNICEF and others; designs and participates in international research, monitoring and evaluation projects; develops strategies to improve human resource skills in target countries; refines immunization strategies; assists with surveillance and laboratory activities; and participates in international planning efforts to finance eradication. The program has also provided auto-disable needles, syringes, and safety boxes for national campaigns. For the STOP campaigns, the program recruits individuals from other countries who use the experience to get better opportunities, such as at the UN, and also individuals from CDC, and US schools of public health and state health departments. Roughly one in three STOP volunteers continue on in public health careers.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program targets polio funds to countries with transmission of wild poliovirus and those at risk of re-established transmission. The program targets these countries through grants to WHO and UNICEF, primarily. The program targets measles activities through UNF, WHO, PAHO and UNICEF. The program's resources are not held up for any period prior to being used to benefit intended beneficiaries. Decisions on allocations between areas are based on epidemiological data (rate of disease and vaccination coverage levels), national program immunization delivery capacity, partner support and gaps in funding. Mass vaccination campaigns, one of the program's main strategies, can achieve high rates of coverage and reach children not regularly served by the healthcare system. The program focuses additional efforts on unvaccinated children in areas where polio and/or measles transmission continues. The program utilizes special outreach, such as collaboration with religious leaders, and efforts, such as house-to-house campaigns when needed.

Evidence: The program estimates that between 95%-97% of their extramural awards respond to countries with high polio or measles transmission or disease outbreaks. The program's main four funding partners, WHO, UNICEF, PAHO and UNF, utilize at least 95% of the funds each budget year. Endemic and recently endemic countries conduct supplemental immunization activities, including national immunization days and sub-national immunization days. In addition to eradication and reduction efforts, the program supports efforts to strengthen routine immunization practices, introduce new and under-utilized vaccines, and promote safe injection practices. The polio and measles laboratories that have been established in part due to the efforts of the program can serve as the platform for the global surveillance of other diseases. Other efforts, such as training of healthcare workers and the mobilization of immunization volunteers, may also help prepare these countries to respond to other health threats.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has long-term outcome performance measures that reflect program achievements and provide key indicators to make programmatic and resource management decisions. The measures are meaningful and capture the most important aspects of the program, namely the reduction of polio and measles associated disease transmission, illness and death.

Evidence: The long-term measures include to reduce the global measles-related mortality and eradicate endemic wild polio virus in all countries of world.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program has challenging but realistic quantifiable targets and timeframes for the long-term outcome measures.

Evidence: The targets for measles include a ten-fold reduction of deaths from 2000 to 2015 worldwide. Achievement of this goal will require involving countries such as India, Pakistan and Nigeria that have no measures control program currently. The long-term target for polio is elimination of endemic polio from all countries by 2007.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has annual performance measures, including an efficiency measure, that reflect program achievements and indicate progress toward meeting the long-term performance measures.

Evidence: The measures include to maintain elimination of endemic measles transmission in all 47 countries of the Americas; eradicate endemic wild polio virus in all countries of world; and an efficiency measure on the portion of the program's annual budget that will directly support the program goals through grants and cooperative agreements, foreign travel, consulting services abroad (e.g., contracts for overseas consultants, Embassy ICAAS charges) and salary/benefits of the program staff that are assigned abroad (including temporary assignments). The remainder of the budget is for salaries and benefits for program staff in headquarters, administrative expenses, overhead costs and other expenses that are not provided to or expended in the field.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program has challenging but realistic quantifiable targets and timeframes for the annual measures, including the efficiency measure.

Evidence: The targets for measles include elimination of endemic measles in the 47 countries of the Americas with less than 500 cases, for polio to eradicate endemic polio in all countries of the world by 2007, and for the efficiency measure to spend no less than 90% of all program funds on direct, front-line activities for the program, including grants and cooperative agreements and foreign staffing and travel.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: As described in more detail in sections I and III, the program's funded partners include UNICEF, WHO, UNF, PAHO and others. The program works with these partners to set common goals and shares in common objectives and targets. The partners are committed to program's long-term and annual goals for the eradication of polio and reduction of measles mortality.

Evidence: Roughly 80% of the program's budget is provided extramurally to partners through cooperative agreements. The program coordinates with the partners regularly and has program staff detailed to the organizations. Both the program and the partners monitor progress toward the achievement of goals through the global polio surveillance bulletin, WHO regional surveillance bulletins, monthly laboratory bulletins. UNICEF also provides quarterly reports on vaccine procurement. Evidence also includes cooperative agreements between the program and major program partners. The cooperative agreements include performance measures with specific, quantifiable, outcome oriented targets that directly relate to and are shared by the program.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program has had multiple independent evaluations of polio eradication, which make up 70 percent of the total program budget. An evaluation by a multinational team of individuals conducted in 2001 documented contributions, achievements and lessons learned and covered the central roles of the program. Another PEI evaluation conducted in 2001 focused more heavily on WHO's performance, but also referenced CDC activities. A 2001 evaluation of the PEI and the role of the Department for International Development in the UK included a review of the overall progress of the PEI. Emory University evaluated the program's Stop Transmission of Polio (STOP) project. Program activities are also reviewed at the country and regional level and circulated among partners. GAO has reviewed global immunization in the developing world, but has not focused directly on the program. A review of global measles and other global immunization activities is warranted to evaluate the program and help provide strategic direction.

Evidence: The evaluation of the STOP campaign, which makes up roughly 3% of the total budget, was conducted in FY 2004 and examined the role of STOP in strengthening surveillance, supporting national immunization days and improving routine immunization. The evaluation incorporated feedback and suggested changes from the program itself and is not entirely independent. Related GAO reports that are not focused on the program and cannot be considered comprehensive evaluations for this question include GAO/NSIAD-00-4, GAO/NSIAD-00-95. The DFID review of the PEI was provided in December 2001. The 2001 Polio Evaluation Report was focused on the accomplishments to date of polio eradication overall, but also examined CDC's role. The local program reviews, referred to by the program as grey literature, provide information to the program and its partners to improve approaches.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program has made progress in this area but has not yet reached an integrated development of the program budget and performance information that meets the standards set out for this question. The program includes outputs and information on program accomplishments in budget documents. The 2006 budget justification also incorporates measures into the budget document. The program has not quantified or estimated the impact of a given change in funding level on specific program outcomes in the budget request. Decisions on allocations between areas are made based on epidemiological data, capacity, partner support, gaps in funding and historical patterns.

Evidence: Evidence includes the GPRA plans and reports and annual Congressional Justifications and budget documents provided to OMB. The program can show that an increase in dollars spent in prior years correlates to an increased level of vaccination coverage through purchase of vaccines and support of operational costs for campaigns, surveillance and social mobilization, and other interventions.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The agency is taking a comprehensive effort to integrate budget and goals agency-wide. As described more fully in section III, the program regularly reviews progress toward polio eradication and implements new or improved strategies to reach the goal.

Evidence: Evidence includes the annual budget submissions to OMB and the Congress and documentation from the agency level on a new process for ceiling letters and other initiatives to advance budget and performance integration.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program regularly collects performance information from surveillance systems and from key program partners and uses the information to manage the program and improve performance. For example, the Global Polio Eradication Initiative regularly reviews progress towards eradication and implements new or improved strategies to reach the goal. The program has identified that in a few densely populated tropical countries with extremely poor sanitation (India), vaccine efficacy with tri-valent OPV may not be adequate to rapidly interrupt transmission. The program supported development of mono-valent OPV which is now being used in India, Egypt and Yemen. Further, CDC is working with WHO for production and use of a second mono-valent vaccine to protect against type 3 poliovirus. The program increases immunization rounds, supports house-to-house campaigns, deploys STOP teams, increases its presence in key areas, implements synchronized supplemental immunization campaigns in response to changes in performance information.

Evidence: Sources of performance information used to manage the program and improve performance include monthly reports from UNICEF on the obligation of CDC funds, partner websites (WHO for polio and ARC for measles), regularly scheduled conference calls with partners, field visits, information from overseas assignees and annual progress reports from grantees.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Accountability for cost, schedule and specific outputs is established through performance appraisals for program managers. The program reports a process by which branch chiefs are held accountable for program results. Senior managers have elements of accountability built into performance evaluation systems, including for the Commissioned Corps, and employees now incorporate one or more general performance measures from the agency or department level into their workplans. Cooperative agreement recipients are required to report on program progress.

Evidence: Evidence includes grantee reports and copies of a work plan for performance evaluations that is tied to the achievement of program goals and objectives.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds for the program are obligated in a timely manner and spent for the intended purpose. At the agency level, CDC consolidated budget execution functions in 2004 into a central office that is now charged with quality assurance and data validation for program execution. The agency approves internal reallocations that vary from spend plans; regularly reviews unliquidated obligations; and established a standard operating procedure for spending plan execution to help ensure program funds are obligated consistently with the overall objective of the program and that allotted funds are fully executed in a timely manner. CDC conducted risk assessments to determine whether specific programs were susceptible to improper payments exceeding $10 million and a 2.5 percent error rate and will estimate improper payments. At the agency level, data validation of commitments is used to help identify whether funds are committed for the alloted purpose and done correctly to track with budget and accounting systems.

Evidence: Each year, about 95% of funds provided by the program to WHO, UNICEF, PAHO and UNF are utilized in each budget year. The remaining 5% is obligated, but carried over by the grantees into the next fiscal year. Evidence includes operating procedures of the budget execution branch at CDC, sample data validation reports, the budget execution standard operating procedures, agency procedures for development and submission of annual spending plans, budget execution spending plans and obligation reports for NCHS, CDC's February 2005 submission for risk assessments under the Improper Payments Information Act. The spending plans are to be used to certify and monitor the status of funds at the program and agency level.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has an efficiency measure that tracks the proportion of the annual budget that is spent on grants/cooperative agreements, foreign travel, consulting services in the field and salary/benefits of program staff in the field. The program is establishing a web-based database to track and manage short-term international consultants and travel. The program is taking steps to eliminate inefficient or redundant arrangements in the field. The program has experimented with synchronized measles and polio immunization and Vitamin A distribution campaigns as well as synchronized measles and polio immunization with insecticide treated bed-nets and de-worming medicines in West Africa, which may increase efficiency and effectiveness where appropriate. The program achieves substantial cost savings for vaccine procurement through UNICEF. The program is studying the cost effectiveness of polio immunizations efforts post eradication.

Evidence: Roughly 80% of the program's budget is provided extramurally to partners. The remaining 20% covers salaries, benefits, travel, consulting and other supportive services. The efficiency measure is that at least 90% of the program's total annual budget will directly support the purpose of the program through grants and cooperative agreements, foreign travel, consulting services abroad (e.g., contracts for overseas consultants, Embassy ICAAS charges) and salary/benefits of the program staff that are assigned abroad (including temporary assignments). Less than 10% will support salaries and benefits for program staff in headquarters, administrative expenses, overhead costs and other expenses that are not provided to or expended in the field.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The global polio eradication effort is lead by WHO, Rotary International, CDC and UNICEF. The program shares eradication and disease reduction performance goals with these partners. Through the polio and measles efforts, the program collaborates closely with WHO and its regional offices, Ministries of Health, UNICEF, Rotary International, World Bank, the US Agency for International Development, the American Red Cross, the International Federation of Red Cross/Red Crescent Societies, the United Nations Foundation, the Bill and Melinda Gates Foundation, Path, and other offices within CDC and HHS. In addition to providing resources, the program provides technical assistance to its partners. Evaluations of the PEI described in sections II and IV confirm the positive contributions of CDC working through key partnerships.

Evidence: Staff from the program serve as members of regional and country level technical advisory groups; as advisors to WHO, Rotary International and UNICEF; and as technical assignees to WHO, UNICEF, the American Red Cross, PAHO and the World Bank. According to the program's polio eradication partners, "the most important contribution of... CDC to polio eradication continues to be deployment of its epidemiologists, public health experts, and scientists to WHO and UNICEF."

Does the program use strong financial management practices?

Explanation: CDC recently underwent a major effort to bring on the new Unified Financial Management System that is intended to reconcile any remaining weaknesses in this area. The system and CDC's associated process changes are to provide more real time data, streamlined processes, absolute funds control, and improved monitoring. UFMS automated funds control has enabled tracking of commitments, including aged balances, to provide better management information; improved financial planning using commitment data; produced tracking of current data on obligations to date, commitments and balances to provide information on spending actions, trends, plans, and the resulting impact on remaining funds availability; and provided historical data on all commitments, undelivered orders, payables, and payment transactions. Continued success will require positive documentation from independent auditors that indicates the new system has resolved weaknesses. Prior Performance and Accountability Reports noted continued weaknesses with CDC's financial systems, including a material weakness.

Evidence: Evidence includes error reports and other preliminary financial controls data from CDC's initial experience with fully implementing the Unified Financial Management System. CDC also automated reimbursable billings, enhanced year end closing transactions, implemented a new indirect cost methodology, and addressed staff needs. A December 2003 report by the OIG noted the agency had not implemented a system to allocate indirect costs until FY 2003, but found the new system to be a significant improvement for equity and accuracy. The OIG recommends CDC periodically review indirect costing methods. CDC has received five consecutive unqualified opinions. CDC issued 64 duplicate or erroneous payments in FY 2002, or 0.042 percent of all payments and has a 97 percent compliance rate for prompt payments. In November 2000, GAO (GAO-01-40) reported the agency's financial management capacity systems and procedures were insufficiently developed to address the agency's mission and budget growth.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has not scored no's in this section and the weighting for this question is adjusted to zero. In areas where further improvement is warranted, the agency is extending the incorporation of performance measures into employee evaluations and work contracts. The agency is also putting considerable effort into setting priorities and reorganizing operations through the Future's Initiative, including to improve CDC's business practices. The agency will continue to monitor the performance of a new financial management system and the impact the system has on resolving prior weaknesses. The program regularly identifies and responds to program management deficiencies and works to make necessary corrections in given timeframes.

Evidence: CDC implemented UFMS in April of 2005. The system is intended to correct the agency's prior weaknesses in financial management. The FY 2003 PAR cites improvements in preparing financial statements. The agency submitted financial statements to the Department ahead of schedule. The program has improved employee evaluations for senior managers to incorporate the achievement of goals.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The weighting for the answer is reduced because the program is part of international eradication campaigns whose partners are uniquely qualified to fulfill their respective roles. For example, PAHO is the only organization in the Americas with a regional mandate for the control and prevention of vaccine-preventable diseases. UNICEF is the only organization in the world with global vaccine procurement and distribution responsibilities. The UNICEF cooperative agreement supports the procurement and distribution of polio and measles vaccine and operational support such as communications for vaccination campaigns, campaign monitors and cold chain storage equipment. The PAHO cooperative agreement supports measles elimination. WHO and PAHO agreements support surveillance officers, laboratory network coordinators, laboratory supplies, training, travel, and monitoring for safe injections.

Evidence: Evidence includes the role, position and mission of the grantees and memorandums providing justification for approval of single eligibility awards. UNICEF is able to procure vaccine for far less than the US private sector cost or CDC domestic market cost. For example, UNICEF pays $1.70 for measles-mumps-rubella compared to $38 in the US private sector and $16 through CDC. UNICEF pays $0.12 for polio vaccine compared to $22 in the US private sector and and $10 through CDC.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Staff from the program provide oversight of grantee activities by participating in global and regional technical advisory groups. Program staff are also detailed to WHO, PAHO and UNICEF. The program conducts annual reviews with grantees, weekly conference calls on progress and periodic site visits and project officers review all grantee proposed activities. UNICEF provides monthly reports on spending by area. WHO and UNICEF provide annual financial reports.

Evidence: The program collects monthly reports from UNICEF that indicate obligation of funds by country, disease, purchase and operations. Evidence also includes copies of a sample trip report on management issues for the national polio surveillance project in India; data on overseas staff assignments for technical oversight.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The surveillance systems supported by the program and its partners provide regional, national and local data on the number of cases of polio and measles that are readily available to the public. These data provide an indication of grantee performance. Case numbers are also influenced by political and epidemiological influences in these countries that are beyond the direct control of program grantees. UNICEF, PAHO and WHO provide numerous documents and reports on the internet and provide financial and end of year reports directly to the program. Information on the grantee and program progress includes surveillance data from WHO (www.polioeradication.org), polio laboratory network quarterly updates from WHO, PAHO measles information such as the surveillance bulletin(www.paho.org/English/AD/FCH/IM/measles.htm), WHO weekly epidemiologic record and CDC Morbidity and Mortality Weekly Report articles on progress toward eradication and surveillance and laboratory activities. UNICEF also maintains information on provision of polio vaccines (www.unicef.org/immunization).

Evidence: Data available to the public that provide information on the performance of key grantees include UNICEF's Annual Reports that include data and descriptions of UNICEF's provision and distribution of oral polio vaccine; WHO surveillance and activities; PAHO measles publications, technical documents and periodicals; country reports; and PAHO regional updates through newsletters.(www.unicef.org/publications/files/UNICEFAnnualReport2004_eng.pdf, www.unicef.org/publications/files/unicef_eng_final.pdf, www.unicef.org/publications/files/pub_ar03_en.pdf) info: www.paho.org/English/PAHO/WHO_region.htm www.afro.who.int/polio/surveillance_maps/index.html www.afro.who.int/polio/nids/niddates2005.pdf www.paho.org/Project.asp?SEL=KW&LNG=ENG&ID=112 www.paho.org/English/AD/FCH/IM/TAG16_FinalReport_2004.pdf, page 9. www.paho.org/English/DD/AIS/cp_index.htm, for polio (scroll to bottom) www.paho.org/English/AD/FCH/IM/sne2701.pdf www.paho.org/English/AD/FCH/IM/sne2702.pdf

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The assessment reflects overall achievement toward meeting long-term outcomes for both polio and measles goals. The program helped reduce measles-related deaths by 46 percent in countries in the WHO AFRO region and 39 percent globally from 2001 to 2004. The program has helped reduce the number of countries with endemic wild polio virus from 20 countries in 2000 to six in 2004. Recent setbacks in polio eradication followed local cessation of vaccination activities in Nigeria.

Evidence: In 2003 and 2004, wild poliovirus spread from Nigeria, where immunization efforts had been halted by local leaders, to other countries in west and central Africa. The World Health Assembly initially set the year 2000 as the target date for eradication with certification by 2005. Endemic polio transmission has been reduced from 125 countries when the global effort began in 1988 to six countries in 2004. In 2004, polio declined by 42% in India, Pakistan, and Afghanistan. The program contributed 500 million doses of oral polio vaccine and over 4.5 million doses of measles vaccine through UNICEF. The program has worked to strengthen polio and measles laboratories and as of January 2005, 145 laboratories were part of the global polio network and 690 laboratories were part of the global measles network. Of the three types of wild polioviruses, type 2 appears to have been eradicated already. Complete polio eradication is at a critical phase.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The assessment of large extent reflects considerable progress in meeting both polio and measles goals, with some setbacks in polio eradication that were largely beyond the control of the program.

Evidence: As noted above, the program has made considerable progress in polio and measles. Performance goals for measles have been fully met through FY 2004 and the program is on target to achieve 2005 goals. Some performance goals related to polio eradication have not been met. The number of countries with polio has exceeded the goal for several years due to events beyond the control of the program.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The assessment of large extent reflects the adoption of an efficiency goal and achievement in meeting the goal, but little additional evidence beyond the agency level of improved efficiencies or cost effectiveness in achieving program goals each year. As described previously, the program has established an efficiency goal measuring the proportion of funds that directly support activities in the field. The program has maintained a large portion of its funds in direct support of field work to accomplish the long-term outcome goals at 94%, 93% and 93% in 2002, 2003 and 2004, respectively. The program implemented a web-based tracking tool to automate recruitment and selection of short-term consultants and travel requests. There are no savings data available, but the system replaces an email and paper based approach and has reduced delays and staff time dedicated to this activity. At the agency level, CDC has conducted or is conducting A-76 studies for library services, office automation, animal care, laboratory glassware and laundry services, printing, and material management services.

Evidence: The program is meeting the efficiency goal of targeting at least 90% of all budget resources to direct field activities, including grants and cooperative agreements, foreign travel and assignments. Less than 10% of the total budget is for administrative support and other activities in the headquarters office. Direct field activities include grants and cooperative agreements, salary and benefits for field staff, salary and benefits for Atlanta staff while on temporary assignment overseas, Atlanta staff and STOP Team overseas deployment costs, international permanent change of station costs, external contracts for overseas consultants and embassy/ICASS costs. The program does not have data indicating savings or efficiency gains over the prior year. At the agency level, CDC has achieved savings in IT and is consolidating graphics/web design services and professional training to provide services centrally. Evidence includes copies of the new web-based staffing tool.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program's immunization campaigns, polio and measles, could be compared to smallpox eradication and other campaigns. WHO declared smallpox eradicated in 1979, following a comprehensive eradication campaign beginning in 1966 that was the largest eradication effort at that time. Polio involves more countries, more people being vaccinated, more doses per person, more difficult surveillance (many asymptomatic infections), greater levels of war, conflicts, and terrorism, HIV/AIDS, international travel, easier transmission, complicated partnerships (as measured by funds raised, volunteers, and advocacy), three distinct types of poliovirus requiring 3 different components (combined in a trivalent vaccine). Measles shares most of these factors. Several other eradication initiatives of the 1990s, including programs to eradicate malaria, yaws, and guinea worm disease, have either failed and been terminated (malaria and yaws) or are many years behind schedule (guinea worm disease).

Evidence: Since 1988, more than 2 billion children have been immunized in over 200 countries thorugh an international investment of more than $3 billion. There were 125 polio endemic countries at the start of the initiative compared to roughy 40 for smallpox. Polio requires nationwide campaigns targeting all children less than 5 years with repeated vaccination. More children are vaccinated on an annual basis in the polio program than were vaccinated during the entire multi-year smallpox eradication program. Polio funding has been an estimated five times higher in inflation adjusted dollars. As noted by GAO, "unlike most of the diseases that are currently candidates for eradication, smallpox had unique characteristics that made it particularly vulnerable to eradication and therefore has limitations as a model for current efforts." Evidence includes GAO/NSIAD-00-4; GAO/NSIAD-98-114; Fenner, F., Henderson, D.A., Arita, I, et al. Smallpox and its Eradication. World Health Organization, Geneva, Switzerland. 1988. Pages 1363-1370.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The small extent reflects that evaluations have been conducted for polio eradication that reflect program contributions, but are not focused on the program sufficiently to provide a full indication of program performance. There are positive findings from the STOP evaluation, but STOP is a small portion of the program's budget and the evaluation conclusions were not entirely independent. The 2001 Polio Evaluation Report found CDC's financial and technical contributions to be invaluable to the eradication effort. Potential weaknesses identified in the effort overall include cost and the diversion of resources from other services, the level of added value to health services and lack of clarity of roles between WHO and UNICEF. The report concluded steps are needed to make use of considerable investments in infrastructure and human resources when polio is eradicated. The program is actively considering post-cessation plans. The 2001 review of PEI and WHO cited an absolute commitment to polio eradication at CDC that translated to technical support and reviews WHO found invaluable. There are no evaluations of measles activities available.

Evidence: The Polio Evaluation Report was conducted by representatives from WHO, Rotary, Nigeria, India and the UK. The team reviewed outcome indicators and WHO documentation and conducted interviews and country visits. The report by Emory University's Rollins School of Public Health concluded that STOP assignees "contributed substantially to the strengthening of district polio programs in their areas of assignment", and "STOP alumni are taking on increasing global health responsibilities in CDC, WHO, and UNICEF." The majority of STOP assignees are from the United States and very few are from polio affected countries. Responses on training were mixed. Nearly half of the STOP assignees reported spending more than 80% of their time with counterparts, which reduces the area covered. The majority report their skills are well utilized. Country level evaluations by the technical advisory group are also available (e.g., www.polioeradication.org/meeting_detail.asp?day=1&month=1&year=2004). GAO reports described in Section II provide little information specific to the program.