Detailed Information on the
Global Environment Facility Assessment

Work with the Facility and donors to fully implement a performance-based funding allocation system based on relative country performance and environmental benefit.

Work with the Facility and donors to establish ambitious long-term performance goals and measures and undertaking more rigorous evaluations of project performance.

Work with the Facility and donors to strengthen anti-corruption mechanisms, including establishing high fiduciary standards and achieving clean annual audits from independent external auditors.

Measure: Increase biodiversity area (million hectares) under protection and/or improved management

Explanation:

Measure: Eliminate persistent organic pollutants (tons of POPs)

Explanation:

Measure: In the very long term, stabilize CO2 emissions (tons).

Explanation:

Measure: Reudce $/ton cost of CO2 emissions avoided for portfolio of GEF projects to below $4.

Explanation:

Measure: Phaseout tons of ozone depleting substances consumed by countries with economies in transition

Explanation:

Is the program purpose clear?

Explanation: The GEF Instrument states the purpose of the Institution: "The GEF shall operate on the basis of collaboration and partnership among the implementing Agencies, as a mechanism for international cooperation for the purpose of providing, new and additional grant and concessional funding to meet the agreed incremental costs of measures to achieve agreed global environmental benefits in the following areas: (a) climate change, (b) biological diversity, (c) international water, (d) ozone layer depletion, (e) land degradation, primarily desertification and deforestation; and, (f) persisternt organic pollutants."

Evidence: www.gefweb.org/documents/instrument. GEF has three Implementing Agencies (IAs): the World Bank, the UN Development Program (UNDP), and the UN Environment Program (UNEP). It also has seven Executing Agencies (EAs) that have expanded opportunities to execute GEF projects and, more recently, Council approval for direct access to GEF funds (expected to be operational shortly). These agencies are: the African Development Bank (AfDB), the Asian Development Bank (AsDB), the European Bank for Reconstruction and Development (EBRD), the Inter-American Development Bank (IDB), the International Fund for Agricultural Development (IFAD), the Food and Agriculture Organization (FAO), and the United Nations Industrial Development Organization (UNIDO).

Does the program address a specific and existing problem, interest or need?

Explanation: GEF addresses global environmental problems in the seven focal areas (listed in Q 1.1), and serves as a financial mechanism for several international environmental conventions to which the U.S. is a party.

Evidence: Polluted water is estimated to affect the health of over 1 billion people and nearly 2 billion face water shortage problems. Invasive species in the U.S. and elsewhere have caused billions of dollars in damages. Species-rich coral reefs are being degraded, and valuable rain forests are disappearing. www.gefweb.org/Outreach/outreach-PUblications/MainBook.pdf

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: GEF is designed to fund only the incremental cost associated with the global environmental benefit of projects put forward by the Implementing and Executing Agencies. All projects must include a calculation of the incremental cost, although the rigor of the calculations vary depending on the type of project and agency. All project documents must include a discussion of coordination with other efforts in the country, and must be endorsed by the country's operational contact for GEF projects. GEF projects are aimed at leveraging other funds in order to maximize its impact, so coordination is essential. Finally, the GEF pipeline and GEF project documents are published on the web prior to Council approval, and this transparency facilitates coordination.

Evidence: www.gefweb.org/Documents/Council_Documents/GEF_C22/Project_Cycle_Update__FINAL__Nov_5_2003.pdf

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: On the positive side, funding is primarily grants which does not add to country debt burdens. Implementing Agencies (IAs) propose projects that fit the development needs of recipient countries while producing the global environmental benefits through the GEF component. However, funds are not allocated to countries on the basis of relative performance or environmental benefit. Instead, projects are funded in the order in which they are proposed, provided that they meet the operational policy criteria and guidelines and fit the functional priorities for that year.

Evidence: GEF is currenty trying to implement a performance-based allocation system, as agreed by the Council in the GEF-3 Replenishment Agreement. (www.gefweb.org/Replenishment/Summary_of_negotiations_-_ENGLISH-11.5.doc) However, the PBAS has encountered procedural delays and operational problems.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The project review criteria, as discussed in Q.1.3, ensures a focus on global environmental benefits in eligible countries, while introduction of a results measurement system is helping to ensure that project targets are achieved. However, funds should be focused more on countries with the best policy performance and potential global environmental benefits, and there should be greater attention to cost effectiveness.

Evidence: www.gefweb.org/Documents/Council_Documents/GEF_C22/Project_Cycle_Update__FINAL__Nov_5_2003.pdf

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: There are two specific long-term performance goals for the GEF -- (1) phase out ozone depleting substances in countries with economy in transition, and (2) eliminate stockpiles of persistent organic pollutants. For the other focal areas, the long-term goals are more general, such as stabilization of CO2 emissions in the very long term and "the conservation of biological diversity" in large part because there is no clear agreement among governments on long-term targets.

Evidence: See operation programs for each focal area: www.gefweb.org/Operational_Policies/Operational_Programs/operational_programs.html

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long term goals do not yet have timeframes, and a number of them do not yet have baselines.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The GEF-3 agreement contains a set of process targets in each focal area for completion by the end of the replenishment period (e.g., 800-1200 million metric tons of CO2 emissions avoided during the lifetime of investments approved and funded by GEF-3 resources), and includes one cost effectiveness measure (decrease in $ cost per ton of CO2 emissions reduction). Interim process targets (to be achieved by fall 2004) have been established for each of the replenishment targets.

Evidence: These goals can be found in the report from the third replenishment of the GEF: (www.gefweb.org/Replenishment/Summary_of_negotiations_-_ENGLISH-11.5.doc) Prior to the next replenishment period, new targets will be developed.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: As indicated in Q.2.3, the baseline and targets are contained in the GEF-3 agreement. However, their ambitiousness is limited by the fact that all are process indicators.

Evidence: (www.gefweb.org/Replenishment/Summary_of_negotiations_-_ENGLISH-11.5.doc)

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The full GEF Council (which represents both donors and recipients) has accepted the GEF-3 agreement (including the periodic/annual goals listed above) and has put in place many of its reforms, e.g., all projects must incorporate results measurement frameworks related to those goals. However, it is unclear whether a key replenishment reform - the establishment of a performance based allocation system (PBAS) - will be implemented.

Evidence: Results measurement requirement and annual goals are contained in the policy recommendations all the GEF-3 replenishment agreement. All projects entering the work program have targets listed in a logical framework presentation contained in the summary of each project proposal document.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: A regular external evaluation is conducted prior to each replenishment. However, the two previous evaluations have not been sufficiently rigorous or independent, due to inadequate baselines and performance indicators, the newness of the portfolio, and the selection process for the outside evaluation team. Therefore, at US urging, the terms of reference for the third external review of the GEF contains a strict conflict of interest provision for the selection of the external evaluators. In addition, the GEF monitoring and evaluation unit is now an independent office, with clear terms of reference and a recently approved (and more senior) new head of the unit. Finally, the GEF also relies on the M&E units of the IAs and EAs, a number of which are independent.

Evidence: Pursuant to the GEF-3 agreement, the terms of reference for the newly independent internal monitoring and evaluation (M&E) unit were approved in August 2003. A new head of the unit was approved by the Council in May 2004. gefweb.org/MonitoringandEvaluation/MEAbout/GEFMETORsRevised.doc The terms of reference for the fourth external review were also approved in May 2004.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The GEF-3 agreement contains a set of process targets associated with different funding levels and performance incentives for achieving those targets. It provides for an incentive contribution ($70 million) if certain interim targets are met. A results measurement system is also being put in place. This will allow the GEF to routinely quantify global environmental achievements and understand the reasons for success and failure.

Evidence: GEF-3 replenishment agreement. Once projects with results measurement frameworks are reporting outcomes, there should be more concrete evidence as to whether budget alignment exists.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: GEF-3 calls for the creation of an annnual strategic business plan, based on the introduction of a system of performance-based allocations. Drafts of the strategic plan have been reviewed by the Council. However, the introduction of the performance-based allocation system (PBAS) is behind schedule, thereby delaying approval of the strategic business plan. In practice, the GEF has operated on the basis of the targets set in the replenishment agreement and the interim targets in schedule 1 of that attachment.

Evidence: We expect decision on the PBAS to take place at the November 2004 Council meeting. The draft strategic plan can be found on www.gefweb.org in the Council documents for the November 2003 meeting.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Monitoring and Evaluation Unit receives from Implementing and Executing agencies annual reports on implementation, all supervision reports and aide memoires relating to the projects, terminal evaluations for all projects and midterm evaluations for all full-sized projects. It uses this input to prepare program studies, thematic studies, and an annual project implementation review. However, the data is not always timely or credible, and the monitoring and evaluation unit has observed that some reports are overly positive. In addition, outcomes are not yet fully reported to the Council on an annual basis, as called for in the GEF-3 Replenishment. Lessons learned are not being adequately incorporated into new projects, but efforts are underway to develop a formal feedback loop. The ongoing development of a performance-based allocation system (if successful) will also help generate metrics for country level performance.

Evidence: All GEF monitoring and evaluation studies are available to the public on the GEF website. The core M&E document is the Annual Project Performance Report (latest 2003) and is drawn from annual portfolio performance reports by Implementing Agecies. The M&E unit will shortly post all midterm and terminal evaluations of GEF projects conducted by Implementing Agencies on the GEF website. www.gefweb.org/MonitoringandEvaluation/MEPublications/mepublications.html

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Once every four years during the GEF replenishment negotiation process, each donor makes an assesment of GEF effectiveness and efficiency in order to determine whether and to what extent to fund GEF. The U.S. is encouraging other donors to tie a portion of their contributions to key outcomes.

Evidence: The GEF-3 replenishment agreement calls for the U.S. to contribute an addition $70 million for GEF in FY 2006 if GEF meets specific benchmarks in establishing the new performance based allocation system and for putting forward projects that make specific progress towards the long-term goals.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The GEF ensures that funds are spent for the intended purpose through financial procedures agreement between the World Bank, which serves as the GEF's Trustee, and all Implementing and Executing Agencies. However, there are substantial (and growing) lags between the time of Council approval and project implementation (e.g., over two years for World Bank projects).

Evidence: Annual Project Performance Report (2003) indicated problems with delays in project implementation. The Secretariat has developed an action plan to improve procedures for medium-sized projects that will be discussed in November 2004. It is working with IA's and EA's to propose measures to streamline the project cycle. These will be put forward for review in November 2004. gefweb.org/C.23.Inf.5_Project_Performance_Report_2003_FINAL.doc; gefweb.org/C.23.Inf.6_Action_Plan_of_MSP_FINAL.pdf.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The GEF has opened up the number of Executing Agencies that can directly access GEF funds, although technical agreements are still being worked out. It is in the process of putting in place a PBAS (although outcome still in doubt). Projects are implemented by agencies such as the World Bank (which is implementing the majority of investment projects) that use international competitive bidding processes. However, GEF does not systematically measure cost effectiveness of projects even though this is a core principle of its operations.

Evidence: Paper on direct access for Executing Agencies: http:/www.gefweb.org/Documents/GEF_C22/C.22.12_Executing_Agencies_FINAL.doc. Draft PBAS paper (if successful): http://www.gefweb.org/Documents/Council_Documents/GEF_C23/C.23.7_PBA_FINAL.pdf Annual reporting of outcomes should help in the development of unit cost comparisions.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The GEF is intended to be a collaborative mechanism. While coordination is not perfect, the three Implementing Agencies work closely with the GEF Secretariat, review GEF policy papers, and provide comments on GEF projects, even those that are proposed by other agencies. The Implementing Agencies have also prepared reports on how their institutuions mainstream environmental considerations into their own operations, and are required to do so on an annual basis under GEF-3. This collaboration is being extended to the 7 incoming Executing Agencies, who are at various stages of completing memoranda of understanding with the GEF Secretariat and the Trustee. The process for the Council to approve the projects also improves coordination with bilateral donors.

Evidence: All three Implementing Agencies have produced reports on how they are mainstreaming global environmental issues into their own operations, e.g., http://lnweb18.worldbank.org/ESSD/envext.nsf/yDocName/Mainstreaming The GEF Secretariat is developing MOUs with the seven new EAs. Project documents contain comments from agencies and responses from the project sponsor.

Does the program use strong financial management practices?

Explanation: GEF has its own financial statements audited by independent external auditors every year. It relies on the World Bank Trustee to disburse funds and provide financial supervision over the Executing Agencies (EAs). The Trustee has financial procedures agreements with each of the EAs. However, GEF essentially relies on IAs' and EAs' own internal controls to ensure sound financial management, recognizing that it is the responsibility of those institutions' governing bodies to oversee their internal controls.

Evidence: Annual audited financial statements for Secretariat and IAs are contained in GEF annual reports. 2003 report is at: www.gefweb.org/Outreach/outreach-PUblications/AR_2003.pdf. Discussion of financial procedures for direct access to GEF and funds by seven new EAs (approved by the Council in May 2004) at: www.gefweb.org/Outreach/outreach-PUblications/Draft_AR_2003.pdf

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The GEF Council reviews procedures for managing the GEF financing program. GEF's donors have made broad changes in GEF's management practices as appropriate. In response to the GEF-3 replenishment agreement, management has put in place results measurement frameworks in all projects, begun a process of retrofit projects to ensure strong M&E frameworks, revised procedures for co-financing, strengthened project review criteria, and established an independent M&E unit. However, more needs to be done to correct deficiencies, e.g., the establishment of a PBAS, the development and use of additional cost effectiveness measures.

Evidence: See GEF-3 replenishment agreement policy recommendations (ref above), and Action Plan document www.gefweb.org/whatsnew/GEFActionPlan0708-04.doc

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: GEF relies on the Implementing and Executing Agencies to report on implementation of projects and flag projects at risk. The World Bank, which has implemented over 50% of GEF funds thus far, has strong supervision and monitoring systems. However, not all of the other agencies have such systems. Also, data is not sufficiently disaggregated to assess country performance.

Evidence: The World Bank has an internal portfolio monitoring and supervision system that tracks all ongoing projects and identifies projects at risk. Internal portfolio monitoring reports - the Annual Report on Portfolio Performance (ARPP) - are submitted to the World Bank Board every year. The other Implementing Agencies have reported on their project at risk systems and are currently taking steps to strengthen them.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Project data are widely available to the public. In addition, project data is being improved pursuant to the GEF-3 Replenishment agreement, which states: "Indicators should be designed with a view to assessing global environmental impacts achieved from the GEF resources. All projects must include clear and monitorable indicators, plans for monitoring and supervision, and identification of risks and other factors designed to improve quality at entry and to maximize impact. There should be a transparent system for the monitoring of these indicators and outcomes and for informing the Council on an annual basis." However, there is inadequate data or analysis on country performance.

Evidence: All project proposals as well as all monitoring and evaluation work conducted by or commissioned by the GEF unit are already available to the public. Terminal evaluations of projects are available in agencies' websites and are in the process of being posted on the GEF website

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: There has been significant progress on the phase out of ozone-depleting substances (4% of the portfolio). However, there is not yet sufficient data to tell whether the results of GEF projects have been adequate across all focal areas.

Evidence: Study of impact of the GEF activities on phaseout of ozone depleting substances (2000; evaluation report #1-00) for ozone results.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The four-year (and interim) targets have only recently been established (agreed to August 2002), and it is too soon to tell whether they have been achieved.

Evidence: GEF-3 requires verification of interim targets in Fall 2004.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Direct access for seven new Executing Agencies and improvements in data management is increasing competition and providing greater choices for project implementation. The introduction of a performance-based allocation system should further improve efficiencies and cost effectiveness. However, there is not a systematic approach to measuring cost effectiveness and maximizing impact per dollar spent, even though cost effectivenes is a core operational principle. In addition, a number of projects in the renewable energy portfolio do not appear to be cost effective.

Evidence: GEF-3 replenishment agreement in cludes a cost effectiveness measure for CO2 emissions, but it is too soon to tell whether it has been met. In addition, GEF-3 includes a commitment to introduce a performance-based allocation system, which should enhance cost effectiveness since funds would be channeled to their best use.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: No common measurments of effectiveness across multilateral and bilateral donors exist. The comparison would be diffcult and costly.

Evidence:  

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: This is a relatively new institutution. The last external review (January 2002) was done with limited number of completed projects (about 100) and inadequate baseline data in many projects. With that caveat in mind, the study concluded that "GEF-supported projects have been able to produce significant results that address important global environment problems."

Evidence: "The First Decade of the GEF: Second Overall Performance Study" pg. ix. See www.gefweb.org/1Full_Report-FINAL-2-26-02.pdf. Previous external evaluations were undertaken in 1994 and 1998.