Detailed Information on the
Drinking Water State Revolving Fund Assessment

Implement recommendations from the second triennial drinking water data quality review which are designed to improve the overall quality of the data in EPA's drinking water compliance reporting system.

Develop a new long-term outcome performance measure to assess the impact of drinking water compliance improvements on public health.

Develop an efficiency measure that is more useful and meaningful for tracking annual programmatic efficiency.

Measure: Percent of population served by community water systems that will receive drinking water that meets all applicable health-based drinking water standards through approaches including effective treatment and source water protection.

Explanation:This measure tracks the percentage of population served by community water systems that meet all applicable health-based drinking water standards. If systems meet such standards, the population's exposure to contaminants is reduced.

Measure: Percent of community water systems that meet all applicable health-based standards through approaches that include effective treatment and source water protection.

Explanation:This measure tracks the percentage of community water systems that meet all applicable health-based standards. If systems meet such standards, the population's exposure to contaminants is reduced. The revised target was developed based on a bottom-up approach, which provides a more realistic target that is based on historical observations of performance since the late 1990's to the present.

Measure: People receiving drinking water that meets all applicable health-based standards per million dollars spent to manage the national drinking water program

Explanation:This measure will be changed for PWSS, UIC, and DWSRF and will include EPA funds, UIC and PWSS grants (with associated state match) , DWSRF grant set-aside funds and DWSRF funds provided for assistance (including state match, bond proceeds, repayments and interest earnings).

Measure: Fund utilization rate for the DWSRF.

Explanation:Cumulative dollar amount of loan agreements divided by cumulative funds available for projects.

Measure: DWSRF Long-Term Revolving Level ($billions/yr)

Explanation:Indicates the amount of funds available to be disbursed from the DWSRF program. The target is an average level of $1.2 B/year for the period 2018-2035

Measure: Average funding (in millions of dollars) per project initiating operations.

Explanation:Dollars include all federal and state DWSRF funds made available to projects that have initiated operations since inception of the program. Average funding per project initiating operations was $1.74 million in 2004 and $1.71 million in 2005.

Measure: Number of additional projects initiating operations.

Explanation:

Is the program purpose clear?

Explanation: The DWSRF provides funds to States to establish State loan revolving funds that finance infrastructure improvements for public water systems and other activities that support State drinking water programs and promote public health protection.

Evidence: The program's authorizing statute (Safe Drinking Water Act amendments of 1996, Section 1452) and final rule (40 CFR 35, Subpart L) provide clear and consistent statements that the purpose of the DWSRF is to further public health objectives under the SDWA and to promote the efficient use of fund resources.

Does the program address a specific and existing problem, interest or need?

Explanation: The DWSRF provides a financial tool to address infrastructure problems associated with the provision of drinking water that is safe, affordable and compliant with SDWA drinking water standards. The program also has a focus on assistance to small systems (including private systems) that have difficulty finding financing.

Evidence: The 1999 Drinking Water Infrastructure Needs Survey identified $151 billion in infrastructure needs for the next 20 years. The DWSRF Report to Congress estimates that 93% of community water systems serve fewer than 3,300 people (19% of total population).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While other programs may provide financial assistance to water systems for infrastructure improvements, none have the program design of a revolving fund which can provide a long-term, sustainable funding source for states to address system needs. Additionally, the DWSRF may provide assistance to privately-owned systems, which most similar programs are not allowed to fund. The program precludes duplicative funding of projects through regulatory constraints and segmentation of multiple sources of funding within EPA.

Evidence: SDWA. GAO report on water funding sources discussed differences between different federal programs. CFR 35.3125 Limitations on SRF Assistance specifies the prevention of double benefit.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The priority-setting process that states are required to use focuses the funds on the most important public health and compliance infrastructure needs. The DWSRFs utilize loans plus a state match, rather than grants, to enable a sustainable source of funding. The program design provides significant flexibility to states to maximize protection of public health (including leveraging of funds). The design ensures short-term sustainability of a state's fund by requiring all repayments and interest earnings to be deposited in the fund.

Evidence: SDWA and regulations describe requirements for fund to operate in perpetuity and target funds to public health and compliance needs.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: States receive grants which are allotted based on their share of the overall national need identified in the most recent infrastructure needs survey. States must develop priority systems which give emphasis to projects needed for public health protection, compliance and economic need on a per household basis. States must then offer funding to those systems with the highest priority that are ready to proceed with construction. During required annual reviews of state programs, EPA regional staff review records to ensure that the state is in compliance with the requirement to address the highest priority projects.

Evidence: SDWA and regulations describe allotment formula and priority-setting requirements

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has an outcome-based, long-term perfomance measure that supports the goals of "Water Safe to Drink" by reducing exposure to contaminants. The program has key long-term output measures that support the goal of developing state funds that are self-sustaining after federal support ends. The program is implementing two new long-term outcome efficiency measures that will be included in the 2006 GPRA strategic plan and a new long-term output efficiency measure that reports the cumulative number of projects initiating operations per cumulative dollars. The program has committed to develop a high level outcome measure (based on GAO's Hierarchy of Indicators) to assess the reduction in waterborne disease outbreaks due to unsafe drinking water.

Evidence: The performance measure tracks the percent population served by community water systems that receives drinking water in compliance with health-based standards. The measure is listed in EPA's 2003 Strategic Plan as a strategic target to track progress on the proposed goal of "Water Safe to Drink." The program tracks the national long-term average revolving level of the fund to assess long-term sustainability. For the efficiency measures (see measures tab) a "Measures Implementation Plan" has been submitted to OMB.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets and timeframes for the outcome performance measure and efficiency measure are ambitious given the existing external factors that limit EPA control and measurement of program progress. The measures emphasizes the importance of sustaining compliance as well as returning systems to compliance. Ambitious targets have been established for one of the key output measures. Targets for new output measures must be developed by next year.

Evidence: The baselines and targets provided in the measures tab for the long-term outcome measure are published in EPA's Strategic Plan, and the program's more detailed Subobjective Implementation Plan.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has an outcome-based annual measure that tracks the rate of compliance of the nation's 53,000 community water systems with drinking water standards. If systems are in compliance, the population's exposure to contaminants is reduced. The program has two output performance measures that demonstrate progress toward the long-term financial and environmental goals. The program has a new efficiency measure as discussed in 2.1.

Evidence: The DWSRF contributes to the performance measure proposed in EPA's 2003 Strategic Plan: Increase the percent of community water systems that meet Pre-2001 and Post-2001 drinking water standards.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The targets and timeframes are ambitious given the the existing external factors that limit collection of reliable data. For example, a percentage of the systems do not submit compliance reports. In future years, EPA must reduce external factors to continue to receive a yes answer.

Evidence: The baselines and targets provided in the measures tab for the annual outcome measure are published in EPA's Strategic Plan, and the program's more detailed Subobjective Implementation Plan.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Insufficient evidence. States submit annual reports on the use of funds, but do not report on how funding is linked to the long-term goal.

Evidence:

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: At the Federal level, GAO has evaluated EPA's DWSRF program. EPA/HQ conducts reviews of EPA regional programs. EPA Regional Offices review state programs annually. At the State level, 43 states conduct separate independent audits with the remainder scheduled for periodic audits by the EPA Inspector General, which also reviews the quality of the other independent audits. These evaluations support program goals for financial performance. None of these audits evaluate the project level data necessary to assess performance with respect to public health goals. Additionally, audit-based approaches do not meet the quality criteria for determing the impact of the program.

Evidence:

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Agency estimates and budgets for the full annual costs of operating its programs, taking into consideration any changes in funding, policy and legislative changes. All spending categories and the resource levels and activities associated with them are included in the annual Congressional Justification. Presentation to Congress of the Agency's budget, including resources for the DWSRF program, include alignment to its Strategic Plan goals. For the DWSRF, EPA also utilizes a planning model to align the budget with the output goal of achieving a target long-term revolving level.

Evidence: Annual Congressional Justification. Budget Automation System (BAS) reports. The SRF planning model has been used by both OMB and EPA to evaluate the impact of changes in appropriations and economic factors on the long-term revolving level.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: EPA has taken steps to strengthen its oversight of States' management and use of the fund, and has developed several key measures of the programs financial performance. But it has not linked this oversight role to performance evaluation related to the long-term goals of protecting public health and establishing funds that are sustainable in the absence of federal support.

Evidence:

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Data for outcome performance measures are compiled from the Safe Drinking Water Information System (SDWIS/FED). EPA's Inspector General recently reported data quality problems with this reporting system and concluded that EPA is not accurately reporting its performance. EPA collects DWSRF financial and progress information annually from States through Annual Reports and through its information management system (DWNIMS). Regions review reports and data to determine if the program is meeting its objectives described in annual intended use plans and to use in discussions with the state on how to improve program performance.

Evidence: OIG, March 5, 2004. EPA Claims to Meet Drinking Water Goals Despite Persistent Data Quality Shortcomings, No. 2004-P-0008. Regulatory requirements for annual reports, information collection, annual reviews by regional staff. Examples of Performance Evaluation Reports (PERs) that show how information collected is used to assess performance.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: EPA has designated DWSRF program managers in all regional offices and at the national level. Additionally, Federal regional grant project officers are held accountable for ensuring that all policies and procedures of the EPA Grants Administration Division are followed. Grantees are accountable through grant agreements with EPA for program costs. EPA Regions' annual review of state performance under the grants and audit results can be used to adjust grant conditions.

Evidence: DWSRF program responsibilities are specified under performance standards in personnel performance appraisals. The final rule (40 CFR 35, Subpart L) specifies performance standards to be included in grant agreements (Section 35.3550).

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The DWSRF program requires states to have a schedule with timing targets to ensure that federal grants are taken in a timely and efficient way. DWSRF funded projects are identified on each State's Priority Projects List. EPA Regional Offices review state programs annually. At the State level, 43 states conduct separate independent audits with the remainder scheduled for periodic audits by the EPA Inspector General, which also reviews the quality of the other independent audits. All grantees are required to submit annual reports and supply data to EPA national database (DWNIMS) that document the activities of loan recipients. The reporting and evaluations confirm that recipients are spending the funds designated to each project for the intended purpose.

Evidence: Specific regulatory requirements governing DWSRF capitalization grant agreements are located at 40CFR Part 35 Subpart L Sections 35.3545 and 35.3550. In 2003 nearly 80% of available funds were committed to projects. This measure has increased every year since program inception; in 1998 it was 33%; in 2000 it was 64%; and in 2002 it was 75%. DWNIMS data are available at: www.epa.gov/safewater/dwsrf/dwnims.html. Many state and regional offices make annual report information available on their web sites.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The DWSRF program requires states to have a schedule with timing targets to ensure that federal grants are taken in a timely and efficient way. States have flexibility to adopt procedures to maximize effectiveness. For example, states may not award loans to systems that do not have the technical, financial and managerial capacity to accomplish the intended results.

Evidence: Operating agreements obtain state commitment schedules to commit and expend all funds as efficiently as possible. On a quarterly basis, EPA regional staff checks federal cash draw requests against negotiated payment schedules to ensure state compliance.

Does the program collaborate and coordinate effectively with related programs?

Explanation: EPA has national agreements with the USDA Rural Utilities Service and HUD Community Development Block Grant programs, which also operate financing programs for water supply, to facilitate coordination within states. However, the effectiveness of coordination at the federal level has not been evaluated. A forthcoming report from EPA's Environmental Finance Advisory Board is expected to indicate that coordination generally has been effective although there are state-specific issues impeding coordination. DWSRF coordinates with related EPA programs (Public Water System Supervision and Underground Injection Control programs) to work toward shared performance goals.

Evidence: DWNIMS records indicate that 27 states report that 15% of DWSRF agreements were coordinated funding. Supporting material from State data submissions to DWNIMS indicates that at least 41 States answer "yes" to the question of whether they coordinate funding for infrastructure. EPA's 2003 Strategic Plan on the proposed goal of "Water Safe to Drink."

Does the program use strong financial management practices?

Explanation: The program follows EPA's financial management guidelines for committing, obligating, reprogramming, and reconciling appropriated funds. Agency officials have a system of controls and accountability, based on GAO and other principles, to ensure that improper payments are not made. At each step in the process, the propriety of the payment is reviewed. EPA trains individuals to ensure that they understand their roles and responsibilities for invoice review and for carrying out the financial aspects of program objectives. EPA received an unqualified audit opinion on its FY02 financial statements and had no material weaknesses associated with the audit. EPA reviewed the Drinking and Clean Water State Revolving Fund (DWSRF and CWSRF) programs and found only isolated instances of erroneous payments (<0.25%). Controls were found to be effective and the majority of overpayments had already been collected. The Office of Water SRF training held in March included a discussion on erroneous payments.

Evidence: Annual Congressional Justification, Budget Automation System (BAS) reports, Data: unqualified audit opinion on EPA FY02 financial statements, Fiscal Year 2002 Advice of Allowance Letter, 2002 Integrity Act Report. October 31, 2002, EPA final report on the review of the DWSRF AND CWSRF Operating agreements obtain state commitment schedules to commit and expend all funds as efficiently as possible.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: EPA is developing an action plan to address data quality issues in SDWIS/FED. EPA has taken steps to address program management deficiencies. For example, EPA has taken action to address findings in a January 2002 GAO report that criticized EPA's slow development of financial indicators for use in assessing state programs and insufficient use of DWNIMS data in conducting annual reviews of state programs. EPA/HQ developed a review strategy to evaluate regional program management effectiveness and EPA Regions conduct annual reviews of state program effectiveness and compliance. EPA also has procedures to address non-compliant state programs.

Evidence: EPA, March 2004. Drinking Water Data Reliability Analysis and Action Plan (2003). EPA finalized a suite of financial indicators in May 2002 and is working to increase use of DWNIMS data in oversight. EPA has provided past examples of non-compliance notices to states.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Annually, data collection through DWNIMS, site visits, program audits and performance evaluation reports track how funds are used. EPA's Integrated Financial Management System tracks federal outlays to grantees.

Evidence: Grantee activities and use of funds are documented in the DWNIMS data reported by states.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: EPA/HQ collects data on grantee activities through DWNIMS and makes them available to the public through a web site. State grantees must submit biennial reports on meeting goals/objectives. EPA Regions conduct annual reviews with each state. States are audited for proper practices.

Evidence: DWNIMS data are available at: www.epa.gov/safewater/dwsrf/dwnims.html. Many state and regional offices make annual report information available on their web sites.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program is implementing new efficiency measures and is committed to a strategy and workplan. The DWSRF is progressing toward its long-term performance target although data quality issues suggest the numbers may under-report actual performance. It also met targets that measure relevant outputs that link to the long-term goal.

Evidence: Summarized in measures tab. EPA's Financial Planning Model projections indicate that a long-term revolving level of $1.2 billion can be achieved under current economic and proposed federal funding conditions.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The DWSRF appears to have met its annual performance targets although data quality issues suggest the numbers may under-report actual performance. It also met targets that measure relevant outputs that link to the annual goal.

Evidence: Summarized in measures tab.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has several output efficiency measures and demonstrates improved efficiencies in meeting its financial management goals. The program has made great progress in developing an outcome-based efficiency measure.

Evidence: The average fund utilization rate nationwide (a key efficiency measure) continues to increase. A high rate indicates that DWSRF funds are expeditiously used. From 1998 to 2003, the rate increased from 33% to 79%.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Administration's Common Measures assessment for rural water activities indicated higher efficiencies for the Rural Utilities Service Water Funding Program and EPA's DWSRF relative to the Bureau of Reclamation Water Funding Program and the IHS Sanitation Program. The common measures selected were population and connections served per million dollars. The measures considered focus on only a small part of the DWSRF program.

Evidence: Findings of Rural Water drinking water funding projects described in the Department of the Interior FY 2004 budget narrative.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Although evaluations have been performed, they do not meet the criteria for independence, scope and quality. The Inspector General (IG) conducts financial audits of selected state programs each year, and evaluates independent audits conducted within each state for consistency with financial accounting standards. No independent evaluations of program achievements with respect to outcomes have been performed, Given that the program represents more than 10% of EPA's budget, more frequent and comprehensive evaluations are warranted.

Evidence: There has been no comprehensive independent evaluation of the DWSRF. A single GAO report has evaluated three key aspects of the program and has identified some deficiencies. GAO concluded that "EPA does not have all the information it needs to monitor the state's implementation of the program or assess the programs overall effectiveness."