Detailed Information on the
National Park Service - Land and Water Conservation Fund State Grants Assessment

Determine a process and schedule for an independent evaluation of this program.

Based on Interior IG report issued Feb. 2008, and other relevant factors, update existing measures and/or add new measure(s) potentially including # of new acres enhanced, cumulative acres protected, population served by new grants, and average unit cost per site inspection.

Establish an effective reporting process to show national results. States would still be responsible for setting their own targets and priorities.

Implement new procedures to measure and achieve efficiencies in workload performance.

Prepare an annual report that uses performance measures to show program results nationally and by state.

Coordinate with State partners to identify appropriate performance measures, consistent with the DOI strategic plan.

Work with States to establish baselines and set targets for performance measures.

Measure: Number of parks where new outdoor recreation facilities were developed.

Explanation:LWCF assistance results in the development of a new recreation facility or area that was not previously present in an existing park.

Measure: Number of parks enhanced through development or rehabilitation.

Explanation:LWCF assistance results in the development of new or rehabilitation of new or rehabilitation of an existing outdoor recreation resource and/or support facility.

Measure: Number of new acres protected.

Explanation:LWCF assistance results in new acres protected under sec. 6(f)(3) of the LWCF Act; including due to both development and acquisition of parkland.

Measure: Average grant application processing time efficiency measure.

Explanation:Average number of days from receipt of complete application through grant approval date. Desired result is increased productivity.

Is the program purpose clear?

Explanation: The LWCF Act clearly states the purpose is "to assist in preserving, developing, and assuring accessibility to all citizens . . . outdoor recreation resources" by "providing Federal assistance to the States in planning, acquisition, and development of needed land and water areas and facilities."

Evidence: Land and Water Conservation Act (LWCF) Act of 1965 (16 U.S.C. 460l-4). Outdoor Recreation Resources Commission Report (1988). National Park System Advisory Board findings (1994).

Does the program address a specific and existing problem, interest, or need?

Explanation: The need for outdoor recreation opportunities is broadly recognized. Almost 40 years ago, the LWCF Act spoke of the need for outdoor recreation resources "to strengthen the health and vitality" of U.S. citizens; today, health advocates continue to cite that need. Many surveys show the importance of outdoor recreation for people's quality of life. States regularly pass bonds for outdoor recreation needs. Demand for hiking, camping, and other outdoor activities continues to increase.

Evidence: LWCF Act (16 U.S.C. 460l-4). See each of the 56 Statewide Comprehensive Outdoor Recreation Plans, or SCORPs.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The program is well designed to assist State and local government efforts in providing outdoor recreation opportunities. Although the program duplicates State and local programs, the gap in non-Federal services is large enough to warrant a Federal program. Requirements for States to match funds and prepare statewide plans help to ensure that the Federal grants mesh with non-Federal responsibilities. Some other Federal programs (e.g., HUD's Community Development Block Grants, or CDBG) can support outdoor recreation activities, but LWCF grants have a much broader population of applicants.

Evidence: See a cross-cut comparison between LWCF and other Federal programs. NPS notes there are 87,000 units of governments that are eligible to receive LWCF State grants, compared to about 1,000 for CDBG grants. NPS also notes that 98% of all counties have received an LWCF grant at some point. NPS argues the gap in non-Federal efforts is best shown through surveys, various capital investment plans, and the large number of applicants willing to meet the 50 percent matching requirement.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program does not have sufficient program measures and reporting requirements to determine the overall effectiveness of the program. As a result, NPS cannot adequately document program results or verify the extent to which Federal funds are well targeted to meet program purposes.

Evidence: Section 6(d) of the LWCF Act (16 U.S.C. 460l-8) authorizes NPS to collect "other necessary information, as may be determined by the Secretary", but so far no performance information has been required from States. NPS will work cooperatively with the States to identify performance measures by 10/1/04 and begin collecting performance data no later than 10/1/05.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: States use the SCORP planning process and an "Open Project Selection" process to identify and select priorities that target beneficiaries most effectively. These processes ensure that no grant is funded without proof that it meets some need as defined by the State in the SCORP. NPS reviews the States' final project selections to ensure that the Federal funds are passed on to the intended beneficiaries.

Evidence: See examples of State SCORP plans and Open Project Selection procedures. NPS notes that the 50/50 matching requirement also helps to ensure that funds go only to serious applicants, since half of the funds for each project must come from non-Federal sources.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: NPS lacks salient, meaningful performance measures that capture the most important aspects of the program. NPS will need extensive coordination with State partners to develop adequate measures that support the goal in the new DOI Strategic Plan to "improve access to appropriate recreation opportunities on DOI managed or partnered lands and waters."

Evidence: No evidence provided to show that the program systematically collects information from States on program outcomes. NPS does track one measure (the number of acres made available for outdoor recreation through LWCF acquisition grants), but it still lacks adequate information on prior performance and future targets.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: See explanation for question 2.1.

Evidence: See evidence for question 2.1.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: NPS lacks salient, meaningful performance measures that capture the most important aspects of the program. NPS will need extensive coordination with State partners to develop adequate measures that support the goal in the new DOI Strategic Plan to "improve access to appropriate recreation opportunities on DOI managed or partnered lands and waters."

Evidence: No relevant evidence available. Although the program has limited measures on its processing of grant applications, these workload measures do not demonstrate progress towards reaching long-term goals.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: See explanation for question 2.3.

Evidence: See evidence for 2.3.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The program cannot measure and report on the performance of its partners as it relates to accomplishing the overall goals of the program. Nevertheless, evidence suggests that most State partners are committed to working with the program through (a) timely updates to their SCORPs, (b) obligation of funds, (c) project completion as outlined in grant agreements, and (d) post-completion site reviews.

Evidence: No relevant evidence available.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: No independent evaluations have been conducted since the 1980s, and no reviews are currently scheduled. As a result, DOI is strongly encouraged to conduct a review as soon as possible.

Evidence: GAO conducted a review of the program in 1981 and cited problems with the program. The Outdoor Recreation Resources Commission in 1988 evaluated the national need for outdoor recreation, and the American Planning Association reviewed the SCORP planning process in 1989, but these reviews did not focus on the overall effectiveness of the program.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget planning cannot be adequately tied to performance planning until sufficient outcome-based performance measures are developed. Program budget documents do not clearly indicate the full costs of achieving performance goals.

Evidence: No evidence was provided to show that budget plans are based on performance or results.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has not completed any formal strategic planning, but has begun to work aggressively with States to identify program performance goals that are consistent with the new DOI Strategic Plan to "improve access to appropriate recreation opportunities on DOI managed or partnered lands and waters." NPS and the States will need to reach agreement on ways to measure performance and collect data that demonstrate progress in addressing these goals.

Evidence: There is no relevant evidence available.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: NPS does not collect performance data related to key program goal and use that information to adjust priorities, allocate resources, or make management decisions. Although NPS does monitor how grantees obligate and use funds, that is a basic requirement and not a systematic process to monitor overall program performance.

Evidence: NPS and the States do not have a systematic process for setting and monitoring results-oriented performance targets. So far, the LWCF Grants Manual (Chapter 600.8) only describes the procedural requirements for State programs to comply with basic Federal grant requirements (e.g., appraisals, financial reporting).

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: NPS does not require partners to set and meet cost, schedule and performance goals. Although the program does require State partners to meet certain Federal grant requirements (e.g., obligate funds within three years), these requirements do not represent specific performance standards. NPS has no systematic policy on how States may use grant funding for administrative purposes, so there is a wide variety of rates used by States to determine the amount of indirect costs charged against Federal funds.

Evidence: So far, the LWCF Grants Manual has no requirement for States to measure results using performance goals. NPS has not provided evidence of specific performance standards or incentives for program partners, or evidence that grant and contract awards consider past performance. NPS did provide limited evidence that it enforces compliance with Federal grant requirements.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The LWCF Act requires States to obligate grants within three years. NPS appears to have adequate procedures in place to encourage timely obligations, including a Special Reapportionment Account for funds that have been withdrawn or deobligated.

Evidence: LWCF Act section 6(b)(4), LWCF Grant Manual (chapter 600.3), reports on apportionments for 02, 03 and unobligated balances for 02 & 03.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has not developed adequate procedures to measure and achieve efficiencies. It has begun to track average grant processing time, but more evidence is needed (e.g., future targets and written explanations of why processing time has not improved). The program has begun to make some IT improvements, and may be able to show improved efficiencies through the use of an electronic grant application and management system.

Evidence: The only efficiency measure shows that the average LWCF grant processing time has increased from 31 days in 2000 to 66 days in 2002. This is due to many factors, including a 168% increase in grants, but NPS needs to provide more information (e.g., written explanation for changes, outyear targets) before this becomes useful evidence. It also needs to find additional measures, such as cost per grant application processed, to track cost effectiveness in program application. NPS and DOI have taken steps toward electronic grant applications, which could improve program efficiencies by next year.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The program coordinates with the National Association of State Outdoor Recreation Liaison Officers (NASORLO) and individual State offices to ensure the grants go to projects consistent with the States' SCORPs. The States, in turn, coordinate with NPS and other Federal agencies to ensure that the SCORPs are consistent with various Federal requirements. NPS conducts a limited review of SCORPs, but does not produce an annual report that compiles information from States on accomplishments and performance.

Evidence: Examples of State SCORPs show some meaningful collaboration with NPS and other Federal agencies. Coordination with other agencies shown in other grant manuals, such as U.S. Fish and Wildlife Service (Grants Manual, 660 FW 4); Federal Highway Administration (Bicycle and Pedestrian Planning); and USDA Forest Service (Eastern Region Recreation Blueprint). NPS still needs to work with grantees to jointly produce an annual report, performance goals, and grant announcements that demonstrate meaningful collaboration.

Does the program use strong financial management practices?

Explanation: The program manages payments through HHS's SMARTLINK system, which is used by many Federal grant programs. This system allows for up-to-date monitoring of grantee payments and draw-downs. Also, the NPS Accounting Operations Center (AOC) tracks obligations and provides regular updates. The program is not aware of any questioned costs or audit exceptions found under the Single Audit Act process. It is also using the Federal Audit Clearinghouse to monitor audits more closely.

Evidence: See description of SMARTLINK system. Also see example of AOC's reports generated through the Document Direct system.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has been slow to address management deficiencies, partly because it is rebuilding after no funding during much of the 1990s. NPS and a NASORLO task force have identified portions of the Grants Manual that require updating, and are revising the State Review Workbook. NPS still needs to identify deficiencies in performance information.

Evidence: No evidence provided of an annual report or summary of accomplishments in meeting performance goals and addressing management deficiencies. However, NPS has made some initial efforts, such as convening a NASORLO task force and drafting updates for the State Review Workbook.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: NPS has a reporting system to track expenditures by grantees. It also reviews SCORPs and conducts post-completion site inspections to verify that funds are used for their designated purpose.

Evidence: The LWCF Grants Manual identifies procedures for site visits, periodic inspections, SCORP reviews, and post-completion inspections. NPS still needs to document the annual accomplishments from grantee activities.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program does not collect performance information in a user-friendly manner. NPS should prepare an annual report that includes information on project accomplishments, annual expenditures by state, workload measures, and performance results.

Evidence: NPS did not provide evidence of grantee performance data. NPS did publish on the internet a three-year summary report (FY2000-02), but this was just a list of approved projects (see: http//www.nps.gov/lwcf/). The program plans to develop an annual report for calendar year 2003.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: NPS lacks salient, meaningful performance measures that capture the most important aspects of the program. NPS will need extensive coordination with State partners to develop adequate measures that support the goal in the new DOI Strategic Plan to "improve access to appropriate recreation opportunities on DOI managed or partnered lands and waters."

Evidence: No evidence provided to show that the program systematically collects information from States on program outcomes. NPS does track one measure (the number of acres made available for outdoor recreation through LWCF acquisition grants), but it still lacks adequate information on prior performance and future targets.

Does the program (including program partners) achieve its annual performance goals?

Explanation: See explanation for question 4.1.

Evidence: See evidence for question 4.1.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The program has not yet developed efficiency measures or been able to demonstrate high levels of efficiency through other means. It has made some initial IT improvements, such as including forms on its website, but it has not provided evidence that management practices have resulted in efficiency gains over the past year.

Evidence: No evidence of meeting performance targets to reduce per unit costs or other steps that result in tangible productivity or efficiency gains. Although NPS did provide data on grant processing time, the trends do not indicate improved efficiencies. NPS also did not set targets or identify strategies for improving processing time.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: The program did not provide sufficient evidence to determine if (a) benchmarks exist to compare performance against other programs, or (b) no comparable programs exist.

Evidence: No evidence of benchmarks to compare performance to other programs. NPS did provide evidence for question 1.3 that shows the program purpose and design is not redundant or duplicative of other Federal programs. Yet, that is not the same as showing that the results of this program cannot be compared to other programs.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: No independent evaluations have been conducted since the 1980s, and no reviews are currently scheduled. As a result, DOI is strongly encouraged to conduct a review as soon as possible.

Evidence: GAO conducted a review of the program in 1981. The Outdoor Recreation Resources Commission in 1988 evaluated the national need for outdoor recreation, but it did not focus on the effectiveness of the program. The American Planning Association reviewed the SCORP planning process in 1989.