Detailed Information on the
In-House Research for Natural Resource Base and Environment Assessment

Re-evaluating the criteria that it uses to determine outyear targets for project quality, for the purpose of establishing more ambitious targets.

Review and revise the planning, implementation and external review processes of the National Research Program 5-year cycle.

Continuing to use independent, external peer-panels to ensure that Congressional earmarks address Administration goals of quality, performance and relevance.

Conducting an independent external retrospective panel to review the Global Change and Air Quality programs during FY 2007.

Measure: Annual number of sustainable production practices and technologies developed by the Program and used by customers and partners to enhance production of food, feed, fiber, and crops for bioenergy and biobased products.

Explanation:The use of research results in itself does not ensure improvements in the environment and the use of natural respurces. There are intervening factors that are beyond the control of the research. Therefore, even though this is an output measure, it is appropriate in this instance. Research will focus on development, evaluation, and transfer of practices (e.g., new methods for fertilizer application) and technologies (e.g., new irrigation equipment) to: enhance soil capabilities to optimize agricultural production; make more efficient use of inputs (water, nutrients, and pest control chemicals); develop more energy efficient practices; increase use of manure and other byproducts; and reuse of scarce water resources. Recent examples of ARS developed practices and technologies that have been adopted include: indicators of rangeland health used by the Natural Resource Conservation Service (NRCS) to assess sustainability of range management practices; conservation tillage systems transferred to NRCS and Extension Services and used to reduce erosion and improve crop yields on highly eroded soils; and standards for estimating water use by crops used by NRCS, Bureau of Reclamation, and consultants for irrigation scheduling.

Measure: Annual number of practices and technologies specifically designed by the Program and used by customers and partners to protect the environment from agriculturally generated pollutants (e.g., sediment, agricultural chemicals).

Explanation:The use of research results in itself does not ensure improvements in the environment and the use of natural respurces. There are intervening factors that are beyond the control of the research. Therefore, even though this is an output measure, it is appropriate in this instance. Research will focus on development, evaluation, and transfer of practices (e.g., erosion control measures) and technologies (e.g., animal waste treatment systems) to control pollutants (sediments, nutrients, agricultural chemicals, pathogens, gaseous and particulate matter emissions, and greenhouse gases) from crop and animal production that could degrade soil, water and air, and contribute to global warming. Recently adopted ARS developed practices and technologies include: the Revised Universal Soil Loss Equation used by NRCS, EPA, USGS, and the Army Corps of Engineers to predict soil erosion and identify erosion control practices; alum treatment of poultry litter used by NRCS as a standard practice for control of ammonia emissions and phosphorus losses to surface waters; and the Soil Water Assessment Tool model used in the U.S. (NRCS, EPA, USGS) and worldwide to assess water quality impairments and to address water quality problems at a watershed scale.

Measure: Annual number of risk reduction decision tools developed by the Program and used by customers and partners to reduce on-farm risks to producers and off-site risks to the public and environment.

Explanation:The use of research results in itself does not ensure improvements in the environment and the use of natural respurces. There are intervening factors that are beyond the control of the research. Research will focus on development, evaluation, and transfer of decision tools (guidelines, models, decision support systems) to allow producers and technical advisors to make on-farm decisions that will reduce their economic risks (e.g., decision tools that will forecast drought and develop strategies to mitigate adverse effects). In addition, tools to predict dispersion of agriculturally generated pathogens, particulate matter, and gases that could cause public health effects will be developed. Recent examples of successfully adopted ARS developed tools include the following: the SITES (Water Resource Site Analysis Software Program) model used by NRCS to design and evaluate the safety of earthen flood control and water storage structures; and the Phosphorus Index used by NRCS in 47 States to identify areas in a field or in a watershed that are most vulnerable to phosphorus losses to surface water.

Measure: Percentage of projects annually assessed to have direct relevancy in contributing to the achievement of ARS long-term goals.

Explanation:In FY 2004, ARS initiated a new assessment mechanism to measure relevancy of its research projects. All of ARS research projects are now assessed annually against the Program Action Plans, the Agency's long-term goals, and the priority needs of U.S. agriculture. The results from the first review conducted in FY 2004, indicated that 97.1% of ARS projects were conducting highly relevant research. Those projects considered of lower relevance (2.9%) were subsequently re-evaluated, and 41% were redirected to higher priority research. In FY 2005, a second evaluation was conducted and the percent of projects considered of lower relevance had been significantly reduced to 1.7%. Information from this evaluation process is used directly for making program management decisions and in formulating the agencys budget.

Measure: Five-year rolling average percent of projects receiving a passing score by independent, external, expert panels on the first submission through the OSQR review process (year on table is the final year).

Explanation:This measure tracks research project and program quality. To ensure quality, 20% of ARS' research projects are reviewed annually. Those receiving passing scores require no further panel review. All projects with failing scores are terminated or are rewritten and re-reviewed. In the last 5 years, all ARS projects have been reviewed, with a statistical rolling average of 76.0% of the projects receiving passing grades. Since ARS now has 5 years of quality data, we have revised the performance measure to use a rolling statistical average, which provides a more accurate baseline. Increases to this average require a significant degree of improvement in any year to raise the overall number. Thus, to achieve an increase of 0.5% in FY02-06 will require a real increase of 2.5% in FY06, a most ambitious target realistically reflecting the entire ARS research program. Getting to 80.5% by 2012 is an ambitious target for 2 reasons: 1) Peer panels and researchers may have valid scientific disagreements that affect the panel's rating. Upon re-review the panel may accept the researchers approach while still differing in their opinion. Hence, 96%+ of all plans are approved. 2) If the goal moves up too quickly, the scientist will be discouraged from proposing highly innovative/creative approaches that might engender such disagreement. However, an annual increase of .5% cannot necessarily be consdered to be ambitious.

Measure: The percentage of annual research project objectives met.

Explanation:To measure the performance of the research program, in FY 2004, ARS adopted the OMB R&D Investment Criteria as its annual measure for all research. All research projects are now assessed annually to determine the number of research objectives, as described in the 5-year project plans, which were met/not met. Information as to why a research objective was not met (including mitigating circumstances) was collected and will be used for making program management decisions. ARS projects a 10% baseline level of research objectives were not met due to personnel attrition. Thus the maximum achievable research target is reduced in real terms to 90%. By implementing this measure, ARS modified its Strategic Plan, incorporated the data requirements into its GPRA process, and modified its data collection process to report project objective information.

Measure: Total program analysis through a retrospective independent external program review and assessment.

Explanation:This measure addresses Q. 2.6 in the PART analysis (Strategic Planning) framed by the R&D Investment Criteria. The score by independent expert panel reflects their judgment of the National Program in its totality at the end of the 5-year cycle (retrospectively). Beginning in FY 2004, about 20% of ARS program will be evaluated annually for a series of factors that include: Relevance, Quality, and Performance. This summary evaluation combines data from a series of pilot assessments (conducted at the end of the 5-year program cycle for several National Programs) serves as the baseline score for future retrospective reviews. ARS is now implementing a policy to govern future retrospective reviews by external panels framed by R&D Investment Criteria. The average score for National Programs ending their 5-year cycle in FY 2004 was 78 out of a possible 100. This information is used in formulating the agencys budget and in making program management decisions.

Measure: Additional research funds leveraged from external sources.

Explanation:There are several sources of additional funds to ARS including grants, trusts, reimbursables, specific agreements, and cooperative development research agreements (CRADAs). Grant funds are variable and inconsistent and are not a direct measure. Leverage (cost-sharing) can however, be directly measured by CRADAs, trusts and reimbursables, which indicate the level of partnerships between ARS and external parties interested in developing, transferring and commercializing ARS technology. The target is the percent of additional funding leveraged directly for research purposes. That is, additional funding over and above the total appropriated ARS budget, including the management overhead. The target is calculated from a running average of the previous 2 fiscal years. Targets for fiscal years 2006-2012 can only be estimated since the appropriated ARS budget is unknown.

Measure: Relative increase in peer reviewed publications.

Explanation:The measure is increased productivity through the more efficient use of research funds. The target is the increase in research publications in peer reviewed scientific journals over the previous year relative to the number of scientists in ARS. The target was calculated from a rolling average of the previous 3 fiscal years. The baseline is calculated from 2001-2003 data. Targets for fiscal years 2006 through 2012 can only be estimated since the appropriated ARS budget and the number of scientists is unknown. Peer reviewed publications as a measure of efficient use of research funds.

Is the program purpose clear?

Explanation: The mission of ARS' Environmental Stewardship Program (Protect and Enhance the Nation's Natural Resource Base and Environment) is to conduct research to provide scientific information, management practices, technologies and decision tools that allow public and private sector individuals and groups to enhance agricultural production, sustain natural resources, and improve environmental quality. ARS' research is used by Natural Resources Conservation Service, Environmental Protection Agency, academia, and trade and commodity groups to make policy and regulatory decisions that improve soil, air, and water resources. ARS research provides producers and technical advisors with tools such as conservation tillage systems to improve productivity and profitability.

Evidence: The ARS Program to Protect and Enhance the Nation's Natural Resource Base and Environment was authorized under Pub. L 104-127, Title VIII, Section 801, "Purposes of Agricultural Research" which includes increasing "the long-term productivity of the U.S. agriculture??while maintaining and enhancing the natural resource base," improving "risk management," and "safe production??that maintain the balance between yield and environmental soundness." ARS' research for this Program is defined by the USDA/REE/ARS Strategic Plans. The ARS Strategic Plan 2003-2007 states: "Agriculture relies on a natural resource base whose sustainability depends on sound, science-based production practices." "Longer-term management strategies combined with adequate knowledge of the complex natural systems can yield maximum sustainable benefits from our resources??" The Bayh-Dole and Stevenson-Wydler Acts of 1980 require all Federal agencies to transfer results of their research programs.

Does the program address a specific and existing problem, interest, or need?

Explanation: Research in this Program addresses natural resource problems that impact agricultural production and environmental quality including droughts and floods, water quality and quantity, air quality, soil degradation, and greenhouse gas emissions. Program Action Plans (www.ars.usda.gov) are designed to address these challenges. Research is being and will be conducted in current interest areas such as (1) management of water quantity/quality problems that affect agricultural production in irrigated areas of the U.S. (16% of croplands are irrigated, produce 60% of crop sales, but consume 85% of the Nation's water supply); (2) control of nutrients, pesticides, and contaminants that cause eutrification/algal blooms in inland and coastal waters, destroy aquatic ecosystems and economically impacts seafood production (agricultural sources provide 65% of the nutrients to the Gulf of Mexico); and (3) correction of overgrazing of range and pastures that results in degradation of over 50% of the land used for livestock production.

Evidence: The Western Governors' Conference identified water scarcity and drought as major problems resulting in $B of losses to U.S. agriculture. The National Academy of Sciences/National Research Council report, Air Emissions from Concentrated Animal Feeding Operations (CAFOs), concluded that insufficient information exists to make policy and regulatory decisions concerning emissions (www.ers.usda.gov/publications/ERR9). The Gulf of Mexico Nutrient Watershed Task Force has reported that improved management practices are needed to mitigate pollutant loadings from 31 States that contribute to hypoxia in the Gulf (www.nos.noaa.gov). Problems with overgrazing have been documented at www.ars.usda.gov. Research is being conducted to solve these problems.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Program does not duplicate research conducted elsewhere. The Program is designed to conduct long-term, multi-disciplinary, high-risk research to enhance agricultural production, make wise use of natural resources, and improve environmental quality. Based on core funding, 69 locations around the country and scientists from a variety of disciplines, the Program can solve national agricultural research problems that cannot be addressed by other Federal agencies, State agencies and universities, and the private sector. The ARS Program does cooperative planning with others including Federal agencies (USDA-agencies, EPA, NASA, NOAA) to more effectively address a specific aspect of a broad research area.

Evidence: The Program's core capabilities, such as long-term watershed research with critical historical databases in all 9 agro-ecosystems in the Nation; specialized equipment to monitor the impacts of global climate change on agriculture, as well as agriculture's impact on global change do not exist elsewhere (www.ars.usda.gov). Program Workshops involving customers, stakeholders, and research partners are used to identify high priority issues, set the research agenda for the next 5 years, and avoid redundancies. Information on the Program Workshops and Action Plans is available at www.ars.usda.gov/research/programs. ARS regularly consults with other Federal and State research agencies, academia, and industry to avoid unnecessary duplication. Evidence of non-duplication can be found through a search of the Current Research Information System (www.cris.csrees.usda.gov) and the scientific literature.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Agency's management of resources is impacted by Congressional add-ons and pass-through funds. Congressional add-ons to the Administration's budget are made at the expense of high priority Administration programs, even though the National Program Staff issues Program Direction and Resource Allocation Memos to labs receiving Congressional funds to ensure the research contributes to the Program's Action Plans (www.ars.usda.gov). Research conducted with Congressional add-ons and pass-through funds is subjected to the same external independent peer review by the Office of Scientific Quality Review (OSQR), performance measures, and Retrospective Program Review required for all other Program projects. Thus, all ARS research is held to the same standards of relevance, performance, and quality. By subjecting Congressionally-directed projects to these processes, the Program is able to utilize these funds to more effectively achieve Administration priorities and goals.

Evidence: In prior years, this program has received a significant number of Congressional earmarks. While the Program efficiently and effectively uses all Congressional add-on and pass-through funding to address Administration goals, theseearmarks do not represent the highest priority use of program dollars. Review of Congressional directives strictly follows the OSQR review process (www.ars.usda.gov/osqr) and retrospective review (Office of Retrospective Review). Add-ons have enhanced the development of environmentally superior technologies that allow producers to treat swine waste water to protect water and air quality and public health by reducing odor and noxious gas emissions, and controlling zoonotic pathogens (www.florence.ars.usda.gov); improved rangeland management in the Western U.S. through the development of management practices and rangeland health assessment tools (http://usda-ars.nmsu.edu/); and addressed the President's Initiative on Drought Mitigation by expanding research on effective use of water from the Ogallala aquifer (http://ogallala.tamu.edu).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Program is targeted to provide customers, stakeholders and partners with scientific information, management practices, technologies and decision tools to enhance agricultural production, sustain natural resources and improve environmental quality. The beneficiaries of the research include Federal (USDA agencies, EPA) and State Agencies, producers, technical advisors, commodity groups and environmental organizations. The Program continually seeks input from a broad range of interested parties at workshops, planning meetings and field demonstration sites to ensure that the correct beneficiaries are targeted and appropriate research priorities are established in the Program's Action Plans (www.ars.usda.gov/research/programs). Broad use of ARS research results by the National Resource Conservation Service (NRCS), EPA and producers is evidence that the correct beneficiaries have been targeted. The issue of subsidies is not applicable to this Program.

Evidence: The Program's research is actively articulated to its beneficiaries by various means, including USDA reports (www.nrcsinfo.gov, www.ers.usda.gov); scientific and publicly targeted publications and presentations; reports; conferences, workshops, and field-days; and reports to Congress. Program beneficiaries are provided scientifically sound, quantified and validated management practices and technologies for effective decision-making. The USDA-NRCS uses the Program's research to make financial and technical assistance decisions in the multi-$B Conservation Security and the Environmental Quality Incentives Programs (www.nrcs.usda.gov/technical/nri/ceap); EPA uses the research for establishing water and air quality regulations; and the U.S. poultry industry uses the research to control ammonia emissions and the movement of phosphorus to surface waters from field applied poultry-waste litter (www.epa.gov, and click on air or water).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Program has three main outcomes: enhanced agricultural production, wise use of natural resources, and improved environmental quality. These outcomes will be achieved through development of management practices, technologies and decision tools that will be used by groups such as producers, technical advisors, and Federal and State Agencies such as the National Resources Conservation Service (NRCS) to enhance the Nation's production of food, feed, fiber, and crops for bioenergy and biobased products; to protect the Nation's soil, water, and air resources; and to reduce on-farm economic risks to producers and off-farm risks to public health and environment. By 2012, the Program will have provided over 50 practical solutions to national and international problems related to environmental stewardship.

Evidence: The long-term Performance Measures of the Program are: annual number of sustainable production practices and technologies developed by the Program and used by customers and partners to enhance agricultural production; annual number of practices and technologies developed by the Program and used by customers and partners to protect the environment from agriculturally generated pollutants (e.g., sediments, agricultural chemicals); annual number of decision tools developed by the Program and used by customers and partners to reduce economic risks to producers and off-site risks to public health and the environment. Examples of past Program accomplishments include: development of the Revised Universal Soil Loss Equation (RUSLE)(http://fargo.nserl.purdue.edu/rusle2_dataweb/RUSLE2_Index.htm) used by NRCS to reduce the average rate of erosion on cropland by 33% and the Wind Erosion Prediction System (WEPS) (www.weru.ksu.edu/weps) used to reduce wind erosion by 36%.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Given the overwhelming complexity of Environmental Stewardship, a target of annually developing through 2012 from 5 to 9 new practices, technologies and decision tools used by Federal and State Agencies and producers to enhance agriculture production, sustain natural resources and improve environmental quality is inherently ambitious. Environmental and natural resource problems are extremely difficult, strongly influenced by site-specific conditions, and driven by uncontrollable natural events (floods, droughts). It takes many years to develop, evaluate, and transfer a practice. This Program addresses these multifaceted challenges by formulating long-term objectives and actionable strategies, coordinating research across multiple scientific disciplines, and utilizing cross-location research. The Program has established research baselines and clearly defined its targets to build upon ARS' capabilities to address the needs of Environmental Stewardship.

Evidence: The Program has set ambitious targets by starting at 5 new products (practices, technologies, decision tools) each year and increasing to 9 products per year by 2012. Each new product will take years to develop and validate before it is usable by customers. For example, the Wind Erosion Prediction System currently used by Natural Resources Conservation Service for soil erosion prediction and control took almost 15 years of research and testing before it was released (www.weru.ksu.edu/weps.html). New research products developed by the program (e.g. tools to predict environmental benefits of conservation practices) will be used to guide funding decisions in NRCS' $1B per year Environmental Quality Incentives Program (www.nrcs.usda.gov/programs). Milestones will be established for each product and progress documented through laboratory and GPRA Annual Performance reports (www.ars.usda.gov).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual measures are the building blocks of the research that is needed to achieve the Program's long-term goals. The Program uses these annual measures to ensure that the research stays on target to meet its long-term goals of providing practices, technologies, and decision tools to enhance agricultural production, sustain natural resources, and improve environmental quality. Each measure is specifically designed, targeted, and tracked to ensure that the research is of the highest quality and performance, while making the most efficient use of available resources to move the science closer to solving the issues identified in the long-term goals. ARS has adopted the OMB R&D Investment Criteria as its annual performance measures that focus on relevancy, quality, performance, and efficiency. High scores in performance, quality and relevance improve the likelihood that the research products that are produced will be used by others, which are the program's long term goals.

Evidence: Tracking the annual measures keeps the scientific work focused on the long-term objectives. Continued relevancy is assessed against the priorities and goals as described in the 5-year Action Plan (www.ars.usda.gov). Quality is established through external independent peer panel project review. Performance is determined by the number of project milestones met or not met each year. Total Program Analysis is determined through a retrospective external peer panel project/program review. Efficiency is gauged by the relative increase in peer reviewed publications per scientist as a measure of efficient use of research funds. Baselines, targets, and actual performance for each measure are based on a 3-year rolling average and can be found in the Measures Tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Quantifiable baselines and ambitious targets have been established. The annual Quality Measure has been revised to more accurately and realistically reflect the entire ARS research program. Annual milestones are the target measures for individual research projects; milestones increase incrementally in difficulty and complexity throughout the project's 5-year life. ARS expects and tracks each project (Annual Project Report; AD-421) to show continued progress and to produce results. Most projects will be used as the basis of national nutrition standards and/or policies. An evaluation of the milestones both annually and after 5 years determines whether a project continues, is modified, redirected, or terminated. Thus, these baselines and measures are ambitious indicators used to gauge the relative success and progress of an individual project and its contribution to the overall progress of the Program.

Evidence: Since FY04, annual Performance Measures and their baselines and targets have been identified in each research project plan. In FY05, baseline and annual targets were included in all projects. With 5 years of quality data, ARS can now present the performance measure as a 5-year rolling average, providing a more accurate baseline. Yearly increases require a significant improvement to raise the overall number. Thus, to achieve an increase of 0.5% in FY02-06 will require a real increase of 2.5% in FY06, a most ambitious target. However, while getting to 80.5% by 2012 can be considered to be an ambitious target for 2 reasons: 1) Peer panels and researchers may have valid scientific disagreements that affect the panel's rating, it can also be argued that an annual increase of .5% can be improved upon. Upon re-review the panel may accept the researchers approach while still differing in their opinion. Hence, 96%+ of all plans are approved. 2) If the goal moves up too quickly, the scientist will be discouraged from proposing highly innovative/creative approaches that might engender such disagreement. Also, ARS has found that not every annual milestone can be met. The major mitigating circumstance for not meeting a milestone is the unanticipated loss of essential personnel (illness, death, retirement, transfer, or vacancies) calculated at 10%. Thus, achieving 90% is a realistic and an ambitious target.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ARS enters into research partnerships with external organizations when a mutual benefit consistent with the ARS mission and its annual and long-term Performance Measures is clearly demonstrated. The Program conducts research at over 60 locations throughout the U.S. Most locations have external partners, such as academia, industry, and State and Federal agencies. All of the 100+ partnership agreements (Specific Cooperative Agreements, Cooperative Research and Development Agreements, etc.) clearly define the scope of the research, milestones, measures, outcomes, and the mechanisms for monitoring performance to ensure support for the overall Program goals. Each partnership agreement is overseen by an Authorized Departmental Officer's Designated Representative to ensure progress toward meeting the annual and long-term Performance Measures of this Program.

Evidence: Cooperative agreements, trusts, etc. for work in support of this Program's research goals can be found at www.ars.usda.gov/research/programs. All research agreements include performance measures directly relevant and complementary to the annual and long-term goals of this Program. Accomplishments are reported in the GPRA APR process. Information on annual meetings, workshops, and quarterly and annual reports are available on file, or at www.ars.usda.gov/research/programs. Research partners are involved in National Program Workshops and planning ongoing research activities. ARS requires its external research partners to provide Annual Project Reports (AD-421s) and identify accomplishments as they relate to the Program's goals. Contractors or agreement partners are required to file quarterly financial, progress, and summary reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The overall program process that ARS has devised covers the life of each project within the program from inception to completion with opportunities for modifications and adjustments to refine or redirect research focus and direction throughout the 5-year cycle. ARS has developed a comprehensive objective external peer review process, described at www.ars.usda.gov/osqr, specifically designed for intramural research, which has been adopted by several countries, i.e., Brazil, Canada, Finland, and France.

Evidence: To date, 4 Retrospective panel reviews, 4 National Program Workshops, 3 Action Plans, and 2 Office of Scientific Quality Review (OSQR) reviews have been completed to begin the 2nd 5-year Program cycle. The Retrospective review by an external panel near the end of the cycle evaluates relevancy, scope, quality, performance, impact, and effectiveness. The concluding findings critique the Program. This assessment is also used to create a framework for the subsequent 5-year Action Plan (research agenda). Each project undergoes a prospective independent external peer review for quality (by OSQR) before implementation. Annual Project Reports (AD-421s) are reviewed by the Area Office and the National Program Staff to assess continued performance, relevancy, and progress. The Action Plans and Annual Reports are available at www.ars.usda.gov/research/programs. ARS maintains on its Web site a comprehensive and detailed description of the OSQR peer review process at www.ars.usda.gov/osqr.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ARS explicitly ties its base budget and requests for increases to its GPRA Strategic Plan performance measures and targets. National Program Leaders identify budgetary needs and implement funding to ensure sufficient resources are available to meet annual/long-term Program needs. The Program's current budget allows it to achieve the targets projected in the measures tab. Increased funding will enable the Program to produce practices, technologies, and tools in a more timely manner to enhance the effectiveness/efficiency of USDA's multi-$B conservation programs. ARS' FY07 budget documents included 3 Performance Measures and numerous targets for the Environmental Stewardship Program and the 08 budget request will include the PART Performance Measures. ARS budget requests are reviewed and approved by USDA's Office of Budget and Program Analysis and OMB. These reviews enhance the alignment of funding to long-term performance goals. The Budget Explanatory Notes (Congressional justifications), mentioned in the evidence also shows indirect costs for strategic goals.

Evidence: ARS has several mechanisms in place that tie budget requests to long-term performance goals. At the Administration/USDA level, budgetary linkage to the accomplishment of the long-term performance goals are documented within ARS' Agency and Department Estimates and Explanatory Notes, ARS' Strategic Plan, and GPRA documents. In the Performance and Resource Plan Exhibit and the Budget and Performance Overview Exhibit of the FY07 Agency and Departmental Estimates, ARS details 27 Performance Measures tying performance targets to funding requests (www.ars.usda.gov/research), indicating the transparent relationship between budget and long-term performance goals.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: GPRA requires ARS to review and revise its Strategic Plan every 5 years. The Strategic Plan aligns ARS' priorities with the outcome oriented USDA Strategic Plan. In FY03, ARS developed a new Strategic Plan that specifically incorporated both PART and the Budget Performance Integration (BPI) initiatives in the President's Management Agenda. The previous Strategic Plan was deficient in providing a good basis for developing a performance-based budget. Beginning in FY05, the ARS budget request followed the goals and performance measures contained in its Strategic Plan. The ARS Strategic Plan makes it easier to align the National Programs with Agency goals and objectives. The previous Strategic Plan did not. This alignment focuses the Agency towards providing tangible results essential for responding to critical problems related to conserving/improving natural resources and balancing/strengthening production efficiency and environmental quality.

Evidence: USDA is completing revision of its Strategic Plan and ARS will revise its Strategic Plan by the end of this calendar year. ARS previously corrected strategic planning deficiencies when it drafted the FY03-07 ARS Strategic Plan to address the needs of the BPI Initiative, PART, GPRA, Research, Education, and Economics, and USDA Strategic Plans' goals and objectives (www.ars.usda.gov). The ARS Strategic Plan now links budget requests with specific performance measures at the Program level. Also, each project within the Program now has annual milestones that must be met to fulfill the overall Program goals and objectives. During FY05-06, about 60% of the Program's milestones were revised to more accurately align them with the Strategic Plan. The project's annual reports (AD-421) now collect information on performance indicators (milestones met/not met; see Performance Measure 6).

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: This Program is coordinated through NPS with complementary programs across the Federal Government, academia, and industry to ensure scientific relevance to the USDA mission. A wide range of formal and informal mechanisms are used to ensure an ongoing dialogue with customers and partners (including researchers from academia and the private sector) to assess potential benefits of alternative research approaches. [MORE DETAIL, INCLUDING EXAMPLES OF PAST REVIEWS AND PROGRAMS AFFECTED. HOW FREQUENTLY ARE THE REVIEWS CONDUCTED?; WHAT WERE THE RESULTS AND CHANGES MADE TO THE PROGRAM, IF ANY?]

Evidence: Four Retrospective Reviews by independent experts and 4 Stakeholder Workshops have been conducted in the last 2 years to assess the benefits of past research and to provide direction for future research. These activities have resulted in restructuring of research programs to place greater emphasis on reuse of wastewater, control of atmospheric emissions and pathogens from animal production operations, and measurement and prediction of soil carbon sequestration. Meetings with other organizations (e.g., university partners, National Aeronautics and Space Administration (NASA), Environmental Protection Agency) to compare research approaches, reduce potential duplication and maximize benefits have resulted in new research approaches within the Program. A partnership involving ARS, NASA and university partners will use new instrumentation and methodologies to measure, control and predict emissions from agricultural operations that degrade air quality.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Program's priorities are clearly identified in 5-year Action Plans which guide future budget requests and funding decisions. Partners and customers, including action and regulatory agencies, continually provide advisory input to help the Program identify funding priorities. The Action Plans, the needs of ARS' partners and customers, USDA's/ARS' Strategic Plans, the R&D Investment Criteria, and OMB's guidelines and budget targets are considered by the Budget Team (comprised of National Program Leaders and budget representatives) in formulating the Agency's budget. The Program receives input from various sources (e.g., OSTP, National Advisory Research, Education, Economics, and Extension Bd., Chesapeake Bay Foundation, So. Florida Everglades Task Force, Agricultural Air Quality Task Force, Society of Range Mgmt.), which it considers when developing future program directions. Budget recommendations are forwarded first to the Administrator and senior Agency policy officials, and then to the Department, and to OMB for review and final decision. Budget prioriies are based on discussions at the Department and with OMB, and include research on air and water quality and climate change.

Evidence: The Program obtains initial input from customers, stakeholders, and partners to prioritize Environmental Stewardship research in the Program's 5-year Action Plan. Research priorities are captured in the Program's mission statement: "will develop the knowledge base, conservation practices, and tools needed to achieve maximum sustainable benefits from our renewable natural resources on both public and private lands." Program managers make funding decisions and redirect resources within the base budget to areas of higher priority based on annual project reviews for relevance, quality, and performance. Budget requests (starting from the Agency Estimates through the Explanatory Notes) undergo a rigorous review at the Agency, Department, and OMB levels to ensure that the proposed program increases are performance based.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: As part of the process to develop specific research Project Plans, ARS establishes milestones, which become the baseline for measuring performance for each year of the 5-year project cycle. Project research leaders submit Annual Progress Reports (AD-421) that provide performance information regarding completion of milestones. This timely and credible performance data is used by NPLs and Area Directors to monitor project performance. NPLs analyze these data and use it to redirect or realign projects, if evidence suggests such a step would enhance the quality and performance of the research. The NPLs provide summary reports on progress towards meeting milestones that are widely disseminated in the GPRA annual performance report and the PART analysis.

Evidence: Project Plans, Annual Reports (AD-421), and other documents are used to manage the Program and realign, redirect, reduce, or terminate [Program Adjustment Decision Item (PADI)] a project, as needed, to maintain quality and performance. For example, in the past 2 FYs, ARS partnered with NRCS in developing the USDA Conservation Effects Assessment Program (CEAP) which is designed to quantify the cost effectiveness of science based practices implemented under the Farm Bill. CEAP serves as a model for interaction among multiple agencies to create long-term scientific databases and to better meet conservation needs (www.nrcs.usda.gov/technical/nri/ceap). In FY06, to improve overall management, the Program is being realigned from 7 to 4 program components. Also, ARS proposed in its FY07 budget the redirection of research from laboratories with limited resources and capabilities to higher priority research at other locations.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: National Program Leaders ((NPLs) program managers) and Center/Area Directors (line managers) are held strictly accountable annually for evaluating Program impact, quality, and performance to ensure that the goals are being met and results are produced in a timely manner to meet expectations. Program and line managers responsible for the Program have specific program management performance standards in their annual evaluations. All ARS grants and agreements include terms and conditions that require recipient performance and financial reports to ensure funds are provided and obligated in accordance with the terms and conditions of the award. All research agreements are continuously monitored by appropriate program and line managers to ensure research activities are meeting project goals and milestones in a timely and fiscally responsible manner.

Evidence: Managers and scientists are subject to annual performance reviews. ARS scientists are subject to an in-depth peer review on a 3- to 5-year cycle under the Research Position Evaluation System. The annual reports from each research project, which contain performance results, are reviewed by both the Area Office and the National Program Staff. NPLs are evaluated on programmatic criteria annually and held accountable for evaluating their Programs for "?? relevance, direction, and rate of progress toward providing solutions to specific high-priority national problems and research objectives." Their Performance Plan also requires them to develop "?? recommendations to line managers for strengthening the Agency's research program." Line managers have additional performance standards holding them accountable for cost, schedule, and performance of program partners.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: ARS allocates, obligates, and manages Agency funds at the project level to ensure adherence to the Program plan while limiting unobligated year-end funds.. All ARS grants and agreements hold partners to strict performance standards and fiscal accountability requirements (financial expenditures and cost management). As part of the annual performance appraisal, Center/Area Directors are held accountable for the enforcement of Agency procedures ensuring the timely and appropriate use of resources. ARS uses the Foundation Financial Information System (FFIS) accounting system to enhance accountability. ARS undergoes annual financial and programmatic audits by USDA/OIG and GAO that review timeliness of obligating and reporting, accuracy in accounting, and intended use of funds and the prompt correction of deficiencies identified. All Program fund awards are reported promptly and accurately via FFIS.

Evidence: The FFIS accounting system is used to process and manage financial data to meet stringent budget and fund control needs, as well as complex multi-fund reporting requirements. This Program does not have carryover authority; all funds are expended or obligated by the middle of the 4th quarter of each fiscal year. ARS uses its own obligation tracking system in conjunction with FFIS to ensure funds are obligated up to, but not in excess of, the assigned funding level. Historically, ARS obligates more than 99% of its assigned funds. In FY05, ARS' obligation rate was 99.7%. In FY05, ARS implemented a Program Direction and Resource Allocation Memo process to ensure the prompt dissemination of new funds and to provide specific program direction for the use of these funds. Scientists/managers are held accountable for financial management of fiscal resources. ARS has received clean audit opinions from OIG and GAO for several years.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ARS has engaged in continuous efforts as part of its competitive sourcing program to pursue Agency strategies to identify efficiencies to better support its research programs. ARS' "Green Plan" is a long-term approach to "feasibility reviews" that evaluate, analyze, and measure commercial functions within the Agency's FAIR Act Inventory. To operate as efficiently as possible, ARS has consistently managed with an overhead of 10% or less and the Program leverages funds to achieve increased cost effectiveness. Analyses conducted by ARS exposed IT infrastructure deficiencies particularly in cyber security, network connectivity, and email configurations. Subsequently, ARS analyzed the costs and benefits of alternative solutions for improving email and web hosting services; recommended email and web hosting solutions that best meet the Agency's needs; and developed an efficient and cost effective IT Consolidation Project Plan for infrastructure improvements. The program has one efficiency measure with baselines and targets. This is a measure of the improvement in the efficiency in the output of scientific staff, as measured by an increase in the peer-reviewed publications per scientist.

Evidence: ARS monitors the cost from A-76 competitions to ensure that projected benefits are achieved. ARS continually works to identify future opportunities through its "Green Plan" which provides a framework and methodology for its continued evaluation of its commercial activities. Leveraged funds have increased to about 9% of the Agency's appropriated funds. In FY05, ARS further upgraded the IT Consolidation Project to incorporate new standards for a USDA-wide email system, and to integrate with the USDA Universal Telecommunications Network (UTN). The project consolidated web hosting, began updating infrastructure for email and web hosting, integrated with UTN, and began planning new enterprise architecture. By FY07, ARS will have unified 52 email systems, completed infrastructure upgrades, and built the enterprise architecture. The benefits of IT consolidation will have provided ARS with an efficient, secure, scalable, robust network, enterprise email, and web hosting capabilities. The efficiency measure for this program, the "relative increase in peer reviewed publications" per sceintist is included in the measures tab, and has baselines and targets.

Does the program collaborate and coordinate effectively with related programs?

Explanation: ARS conducts intramural research and is not a grants-based program. Aspects of the Program are coordinated with related programs across the Federal Government (Natural Resources Conservation Service (NRCS), Cooperative State Research Education and Extension Service (CSREES), and EPA), academia, State agencies, industry, and commodity groups. Many of these groups fund extramural research that is complementary to the work of ARS. Whenever possible, ARS collaborates with these organizations to meet their needs and, where appropriate, enters into Cooperative Research and Development Agreements, and agreements to transfer the research knowledge for implementation and use. These efforts assure close coordination of ARS research to eliminate program overlap and duplication. To maintain close coordination, ARS continually works with its partners to reevaluate the environmental and conservation needs based on changing priorities

Evidence: Program managers regularly meet with ARS partners to ensure coordination of effort. Conservation Effects Assessment Project (www.nrcs.usda.gov/technical/nri/ceap/), a nationwide effort to measure and predict the effectiveness of conservation practices, has been jointly planned and implemented by ARS, CSREES, NRCS, U.S. Geological Survey, and EPA. This effort will guide regulatory and policy decisions relative to natural resource management. ARS has redirected $16M to this high priority activity which will result in more effective use of the $4B/year conservation programs administered by NRCS (www.nrcs.usda.gov). ARS and EPA cooperate to evaluate methodology for measuring gaseous emissions from animal production operations in the EPA's Environmental Technology Verification (ETV) Program (www.epa.gov/etv/). Research efforts to protect air quality by reducing emissions from agricultural operations have been enhanced through cooperation with university and NASA partners.

Does the program use strong financial management practices?

Explanation: Agency expenditures are continuously tracked using the ARS' Agricultural Research Information System (ARIS) and the Foundation Financial Information System (FFIS) to monitor scientific and fiscal progress. ARS maintains extensive financial management policies and procedures on funds management and accountability, including resource management, funds control, and semiannual reviews of unpaid obligations. ARS is free of material internal control weaknesses. FFIS is used to track accuracy and timeliness of payments; this system has received a clean audit opinion.

Evidence: ARIS tracks scientific funding, objectives, milestones, and accomplishments across the Program. FFIS is used to process and manage financial data to meet stringent budget and funds controls, as well as complex reporting requirements. Implementation of FFIS and its associated data warehouses offers the integration and capacity to improve the delivery of timely and meaningful financial management information, putting ARS in compliance with legislation, including the CFO Act of 1990. FFIS helped ARS achieve a clean OIG audit of its financial statements for FY05. In addition, ARS is part of the USDA Consolidated Audit. For the last 4 years, USDA has received an "unqualified (clean) audit opinion." In FY05, ARS implemented a new Program Direction and Resource Allocation Memo process that provides specific guidance to field managers on the direction of research programs and the allocation of resources (fiscal, personnel, and equipment) to achieve program goals.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Agency continually seeks management improvements. In FY 2000 and 2005, ARS revised its research management structure to ensure coherent and relevant research. A 5-year program planning cycle now includes an external retrospective assessment of relevance, quality, performance, and management of the Program. This assessment directs development of the next 5-year Action Plan. National Program Leaders, with input from customers, stakeholders, and partners, establish the National Program (NP) research agendas to ensure program goals remain relevant. Area Directors are responsible for performance and scientific quality. Each new project undergoes a rigorous, independent external peer review that provides scientific credibility to the Program's research agenda and identifies project level management deficiencies. Area Offices and the National Program Staff (NPS) conduct in-depth external location reviews during the NP cycle to identify and correct management and program deficiencies. NPS is tasked with continually monitoring management procedures for improvement.

Evidence: In 2005, NPS was restructured from 3 to 4 program areas to better manage Agency goals and to highlight the importance of Environmental Stewardship research. In FY06, the Environmental Stewardship Program is being realigned from 7 to 4 components and expanded to include bioenergy research to improve overall management. In addition, the Agency proposed in its FY07 budget to redirect research from laboratories with limited resources and capabilities to higher priority research at other locations. The Program has also initiated cross-location projects to strengthen cooperation among ARS units addressing problems related to water/air quality and global climate change. This action has improved management of personnel, fiscal, and facility resources to better address the Program's objectives as outlined in the new Action Plans. Matrix management within ARS requires continuous communication between Progam managers and line managers.

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: This is an intramural program. The process used to allocate funding employs peer review panels that ensure performance, quality and relevance in the selection of spefici projects, as would a competitve review process. The statutory National Agricultural Research, Extension, Education, and Economics,Advisory Board (NAREEEAB) regularly provides ARS with both general and specific guidance on how to allocate its resources. This input, plus Customer/Stakeholder input via Program Workshops, helps determine which research priorities to fund. Then all projects are reviewed by an independent external peer panel (Office of Scientific Quality Review (OSQR)) to ensure quality. Research plans are revised to respond to the review panel's critique. Projects required to make major revisions are re-reviewed. The panel may indicate that a project is not feasible. Under these circumstances, the National Program Staff (NPS) and the Area Office jointly consider whether the project should be revised or discontinued and whether the scientist(s) and funds should be redirected. Periodic location reviews and retrospective program assessments, by outside experts, monitor quality throughout the 5-year program cycle.

Evidence: The NAREEEAB has reviewed and approved the method ARS uses for fund allocation and copies of their recommendations are available at www.nareeeab.com. To ensure the quality of ARS' research program, each project undergoes periodic review by independent external peer panels. The OSQR process invites leading scientists both nationally and internationally from industry, academia, and Federal and State agencies to evaluate the prospective research plan for the quality of the science proposed and the suitability of the scientists to conduct the work (www.ars.usda.gov/osqr). To date, OSQR has completed review of all ARS projects through one cycle. The 2nd 5-year cycle of review began in FY05 with the review of 2 Program components. Eighty-two percent of the projects were approved by the peer panels on the 1st review and after revision and re-review, 100% of the projects were approved.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has achieved its targets for each of its three long term goals. For each of the three long term The Program's long-term performance is measured annually by tracking practices, technologies, and decision support tools developed and used to enhance agricultural production, sustain natural resources, and protect the environment. The Program is on target to meet the goals identified in the Measures tab. These address the most critical Environmental Stewardship concerns expressed to ARS by its customers and partners, such as Federal agencies (USDA agencies, EPA), academia, State Departments of Agriculture and Natural Resources, commodity groups, and producers. Long-term partnerships are in place to determine the effectiveness of conservation practices with Natural Resources Conservation Service (NRCS), EPA and universities; control wind and water erosion with NRCS; and prevent contamination of soil, water and air by animal production operations with EPA and universities.

Evidence: In the last 3 years, the Program has met its PART targets by developing practices, technologies, and decision support tools to address agricultural production, natural resources conservation, and environmental protection concerns. The Program has produced: animal waste treatment technologies used by 15% of U.S. poultry producers to reduce ammonia emissions; the Revised Universal Soil Loss Equation used in all 50 States by NRCS and private consultants; the Soil and Water Assessment Tool, a model used by EPA, NRCS, and Farm Services Administration for Total Maximum Daily Load determinations; "SITES" technologies used nationally by NRCS and the U.S. Army Corps of Engineers to assess dam and reservoir safety; and soil carbon sequestration data generated to support the President's Global Climate Change Initiative and used by the State Dept. This Program had 15 indicators aligned under 7 performance goals with 28 accomplishments in the FY05 GPRA Report.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Despite the fact that most of its measures acheived their targets , this question received a "small extent." That is because of a "no" for question 2.4, since one of the measures, the percentage of annual milestones met, did not appear to have ambitious targests. However, ARS strives to meet its target goals. ARS uses independent external panels for review purposes. In FY04, ARS began annually evaluating all research projects for relevance and performance against priorities and goals in the 5-year Action Plan. ARS modified its annual reporting mechanism (AD-421) to specifically collect quantifiable milestone data to better measure annual research performance. Data on mitigating circumstances for milestones not met described in Section 2.4 were also collected. Beginning in FY05, ARS has specifically included program milestones in all new Action Plans. All partners, including Federal Agencies and universities, are committed to the annual targets. The Office of Scientific Quality Review quantifiably measures quality based on a 5-year rolling statistical average since 20% of the ARS research program is evaluated annually.

Evidence: ARS successfully met 4 of its 6 PART annual Performance Measures in this Program in FY05, almost met a fifth, and improved in a 6th. The relevance measure improved from 97.1% in FY04 to 98.3% in FY05. While 100% is the ultimate goal for relevance, this may not be realistically achievable. 87.0% of total annual milestones were met. Of those milestones not met, two-thirds were due to critical scientific vacancies, program redirection, or insufficient resources. 76.8.9% of ARS projects passed on the first quality review. After re-review by the peer panel, a total of 95.5% of all projects were approved. Beyond the R&D criteria, ARS met or exceeded additional annual measures for external independent total program assessment and for the new efficiency measure which gauges relative increases in peer reviewed publications per scientist as a measure of efficient use of research funds. Additional information and future targets are described in the Measures tab.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Program has identified areas of research and resources that can be shared to meet common goals and needs of U.S. agriculture. Trusts, reimbursables, Cooperative Agreements, and Cooperative Research and Development Agreements that ARS has entered into indicate the level of partnership with external parties interested in developing, transferring, and commercializing ARS research and technologies. To increase efficiency and cost effectiveness the Program makes significant effort through leveraging activities, for example: co-locating with university partners, shared laboratory equipment, formation of core facilities, and sharing support staff. The Program annually reviews its operations for opportunities to further improve efficiency and cost effectiveness. However, ARS routinely sets aside 10% of project allocations for administrative expenses, when smaller amounts may be attainable. The efficiency goal has also shown improvement during the last two years, by increasing the number of peer reviewed publications, measured on a per-scientist basis. It also met its target for both 2004 and 2005.

Evidence: To improve efficiency and cost effectiveness, the Program makes significant effort through leveraging activities, for example: co-location or joint operation with university partners to share laboratory equipment, core facilities, support staff, and to extend scientific expertise available to the Program. In FY05, the Program increased leveraged funds by approximately 6% relative to FY04. The Program recently restructured research units at Oxford, MS, and Beltsville, MD, to increase efficiency and reduce cost; University scientists, working with the National Sedimentation Laboratory in MS, were critical to developing practices and modeling capabilities that improve erosion and sedimentation prediction; new facilities were completed at Maricopa, AZ, to consolidate research units currently scattered at multiple locations; and new cost sharing research partnerships were developed. ARS continues to maintain its overhead at <10%. The efficiency measure of the number of peer reviewed publications per scientist has increased from 1.25 for the period from 2001 through 2003, to 1.35 for the period from 2002 to 2004, to 1.52 for the period from 2003 through 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The core long-term components of this Program are not fully addressed by any comparable agricultural research entity in government or the private sector. The Program addresses the broad scope of research related to agricultural, natural resources, and environmental issues. ARS research covers a wide range of issues affecting the agricultural economy, sustainability, and productivity, including reducing risks, and the ability to make the resulting knowledge available for practical use by producers, Cooperative State Research Education and Extension Service (CSREES), consultants, and industry. CSREES and other agencies may fund some extramural research with short-term objectives in these areas. Private sector investment is typically short-term, motivated by profits, and geared to specific commercial products. However, the limited external studies that have been done recognize the value and importance of this national research Program compared to others.

Evidence: The Program's core components address the broad scope of national agricultural priorities through long-term, high-risk, fundamental, applied, and developmental research, and technology transfer to potential users. ARS is distinguished from other research organizations by its databases used worldwide to understand basic soil processes; ability to measure odors, particulates, and microbial components from agricultural production; and ability to evaluate large-scale cropping/grazing/mixed land use practices. NAS/NRC envisions Environmental Stewardship as a new frontier for research. Several GAO reports (www.gao.gov) stress the significance of ARS watershed and natural resource data for decision making. In a 2005 NAS-sponsored workshop, ARS was recognized as a significant international contributor to recent advances in soil science (http://soilsassociation.org/new/NAS-Workshop2005.htm). ISI Essential Science Indicators list ARS as the most cited agricultural research institution.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: ARS has developed a comprehensive objective external peer review process, specifically designed for intramural research that evaluates all aspects and stages of the Program, as discussed in the evidence section. Initially, Customer/ Stakeholder Workshops are held to provide independent external input about program performance and research needs that contribute to the development of an Action Plan to guide research over the next 5 years. Each ARS project is evaluated by an independent external expert panel to ensure project quality and validate relevance. Tracking project milestones annually ensures the performance of the research Program. At the end of the 5-year cycle, an independent external panel retrospectively evaluates the Program's research performance for scope, focus, productivity, and impact. This array of checks/oversight assures the relevance, quality, and performance of the research conducted by ARS.

Evidence: In FY05, 76.1% of all ARS projects peer reviewed for quality passed the first review. The remaining 23.9% were revised and re-reviewed. After the second review a total of 95.5% of the projects were approved by the peer panel. The remaining 4.5% were redirected to areas of higher priority. In FY05, 87% of milestones were met for all ARS National Programs and 20% were evaluated retrospectively for overall performance with an average score of 86.6%. Four Retrospective panel reviews, completed within the 2nd 5-year Program cycle, have indicated high impact of the Program's research. One of the panels concluded, "ARS is the leader in water resources research...and techniques, tools, and models developed have become global standards for agriculture" (ftp://ftp.nps.ars.usda.gov/upload/201/). In 2004, NAS/NRC commended the Program for relevancy, acknowledging that it addressed 19 or 27 high priority research issues that the Academy identified.