Detailed Information on the
In-House Research for Nutrition and Health Assessment

Re-evaluating the criteria that it uses to determine outyear targets for project quality, for the purpose of establishing more ambitious targets.

Conducting an independent external peer review during FY 2007.

Review and revise the planning, implementation, and external review processes of the National Research Program 5-year cycle.

Conducting an independent external peer review during FY 2007.

Continuing to monitor the actual use of research outputs (new knowledge and technologies).

Measure: Annual number of Federal and Institute of Medicine reports used to establish Federal nutrition policy and regulations that employ Program research results in formulating recommendations to safeguard the health of the American people.

Explanation:The use of research results in itself does not ensure improvements in overall nutritional well being. There are intervening factors that are beyond the control of the research. Therefore, even though this is an output measure, it is appropriate in this instance. The purpose of this measure is to assist ARS customers in reaching authoritative positions on nutrition and health issues ranging from nutrient requirements to prevention of obesity and chronic diseases. Research focuses on: nutrients such as vitamins, minerals, protein, fats; nutrient needs for different age and gender groups; optimal levels of nutrients to promote health and prevent disease; and regulation of healthy body weight through diet and physical activity. Program results have been used to establish sound dietary recommendations for the American public, such as the Dietary Reference Intakes and the USDA/HHS Dietary Guidelines, as well as to formulate policy for guiding USDA food assistance, DoD feeding programs, and FDA nutrition labeling.

Measure: Annual number of food intake and nutrient content databases released by the Program and used by ARS customers to establish Federal dietary policy guidelines, food assistance and feeding programs, and food labeling to safeguard the health of the American people.

Explanation:The use of research results in itself does not ensure improvements in overall nutritional well being. There are intervening factors that are beyond the control of the research. Therefore, even though this is an output measure, it is appropriate in this instance. The purpose of this measure is to assist ARS customers in evaluating the healthfulness of the American food supply and the nutrient content of the American diet. The U.S. food supply is dynamic with thousands of new and reformulated foods introduced each year. New foods combined with changing food choices and increasing consumption of foods away from home require continual food consumption monitoring and food supply nutrient analyses to evaluate the healthfulness of the American diet. Recent database releases include: What We Eat in America (NHANES 2001-2002), Standard Release 18 of the National Nutrient Databank, and special interest databases including the content of fluoride, added sugars, and other food components with health effects. These databases are used by USDA (FNS, CNPP) and other Federal agencies (NIH, FDA, CDC, DoD, EPA), private industry, and individuals who can download any of them to their computers, PDAs, or use them from a web page.

Measure: Percentage of projects annually assessed to have direct relevancy in contributing to the achievement of ARS long-term goals.

Explanation:In FY 2004, ARS initiated a new assessment mechanism to measure relevancy of its research projects. All of ARS research projects are now assessed annually against the Program Action Plans, the Agency's long-term goals, and the priority needs of U.S. agriculture. The results from the first review conducted in FY 2004, indicated that 97.1% of ARS projects were conducting highly relevant research. Those projects considered of lower relevance (2.9%) were subsequently re-evaluated, and 41% were redirected to higher priority research. In FY 2005, a second evaluation was conducted and the percent of projects considered of lower relevance had been significantly reduced to 1.7%. Information from this evaluation process is used directly for making program management decisions and in formulating the agencys budget.

Measure: Five year rolling average percent of projects receiving passing scores by independent, external, expert panels on the first submission through the OSQR review process (final year is shown on the table).

Explanation:This measure tracks research project and program quality. To ensure quality, 20% of ARS' research projects are reviewed annually. Those receiving passing scores require no further panel review. All projects with failing scores are terminated or are rewritten and re-reviewed. In the last 5 years, all ARS projects have been reviewed, with a statistical rolling average of 76.0% of the projects receiving passing grades. Since ARS now has 5 years of quality data, we have revised the performance measure to use a rolling statistical average, which provides a more accurate baseline. Increases to this average require a significant degree of improvement in any year to raise the overall number. Thus, to achieve an increase of 0.5% in FY02-06 will require a real increase of 2.5% in FY06, a most ambitious target realistically reflecting the entire ARS research program. Getting to 80.5% by 2012 can be considerd ambitious for 2 reasons: 1) Peer panels and researchers may have valid scientific disagreements that affect the panel's rating. Upon re-review the panel may accept the researchers approach while still differing in their opinion. Hence, 96%+ of all plans are approved. 2) If the goal moves up too quickly, the scientist will be discouraged from proposing highly innovative/creative approaches that might engender such disagreement. However, it can be argued that annual increases of only .5% , with an overall goal of 80.5% for FY 2008 through FY 2012 can still be improved upon without causing a lack of innovation.

Measure: The percentage of annual research project objectives met.

Explanation:To measure the performance of the research program, in FY 2004, ARS adopted the OMB R&D Investment Criteria as its annual measure for all research. All research projects are now assessed annually to determine the number of research objectives, as described in the 5-year project plans, which were met/not met. Information as to why a project objective was not met (including mitigating circumstances) was collected and will be used for making program management decisions. ARS projects a 10% baseline level of project objectives not met due to personnel attrition. Thus the maximum achievable project target is reduced in real terms to 90%. By implementing this measure, ARS modified its Strategic Plan, incorporated the data requirements into its GPRA process, and modified its data collection process to report project objective information.

Measure: Total program analysis through a retrospective independent external program review and assessment.

Explanation:This measure addresses Q. 2.6 in the PART analysis (Strategic Planning) framed by the R&D Investment Criteria. The score by independent expert panel reflects their judgment of the National Program in its totality at the end of the 5-year cycle (retrospectively). Beginning in FY 2004, about 20% of ARS program will be evaluated annually for a series of factors that include: Relevance, Quality, and Performance. This summary evaluation combines data from a series of several pilot assessments (conducted at the end of the 5-year program cycle for several National Programs) serves as the baseline score for future retrospective reviews. ARS is now implementing a policy to govern future retrospective reviews by external panels framed by R&D Investment Criteria. The average score for National Programs ending their 5-year cycle in FY 2004 was 78 out of a possible 100. This information is used in formulating the agencys budget and in making program management decisions.

Measure: Additional research funds leveraged from external sources.

Explanation:There are several sources of additional funds to ARS including grants, trusts, reimbursables, specific agreements, and cooperative development research agreements (CRADAs). Grant funds are variable and inconsistent and are not a direct measure. Leverage (cost-sharing) can however, be directly measured by CRADAs, trusts and reimbursables, which indicate the level of partnerships between ARS and external parties interested in developing, transferring and commercializing ARS technology. The target is the percent of additional funding leveraged directly for research purposes. That is, additional funding over and above the total appropriated ARS budget, including the management overhead. The target is calculated from a running average of the previous 2 fiscal years. Targets for fiscal years 2006-2012 can only be estimated since the appropriated ARS budget is unknown.

Measure: Relative increase in peer reviewed publications.

Explanation:The measure is increased productivity through the more efficient use of research funds. The target is the increase in research publications in peer reviewed scientific journals over the previous year relative to the number of scientists in ARS. The target was calculated from a rolling average of the previous 3 fiscal years. The baseline is calculated from 2001-2003 data. Targets for fiscal years 2006 through 2012 can only be estimated since the appropriated ARS budget and the number of scientists is unknown. Ppeer reviewed publications as a measure of efficent use of reseach funds.

Is the program purpose clear?

Explanation: The mission of the Agricultural Research Service (ARS) Human Nutrition Research Program (the "Program") is to conduct high priority national research to better understand the role of food, its components, and appropriate diet choices to optimize health throughout the human life cycle. Outcomes of the Program's research are used by the National Academy of Sciences to establish Dietary Reference Intakes for children, adults, and the elderly. Subsequently, this information, plus Program food and nutrient databases, are used by Federal agencies to design food assistance and food labeling programs. Program outcomes are also translated into dietary recommendations and policies to improve the overall nutritional level of the U.S. population.

Evidence: The Human Nutrition Research Program was authorized under Public Law 104-127, Title VIII, Section 801 of the Farm Bill to "maintain an adequate, nutritious, and safe supply of food to meet human nutritional needs and requirements," and Public Law 101-445, Nat'l. Nutrition Monitoring and Related Research Act of 1990 (currently pending re-authorization) which directs USDA "to assess the dietary and nutritional status of the U.S. population. . . to establish dietary guidelines and for other purposes." The vision and mission statements of the Human Nutrition National Program (NP) 107 are explicitly stated on the ARS Web site at www.ars.usda.gov/research/programs/programs.htm?NP_CODE=107. ARS' research for Human Nutrition is clearly defined by the current USDA/REE/ARS Strategic Plans. The Bayh-Dole and Stevenson-Wydler Acts of 1980 require all Federal agencies transfer results of their research programs.

Does the program address a specific and existing problem, interest, or need?

Explanation: Health related problems such as obesity and poor nutrition directly contribute to 4 of the 6 major causes of morbidity and mortality: heart disease, cancer, stroke, and diabetes. Nearly 2/3 of American adults are overweight or obese and the incidence of obesity in children age 6 to 11 has tripled in the last three decades. Poor nutrition and obesity impact national defense in that 18% of men and 43% of women exceed screening weight limits for military service. This Program plays a critical role, as identified by Congress and the Administration, in providing science-based approaches to establish the nutritional needs of the American people. Currently, the annual health care costs related to poor nutrition alone exceed $265B (16% of total health care costs) indicating the Program purpose is highly relevant.

Evidence: Evidence of these problems is highlighted in the President's HealthierUS Initiative, the DHHS Healthy People 2010 goals, and by the recent establishment by the Office of Science and Technology Policy (OSTP) of an Interagency Work Group (IWG) on Overweight and Obesity Research, and substantiated in numerous research journals (www.PubMed.gov). Further, research from ARS' Program provides the basis for the U.S. calorie and nutrient requirements set by the Institute of Medicine (IOM) (affiliated with the Nat'l. Academy of Sciences) and the U.S. Dietary Guidelines established by USDA/DHHS. This research is used in policy formulation to guide USDA food assistance, DoD food service, and FDA nutrition labeling programs, and serves as the nutritional basis for food product formulation by private industry in an effort to reduce these diet-associated conditions/diseases.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: ARS' Human Nutrition Program is designed to be the sole, foremost and/or preferred authority for providing research data used by others to set both nutrient and dietary requirements for the U.S. population. This Program is coordinated through National Program Staff with complementary programs across the Federal Government, academia, industry, producers, processors, and consumer groups (end-users) to ensure scientific relevance. Relevance to the USDA mission is regularly evaluated and coordinated with other Federal, industry, and university programs both nationally and internationally. The Program avoids unnecessary duplication through continual interactions with its research customers and partners. Some activities are unique and not conducted by any other entity (see 4.4). Where possible and appropriate, purposeful duplicative research is undertaken with collaborators to substantiate original Program findings.

Evidence: This Program is the principal source of research and the only source of nationally representative nutrition monitoring and food composition information for establishing the Dietary Reference Intakes (U.S. nutrient requirements) and Dietary Guidelines for Americans (food recommendations). The ARS Program is the primary source of food consumption data, and conducts the only nutrient analysis of foods commonly eaten by high-risk populations, such as American Indians and the poor of the Lower Mississippi Delta region. Approximately 2/3 of the data used by the IOM for establishing U.S. nutrient requirements are derived from ARS' Program. ARS has the unique role for national nutrition monitoring ("What We Eat in America" survey) and food composition analyses. Collaborative projects can be found at http://www.ars.usda.gov/research/programs/programs.htm?NP_CODE=107.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Agency's management of resources is impacted by Congressional add-ons and pass-through funds. However, only about 1.3% of all funding for this program is provded through earmarks. In addition, Congressional priorities are addressed by incorporating them in the Program's Action Plan (www.ars.usda.gov). National Program Leaders set objectives to meet both Congressional and Administration priorities via a Program Direction Memo. These are subjected to the same required external independent peer review by the Office of Scientific Quality Review (OSQR), performance measures, and separate, external Retrospective Program Review. In this way, all ARS research is held to the same standards of relevance, performance, and quality. The Program has no evidence that another approach would be more efficient or effective to achieve the intended purpose.

Evidence: The Program efficiently and effectively uses all Congressional add-ons and pass-through funding to address Administration goals. Review of Congressional directives strictly follows the OSQR review process (www.ars.usda.gov/osqr) and retrospective review (Office of Retrospective Review). Add-ons have enhanced the Program's access to medical research expertise otherwise unavailable to ARS. Redirection of existing resources doubled research on obesity in response to the Presidential HealthierUS Initiative between FY02-05 and now accounts for 23% of Program expenditures. Leveraging by the Program brings in substantial additional funding. Program Centers operated cooperatively with universities bring in $1.12 in other funding for every $1.00 of ARS support, thus expanding our research portfolio.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Program is effectively designed to provide research data to other agencies that establish National food assistance programs and formulate dietary recommendations and policy which helps the U.S. public make better food choices. Research outputs are made available through peer reviewed publications and Web databases for direct use by Program beneficiaries. To ensure that the right beneficiaries are targeted, the Program continually seeks input from its customers (USDA, FDA, Institute of Medicine), commodity and voluntary health organizations (e.g., American Heart Assn. (AHA), American Dietetic Assn. (ADA)), and food producers. The input provides the basis for establishing and redirecting priorities that are identified in the Action Plan that guide the Program during its 5-year cycle (www.ars.usda.gov/research/programs), in order to adequately reach the intended beneficiaries. The issue of subsidies is not applicable to this Program.

Evidence: The Program's Action Plan provides a targeted yet flexible process to reaching the needs of all and only the intended beneficiaries. Program research is actively articulated to its beneficiaries by appropriate means, e.g., peer reviewed publications, authoritative reports from the Institute of Medicine, and Web sites such as www.Nutrition.gov. Also, research from the Program on nutritional needs of individuals, dietary intake ("What We Eat in America" survey of National Health and Nutrition Examination Survey), and food composition (U.S. Nat'l. Nutrient Databank), is used extensively by other Federal agencies (USDA, NIH, FDA, CDC, EPA), health organizations (ADA, AHA, American Cancer Society, etc.), and academia. Examples include: USDA as the basis for food assistance program guidelines and allotments (www.fns.usda.gov), FDA for food labeling (www.fda.gov), and EPA in regulating pesticide residues in children's foods (www.epa.gov).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Program has 2 long-term outcome Performance Measures that provide information to understand the role of food, its components, and appropriate diet choices in optimizing health. Over the next 5 years, ARS data will be released and used in over 70 new food survey and composition databases and in Federal and Institute of Medicine (IOM) reports for establishing nutrition policy and regulations. For example, this Program directly supports the President's HealthierUS Initiative and the DHHS Healthy People 2010 goals. The outcomes of this research benefit and improve the overall nutritional level of the U.S. population through improved dietary guidance and food assistance programs. In the last 3 years, cumulative usage of Program products resulted in 21 releases and reports by others that underpin the development of Federal nutrition policy and regulations, such as "MyPyramid" (www.MyPyramid.gov) and the release of 6 IOM Dietary Reference Intakes reports between 1997-2005 (www.nas.edu).

Evidence: The outcomes as described in the Performance Measures tabs are: (1) cumulative number of food intake and nutrient content databases released by the Program and (2) cumulative number of Federal and Institute of Medicine reports used to establish Federal nutrition policy and regulations. Products of the Program are: regular update releases to the National Nutrient Databank and national food consumption surveys; and expansion of the knowledge base on human nutrient requirements, obesity prevention, and health promotion. Specifically, a new food composition database for choline led IOM to establish, for the first time, a requirement for this nutrient. The IOM set all dietary standards using Program results which have led to changes in all USDA food assistance programs (leading to a reduction in saturated fats, added sugars, and an increase in fruits and vegetables) and in FDA nutrition labeling that better informs the public about healthy food choices.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The challenges confronting the health of most Americans are complex and require long-term objectives. Additionally, behavioral changes to meet dietary recommendations are notoriously difficult to implement. Some aspects of the Program's role are to provide research data that enables the Institute of Medicine (IOM) to establish dietary recommendations and other Federal agencies to improve food assistance programs. The IOM has exacting expectations for research and a mechanism that involves panels of international experts who review all sources of data and decide those that meet the quality standards needed to set U.S. dietary recommendations. During the 5-year Program cycle, a very limited number of Federal and IOM reports are produced for establishing Federal nutrition policy and regulation. Therefore, since the process takes 18-24 months per report, the targets and timelines are highly ambitious.

Evidence: The Program's research and survey databases capture long-term and nationally representative information requiring years to collect and analyze. The Program has set an ambitious target of at least 6-8 new IOM and other Federal reports and databases each year through 2012 especially since IOM only released 6 dietary recommendation reports from 1997-2005 (www.nas.edu) which drew heavily upon program results as detailed in 2.1 evidence. These ambitious targets include use of Program data to establish sound dietary recommendations for the U.S. public, (Dietary Reference Intakes and USDA/DHHS Dietary Guidelines), and to formulate policy for USDA food assistance, DoD feeding programs, and FDA nutrition labeling. Database releases include: the National Nutrient Databank, special interest food tables, and releases from "What We Eat in America" national dietary intake survey.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual measures are the building blocks of the research that is needed to achieve the Program's long-term goals. ARS uses these annual measures to ensure that the research stays on target to meet the Program's long-term goals of providing data for Federal and Institute of Medicine reports and databases needed to establish nutrition policy and regulations to safeguard the health of Americans. Each measure is specifically designed, targeted, and tracked to ensure that the research is of the highest quality and performance, while making the most efficient use of available resources to move the science closer to solving the issues identified in the long-term goals. ARS has adopted the OMB R&D Investment Criteria as its annual performance measures that focus on relevancy, quality, performance, and efficiency. Programs that demonstrate high levels of quality, performance and relevance are more likely to produce products that ARS customers will use.

Evidence: Tracking the annual measures keeps the scientific work focused on the long-term objectives. Continued relevancy is assessed against the priorities and goals as described in the 5-year Action Plan. Quality is determined through an external independent peer-panel project review. Performance is determined by the number of project milestones met or not met each year. Total Program Analysis is determined through a retrospective external peer panel project/program review. Efficiency is gauged by the relative increase in peer reviewed publications per scientist as a measure of efficient use of research funds. Baselines, targets and actual performance for each measure are calculated on a 3-year rolling average and can be found in the Measures Tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Quantifiable baselines and ambitious targets have been established. The annual Quality Measure has been revised to more accurately and realistically reflect the entire ARS research program. Annual milestones are the target measures for individual research projects; milestones increase incrementally in difficulty and complexity throughout the project's 5-year life. ARS expects and tracks each project (Annual Project Report; AD-421) to show continued progress and to produce results. Most projects will be used as the basis of national nutrition standards and/or policies. An evaluation of the milestones both annually and after 5 years determines whether a project continues, is modified, redirected, or terminated. Thus, these baselines and measures are ambitious indicators used to gauge the relative success and progress of an individual project and its contribution to the overall progress of the Program. However, the percent of projects that receive a passing score on their first submission to the OSQR process increases by only .5% a year,a rate which does not seem to be ambitious.

Evidence: Since FY04, annual Performance Measures and their baselines and targets have been identified in each research project plan. In FY05, baseline and annual targets were included in all projects. With 5 years of quality data, ARS can now present the performance measure as a 5-year rolling average, providing a more accurate baseline. Getting to 80.5% by 2012 is an ambitious target for 2 reasons: 1) Peer panels and researchers may have valid scientific disagreements that affect the panel's rating. Upon re-review the panel may accept the researchers approach while still differing in their opinion. Hence, 96%+ of all plans are approved. 2) If the goal moves up too quickly, the scientist will be discouraged from proposing highly innovative/creative approaches that might engender such disagreement. Also, ARS has found that not every annual milestone can be met. The major mitigating circumstance for not meeting a milestone is the unanticipated loss of essential personnel (illness, death, retirement, transfer, or vacancies) calculated at 10%. Thus, achieving 90% is a realistic and an ambitious target.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ARS enters into research partnerships with external organizations when a mutual benefit is clearly demonstrated that is consistent with the ARS mission and its annual and long-term goals. The Human Nutrition Program operates 6 Centers, half of which are Federally-operated and half of which are operated through cooperative agreements with Universities/Colleges of Medicine. All partnership agreements (Specific Cooperative Agreements, Cooperative Research and Development Agreements, etc.) clearly define the scope of the research, milestones, measures, outcomes, and the mechanism for monitoring performance to ensure support for the overall Program goals. Each partnership agreement is overseen by an Authorized Departmental Officer's Designated Representative to ensure progress toward meeting the annual and long-term goals of this Program.

Evidence: Cooperative agreements, trusts, etc. for work in support of this Program's research goals can be found at http://www.ars.usda.gov/research/programs/programs.htm?projectlist=true&np_code=107 . All research agreements include performance measures directly relevant and complementary to the annual and long-term goals of this Program. Accomplishments are reported in the GPRA APR process. Information on annual meetings, workshops, and quarterly and annual reports are available on file, or at www.nps.ars.usda.gov/programs. Research partners are involved in National Program Workshops and planning ongoing research activities. ARS requires its external research partners, including the 3 university-based Centers, to provide Annual Project Reports (AD-421s) and identify accomplishments as they relate to the Program's goals. In addition, contractors or agreement partners are required to file quarterly financial, progress, and summary reports.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The overall program process that ARS has devised covers the life of each project within the program from inception to completion with opportunities for modifications and adjustments to refine or redirect research focus and direction throughout the 5-year cycle. ARS has developed a comprehensive objective external peer review process specifically designed for intramural research, which has been adopted by several countries, i.e., Brazil, Canada, Finland, and France (http://cordis.europa.eu.int/innovation/en/smeip.htm).

Evidence: At the onset of the Program cycle, relevancy is established in consultation with customers and partners. From this input, ARS develops an Action Plan to establish the 5-year research agenda. Projects undergo a prospective independent external peer-review for quality by the Office of Scientific Quality Review (OSQR) before implementation. The Program undergoes retrospective assessment by an external review panel near the end of the 5-year cycle to evaluate for relevancy, scope, quality, performance, impact, and effectiveness. The findings critique the Program. This assessment is also used to create the 5-year Action Plan. Annual Project Reports (AD-421s) are reviewed by the Area Office and the National Program Staff to assess continued performance, relevancy, and progress. The Action Plans and Annual Reports are available at www.ars.usda.gov. ARS maintains on its Web site a comprehensive and detailed description of the OSQR peer review process at www.ars.usda.gov/osqr.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The ARS budget planning process makes every attempt to ensure that its research activities are directly linked to annual and long-term Performance Measures. ARS and USDA Estimates and Explanatory Notes detail many of ARS' strategic performance measures and targets which are tied to the Program's annual budget request. National Program Leaders identify budgetary needs and implement funding to ensure sufficient resources are available to meet annual/long-term Program needs. ARS' FY07 budget documents included 2 Performance Measures and numerous targets for the Human Nutrition Program and the 08 budget request will include the PART Performance Measures. Increased funding may not necessarily increase the number of outcomes, but would significantly reduce the time to achieve the outcomes. However, since different projects may be proposed each year, it is not always possible to draw a direct relationship between funding for individual projects and performance from year to year. In addition, the Budget Explanatory Notes (Congressional justification), as mentioned in the evidence, includes both the direct and indirect costs associated with strategic goals.

Evidence: ARS has several mechanisms in place that tie budget requests to long-term performance goals. Budgetary linkage to the accomplishment of the long-term performance goals is documented within ARS' Agency and Department Estimates and Explanatory Notes and GPRA document. In the Performance and Resource Plan Exhibit and the Budget and Performance Overview Exhibit of the FY07 Agency and Departmental Estimates, ARS details 27 performance measures tied to performance targets to dollars for each (www.ars.usda.gov/research) indicating the complete and transparent nature of the relationship between research requests and budget.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: GPRA requires ARS to review and revise its Strategic Plan every 5 years. The Strategic Plan aligns ARS' priorities with the outcome oriented USDA Strategic Plan. In FY03, ARS developed a new Strategic Plan that specifically incorporated both PART and the Budget Performance Integration (BPI) initiatives in the President's Management Agenda. The previous Strategic Plan was deficient in providing a good basis for developing a performance-based budget. Beginning in FY05, the ARS budget request followed the goals and performance measures contained in its Strategic Plan. The ARS Strategic Plan makes it easier to align the National Programs with Agency goals and objectives. The previous Strategic Plan did not. This alignment focuses the Agency towards providing tangible results essential for responding to the critical nutrition and health-related needs of the U.S. population.

Evidence: USDA is completing a revision of its Strategic Plan and ARS will revise its Strategic Plan before the end of this calendar year. ARS previously took steps to correct earlier strategic planning deficiencies when it drafted the FY03-07 ARS Strategic Plan to address the needs of the BPI Initiative, PART, GPRA, Research, Education, and Economics, and USDA Strategic Plans' goals and objectives (www.ars.usda.gov). The ARS Strategic Plan now links budget requests with specific performance measures at the Program level. Also, each project within the Program now has annual milestones that must be met to fulfill the overall Program goals and objectives. During FY05, 20% of the Program's milestones were revised to more accurately align them with the Strategic Plan. The Project's Annual Reports (AD-421s) now collect information on performance indicators (milestones met/not met; see Performance Measure 5).

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: This Program is coordinated through the National Program Staff (NPS) with complementary programs across the Federal Government, academia, and industry to ensure scientific relevance to the USDA mission. A wide range of formal and informal assessments, including independent retrospective and quality reviews, Stakeholder Workshops, and interagency working group (IWG) meetings/listening sessions, are used to ensure an ongoing dialogue with customers and partners (including researchers from academia and the private and non-profit sectors) to assess potential benefits of alternative research approaches. These assessments occur on a regular basis depending on the type of review, ranging from monthly, annually to every 5 years. This information is used by NPS to redirect/restructure research activities to meet customer needs and strengthen the overall Human Nutrition Program. Program analyses are conducted by independent experts and are utilized by NPS. Reports are fully evaluated, and where appropriate are incorporated into the new Action Plan for subsequent implementation.

Evidence: The Program utilizes a number of mechanisms to assess scientific benefit. NPS meets throughout the year with a range of commodity, professional, and trade organizations, as well as representatives from international research organizations and academic institutions. An independent external 5-year Retrospective Review of the Program will be conducted in Fall of 2006. Besides evaluating the past research agenda, it will assess the direction and potential benefits of future research efforts. In 2007, a Stakeholder Workshop will be convened to determine the unique role the Program can play and the benefits that can be achieved in the field of human nutrition. In addition, the National Program Leaders participate with other agencies to compare research approaches, reduce potential duplication, and maximize benefits, e.g., OSTP's IWG on Overweight and Obesity Research, NIH Nutrition Coordinating Committee, and Federal Dietary Reference Intake Steering Committee.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Program's priorities are clearly identified in the 5-year Action Plans which guide future budget requests and funding decisions. Partners and customers, including the action and regulatory agencies, continually provide advisory input which helps the Program identify funding priorities. The Action Plans, the needs of ARS' partners and customers, USDA's/ARS' Strategic Plans, the R&D Investment Criteria, and OMB's guidelines and budget targets are all considered by the Budget Team (comprised of National Program Leaders and budget representatives) in formulating the Agency's budget. The Budget Team's recommendations are forwarded first to the Administrator and senior Agency policy officials, and then to the Under Secretary's office, the Secretary's office, and to OMB for review and final decision. Obesity prevention, nutrient requirements, and nutrition monitoring are examples of identified priorities.

Evidence: ARS obtains input from customers, stakeholders, and partners which enables National Program Staff to prioritize the human nutrition research in the Program's 5-year Action Plan (www.ars.usda.gov/research). In addition, ARS uses the research needs as defined in the authoritative Institute of Medicine reports. The research priorities are captured in the Program's mission statement "??to better understand the role of food, its components, and appropriate diet choices to optimize health throughout the human life cycle." Program managers make funding decisions and redirect resources within the base budget to areas of higher priority based on annual project reviews for relevance, quality, and performance. Budget requests undergo a BPI review at the Department level which ensures they are performance-based.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: To develop specific research Project Plans, ARS establishes milestone targets for measuring performance for each year of the 5-year project cycle. Research Leaders submit Annual Project Reports (AD-421s), reviewed and approved by the Center and Area Directors, that provide performance information on milestones that is used to monitor progress. National Program Leaders use these data to redirect or realign projects, if evidence suggests such a step would enhance the quality and performance of the research. The National Program Staff provides summary data on progress towards meeting milestones that is disseminated in the GPRA annual performance report and the PART analysis. All ARS grants and agreements (partners) include terms and conditions specifying performance standards, fiscal accountability requirements, and regular reporting of accomplishments and financial expenditures to ensure recipient accountability for cost management, timely completion of the project, and results.

Evidence: Information from Project Plans, Annual Reports , and internal reviews are used in managing the Program and deciding whether to realign, redirect, reduce, or terminate a project to maintain quality, accountability, and performance. The Program refocused research at the Human Nutrition Research Centers to emphasize the social and behavioral aspects of obesity resulting in more efficient program management and use of resources. This realignment of personnel, fiscal, & facility resources has been accomplished to better fit the Program's objectives in the Action Plan. Additional realignments are being considered at other locations within the FY07 and 08 budgets to improve quality, performance, and/or accountability. Fiscal accountability is closely tracked to ensure that funds are properly used in accordance with the program's priorities and the terms and conditions of each agreement.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The National Program Leaders ((NPLs) program managers) and Center/Area Directors (line managers) are held strictly accountable, annually, for evaluating the impact, quality, and performance of the Program to ensure that the goals of the Program are being met and that the results produced meet expectations in a timely manner. Line managers have additional responsibility for the fiscal accountable (cost and schedule) of program partners. In addition, all ARS grants and agreements include terms and conditions requiring recipient performance and financial reports to facilitate monitoring to ensure that funds are provided and obligated in accordance with the terms and conditions of the award. All research agreements are continuously monitored by the appropriate program and line managers to ensure research activities are meeting the project goals and milestones on time and in a fiscally responsible manner.

Evidence: Managers and scientists are subject to annual performance reviews. Also, ARS scientists are subject to an in-depth peer review on a 3- to 5-year cycle under the Research Position Evaluation System. The annual reports from each research project, which contain performance results, are reviewed by both the Area Office and the National Program Staff. The Performance Plan for NPLs holds them accountable for evaluating their Programs for "?? relevance, direction, and rate of progress toward providing solutions to specific high-priority national problems and research objectives." The Performance Plan also requires them to develop "?? recommendations to line managers for strengthening the Agency's research program." NPLs are evaluated on these and other criteria annually. Line managers have additional performance standards holding them accountable for cost, schedule, and performance of program partners.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: ARS allocates, obligates, and manages Agency funds at the project level to ensure adherence to the Program plan while limiting unobligated year-end funds.. All ARS grants and agreements hold partners to strict performance standards and fiscal accountability requirements (financial expenditures and cost management). As part of the annual performance appraisal, Center/Area Directors are held accountable for the enforcement of Agency procedures ensuring the timely and appropriate use of resources. ARS uses the Foundation Financial Information System (FFIS) accounting system to enhance accountability. ARS undergoes annual financial and programmatic audits by USDA/OIG and GAO that review timeliness of obligating and reporting, accuracy in accounting, and intended use of funds and the prompt correction of deficiencies identified. All Program fund awards are reported promptly and accurately via FFIS.

Evidence: The FFIS accounting system is used to process and manage financial data to meet stringent budget and fund control needs, as well as complex multi-fund and reporting requirements. This Program does not have carryover authority; all funds are expended or obligated by the middle of the 4th quarter of each fiscal year. ARS, as an Agency, uses its own local obligation tracking system in conjunction with FFIS to ensure funds are obligated to the assigned funding level without exceeding it. Historically, ARS obligates more than 99% of its assigned funds. In FY05, the ARS obligation rate was 99.7%. In FY05, ARS implemented a Program Direction and Resource Allocation Memo process to ensure the prompt dissemination of new funds, and to provide specific program direction for the use of these funds. Scientists/managers are held accountable for financial management of assigned fiscal resources. ARS has received clean audit opinions from OIG and GAO for several years.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ARS has engaged in continuous efforts as part of its competitive sourcing program to pursue Agency strategies to identify efficiencies to better support its research programs. ARS' "Green Plan" is a long-term approach to "feasibility reviews" that evaluate, analyze, and measure commercial functions within the Agency's FAIR Act Inventory. To operate as efficiently as possible, ARS has consistently managed with an overhead of 10% or less and the Program leverages funds to achieve increased cost effectiveness. Analyses conducted by ARS exposed IT infrastructure deficiencies particularly in cyber security, network connectivity, and email configurations. Subsequently, ARS analyzed the costs and benefits of alternative solutions for improving email and web hosting services; recommended email and web hosting solutions that best meet the Agency's needs; and developed an efficient and cost effective IT Consolidation Project Plan for infrastructure improvements. The Program includes an efficiency measure, based upon the increase in the output of scientists, as measured by the change in the average number of peer-reviewed scientific reports per scientist.

Evidence: ARS monitors the cost from A-76 competitions to ensure that projected benefits are achieved. ARS continually works to identify future opportunities through its "Green Plan" which provides a framework and methodology for its continued evaluation of its commercial activities. Leveraged funds have increased to about 9% of the Agency's appropriated funds. In FY05, ARS further upgraded the IT Consolidation Project to incorporate new standards for a USDA-wide email system, and to integrate with the USDA Universal Telecommunications Network (UTN). The project consolidated web hosting, began updating infrastructure for email and web hosting, integrated with UTN, and began planning new enterprise architecture. By FY07, ARS will have unified 52 email systems, completed infrastructure upgrades, and built the enterprise architecture. The benefits of IT consolidation will have provided ARS with an efficient, secure, scalable, robust network, enterprise email, and web hosting capabilities.

Does the program collaborate and coordinate effectively with related programs?

Explanation: ARS conducts intramural research and is not a grants-based program. Aspects of the Program are integrated while others are coordinated with related programs across the Federal Government, including USDA agencies, DHHS-NIH/FDA/CDC-National Center for Health Statistics, EPA, academia, and industry groups. Many of these groups fund extramural research that is complementary to the work of ARS. Whenever and wherever possible, ARS collaborates with these organizations to meet societal needs, and where appropriate enters into CRADAs to transfer the research knowledge. These efforts assure close coordination of ARS research to eliminate program overlap and duplication. To maintain this close coordination, ARS continually works with its partners to reevaluate the Nation's nutrition research needs based on changing priorities.

Evidence: Program managers regularly meet with ARS partners to ensure coordination of effort. The Program receives continual input from USDA-Center for Nutrition Policy and Promotion/CSREES/ERS/FNS, DHHS, EPA, etc., through various interagency coordinating committees to determine how ARS can address national nutrition needs and research priorities. For example, the ARS "What We Eat in America" survey is now integrated with NHANES through a partnership with DHHS to achieve continuous national dietary and health monitoring without duplication of effort and fiscal resources. Previously, both agencies had independent dietary surveys. Also, the National Nutrient Databank, which USDA began 100 years ago, establishes standards/protocols that industry, academia, and are used by States in analyzing the nutrient content of food. The annual USDA National Nutrient Databank Conf. coordinates this effort ensuring that all nutrient analyses in the U.S. meet the same high standards and criteria.

Does the program use strong financial management practices?

Explanation: The Program expenditures are continuously tracked using the Agricultural Research Information System (ARIS) and the Foundation Financial Information System (FFIS) to monitor scientific and fiscal progress. ARS maintains extensive financial management policies and procedures on funds management and accountability, including resource management, funds control, and semiannual reviews of unpaid obligations.

Evidence: ARIS tracks scientific funding, objectives, milestones, and accomplishments across the Program. FFIS is used to process and manage financial data to meet stringent budget and funds controls, as well as complex reporting requirements. Implementation of FFIS and its associated data warehouses offers the integration and capacity to improve the delivery of timely and meaningful financial management information, putting ARS in compliance with legislation, including the CFO Act of 1990. FFIS helped ARS achieve a clean OIG audit of its financial statements for FY05. In addition, ARS is part of the USDA Consolidated Audit. For the last 4 years, USDA has received an "unqualified (clean) audit opinion." In FY05, ARS implemented a new PDRAM process that provides specific guidance to field managers on the direction of research programs and the allocation of resources (fiscal, personnel, and equipment) to achieve program goals.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: In both FY00 and 05, ARS revised its research management structure to ensure coherent and relevant research. The ARS program planning cycle now includes an external retrospective assessment of the relevance, quality, performance, and management of the Program. This assessment is used to direct the development of the next 5-year Action Plan. National Program Leaders (NPLs) establish the research agendas to ensure that program goals remain relevant. Area Directors are responsible for assuring performance and scientific quality. In addition, each new project undergoes a rigorous, independent external peer review that provides scientific credibility to the Program's research agenda and identifies management deficiencies at the project level. Area Offices and the National Program Staff (NPS) conduct in-depth external location reviews during the Program cycle to identify management and program deficiencies. NPS is tasked with continually monitoring management procedures for improvement.

Evidence: In 2005, NPS was restructured from 3 to 4 program areas to better manage Agency goals and to highlight the importance of human nutrition research in ARS. A new Deputy Administrator, with a background in nutrition, was selected and a second NPL position was added to emphasize the growing importance of this Program. To ensure relevance, quality, and performance, program realignments and redirections are implemented by Program Adjustment Decision Items to address and correct program management deficiencies. For example, in the last 2 years, ARS enhanced Program relevance by realigning resources to double the support for obesity research, a priority in the President's HealthierUS Initiative. This action improved management of personnel, fiscal, and facility resources to better address the Program's objectives as outlined in the Action Plan. Matrix management within ARS requires continuous communication between Progam managers and line managers.

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: This is an intramural program. The process used to allocate funding employs peer review panels that ensure performance, quality and relevance in the selection of spefici projects, as would a competitve review process. The statutory National Agricultural Research, Extension, Education, and Economics,Advisory Board (NAREEEAB) regularly provides ARS with both general and specific guidance on how to allocate its resources. This input, plus Customer/Stakeholder input via Program Workshops, helps determine which research priorities to fund. Then all projects are reviewed by an independent external peer panel (Office of Scientific Quality Review (OSQR)) to ensure quality. Research plans are revised to respond to the review panel's critique. Projects required to make major revisions are re-reviewed. The panel may indicate that a project is not feasible. Under these circumstances, the National Program Staff (NPS) and the Area Office jointly consider whether the project should be revised or discontinued and whether the scientist(s) and funds should be redirected. Periodic location reviews and retrospective program assessments, by outside experts, monitor quality throughout the 5-year program cycle.

Evidence: Once objectives are set by NPS, each project undergoes periodic review by independent external peer panels to ensure the quality of ARS' research program. The OSQR process invites leading scientists both nationally and internationally from industry, academia, and Federal and State agencies to evaluate the prospective research plan for the quality of the science proposed and the suitability of the scientists to conduct the work (www.ars.usda.gov/osqr). To date, OSQR has completed review of all ARS projects through one cycle. The second 5-year cycle of review began in FY05. Projects that were not approved by the peer panels were returned to NPS for administrative action, which included project termination and redirection of funds. Copies of NAREEEAB recommendations are available at www.nareeeab.com.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program has acheived the long term goals for both of its long term measures. The long-term PART performance of the Program, encompassing all Program Centers including those operated cooperatively with universities, is measured by tracking the use of ARS' research products (publications and databases) by customers in reaching authoritative nutrition and health positions ranging from nutrient requirements to preventing obesity or chronic diseases, and in evaluating the healthfulness and nutrient content of the American food supply and diet. These accomplishments address the most critical concerns as expressed to ARS by its customers and stakeholders, e.g., Federal agencies (USDA, EPA, NIH, FDA, CDC), National Academy of Sciences/Institute of Medicine (IOM), health organizations (American Dietetic Association, American Heart Association, American Cancer Society, etc.), State agencies, industry, and academia. The Program and its partners (e.g., Centers operated cooperatively with universities) commit to long-term outcome targets and achieve them.

Evidence: In the last 3 years, the Program successfully met its ambitious PART targets and timeframes. 13 ARS databases were released and 8 authoritative reports, substantially based on the Program's research, were used in the IOM Dietary Reference Intakes and the USDA/DHHS Dietary Guidelines, to formulate USDA food assistance policy, DoD feeding programs, and FDA nutrition labeling. The President's HealthierUS Initiative and OSTP IWG on Overweight and Obesity Research are also underpinned by ARS research and databases. Program databases are widely used by Program customers, including USDA, other Federal agencies (NIH, FDA, CDC, DoD, EPA), academia, and private industry. Individuals can also download this information to their computers or use it from the Web. In the FY05 GPRA Annual Performance Report for Human Nutrition, ARS had 6 indicators of progress aligned under 3 performance goals with 17 accomplishments under these indicators.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The question received a "small extent" because of the "no" it received for question 2.4. One of the annual, the percentage of annual project milestones met did not appear to gave ambitious goals. However, ARS strives to meet its target goals. ARS uses independent external panels for review purposes. In FY04, ARS began annually evaluating all research projects for relevance and performance against priorities and goals in the 5-year Action Plan. ARS modified its annual project reporting mechanism (AD-421) to specifically collect quantifiable milestone data to better measure research performance. Data on mitigating circumstances for milestones not met described in Section 2.4 were also collected. Beginning in FY05, ARS has specifically included program milestones in all new Action Plans. All partners commit and are held responsible to annual targets. The Office of Scientific Quality Review (OSQR) process quantifiably measures quality based on a 5-year rolling statistical average since 20% of the ARS research program is evaluated annually. However, one annual measure still has targets that do not appear to be ambitious (see 2.4).

Evidence: ARS met 4 of 5 PART annual Performance Measures and improved in a 5th. The 5th measure reflects project relevance where 100% is the ultimate goal which may not be realistically achievable (FY05 98.3%). 87.0% of the total annual milestones were met. Of those milestones not met, two-thirds were due to critical scientific vacancies, program redirection, or insufficient resources. 76.8% of ARS projects passed on the first quality review (described in 3.RD1 Evidence). After the re-review by the peer panel, a total of 95.5% of all projects were approved. Beyond the R&D criteria, ARS met or exceeded additional annual measures for external independent total program assessment, and for the new efficiency measure which gauges relative increases in peer reviewed publications per scientist as a measure of efficient use of research funds. Baselines, targets and actual performance for each measure are calculated on a 3-year rolling average and can be found in the Measures Tab.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Program makes a concerted effort to identify areas of research and resources that can be shared to meet common goals. Trusts, reimbursables, Cooperative Research and Development Agreements, and Cooperative Agreements are various ways ARS leverages funds and human capital resources from external parties interested in developing, transferring, and commercializing the Program's research and technologies. This Program is more complex than other ARS programs because of the Centers operated cooperatively with Tufts University, Baylor College of Medicine, and University of Arkansas Medical Center. This structure lends itself to leveraging additional resources to operate a more comprehensive research program than would be possible with appropriated funds alone. The Program annually reviews its operations for opportunities to further improve efficiency and cost effectiveness. However, ARS routinely sets aside 10% of project allocations for administrative expenses, when smaller amounts may be attainable. The program also has an efficiency measure that has demonstrated improvement during the last two years, and has shown improvements during that same time period..

Evidence: To increase efficiency and cost effectiveness, the Program makes significant effort through leveraging activities, for example, co-located or jointly operated with university partners to share laboratory equipment, core facilities, support staff, and to extend medical/scientific expertise available to the Program. In 2005, the Program leveraged $36.6M in appropriated funds (total Program appropriated funds were $83.7M) at the 3 Centers, operated cooperatively with universities, by bringing in $39M from external sources to support research. New facilities were substantially completed at the Western Human Nutrition Research Center (CA) and the Arkansas Children's Nutrition Center (constructed with non-ARS funds) in FY05. In FY06, ARS will occupy these facilities, thus consolidating research units currently scattered at many locations and improve Program efficiency and effectiveness. In addition, ARS continues to maintain its overhead at less than 10%. The efficiency measure of the number of peer reviewed publications per scientist has increased from 1.25 for the period from 2001 through 2003, to 1.35 for the period from 2002 to 2004, to 1.52 for the period from 2003 through 2005.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The core functions of this Program are unique because they are not performed by any other government, academic, or private sector research institution, and therefore, no comparisons can be made. For example, this Program conducts the "What We Eat in America" survey which is the only nationally representative dietary survey in the U.S.; the Nat'l. Nutrient Databank is the only comprehensive source for U.S. food composition and nutrient data; metabolic units at 5 of 6 Program Centers conduct extended residential human nutrient requirement studies. Non-unique aspects of the Program, such as nutrient bioavailability, chronic disease prevention through nutrition, and community-based dietary interventions, are conducted by others. USDA, NIH, and CDC fund short-term research through extramural grant programs. Private sector research is geared to specific commercial products.

Evidence: The Program's unique core components (considered "Gold Standards") address the full scope of national nutritional health priorities with emphasis on nutrition monitoring ("What We Eat in America," Nat'l. Nutrient Databank) and human nutrient requirements from pregnancy to old age. Some States and private industry conduct specific/local food consumption surveys, however, only ARS has the capacity to conduct a national dietary survey with the requisite nutrient database. The Program is the authority for methodology used by many nations (e.g., Russian State Statistics Service, www.fsgs.ru/wps/portal/english) to establish nutrition monitoring programs. The majority of the research on U.S. nutrient requirements is conducted by the Program and used by Institute of Medicine (IOM) in establishing Federal nutrition recommendations/standards. IOM's expert committees historically have selected Program data for the consistent high quality of its Program components.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: ARS has developed a comprehensive objective external peer review process specifically designed for intramural research that evaluates all aspects and stages of the Program. Initially, each ARS project is evaluated by an independent external expert panel conducted by the Office of Scientific Quality Review (OSQR) to ensure project quality and validate the Program's priorities. In-depth research site reviews are regularly conducted by external panels to evaluate program quality, effectiveness, productivity and impact. Retrospective reviews conducted at the Program level use independent external panels to assess the accomplishments and benefits of the research against what was identified in the 5-year Action Plan. Retrospective panels frequently opine on future research directions to the National Program Staff which are then presented to the Stakeholder Workshop. The Workshop provides an additional independent, external review and input for the next 5-year Action Plan.

Evidence: The calculated rolling average of all ARS projects reviewed by OSQR that passed the first review was 76.8%. The remaining projects were revised and re-reviewed, or terminated. After re-review, 95.5% were approved. The remaining 4.5% were redirected to areas of higher priority. In FY05, 20% of the total ARS research program was evaluated retrospectively and the average score for all ARS National Programs ending their 5-year cycle was 86.6%, a significant increase from 78% in FY04 (a rolling 5-year baseline will be established in FY09). These evaluations are used to formulate ARS' budget initiatives, modify the base budget, and to make program management decisions. The Nutrition Program will undergo an independent, external, 5-year retrospective review in the fall of 2006 to evaluate the scope, focus, productivity, and impact as described in the Action Plan. In FY08, the projects in the Nutrition Program will be subject to OSQR prospective quality review for the next 5-year cycle.