Detailed Information on the
USDA Foreign Market Development Programs Assessment

Include these performance measures in the Foreign Agricultural Service's strategic plan.

Work with cooperators to systematically collect long-term performance information for use in the OTP strategic plan.

Complete enhancement of the Unified Strategic Planning System currently being reprogrammed to be completed for the 2010 program year.

Reallocate funding to target funds to those that would benefit the most from the programs.

Measure: Participant program management rating

Explanation:The original baseline reflected FAS's expectation that more than half of the participants would rate at least "satisfactory" in the Results Oriented Management (ROM) program management/implementation category given their long-term participation in the market development programs and their ongoing work with FAS marketing specialists. Marketing specialists assess the adequacy of a program participant's marketing plan, soundness of the participant's accounting procedures, risk of compliance issues and whether the participant has appropriate staffing to handle the size and type of proposed programming. In the current review methodology, participant programs receiving a score of 70 percent or higher are considered "satisfactory".

Measure: # of activities supporting small companies

Explanation:Measures the number of activities conducted by small companies (as defined by the Small Business Administration) participating in the Market Access Program (MAP) branded program. The FAS logic model shows the linkages between annual and long-term performance measures, and between inputs and outputs (funded participant activities) and long-term outcomes (exports). For example, the model measures the number of activities in a given market as an output measure, including activities supporting small companies. Activities are intended to overcome constraints, so FAS measures the percentage of results that provide program implementation feedback as short-term outcomes. If these program management goals are met, there is a greater likelihood that activities will generate results that will demonstrate progress towards or overcoming existing constraints (intermediate outcomes). If there is progress towards mitigating constraints, participants become more willing to invest their own resources in export promotion (intermediate outcome). If consumption and participant investment (both intermediate outcomes) increase, exports should also increase (long-term outcome). FAS used the most current data series available at the time of the initial PART review to establish the baseline, that is, 2000-2002.

Measure: Total Exports ($) in countries and product lines where MAP/FMD are expended

Explanation:

Measure: Industry Strategic Planning rating

Explanation:OTP has identified three "critical control points" based on Results Oriented Management (ROM) principles - strategic planning, implementation/program management, and results/evaluation. OTP has determined that there are "best practices" under these critical control points that can help ensure that program participants are operating at their full potential. OTP marketing specialists assess whether the program participant has a sound strategic planning process with broad industry involvement. Higher scores are given to those program participants that have a Comprehensive Industry Strategic Plan (CISP). This is a broader document than the Unified Export Strategy (UES) and is developed by participation of industry and affiliated industries beyond the participant's actual membership, includes domestic constraints that may also affect U.S. export competitiveness, and is updated regularly and used to guide the UES annual submission. Marketing specialists assess whether the UES reflects industry-identified country/region/issue priorities and whether budget requests are reasonable in relation to export levels and the participant's strategically-identified priorities. A key factor in the rating is whether there is evidence that past results and evaluation have been used to prepare the current UES plan and requested programming. Using the current review methodology, program participants receiving a score of 70 percent or higher are considered successful.

Measure: $ of actual sales for small companies. (Reflects total dollars in export sales for small companies participating in the 'Market Acess Program' branded program)

Explanation:Reflects total dollars in export sales for small companies participating in the MAP branded program. FAS used the most current data series available at the time of the initial PART review to establish the baseline, i.e., 2000-2002. With the impending expiration of the 2002 Farm Bill, FAS conservatively estimated that MAP funding levels could be reduced to pre-2002 Farm Bill levels. With possible reduced MAP funding available to support small company market development efforts, export sales were anticipated to grow at a slower rate or even decline.

Measure: # of small companies making 1st export sale

Explanation:Reflects the number of small branded companies (new or old) making their first export sale in a new market. Targets are affected by how long it takes for companies to make their first sale. Often, it may take a company several years to start exporting to a new market from the time the company first starts to explore that market.

Measure: % of all United Export Strategy (UES) constraint measures meeting stated goal

Explanation:The Unified Export Strategy (UES) is an annual application for all market development program funding - an annual marketing plan that outlines past performance and current performance objectives. The plan is organized by country/region, product, constraint or opportunity. Each market constraint or opportunity has a corresponding set of activities, and each constraint or opportunity has associated performance measures. FAS' Office of Trade Programs (OTP) annually reports the percentage of cooperators/participants meeting the established performance measures. OTP has established a program-wide target of 59 percent to encourage ambitious cooperator/participant targets. Participants could establish goals that would be met 100 percent of the time. However, that wouldn't be as effective as more challenging goals that can generate meaningful feedback to ensure continued improvement and modification of programming. For the PART, OTP only reports results that met the stated goal. However, for the market development program funding allocation process, OTP considers the full measure of progress made against established constraint/opportunity goals, as well as those results that actually met the stated goal. This review more fully reflects whether the program is succeeding and if additional funding is warranted. It rewards participants that make progress and doesn't penalize participants that set ambitious goals. In addition, in assessing the critical control point of evaluation/results, OTP also takes into consideration: 1) whether the participant has a system of evaluation in place; 2) whether third-party evaluation is used as needed; 3) the quality of success stories; and 4) whether programming is moving along the market development continuum from product awareness to regular purchasing as appropriate for the product cycle in the market. The most effective way for FAS to work with program participants to improve their UES submission is to push for specific constraints/opportunities that lend themselves to measurement and measures that are within an identifiable context that demonstrates progress and/or provides feedback for program adjustment.

Measure: Industry Investments ($)

Explanation:Industry investment in market development programs includes time and travel of individuals spent in planning and activity participation, as well as membership dues and commodity assessment funds (check-off programs) that pay for domestic staff, international officials and various program activities. OTP has determined that the more diversified the investment and the broader the participation, the greater the commitment to international markets and the better the program. An organization must have at least a minimal level of contributions in order to participate in the market development programs. For example, an industry must be able to support a viable domestic staff to manage an international program. Industry contribution is a key factor in allocating market development funds - the higher the level of contributions (both in-kind and cash), the more competitive the program will be in competing for market development funds.

Measure: Total # of activities conducted

Explanation:Reflects the total number of activities conducted under the foreign market development programs, including activities by small branded companies. The FAS logic model that was developed for the PART review identifies linkages between annual and long-term performance measures, and between inputs and outputs and long-term outcomes. Participant activities (outputs) are developed and implemented to address specific market constraints and/or increase import demand. If program management goals are met, there is a greater likelihood that an increase in activities will speed up progress towards or overcome existing constraints (intermediate outcome). If there is progress towards mitigating constraints, participants also become more willing to invest their own resources in export promotion (intermediate outcome). If import demand and participant investment (both intermediate outcomes) increase, exports should also increase (long-term outcome).

Measure: # of small companies with increased sales >20%

Explanation:Reflects the number of small companies reporting export sales increases above a level of exports that might normally be expected on an incremental basis.

Measure: Export Multiplier Ratio

Explanation:Using the UES submissions, targeted markets are identified, and total exports are calculated and divided by total expenditures (FAS and industry reported program expenditures). Baselines were established using actual levels in previous years. FAS targets reflect gradual improvements in administrative efficiencies. OTP has already begun to institute simplified and streamlined processes to improve program management efficiencies and is in the early stages of developing a new UES electronic system. In addition, OTP is assessing the practicality of moving to a cost-benefit measure to replace the export multiplier ratio. The cost-benefit measure is adjusted for currency fluctuation as well as other economic variables. OTP contracted a third-party cost-benefit program-wide study in 2006 and is looking into repeating this exercise every few years to provide a more meaningful long-term trade measure of market development program success. OTP maintains that an export-multiplier is a more meaningful measure of overall market development program success than market share or total exports. However, a cost-benefit measure would be even more accurate than the export-multiplier ratio.

Measure: Administrative cost per $ of program funds

Explanation:Reflects administrative cost per dollar of program funds allocated under the market development programs. OTP calculates the "target" as the number of expected staff personnel who will administer the programs, multiplied by an average salary amount of $100,000/person, divided by anticipated program allocations. The "actual" is based on the actual staffing level, multiplied by the same average salary of $100,000/person, divided by actual total funds allocated. Following the FAS reorganization in 2007, the number of personnel administering the market development programs was reduced by nearly 50 percent, which accounts for the sharp decline in the "actual" measure in 2007.

Is the program purpose clear?

Explanation: There are five USDA Foreign Agricultural Service (FAS) market development programs with related purposes of maintaining and expanding markets for U.S. agricultural commodities. The Market Access Program (MAP) and Foreign Market Development Cooperator Program (FMD) are the two largest (account for 93 percent of FY 2005 actual funding). MAP and FMD provide seed capital, leverage government and industry dollars, encourage industry partners to pursue opportunities in less developed or riskier markets, and buy down the risk of private investment in those markets. The Quality Samples Program (QSP) provides small samples of U.S. commodities to potential importers in emerging markets; the Technical Assistance for Specialty Crops (TASC) program, assists with the export of specialty crops; and by the Emerging Market Program (EMP), supporting technical assistance to emerging market countries furthering U.S. exports to those countries.

Evidence: "Evidence 1: Agricultural Trade Act of 1978, as amended??MAP (Section 203): The CCC ""shall establish and carry out a program to encourage the development, maintenance, and expansion of commercial export markets for agricultural commodities through cost share assistance to eligible trade organizations and implement a foreign market development program."" Evidence 2: Agricultural Trade Act of 1978, as amended??FMD (Section 702): The Secretary of Agriculture is directed to ""establish and, in cooperation with eligible trade organizations, carry out a foreign market development cooperator program to maintain and develop foreign markets for United States agricultural commodities and products..."" Evidence 3: USDA FY 2006 Budget Summary and Annual Performance Plan (p. 36) ?? ""FAS administers a number of programs in partnership with private sector cooperator organizations, which support the development, maintenance, and expansion of commercial export markets for U.S. agricultural commodities and products."" "

Does the program address a specific and existing problem, interest, or need?

Explanation: The future of American farmers and agriculture will increasingly depend on exports. About ninety-five percent of the world's consumers live outside of the U.S. America's agricultural production is growing approximately 2% per year, while U.S. demand for agricultural products is increasing only 0.8% per year. In the next 20 years, there will be a 25% increase in global population. As a result, the U.S. needs to be well equipped to share in meeting the increased demand for agricultural goods. In addition, many countries have erected trade barriers, technical and non-technical, which limit US agricultural exports. As a result, potential foreign buyers are unaware of the value and quality of American agricultural products, and many potential U.S. exporters are unaware of foreign markets and the means for penetrating them. These programs help reduce market constraints and connect U.S. exporters with foreign buyers. The FAS market development programs also help level the playing field in that other governments assist their agricultural producers and distributors in developing foreign markets.

Evidence: "Evidence 1: The Competition in 2002 (USDA FAS August 2004) ?? In 2002, agricultural, forestry and fishery product exporters in competitor nations received $341.9 million in government funding to conduct export promotions. ""Most of our competitors have developed a government export strategy for agricultural products...Our competitors use a number of other tools like export subsidies, export credits and credit guarantees, single desk marketing boards, and tax policies to help the exporter make the sale."" Evidence 2: NASDA Market Access Program Evaluation Final Report (Deloitte & Touche to National Association of State Departments of Agriculture, 1999) ?? U.S. exporters face constraints such as lack of consumer and importer awareness in a target country of U.S. agricultural products; lack of U.S. industry awareness, knowledge, and skills related to exports; trade and macroeconomic policy barriers in a target country; and strong foreign competition. MAP and FMD funded activities are designed to overcome or reduce these constraints. "

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: MAP and FMD were designed to be complementary programs. However, the funding for both programs tends to go to many of the the same groups. In 2004, 18 of the 23 cooperators that received FMD funding (about $34 million, or 98 percent of the 2004 FMD funding) also received MAP funding (totalling over $54 million). FAS gives preference to nonprofit U.S. agricultural and trade groups that represent an entire industry or are nationwide in membership and scope and those organizations with the broadest base of producer representation. Many of the non-profit trade groups receiving MAP and FMD funds are partly supported by large corporations that can afford to fund their own export promotion efforts. See point number 3 in Evidence section.

Evidence: Evidence 1: FAS FMD fact sheet (www.fas.usda.gov/info/factsheets/coopertr.html) listing the interest groups collecting FMD funds. Evidence 2: FAS MAP fact sheet ( www.fas.usda.gov/info/factsheets/mapfact.html) listing of interest groups collecting MAP funds. Evidence 3: US Meat Export Federation (largest recipient of MAP in 2004) has roughly 175 members, which include non-profit individual state beef, pork, soybean and corn producer organizations (i.e. Idaho Beef Council, Illinois Pork Producers, Iowa Corn Grower'sAssociation, Nebraska Soybean Board), non-profit national farm and producer organizations (i.e. American Farm Bureau, National Cattlemen's Beef Association, National Pork Producer's Association, National Sheep Industry Association), State Departments of Agriculture (i.e. Minnesota,??) and both small (i.e. Washington Beef, Premium Standard Farms, Lincoln International, Bay World, Bell Export Foods Group) and large (Cargil, Sara Lee, Tyson) packers, processors and traders. USMEF also has members in allied transportation, handling and storage industries, i.e. Port Newark Refrigerated Warehouse.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: FAS requires all participants to self-certify (in accordance with 7 CFR 1484.20 and 1485.13) that MAP and FMD funds received supplement, not supplant, their funding of activities to develop new foreign markets for their products. However, there is no way to know whether participants would or wouldn't fund the promotion activities that are carried out under MAP and FMD. In the period FY 1997-2002, industry contributions nearly doubled while federal funding increased slightly (less than 15 percent). In addition, MAP funding is made available to products from branded cooperatives which should provide their own market promotion. Though there are program requirements with respect to companies exporting branded products designed to assure that MAP funding does not continue beyond five years for any company in any one country/market combination, some cooperatives receive funding every year; they just target it to different markets.

Evidence: Evidence 1: NASDA Market Access Program Evaluation Final Report (Deloitte & Touche 1999.) ?? MAP funds, along with other FAS financial resources, help U.S. associations address potential "market failures" such as trade barriers and/or constraints to U.S. agricultural exports. The study identified many examples of beneficial economic effects on the U.S. economy due to MAP promotion. Evidence 2: 7 CFR 1484.20 and 1485.13 FAS requires all participants to self-certify that MAP and FMD funds supplement, not subplant, their funding activities.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The market development programs are designed to reach, on a competitive basis, program participants whose membership has the broadest base of producer representation. However, it is not easy to determine whether any individual activities would have been conducted with or without the program. Much of the funding for the MAP program is going to large national promotion groups that are broadly supported by a number of large corporations and producer groups. Furthermore, some MAP funding is provided to branded products and multinational corporations. These promotion entities and corporations do not need scarce Government funding to promote their products overseas.

Evidence: "Evidence 1: Agricultural Trade Act of 1978, as amended: "The Secretary shall develop a strategy [that increases] (3) ?? the value of United States high value and value-added agricultural exports each year ?? [and] (4)?? the value of United States high value and value added agricultural exports each year at a faster rate than the rate of increase in the value of overall world export trade in high-value and value-added agricultural products??" (Sec. 103) In the FMD program, the Secretary is specifically instructed to place "continued significant emphasis on the importance of the export of value-added United States agricultural products into emerging markets."(Sec 702) Evidence 2: 7 CFR 1484.21 and 22 (FMD) ?? The FMD application approval process includes, among other things, (1) the ability of the participant to provide an experienced U.S.-based staff; (2) the organization's willingness to contribute resources; (3) the conditions or constraints affecting the level of U.S. exports and market share for the agricultural commodities and products; (4) the degree to which the proposed project is likely to contribute to the creation, expansion, or maintenance of foreign markets; (5) the degree to which the strategic plan is coordinated with other private or U.S. government-funded market development projects; (6) past program results and evaluations, if applicable; and (7) previous Cooperator program funding. The FAS formula for allocating funds is: (a) applicant's historic and proposed contribution level (40%); (b) past export performance (20%); (c) past demand expansion performance (20%); (d) future demand expansion goals (10%); and (e) accuracy of past demand expansion projections (10%). Evidence 3: 7 CFR 1485.14 (MAP) ?? The MAP application approval process includes, among other things, (1 and 2) the size of the budget request in relation to the projected, and where applicable the prior year, value of exports; (3) the participant's past projection of exports compared with actual exports; (4) the level of the participant's contributions, (5) the market share goals in target countries; (6) the degree to which the product to be exported consists of U.S. grown agricultural commodities; (7) the degree of value added processing in the U.S.; and (8) general administrative and overhead costs compared to direct promotional costs. The FAS formula for allocating funds is: (a) contribution levels (40%), (b) past export performance (30%), (c) export goals (15%), and (d) accuracy of past projected export goals (15%). Evidence 4: USDA FY 2006 Budget Summary and Annual Performance Plan (p. 36): MAP and FMD ?? ""Historically, more than 80 percent of MAP funding has been devoted to building export markets for high value products, the fastest growing component of U.S. agricultural products."" "

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: FAS has identified the following long-term outcome measures to determine market development program-wide impacts on exports: (1) the export multiplier ratio (total exports for targeted markets divided by expenditures for MAP/FMD/EMP/TASC/QSP, FAS salaries and expenses, and industry contributions from the previous year); (2) exports ($) in countries and product lines where market development program funds are expended; and (3) $ of actual sales for small companies. The export value and export multiplier ratio are calculated on the basis of all export/product combinations of a product in a target country supported by recipients of market development program funds. This is a good proxy in that most program participants represent 80-90 percent of U.S. exports of a product, and the program regulations specifically provide that participant marketing activities are open to non-members of the participant. Upon approval, the new measures will become operational in FY 2006 and be incorporated into the FAS FY 2006 Strategic Plan. The measures build on, and improve, the existing system of measuring participant outcomes and allocating funds based on the formulas contained in regulations. The FAS formula for allocating FMD funds is: (a) applicant's historic and proposed contribution level (40%); (b) past export performance (20%); (c) past demand expansion performance (20%); (d) future demand expansion goals (10%); and (e) accuracy of past demand expansion projections (10%). The FAS formula for allocating MAP funds is: (a) contribution levels (40%), (b) past export performance (30%), (c) export goals (15%), and (d) accuracy of past projected export goals (15%).

Evidence: "Evidence 1: Market Development Program Logic Model ?? The Logic Model that describes overall market development program strategic planning measures, discussed by OMB, USDA, and FAS in March 2005 contains these measures, as well as related intermediate term and short term outcome and output measures. Evidence 2: FAS Performance and Budget Integration Plan 2003-2006 - The relevant objective (1.2 at p. 4) is to "[f]ocus and expand foreign market development, promotion, and outreach activities to U.S. exporters and foreign buyers." The performance goals for this objective are (i) estimated exports generated from market development programs ($millions) and (ii) administrative cost per dollar of program funds committed under market development programs. "

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: "Based on the President's FY 2006 Budget, FAS assumes essentially a flat budget for FY 2006, and an increase to 200 million for the MAP in FY2007/2008. A doubling of participant contributions in the FY 1997-2002 period indicates increasing private commitment to MAP and FMD. Export multiplier ratio is total exports for targeted markets divided by expenditures for MAP/FMD/EMP/TASC/QSP, FAS salaries and expenses, and industry contributions from the previous year. Total Exports ($) represent exports in countries and product lines where market development program funds are expended. Upon agency approval, the above targets will be incorporated in the FY 2006 FAS GPRA performance plan/performance budget. "

Evidence: Evidence 1: Logic Model ?? FAS has adopted the linked Logic Model which will show targets and timeframes for long-term measures. Evidence 2: Measures -- Based on past actual and estimated data, the targets and timeframes for the market development programs are shown in measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: FAS has identified annual performance measures to demonstrate progress toward achieving its long-term goals. These measures monitor the chain of events that influence FAS' long-term outcomes of increased exports generally, and exports by small businesses participating in the programs. The annual measures are: (1) % of constraint performance measures meeting stated goals; (2) # of small companies making first export sales; (3) participant investments in export promotion; (4) # of small companies with increased sales greater than 20%; (6) participant program management rating; and (7) the industry strategic planning rating. Constraints include foreign consumers' lack of awareness of US agricultural product quality, technical barriers to trade (e.g., codes and standards), and inadequate distribution chains and industry export management and funding. Participant investments show industry commitment to the MAP and FMD programs and pursuit of export opportunities. The participant program management rating is the effectiveness of the participants in directing and adjusting their marketing programs appropriately, and industry strategic planning rating reflects scope and breadth of industry planning processes and levering of resources. The newly established Logic Model shows how MAP/FMD annual performance measures enable the programs to achieve their long-term program goals. FAS will enhance its UES system in FY 2006 so as to enable it to categorize and track constraint measure progress on line. Upon approval, these annual performance measures will be incorporated in the FY 2006 FAS GPRA performance plan/performance budget.

Evidence: Evidence 1: Logic Model ?? The new FAS Logic Model shows the linkages between its annual and long-term performance measures and between inputs and outputs (funded participant activities) and long term outcomes (exports). For example, it measures the number of activities in a given market as an output measure. Activities are intended to overcome constraints, so FAS measures the % of results that provide program implementation feedback as short-term outcomes. If these program management goals are met, there is a greater likelihood that activities will generate results that will make progress towards, or overcome existing constraints, these are intermediate outcomes. If there is progress towards mitigating constraints, participants become more willing to invest their own resources in export promotion (intermediate outcome). If consumption and participant investment (both intermediate outcomes) increase, so should exports (long-term outcome). Consumption data is not currently available; UES enhancement will enable data collection.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: "FAS has developed new annual measures that better align with its long-term performance goals. An FAS Working Group has recently developed baselines and targets for these measures. The working group researched historical data to determine past performance and has proposed ambitious targets for the future. Based on its Working Group's efforts, FAS promulgated targets for FYs 2006, 2007, and 2008, and has assigned responsibility for the collection and reporting of performance data to FAS offices. To the extent available, the data is shown below. Data not currently available in automated format will be reported when the enhanced Unified Export System (UES) is available. Some data was estimated and FAS plans to change policies so that more accurate and consistent data will be available in the future. Upon approval, the targets will be implemented in FY 2006 and incorporated in the FY 2006 FAS GPRA performance plan/performance budget. See Table 2.4 in Word document ""FAS_Supporting_Tables.doc"" "

Evidence: "Evidence 1: Logic model ?? The logic model shows baselines and targets for annual performance measures. Evidence 2: Description of Participant program management rating and Industry strategic planning rating."

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Participant grant agreements have in the past assured, and will continue in the future to assure, that participants are held accountable for fulfilling their parts of the agreements, and that the agreement goals, objectives, targets, and measures align with program strategic performance goals. The Country Progress Reports (CPRs) and the UES will continue to serve as reporting systems to assess progress toward the goals. FAS is in the process of adopting new long-term and annual performance goals?? measures that build on and better articulate existing participant goals. Some of these measures will require more specific data reporting from the participants. FAS will issue guidance that explains the new reporting requirements (which will become operational in FY 2006). As in the past, participants will be evaluated based upon the extent they meet agreed upon performance goals and strategies. FAS will review the grant applications to ensure the participants use performance measures that align to the program-wide performance measures. FAS has met with its program participants on these matters and considered their comments.

Evidence: "Evidence 1: CCC Market Access Program Agreement ?? Cooperative agreements and allocation letters set forth objectives and other program requirements and direct that funds be put toward activities that further those objectives. All activities and evaluations must be approved by FAS in advance. Participants must supervise any organizations with which they contract to carry out FAS approved activities. Evidence 2: Country Progress Reports (CPRs) ?? To assure accountability of performance and funding, CPRs are required of each participant for each country or region each year. CPRs contain five sections: (i) market assessment; (ii) financial allocation; (iii) actual vs. forecast exports; (iv) strategic focus and results; and (v) program highlights. CPRs are assessed by FAS staff to determine whether the participants have met relevant goals and objectives. "

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The National Association of State Departments of Agriculture contracted for an independent evaluation of the Market Access Program in 1999. The evaluators, Deloitte & Touche, concluded that the program was effective in achieving its principle objectives of increasing U.S. exports (see answer to Question 4.5)). FAS has initiated the process to contract for a new overall market development program-wide independent evaluation. Program regulations require that participants perform annual evaluations, usually reported as a CPR, on at least one aspect of their programs. FAS carefully reviews participant CPRs, to assess participant success in implementing approved activities and in meeting MAP/FMD strategic plan goals and objectives. The results of these program assessments are then used in considering future funding decisions, program refinements, and initiatives. Over the past three years, at least 72 participant sponsored independent evaluations have been conducted on participant programs. The CPRs and these independent evaluations are used to guide the design and execution of program improvements and initiatives.

Evidence: "Evidence 1: Evaluating Market Development Programs - Guidelines for FAS and Its Industry Partners ?? The Guidelines emphasize that evaluation is an integral part of each participant's program. Evaluation includes impact forecasting, monitoring, activity evaluation (primarily undertaken by program staff), annual country progress reports (CPRs), and program evaluation studies (on an occasional basis, by third party evaluators). The CPRs are required to include market assessments, constraint identification, performance measures (including sales), results of activity evaluations, conclusions, and recommendations for future efforts. FAS uses the CPRs as a principal mechanism for evaluating participants and their activities. Evidence 2: NASDA Market Access Program Evaluation Final Report (Report submitted to the National Association of State Departments of Agriculture by Deloitte & Touche 11/29/99.) ?? As a result of MAP support, U.S. agricultural exports among participants increased significantly in 83 percent of the cases examined. Market share increased, or market share loss was avoided, as a result of MAP-funded promotional activity. MAP and FMD improved the export organization and marketing strategies of the U.S. agricultural industry. MAP increased private investment in U.S. export activity. MAP and FMD helped U.S. producers overcome marketing challenges particular to agriculture. Evidence 3: United States Department of Agriculture, Evidence 3: Three samples of independent evaluations. 1) ""An Evaluation for U.S. Wheat Exports Impact of Technical Service on Sales and Market Share,"" by HCI International Marketing Consultants. 2) ""Evaluation of the American Soybean Association's China Office Feed Industry Program,"" by Informa Economics. 3) Evaluation of US Meat Export Federation's Market Development Program in Russia, Europe and Mexico. All three of these evaluations were conducted by Promar International. "

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The annual budgets of the MAP and FMD are authorized by the Farm Security and Rural Investment Act of 2002. While this program does not require annual appropriation requests, alternative funding levels to those contained in law were considered in the development of the Administration's annual program level request for USDA export programs. The budget requests were not explicitly tied to the programs' accomplishments of the annual and long-term performance goals.

Evidence: Evidence 1: The Farm Security and Rural Investment Act of 2002.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: It has to date been a challenge for FAS to aggregate the measurable performance results of 80 unique organizations into one macro outcome. The new draft FAS Strategic Plan, however, aligns MAP and FMD goals with the goals of the USDA's overall foreign market development, promotion, and outreach activities. FAS has also reviewed and assessed participant activities in order to achieve mutually agreed program specific outcomes that align with the FAS draft strategic plan. Over the last 3-5 years, FAS strategic planning activities have also included participant training and workshops. In 1999, FAS hired Promar to conduct Result Oriented Management training for the participants; this training provided the participants a framework for completing their UES applications. Additionally, beginning in 2004 and still continuing, the Hale Group is providing Comprehensive Industry Strategic Planning (CISP) training to increase industry-wide cooperation, planning and integration. To improve strategic planning further, FAS is in the process of adopting a limited number of specific, ambitious long-term and annual performance goals, as well as baselines, targets, and timeframes. These will be put into effect in FY 2006.

Evidence: "Evidence 1: Results Oriented Management - A Guide for Partners - This guide, created by Promar, describes the framework for establishing performance measures within UES applications. Evidence 2: The Hale Group Comprehensive Industry Strategic Planning (Apr 4-5, 2005) - This is the training package for one of the Hale Group CISP sessions. "

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: FAS collects, through its grant application UES system and through evaluations, market development program performance information on a participant basis. Market development program grant applications include both strategic and activity level country specific information and data. Including specific U.S. export and market share data (5 years historical and 3 years projected), constraints impeding U.S. exports, strategies to overcome the constraints, previous FAS activities in the country, and the projected impact of the proposed program on U.S. exports to the country. This information is used to evaluate the activities proposed in the grant applications and ultimately to allocate available CCC funds in accordance with MAP and FMD regulations. In addition, FAS monitors participant performance and uses participant-submitted annual CPRs to gauge participant success. Program evaluations are conducted annually in both MAP and FMD, on a participant basis, to determine the effectiveness of the participant's strategy in meeting specified goals (including additional export sales and the ratio of additional export sales to MAP funding). In order to improve its collection and use of MAP and FMD performance information and data, FAS has recently adopted refined measures and collected baseline performance data so as to set meaningful, ambitious performance targets. FAS is also enhancing the UES system to assess regularly program-wide activity across all funded industry partners and all countries. The new system will allow FAS to evaluate and improve program-wide, as well as participant, performance.

Evidence: "Evidence 1: 7 CFR 1484.70 ?? FAS cooperators are required to submit an end-of-year contribution report that ""identifies contributions made by the Cooperator and the U.S. industry during that marketing plan year."" (7 CFR 1484.70(a)). Country Progress Report - Program Participants Submission - This document explains the components and purpose of the CPR for program participants. Evidence 2: FAS FY 2004 Management Control Program Guide ?? Participants are required to submit end-of-year contribution reports, travel reports, and research reports (where relevant). Evidence 3: Instructions for the UES Internet-Based Application ?? Participants document their marketing strategy within their UES application. This includes past performance. Evidence 4: MAP program evaluations. "

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: FAS has identified, and established performance standards for, program managers responsible for monitoring and assessing participant activities. Program managers are directed to assess participant strategic planning, management and evaluation and ensure that funding recommendations acknowledge the participants that achieve management success. In place as of FY2005, FAS managers are held accountable for meeting targets associated with FAS's annual and long-term program-wide performance goals. FAS also holds program participants accountable for cost, scheduling, and specific performance results through formal procedures and requirements. FAS managers evaluate and determine participants' progress from data entered into the UES, and in CPRs, by product and by country. Monitoring and assessing participant performance helps FAS managers determine future participant funding allocations and where to make improvements in program implementation. Participants that fail to meet the required levels of performance undergo further review to determine continuing eligibility for MAP and FMD assistance.

Evidence: "Evidence 1: MOS Allocation Formula Results Allocation ?? The allocation formula adjusts the commodity divisions' individual funding recommendations to determine the final allocations to participants. Funding is shifted between participants based on four factors: Participant Contributions, Past Performance, Export Goals, and the Accuracy of Previous Export Goals. For the 2004/2005 program, six participants lost at least one percent of their funding due to poor performance in the allocation formula and six participants' funding were increased by at least one percent due to good performance in the allocation formula. Evidence 2: Country Progress Report - Program Participants Submission - This document explains the components and purpose of the CPR for program participants. Participants annually track export programs against overall market objectives and assess strategic direction of the programs. Evidence 3: Instructions for the UES Internet-Based Application ?? Program participants are required to submit performance results through the UES system. "

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Market development program funding is obligated in a timely manner consistent with CCC authorizing legislation and the resource needs of the FAS overall program plan. All funds are obligated by the end of the fiscal year. FAS and program participants establish schedules for obligations that properly correspond to the resource needs of their plans/marketing years (which vary by crop cycle). Participants use the UES to propose activities. FAS marketing specialists review the proposed activities in the UES to ensure that they are consistent with the intended purpose of the grants. If they are, the marketing specialist approves the proposed activities in the UES. FAS issues a grant approval letter to the participant and an agreement is entered into between FAS (through the CCC) and the participanting organization. FAS has procedures in place for reporting actual expenditures, comparing them against intended/authorized use, and taking timely and appropriate action to correct audit findings when funds are not spent as intended or in accordance with program regulations. These accounting procedures include tracking against unique project numbers traceable to each participant grantee and industry source documents. Program participants submit claims electronically to FAS for reimbursement. Payments are routinely made within 2-3 days of receipt.

Evidence: "Evidence 1: Management Control Program Guide ?? FAS program fund accounting records reflect approved grant purposes, and grantee expenditures are monitored and audited. Participants submit reimbursement claims throughout the year to FAS that show actual spending in relation to activities set out in MAP and FMD grant approval letters. All funds are obligated prior to disbursement and expenditures are validated to confirm compliance with the program's purpose. Internal and external audits of funds obligated by purpose are performed periodically in accordance with generally accepted accounting practices. There have been no audit findings of erroneous payments, loss claims, or anti-deficiency violations in the last two audits. Evidence 2: Financial and Compliance Review Guide ?? Participants are reimbursed once they carry out activities and expend their own funds; participants may receive advances for activities but must submit reimbursement claims within 90 days after receipt of the funds or return the unexpended portion. Reimbursement requests require the participant to keep documentation to serve as evidence in potential compliance audits. Evidence 3: MAP FY 2005 Risk Assessment - Beginning in FY 2005, most MAP participants will begin receiving annual audits in accordance with OMB Circular A-133, providing an even higher level of assurance that MAP funds are used appropriately and that participants receive only those payments to which they are entitled. Evidence 4: FAS Approval Letters ?? MAP grant approval letters specify funding levels and state that, to be eligible for reimbursement, expenditures must be in accordance with 7 CFR 1485. They also define the industry partner's contribution requirement. Some approval letters put restrictions on the use of funds, e.g., caps on administrative expenses. Evidence 5: CCC Market Access Program Agreement ?? Agreements with program participants set forth objectives and direct that funds be put toward activities that further those objectives. Activities and evaluations must be approved by FAS. Participants must supervise any organizations with which they contract to carry out the activities. Agreements include funding authorizations. Participants agree to be subject to the provisions of the Single Audit Act Amendments of 1996. Evidence 6: Sections 1211 and 1512 of the General Appropriations Act, as amended ?? FAS obligates funds apportioned by OMB in accordance with current federal law. "

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: "FAS program procedures measure and achieve efficiencies and cost effectiveness in program execution. All five market development programs are competitive grant programs. Competitive bidding is also required for all contracts over $25,000. Participants and USDA trade offices overseas share resources in support of country strategies. FAS also provides incentives that encourage coordination and collaboration on multiple industry promotions (e.g. wine and cheese and sales of 50 different products to grocery stores in Mexico). The measures of efficiency and effectiveness are currently being updated and enhanced (see Answers to Question 2.1-2.4). The FAS UES system (initiated in 1997) provides an automated single portal for MAP and FMD and the other three market development programs, facilitating applicant access and allowing cross-cut measurement and comparison of efficiency and effectiveness of program participants. FAS is, in addition, sponsoring a requirements study to upgrade its UES system which will improve and streamline monitoring of participant performance and leverage technology for efficiency improvements and will assist FAS in meeting its efficiency measure. FAS also reviews and rates participant's effectiveness in implementing marketing programs (participant program management rating) and industry effectiveness in leveraging resources through collaboration and cooperation (industry strategic planning rating). FAS also rewards those program participants that collaborate on specific projects that address constraints that are multi-industry and/or multi-market in nature, providing programming efficiencies and synergies. "

Evidence: "Evidence 1: Marketing Communications Extranet ?? This contains the following information generated by both FAS and cooperators: FAS program information including program regulations, Q&A's on program regulations and processes; Global Based Initiative activity information including approved activities for past years and concept papers in preparation for the coming UES cycle; a directory of market research conducted by FAS and/or with FAS funds; a reference materials section that includes links to various databases and presentations ranging in subject from FAS marketing initiatives to cooperator activities; and success stories. Plans for the future include an interactive activities calendar; Best Practices, providing guidance in developing program planning, implementation, and evaluation; discussion threads capabilities; and posting Industry Strategic Overviews. Evidence 2: FY 2004 UES IT Contract ?? FAS has contracted for a needs assessment to identify areas needing improvement in its UES automated information management process system. The enhanced system (expected in FY2006) will streamline application and reporting processes and track progress and performance. This will save participants time and resources while providing FAS with better information for program evaluation and funding decisions. "

Does the program collaborate and coordinate effectively with related programs?

Explanation: [A "Yes" requires examples of how collaborations have led to meaningful actions in management and resource allocation. Please provide evidence. Also, provide info on how collaboration has let to changes in management or costs.] FAS coordinates with related programs in the Trade Promotion Coordinating Committee (TPCC) in implementing the Administration's Trade Promotion Agenda and National Export Strategy. All TPCC agencies now share one export-focused Internet portal (export.gov). In 2001, FAS, the Commerce Department's Foreign Commercial Service (FCS), and the State Department agreed on a standard operating procedure (SOP) for increased collaboration in export promotion, domestically and abroad. The SOP (further clarified in 2004) includes FAS, FCS, the National Marine Fisheries Service (NMFS), state regional trade groups, state departments of agriculture, and the National Association of State Departments of Agriculture??the Ag Team. The purpose of the SOP is to assure that U.S. agricultural producers and distributors can draw on the full range of federal and state government resources to assure best outcomes. It divides up lead coverage but requires collaboration and coordination. Additionally, FAS serves as the umbrella organization that unifies independent US organizations (e.g., States, local governments, and private companies) into a national effort. Besides participation in the TPCC, FAS offers a complete portfolio of programs and services to identify opportunities for the growth of agricultural exports to overseas markets, to ensure that the resources, programs, and policies of the Department are coordinated with those of other agencies, and to remove barriers to agricultural trade in overseas markets. These resources and programs include Market Intelligence and Analysis; Trade Negotiations and Enforcement; Cooperation with International Organizations (such as the Cochran Fellowship Program, and Food Aid Programs); and Export Credit Programs. FAS draws on the expertise of its Washington-based staff, its overseas officers in 72 Embassies abroad, and the commodity expertise availed by its unique partnership with 70 commodity trade organizations to adapt and weave its various programs into cohesive market development strategies that are both country and commodity specific.

Evidence: "Evidence 1: 2004 National Export Strategy ?? TPCC agencies will closely coordinate ""their efforts in support of major Administration priorities to enable American businesses, workers, and farmers to succeed in the global marketplace and by playing a critical role in strengthening new democracies."" (p.v) All TPCC agencies share one export-focused Internet portal (p.vii). For example, the TPCC is now developing government-wide strategies to assist U.S. companies in taking advantage of the China export strategy and recently negotiated FTAs (pp.viii, 30-32, 35). A TPCC training course trained 92 staff from 10 federal agencies (including USDA and DOC) in 2003 (p.13). DOC, USDA, and SBA are collaborating in California on training and trade financing (pp.19-22). In January 2004, FAS opened an Agricultural Trade Office in Beijing in support of the National Export Strategy (p.26). FAS and FCS are cooperating in Singapore and Morocco under newly negotiated FTAs (pp.37-40). Evidence 2: FAS/FCS Agreement dated 1/23/01 ?? FAS and FCS have committed to increased collaboration with organizations including state departments of agriculture, NMFS, USDA traditional and non-traditional trade groups, and other export entities. The goals of the increased collaboration are to: 1) provide U.S. food and agricultural companies with more complete customer service, 2) maximize the number of U.S. companies exporting, and 3) capitalize on trade opportunities. Domestically, FAS, FCS, and NMFS will co-develop a matrix that identifies export programs (market intelligence and development, finance, buyers, trade shows, etc.), their costs, and information on the export process. FCS will use an FAS questionnaire to assist in determining the level of export readiness. Non-export ready companies will be referred to appropriate local training providers. Export ready companies will be counseled on the array of programs available (through all agencies) and directed to use the appropriate participant group, as well as resources from FAS, FCS, NMFS, and state and other service providers. Internationally, FAS will refer non-agricultural product companies to FCS, and FCS will refer agricultural product companies to FAS, with some exceptions regarding fishery and forest products. Evidence 3: FCS/FAS Worldwide Cable dated 3/16/04 ?? The purpose of this message is to lay out in clear and concise terms the procedures to assure FAS/FCS cooperation and make clear where cooperation and mutual support are expected. The Cable clarifies lead responsibilities, obligations to refer inquiries to the lead agency, cooperation in countries where FAS does not have an office, and sharing of FAS and FCS expertise so as to provide maximum support to both agricultural and non-agricultural exporters. "

Does the program use strong financial management practices?

Explanation: The market development programs are funded by CCC with matching funding from program participants. The programs are executed in compliance with legislative, regulatory, and administrative rules and guidelines and have been judged to be of low risk for improper payments. The programs have: (1) procedures in place to ensure that payments are made properly for the intended purpose to minimize erroneous payments; (2) financial management systems that meet statutory requirements; (3) financial information that is secure, accurate and timely; (4) integrated financial and performance systems that support day-to-day operations; and (5) financial statements with clean audit opinions and no material internal control weaknesses. FAS has a structured process for addressing audit findings that includes corrective action plans and periodic follow up after completion of the corrective action. FAS inputs front end data and ensures it is accurate and documented. Risk assessments are conducted as required by the Improper Payment Act of 2002. The programs meet the requirements of the Federal Credit Reform Act of 1990, the Debt Collection Improvement Act, and applicable guidance under OMB Circular A-129.

Evidence: "Evidence 1: Part of USDA consolidated financial audit, which has clean opinion. Evidence 2: FAS Approval Letters ?? MAP and FMD grant approval letters specify program funding levels and note that, to be eligible for reimbursement, expenditures must be in accordance with 7 CFR 1485 and 7 CFR 1484. They also define the participant's contribution requirement. Evidence 3: FAS Financial and Compliance Review Guide ?? The Guide facilitates reviews of participants to ensure (i) they meet their financial and administrative responsibilities under grant agreements, (ii) only authorized, reasonable, and documented expenses are claimed; and (iii) only eligible and documented participant contributions are reported and count toward matching requirements. Evidence 4: FAS FY 2004 Management Control Program Guide ?? The FAS Marketing Operations Staff is tasked (i) to coordinate, monitor, and ensure fiscal and operational adequacy and consistency in the administration of market development programs; (ii) to establish and manage a computerized billing reimbursement system to ensure fiscal responsibility, accountability, security, and reporting of market development program information and financial resources; (iii) to develop and implement training programs for FAS employees and program participants with respect to all aspects of market development program administration (including fiscal management); and (iv) to respond to budget, compliance, and audit reports related to market development programs. The MCP reports that FAS has had no outstanding material weaknesses identified in previous fiscal years. "

Has the program taken meaningful steps to address its management deficiencies?

Explanation: "FAS has significantly enhanced its ability to assess market development program effectiveness and management deficiencies and then make management improvements. The FAS UES and Management Control Program (MCP) systems identify, and provide information on the basis of which program management deficiencies might be corrected and the corrections tracked. The UES is one automation umbrella for these programs that has reduced the costs of administration of these programs and allowed for more reliable and readily produced program data/reports to facilitate effective program management, fiscal accountability and compliance with evaluation requirements mandated by Congress and the Government Performance and Results Act. The UES is designed to capture program participant information from the application to the evaluation stage. It serves as a knowledge management tool by the very nature of the information we collect, encompassing: Financial/Accounting; U.S. domestic situation by commodity (Current market conditions, Outlook for production, Strengths and weaknesses of U.S. industry); International market developments by commodity (Current market conditions, Outlook for production, Competition); Country Market Assessment by Commodity (Consumer trends, Distribution, Competitor analysis, Domestic developments); Market strategy (Past performance and evaluation results, Summary, Recommendations); Export Data (Constraints and Performance measures, Marketing activities, Expected Result/Time Frame, Activity budgets ,Industry contributions); Overseas post comments on participants' activities and market conditions.Program management is regularly reviewed through the UES and CPR processes, and adjustments and improvements are made in conjunction with program participants as needed. The UES system provides for a complete application, review, and performance process for all participants in the programs. "

Evidence: "Evidence 1: Management Control Program Guide ?? The MCP Guide provides a framework for addressing and resolving management deficiencies as they arise. "

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Awards are distributed according to a fair and open competitive process with a reasonable amount of outreach to encourage the participation of new grantees. The criteria for award and resource allocation are described in 7 CFR 1485 (MAP) and 1484 (FMD). Ninety-five percent or more of funding is distributed according to a competitive process. The program awards process includes independent merit reviews and ranking of applications with heavy emphasis on export performance. MAP and FMD tend to provide grants to the same grantees in successive years, but the grantees are nonprofit U.S. agricultural trade associations, cooperatives, and state-regional trade groups, and small businesses that represent approximately 90% of U.S. production of the relevant products. To re-qualify for a grant, orgainzations must re-apply for the program through the same competitive UES process as new applicants. Grant allocations, however, vary according to assessments on the merits of individual applications. Further, MAP regulations limit the promotion of branded products by a company in a single country/market to no more than five years.

Evidence: "Evidence 1: 7 CFR 1485.14 (MAP) ??The MAP application approval process includes, among other things, (1 and 2) the size of the budget request in relation to the projected, and where applicable the prior year, value of exports; (3) the participant's past projection of exports compared with actual exports; (4) the level of the participant's contributions, (5) the market share goals in target countries; (6) the degree to which the product to be exported consists of U.S. grown agricultural commodities; (7) the degree of value added processing in the U.S.; and (8) general administrative and overhead costs compared to direct promotional costs. The FAS formula for allocating funds is: (a) contribution levels (40%), (b) past export performance (30%), (c) export goals (15%), and (d) accuracy of past projected export goals (15%). Evidence 2: 7 CFR 1484.21 and 22 (FMD) ?? The FMD application approval process includes, among other things, (1) the ability of the participant to provide an experienced U.S.-based staff; (2) the organization's willingness to contribute resources; (3) the conditions or constraints affecting the level of U.S. exports and market share for the agricultural commodities and products; (4) the degree to which the proposed project is likely to contribute to the creation, expansion, or maintenance of foreign markets; (5) the degree to which the strategic plan is coordinated with other private or U.S. government-funded market development projects; (6) past program results and evaluations, if applicable; and (7) previous Cooperator program funding. The formula FAS uses for allocating funds are: (a) applicant's historic and proposed contribution level (40%); (b) past export performance (20%); (c) past demand expansion performance (20%); (d) future demand expansion goals (10%); and (e) accuracy of past demand expansion projections (10%). "

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program regulations (7 CFR 1485.20 plus FMD EMP citations) contain specific evaluation requirements that are binding on all program participants. Evaluation is an integral element of program planning and implementation and guides the development and scope of a participant's program. Evaluation and performance are primary factors in the funding process. The annual Country Progress Report (CPR) is the vehicle by which participants document their country and program evaluations. Based on the performance measurement data/information, FAS and the participants assess their overall effort in the country market and recommend changes to their programming efforts as warranted. In short the CPR is the touchstone document in the UES process that assesses past performance and suggests a direction for future marketing efforts, including defining market priorities and constraints, identifying performance measures against the identified constraints, and evaluating the performance measures to demonstrate progress towards overcoming market constraints. In addition, the FAS/Compliance Review Staff conducts uniform and thorough reviews of the program participants to: (1) ensure that they met their financial and administrative responsibilities under their cooperative agreements; (2) ensure only authorized, reasonable, and documented program expenses are claimed; and (3) ensure only eligible and documented contributions are reported and the required cost-share requirement is met. Each review includes: (1) specific evaluations of, among other things, the participants financial management and accounting system to determine whether it complies with generally accepted accounting principles; (2) program advances to determine whether advances were properly accounted for and fully expended within 90 days of receipt; (3) program income to determine whether the participant properly identified and applied program income; and (4) activity expenses to determine whether program resources were used only to reimburse expenditures made to further an approved activity (and not reimbursed from any other source).

Evidence: "Evidence 1: FAS Financial and Compliance Review Guide ?? The Guide outlines the procedures for determining whether a grantee has a financial management and accounting system in place that conforms to generally accepted accounting principles. FAS determines, in accordance with the Guide, whether (1) a grantee has sufficient U.S.-based and overseas staff to manage the programs effectively; (2) the adequacy of grantee instructions involving the use of program funds; and (3) the adequacy of the grantee's accounting system. The procedures are intended to ensure that participants have GAAP-compliant financial and accounting systems, apply funds appropriately, and claim only allowable expenses. Evidence 2: Management Control Program Guide ?? FAS has had no outstanding material weaknesses in previous fiscal years. The Guide stipulates GAO standards for testing management controls and steps for corrective actions. Evidence 3: UES documentation and CPR reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: FAS collects grantee performance data on a continuing basis through its UES and CPRs. However, this participant level performance data is not published in order to protect proprietary interests. FAS does publish aggregate performance data as part of its strategic plan and in accordance with GPRA reporting requirements. General information about the programs and funding levels is available as FACT SHEETS published on the FAS web site. Names of grantees and amounts of awards are posted annually on the website and in press releases. Upon approval, FAS will make available on its website the new program-level long term and annual performance goals, targets, and measures when they are incorporated into the FAS strategic plan.

Evidence: "Evidence 1: 7 CFR 1484.70 ?? FAS cooperators are required to submit an end-of-year contribution report that ""identifies contributions made by the Cooperator and the U.S. industry during that marketing plan year."" (7 CFR 1484.70(a)). Following travel, they must submit trip reports that include ""a brief summary of findings, conclusions, recommendations, or specific accomplishments."" (7 CFR 1484.70(b)). If a cooperator performs research as part of its agreement with FAS, it must also submit a research report. (7 CFR 1484.70(c)) Evidence 2: CPR Guidance ?? Program participants are required to submit performance information through CPRs. CPRs should include information on financial allocations and resources and market level impact (i.e., whether the participant met its export goals for a country/product market). Participants should evaluate their performance in the context of identified opportunities and constraints. Based on their performance, participants are asked to make future recommendations for strategies within their market. "

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: FAS staff have always collected data about the dollar value of exports by commodity and the dollar value of exports by small businesses. This data demonstrates an upward trend in exports. FAS staff have used this type of data in its strategic planning and GPRA reporting. What the new measures do is to standardize performance measures for all participants across each commodity division in markets categorized by income level. The new measures more transparently link, through a logic model format, market development investments and outputs to short-term and intermediate outcomes to export goals. FAS'adoption of the new measures provide FAS with a program-wide perspective to complement FAS' participant performance assessments and country specific analyses. The UES submitted by a prior year program participant must report performance data against the export goals and performance measures included in the prior year's application. The Division reviews progress against the multi-year performance measures, to determine program effectiveness and assess the degree to which the industry has adjusted the program to address shortcomings. The allocation formula actually awards or deducts funds based on the applicant's export performance against goals relative to other applicants In addition, over the past five years, FAS has worked diligently to strengthen then strategic planning, management and evaluation processes of program participants. Specifically, in 1999 all program participants and FAS marketing personnel were provided training in results-oriented management. As a result of this effort, program participants have consistently improved their performance in identifying constraints and opportunities, establishing appropriate performance measures and tracking results. This past year, FAS also provided strategic planning training to industry partners in a series of regional workshops, aimed at developing a Comprehensive Industry Strategic Plan - meant to integrate strategic planning for an industry where domestic issues effect global competitiveness and determine industry priorities among global and domestic markets..

Evidence: "Evidence 1: Logic Model ?? FAS has created a logic model which shows measures for program-wide performance and progress. Evidence 2: Measures -- Based on past actual and estimated data, the targets and timeframes for the MAP and FMD programs are shown in Table 4.1 in Word document ""FAS_Supporting_Tables.doc"" "

Does the program (including program partners) achieve its annual performance goals?

Explanation: FAS staff have always collected data about the dollar value of exports by commodity and the dollar value of exports by small businesses. This data demonstrates an upward trend in exports. FAS staff have used this type of data in its strategic planning and GPRA reporting. What the new measures do is to standardize performance measures for all participants across each commodity division in markets categorized by income level. The new measures more transparently link, through a logic model format, market development investments and outputs to short-term and intermediate outcomes to export goals. FAS'adoption of the new measures provide FAS with a program-wide perspective to complement FAS' participant performance assessments and country specific analyses. The UES submitted by a prior year program participant must report performance data against the export goals and performance measures included in the prior year's application. The Division reviews progress against the multi-year performance measures, to determine program effectiveness and assess the degree to which the industry has adjusted the program to address shortcomings. The allocation formula actually awards or deducts funds based on the applicant's export performance against goals relative to other applicants which is captured in the UES system. The allocation formula criteria includes: the applicant's contribution/industry resources devoted to export promotion activities; applicant's past export performance; applicant's projected export performance; and applicant's actual export performance. The Country Progress Report (CPR) is the vehicle by which participants document their country and program evaluations on annual basis. The CPR is an integral part of FAS'annual review and funding process, and it is one of the most critical pieces used in assessing the participant's results-oriented management process and their likelihood of planned success. The Based on the performance measurement data/information, FAS and the participants assess their overall effort in the country and recommend changes to their marketing efforts as warranted. In short the CPR is the touchstone document in the UES process that assesses past performance and suggests a direction for future marketing efforts, including defining market priorities and constraints, identifying performance measures against the identified constraints, and evaluating the performance measures to demonstrate progress towards overcoming market constraints.

Evidence: "Evidence 1: Logic Model ?? FAS has created a logic model which shows measures for program-wide performance and progress. "

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: FAS market development programs have grown by 50 million dollars between FY2000 and FY2005 (due to increases in the MAP), which has contributed to increased U.S. agricultural exports. Over the same period, the number of FTE's assigned to the market development programs has not grown, remaining almost constant at 80 employees. With the creation of the Unified Export Strategy in1996, FAS has used and improved the web-based financial and program management system to refocus FAS and participant staff from administrative tasks to strategic planning and accountability for results. This has allowed FAS to focus more of its time and resources on strategic planning and achieving overall program results. FAS has in recent years standardized across programs a number of "best practices" relating to strategic planning and program evaluation, focusing more efforts on evaluation of entire programs rather than on individual activities. In particular, FAS has implemented the Participant Program Management Rating, and the Industry Strategic Planning Rating which are aimed at identifying the best practices, standardizing them and further reward them with additional funding. The standardization and integration of these "best practices" has allowed FAS to influence decisions by Cooperators and MAP participants to ensure that funded activities are not only consistent with overall U.S. objectives but also operate in the most cost-effective manner across markets and products.

Evidence: Evidence 1: FY2005 Staffing Pattern for Commodity and Marketing Programs. Evidence 2: The Farm Security and Rural Investment Act of 2002. Evidence 3: MAP Program participants' Country Progress Reports.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While there are no truly comparable programs (because only FAS provides funding to participant organizations to carry out export promotion and market development activities) the market development program export multiplier compares favorably with USTDA's ratios. The market development program multiplier ratio is approximately 1:111 in FY2003, with targets of 1:114, 1:117 and 1:120 for FY04-06 respectively. By comparison, USTDA had a target of 1:40 in FY 03 and achieved 1:33. For FY 04 its target is 1:35 and for FY 05 1:33. The most similar programs are the Department of Commerce's U.S. Commercial Service (which provides U.S. representation abroad for non-agricultural products but no funding of market promotion activities) and the U.S. Trade and Development Agency (which advances economic development and U.S. commercial interests in developing and middle income countries through technical assistance, feasibility studies, training, orientation visits, and business workshops that support the development of a modern infrastructure and a fair and open trading environment). In the future, the FAS market development programs will be using the export multiplier ratio (total exports for targeted markets divided by expenditures for MAP/FMD/EMP/TASC/QSP, FAS salaries and expenses, and industry contributions from the previous year) as one of their long-term performance measures. Commerce and USTDA use similar measures, thus facilitating, to some degree, meaningful comparisons with their programs.

Evidence: "Evidence 1: USTDA Annual Report 2004 ?? This report states "the export multiplier ratio measures U.S. exports associated with foreign projects that USTDA has assisted relative to the amount invested in USTDA activities. Since its inception, USTDA-supported projects have generated over $22 billion in U.S. exports. Over the last decade, USTDA data shows the agency's projects have resulted in U.S. companies exporting more than $36 for every $1 of USTDA funds invested. The mission of the USTDA is ""to advance economic development and U.S. commercial interests in developing and middle income countries."" (p. 9) Evidence 2: ITA Strategic Plan 2002-2006 ?? The International Trade Administration of the Department of Commerce is charged with the mission of creating ""economic opportunity for U.S. workers and firms by promoting international trade, opening foreign markets, ensuring compliance with trade laws and agreements, and supporting U.S. commercial interests at home and abroad."" It measures success in terms of the number of export transactions made as a result of ITA involvement. "

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A program-wide independent evaluation on MAP was conducted by Deloitte and Touche in November 1999. It concluded that: (1) MAP support significantly increased U.S. agricultural exports among participating cooperators in 83 percent of the cases examined; (2) market share increased or market share loss was avoided as a result of MAP funded promotional activity; (3) MAP and FMD improved the export organization and marketing strategies of the U.S. agricultural industry; (4) MAP increased private investment in U.S. export activity; and (5) MAP and FMD helped U.S. producers overcome marketing challenges particular to agriculture. FAS is in the initial stages of contracting for a new market development program-wide independent evaluation. In addition, each year independent evaluations are conducted on program participant performance in selected markets. These evaluations are sponsored by the participant and depend on the resources available to the participant and market interest areas. Over the past three years, over 72 independent evaluations have been conducted on the participants' programs.

Evidence: "Evidence 1: NASDA Market Access Program Evaluation (Deloitte Touche 11/29/99) Evidence 2: List of Independent evaluations. Evidence 3: Three samples of independent evaluations. 1) ""An Evaluation for U.S. Wheat Exports Impact of Technical Service on Sales and Market Share,"" by HCI International Marketing Consultants. 2) ""Evaluation of the American Soybean Association's China Office Feed Industry Program,"" by Informa Economics. 3) Evaluation of US Meat Export Federation's Market Development Program in Russia, Europe and Mexico. All three of these evaluations were conducted by Promar International. "