Detailed Information on the
Trade Adjustment Assistance Assessment

Adjusting the formula for allocating training funds to the states to better reflect the current need for training.

Developing an internal review process to verify the accuracy of trade petition certifications and denials.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Measure: Entered Employment Rate

Explanation:Percent of participants employed in the first quarter after exit. This is a Federal job training program common measure, which enables the Trade Adjustment Assistance (TAA) program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the TAA program is workers who have lost their employment due to the effects of foreign trade, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants got a job according to the following formula: Of those who are not employed at the date of participation - The number of participants who are employed in the first quarter after the exit quarter divided by the number of participants who exited during the quarter. This definition for entered employment was adopted in PY 2001. Actual performance results for this measure fluctuate year to year. The program exceeded its goal for this measure in FY 2006 and met the measure in FY 2007. Last year 27,663 participants reentered the workforce after participating in TAA programs, representing 70% of all individuals exiting the program. The goal for this measure is 73% for FY 2008.

Measure: Employment Retention Rate

Explanation:Percent of participants employed in the first quarter after program exit still employed in the second and third quarters after exit This is a Federal job training program common measure, which enables the Trade Adjustment Assistance (TAA) program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the TAA program is workers who have lost their employment due to the effects of foreign trade, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures how many participants retained their employment once placed in a job, according to the following formula: Of those who are employed in the first quarter after the exit quarter - The number of participants who are employed in both the second and third quarters after the exit quarter divided by the number of participants who exited during the quarter. This definition for employment retention was adopted in FY 2007, and made the measure more challenging by including an additional quarter (three months) in which a participant needs to be employed. Participants who found employment while participating in this program were more likely to stay employed. During FY 2007, 88% of participants remained employed after exiting the program. This percentage translates to 26,808 participants. The goal for employment retention for FY 2008 is 91%.

Measure: Average Earnings

Explanation:The average six-month earnings This is a Federal job training program common measure, which enables the Trade Adjustment Assistance (TAA) program to describe in a similar manner the core purposes and results of the program compared to other education, employment and job training programs. For example, while the target population for the TAA program is workers who have lost their employment due to the effects of foreign trade, the ultimate outcomes for this program are the same as for all other employment and training programs. Common measures remove a barrier to service integration among programs by ensuring that programs no longer have different definitions and methodologies for measuring performance. In this case, the performance indicator measures participants' average six-month earnings once placed in a job (note: the average earnings for a year can be obtained by doubling the performance measure result), according to the following formula: Of program participants who are employed in the first, second, and third quarters after the exit quarter - Total earnings in the second quarter plus total earnings in the third quarter after the exit quarter divided by the number of participants who exited during the quarter. The common measure definition for average earnings was adopted in FY 2007, while prior year results were for a different earnings measure tracking wage replacement. Participants exiting the program in FY 2008 are expected to earn $348 more per year than participants who exited the program in FY 2007. The 2.5% target increase in average earnings for this program means that an individual exiting the program will earn $28,526 annually, $2,377 monthly or $594 on a weekly basis.

Measure: Cost per participant

Explanation:The program calculates the average cost for each participant by dividing the total annual appropriation for the program by the number of participants.

Is the program purpose clear?

Explanation: The Trade Adjustment Assistance (TAA) for workers program was established to help workers who have lost their jobs as a result of imported products or shifts in production to certain other countries. This Department of Labor (DOL) program provides training, income support, and other assistance to help these workers gain the skills necessary to find suitable employment.

Evidence: Trade Act of 1974 (PL 93-618); Omnibus Trade and Competitiveness Act of 1988 (PL 100-418); NAFTA Worker Security Act (in North American Free Trade Agreement Implementation Act, PL 103-182); Trade Adjustment Assistance Reform Act of 2002 (PL 107-210).

Does the program address a specific and existing problem, interest, or need?

Explanation: For the past four decades, policy officials have recognized that increases in global trade and trade agreements, in particular, can lead to concentrated job losses in some domestic firms, industries and locations, and that affected workers may need special assistance to obtain new employment. Workers certified eligible by DOL's Employment and Training Administration (ETA) may receive training, income support (at the weekly Unemployment Insurance level in the state) while in training, and allowances for out-of-area job search and relocation. Related benefits are a health coverage tax credit for TAA participants and wage insurance for older workers. Wage insurance is intended to partially offset earnings losses that may result from taking a new job.

Evidence: In FY 2006, an estimated 120,000 workers were employed in over 1,400 trade-impacted firms. See TAA performance highlights for FY 2006 at http://www.doleta.gov/Performance/results/Quarterly_report/PerformanceHighlights06.pdf For additional information on workers served under the TAA program, see http://www.doleta.gov/tradeact/taa/WhoWeServe.cfm

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: TAA is partially duplicative of other Federal job training programs, in particular the job training services available to dislocated workers under the Workforce Investment Act (WIA). TAA provides training to the subset of dislocated workers who had been employed in TAA-certified firms. However, WIA provides important reemployment assistance that is not available under TAA, and DOL encourages the states to provide these "wrap-around" services to TAA participants by co-enrolling them in TAA and WIA. In addition, TAA augments its offer of training with additional benefits that are not available under WIA. These additional benefits are designed to help trade-impacted workers reenter the workforce. TAA supplements the core and intensive services available under WIA with extended income support for those in longer term training. TAA also provides access to a Health Coverage Tax Credit (HCTC); out-of-area job search and relocation allowances; and wage insurance to mitigate earning losses for older workers who choose to go back to work instead of enrolling in training.

Evidence: TAA provided $220 million for training in FY 2006 and had 80,700 workers participating in training. In comparison, for program year 2005, the Dislocated Worker authorized under the Workforce Investment Act had over $1.3 billion available to cover training plus reemployment services. An estimated xxx thousand workers were served by the Dislocated Worker program, of which yyy thousand were enrolled in training.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The TAA program is designed to serve trade-impacted workers identified through a DOL certification process and to provide benefits using entitlement funding. However, certain statutory flaws in program design limit its ability to provide the most appropriate mix of services to meet workers' adjustment needs. For example, TAA sets early deadlines for workers to enroll in training in order to qualify for extended income support. This encourages some workers to enroll in training even before they have fully tested the job market and their remployment prospects. Other workers may lose out on income support if they delay enrolling in training until they have fully explored their job options. In addition, under the current rules, TAA funds only full-time training, so workers who need training but also need the income from a job are unable to enroll in part-time training.

Evidence: 19 USC Sections 2272, 2273 and 2291, GAO Audits: GAO-04-1012; GAO-04-1029; and GAO-06-43

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: There are no unintended subsidies in the TAA program. The TAA certification process ensures that resources are only available to workers laid off from firms that were identified by DOL as being trade-impacted. TAA provides a focused set of expensive benefits (training, income support, wage insurance, and a health insurance tax credit) which are unique to TAA. The TAA program relies on other programs to provide the less expensive wrap-around job search assistance and other reemployment services (e.g., the WIA Adult and Dislocated Worker programs).

Evidence: GAO-06-43, Current Investigation Process, and DOL FY 2008 CBJ http://www.doleta.gov/budget/07ETA_App.pdf DOL conducts an intensive certification investigation for all petitions to ensure that only those workers who are trade impacted are certified. The investigation process verifies that the statutory criteria are met, and that workers are employed by a firm that produces an article, and that imports of like or competitive products have caused a loss of employment, or that a shift in production has occurred, or that workers are secondarily affected by upstream suppliers or downstream finishers of the trade affected article. In many instances, the investigation also includes making contact with customers of the trade affected firm to ensure that they have, in fact, been importing the article in question before DOL certifies the workers as trade affected. Since only trade-certified workers are eligible for services under the TAA program, only the targeted beneficiaries can receive program resources. Rapid Response assistance is provided to all workers for whom a TAA petition is filed, and provides the first notice to workers that they may be eligible for the program. Once a worker group is certified as trade affected, the appropriate state is notified of the certification so that it can begin notifying the workers. This notification is done by letter using a list of workers obtained from the employer and in some cases unemployment insurance wage record files and also through publication in appropriate local newspapers. When appropriate, local labor organizations are also used to make trade affected workers aware of the certification and availability of services under the TAA program.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: TAA is part of the Common Measures for Job Training and Employment programs initiative. Job Training Common Measures is an OMB-led multi-program, multi-agency initiative for performance reporting to provide a comparable outcome measures across agencies and programs. The precise definitions of the measures have evolved over time since they were first developed in 2002. As part of this initiative, the TAA program has adopted 3 specific long-term measures that allow comparisons of employment outcomes across similar adult job training programs. These measures are: 1) Entered Employment Rate; 2) Employment Retention Rate; and 3) Average Earnings after Program Exit. The third performance measure will replace the Earnings Replacement Rate, which measured earnings pre- and post-program participation.

Evidence: DOL's FY 2006-2011 Strategic Plan (http://www.dol.gov/_sec/stratplan/main.htm) Page 40; OMB Director's Memo M-02-067; TEGL 17-05 Common Measures Policy for the Employment and Training Administration's Performance Accomplishment System and Related Performance Issues.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: DOL has revised the TAA program's long-term targets beginning in FY 2008 in an effort to push the program toward more ambitious goals. The targets provide stretch goals that exceed past results and increase over time but that are realistic, based on the experience of other, more successful DOL training programs. The long-term target for the Entered Employment Rate is 76% in FY 2011, versus actual performance in FY 2006 of 72%. The long-term target for the Employment Retention Rate is 92.5% in FY 2011 versus FY 2006 actual performance of 90%. Since 2001, the program has shown a steady increase in the Entered Employment Rate, and the Employment Retention Rate has remained high. Average Earnings after Program Exit is a new measure, replacing the previous Earnings Replacement Rate. The baseline year for this new measure is FY 2007, and outyear targets will be established thereafter. The program achieved steady improvements from 2003 to 2006 on the old Earnings Replacement Rate measure.

Evidence: DOL FY 2006-2011 Strategic Plan (http://www.dol.gov/_sec/stratplan/main.htm) Performance Goal 2-A. Updated targets found in performance measure section for the PART.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Annual measures are identical to the long term measures identified in Question 2.1. DOL has taken additional steps to assure that the common outcome measures reflect all employment and training services provided, including services from programs in which the TAA participant may be co-enrolled. Definitions for the Employment and Training Common Measures have been reinterpreted so that states are reporting a single participant for all programs, and thus a common entrance and exit date for each individual. This will ensure that progress toward goals is measured after all programs (which might be providing placement and other reemployment services) have finished adding value and provide a clearer picture of program achievement.

Evidence: DOL FY 2008 PBO (http://www.dol.gov/_sec/Budget2008/overviewpb-toc.htm)

Does the program have baselines and ambitious targets for its annual measures?

Explanation: DOL has not set ambitious targets in recent years for its annual performance goals. In particular, the target set for the Entered Employment Rate measure continued at 70% in 2006 even when that target was achieved in 2005. For the Employment Retention Rate measures, the target for 2005 was the same as the actual performance for 2004, and the target for 2006 was lowered. Nonetheless, DOL has now established ambitious targets for future TAA program performance (as discussed in Question 2.2) and will develop the baseline for its new Average Earnings measure in 2007.

Evidence: DOL FY 2008 PBO (http://www.dol.gov/_sec/Budget2008/overviewpb-toc.htm) Data reported on the Fiscal Year 2006 Trade Act Participant Report indicates that 72% of participants who exited the program entered employment in the first quarter after exit. Of those who entered employment, 90% were employed six months later. On average, those who were reemployed earned 77% of their previous wage. Targets for FY 2007 remained steady from 2006 - 2007 and are entered employment 70%; and retention 85%. The average earnings rate is a new measure under common measures, and 2007 will constitute the base year with a goal of $12,000 earned in the second and third quarter after program completion.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: States and local areas work toward meeting national TAA goals. DOL has authority under section 239 of the Trade statute where States carry out the program as DOL agents and "otherwise cooperate with the Secretary in providing payments and services" (section 239(a)(4)). The goals established are a means of assessing the provision of services, and reflect an element of cooperation. In addition, establishing goals would be justified as promoting coordination with WIA (section 239(e)). While it is questionable whether or not we could impose sanctions, monitoring these goals is part of our program management. TAA has encouraged states to work toward achieving desired outcomes and goals in a number of ways. These include issuing a performance Training and Employment Guidance Letter (TEGL) to help highlight the need for accurate data; publication of the performance goals on the Trade Website so that the general public is aware of the program goals; focusing on performance at the State Trade Coordinators National meeting held in conjunction with the Rapid Response Summit in May of 2006; and focusing on performance enhancement during monthly Regional Trade Coordinator Conference Calls.

Evidence: Agreement between the Secretary of Labor and the Governor of each state; TEGL 17-05 Common Measures Policy for the Employment and Training Administration's Performance Accomplishment System and Related Performance Issues; TEGL 32-04 State Accomplishment of Performance Goals for Trade Adjustment Assistance (TAA).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ETA has not conducted its evaluations on a sufficiently regular basis to support program improvements. The last evaluation was completed in 1993. The authorization for TAA expires September 30, 2007. ETA has contracted with two independent research organizations (Mathematica and Social Policy Research) to conduct a five-year impact analysis of TAA, but the results will not be available until 2011. In addition, ETA has not initiated an evaluation of the Alternative Trade Adjustment Assistance (ATAA) wage insurance program, which was created in the 2002 Trade Act as a five-year demonstration. This omission means that no evaluation or other programmatic information is available to determine the success of the demonstration -- information that could have informed TAA reauthorization.

Evidence: ?? Latest interim report on SPR/Mathematica study NEED WEB LINK ?? WIA and TAA Co-Enrollment Pilot Project Draft Design Report of September 15, 2006 ?? The information collection request for the TAA evaluation was approved by OMB in November 2006. The Mathematica/Social Policy Rsearch evaluation is designed to compare individuals who received TAA services with a demographically matched control group who were not TAA eligible and received only UI and WIA services. In addition, ETA tasked Social Policy Research with conducting a six-state pilot project to determine the effect of 100% co-enrollment of TAA applicants in the WIA system where wrap-around services are available to augment TAA training services. Results from this study, concluded in December of 2006, will be available late in 2007.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: TAA is an entitlement program, and resource levels requested depend on the number of eligible individuals expected to be served -- not labor market outcomes of program participants. The TAA budget is not linked to performance outcomes in terms of its performance goals for entered employment rates, retention rates, or average earnings. In the current budget submission, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. This program performance and cost integration does not yet permit assessment of the impact of budget levels on performance outcomes.

Evidence: DOL FY 2008 CBJ

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: DOL has taken steps to correct many of its strategic planning deficiencies. Notices of proposed rulemaking for TAA have been developed and published in 2006 for comment. However, Congress has stopped DOL from finalizing those rules until TAA has been reauthorized. DOL has also contracted for a Five Year TAA Evaluation with Social Policy Research to provide an independent analysis of the effect of the TAA program. This study will compare a TAA treatment group with a control group that is demographically matched but received only Dislocated Worker services. As part of this evaluation, our contractor will look at the new ATAA program as well as a follow up to the initial implementation review of ATAA which they performed when the evaluation was begun. In addition, DTAA recently revised its performance goals to reflect a more ambitious approach in the entered employment and retained employment areas. The earnings measure is a new measure adopted across program lines in the Common Measures effort, and baseline data is being established. In the interim, the Department has issued formal operating instructions in the form of Training and Employment Guidance Letters (TEGLs) to the states, which under the Secretary -- Governor's Agreements with the states, must be adhered to in the operation of the TAA program. Issues addressed include: ?? Developing a formula funding allocation process which will allow states to have a base allocation upon which to plan for activity in the coming year. ?? Addressing performance issues through use of a common measures performance system which ensures that all programs are measuring the same thing at the same time and in the same way, while also making it clear to the workforce system that performance counts and incremental improvement is expected. ?? Updating the policy for distance learning to reflect the availability of online education in the 21st century. ?? Implementing a TAA for Farmers joint effort with USDA.

Evidence: TEGL 9-06 Fiscal Year (FY) 2007 State Base Allocations and the process for Requesting Additional Trade Adjustment Assistance (TAA) Reserve Funds ?? [17-05] Common Performance Measures Policy for the Employment and Training Administrations (ETA)'s Performance Accountability System ?? [9-05] Approval of Distance Learning Under the Trade Adjustment Assistance (TAA) Program ?? [4-05] Fiscal Year (FY) 2006 State Base Allocations and the process for Requesting Additional Trade Adjustment Assistance (TAA) Reserve Funds ?? [32-04] State Accomplishment of Performance Goals for Trade Adjustment Assistance (TAA) and North American Free Trade Agreement Transitional Adjustment Assistance (NAFTA-TAA) Participants ?? [9-04] Services Authorized by the U.S. Department of Agriculture's USDA Trade Adjustment Assistance (TAA) for Farmers Program ?? [6-04] Fiscal Year (FY) 2005 State Base Allocations and the process for Requesting Additional Trade Adjustment Assistance (TAA) Reserve Funds

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DOL does not regularly collect timely and credible performance information. Performance information is needed from two sources: (1) the state partners who serve trade-impacted workers, and (2) DOL managers of the petition review process (1) DOL has not been able to collect timely and accurate performance information from its program partners. As verified by a recently published GAO study, several data issues existed in the past including reporting errors by states, and confusion over the definition of a program "exiter." An earlier DOL Office of Inspector General (OIG) audit (September 2005) found that five of seven data elements were not complete or reliable due to inadequate verification of participants' exit dates. These findings were instrumental in guiding efforts in the implementation of the common measures process and issuing guidance on exit dates (which establishes that the exit date is the same date as used in the WIA system). In addition, all states now provide quarterly performance information on a timely basis through the Trade Assistance Performance Report with regional and national office follow-up when necessary. Beyond this, a data validation effort was conducted in conjunction with the common measures implementation effort during the past year. Use of wage records has increased reporting success, and ongoing grants monitoring and oversight efforts are led by ETA Regional Offices with follow-up action taken, where necessary. Finally, two rounds of common measures training were conducted throughout the nation to ensure that states were familiar with the new definitions and were providing valid and consistent responses in their reports. (2) DOL collects information on how long it takes to make final decisions whether or not to certify a petition for TAA. While the petition certification process has significant checks and balances in place to improve the quality of decisions, the program does not currently have a formal internal review process to ensure decisions already made followed appropriate protocol and guidance. An internal process is needed, as the certification process has been criticized by the Court of International Trade as being perfunctory. This process should include taking a random sample of decisions and determining if these decisions were made using the correct procedures, protocol, and guidance.

Evidence: GAO-06-496, OIG 22-05-007-03-330, Information on DOL's Data Validation effort is accessible at (http://wwww.ows.doleta.gov/dmstree/ten/ten2K2/ten_14-02.htm); and TEIN 14-02 Data Validation. Court of International Trade http://www.cit.uscourts.gov/slip_op/Slip_op06/06-132.pdf

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: All ETA managers are held accountable through incorporation of relevant elements and performance standards in the appraisal process. States are held accountable for program performance and Regional Offices routinely monitor state status related to cost, performance results, and program adherence to published directives and regulations. Findings are distributed to appropriate program managers and the states for the purpose of enhancing performance. One means of enhancing accountability in the states is the reserve funding request process (additional funding). Performance factors that are taken into consideration during the review of these funding requests include: the number of participants with approved training plans who have not started training; the average take-up rate for training; the average training cost per participant; the average training duration; and the average training completion rate. In addition, states that are found to be out of compliance and refuse to take corrective action proscribed by Regional Office staff may be issued a letter putting their funding at risk if the required action is not taken. It has been several years since the program has had to issue such a letter to gain compliance from a state, although such action has been threatened. Finally, the Department is considering adding a performance component to the funding formula for base allocations each year, with several options for doing so under consideration. Reports for the past year from the Chicago Regional Office for the states of Missouri, Wisconsin, Kansas and Iowa indicate that the states are doing a good job of rapid intervention and assessment. Isolated issues surrounding participant payment for indicental training costs and approval of training for more than 104 weeks have been detected as well as an aggressive use of waivers are also concerns. Based upon these reviews, states have been directed to take corrective action and in some instances make retroactive payment, or establish overpayments. States operate as agents of DOL and, as discussed earlier, DOL has adopted a funding structure for training similar to that used in the WIA Dislocated Worker program. States are required to meet cost, schedule, and performance results established nationally. TAA benefits are available to a restricted set of eligible trade affected workers, and states cannot use TAA program funds for other dislocated worker groups.

Evidence: Revised DOL performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001), State agreements, and Illinois disallowed cost finding available. Regional Office review reports for Missouri, Wisconsin, Kansas and Iowa.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Congress appropriates funds for TAA training and income support based on statutory training caps and estimated need for income support and other benefits during the fiscal year. To improve management of the $220 million in capped training funds, since FY 2004, DOL has distributed 75% of the funds at the beginning of the fiscal year while holding 25% in reserve. This approach enables states to receive a base allocation for training at the start of the fiscal year to better plan and manage funding resources necessary to meet the training needs of trade-certified workers. The reserve fund is available for distribution to states experiencing large, unexpected layoffs or having training needs that exceed available funds. States draw down funds as needed to pay the training and income support for eligible individuals and submit quarterly financial reports to DOL. DOL verifies proper obligation with a fund disbursement process and tracks expenditures with accrued accounting procedures that were implemented in FY 2004. Program funding at the state level is available for three years, and states carry over unobligated training funds. DOL's report of unobligated balances as of 9/30/06 indicates that unobligated balances from 2005 and 2006 total $120 million, compared with the $518 million appropriated by Congress for training, administrative expenses, and job search and relocation allowances for those years. DOL has observed wide variances in the unobligated training balances (ranging from $0 in five states to less than $1 million in seventeen states to over $23 million in the state of Washington). As a result, DOL is reviewing obligations data to make modifications to that formula for FY 2008 funding, with particular attention being paid to the "hold harmless" feature implemented when the formula was developed to ensure that no state had funding cut prematurely. DOL publishes the funds made available to each state in several ways. TAA grant awards are released annuall in a Training and Employment Guidance Letter, which is a public documetn available on DOL's web site. Grant awards from the reserve account are announced publicly by press release when the award is made to a requesting state. DOL also reports TAA awards promptly and accurately to the Federal Assistance Award Data System. State use of TAA funds is reviewed under the Single Audit Act. A recent audit of the TAA program in Illinois resulted in a finding of disallowed costs with a negotiated settlement of $7 million to be repaid to the TAA program by the state.

Evidence: Spending plans and reports, recent audit reports. Fiscal Year 2005-2006 Unobligated Balances report (as of 9/30/06)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While DOL has taken steps to make its petition review process more efficient and timely, there are no procedures for improving the efficiency of the provision of training and other TAA services to workers. As part of the Common Measures initiative, the program has adopted a program-wide efficiency measure of cost per participant. However, DOL does not use this efficiency measure to manage the program and improve its efficiency. At this time, no incentives exist for states to become more efficient. In addition, DOL has been lax in its oversight of the use of waivers by the states, which allow trade-impacted workers to receive income support without being enrolled in training. Waivers increase the cost of the program with assuring that the workers are gaining skills or better jobs. DOL has focused on improving the timeliness of the petition review process. DOL uses competitive sourcing for some Federal administrative activities such as petition review and audit close-out. TAA measures its administrative efficiency by average processing time for petition review. DOL has focused on reducing the processing time for petition review and has reduced the average time from 96 days in 2002 to 31 days currently - which is below the 40 day statutory requirement embodied in the Trade Reform Act of 2002. TAA is currently reviewing options and working on developing an outcome-based efficiency measure, such as cost per entered employee, to complement the cost per participant measure. TAA is also considering ways to measure output in the petition investigation area as well as ways to measure the true cost of providing training for TAA certified participants.

Evidence: DOL FY 2008 CBJ - ETA Performance Chapter.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Pursuant to requirements of the Trade Reform Act of 2002, the program collaborates with the Department of Agriculture in providing services for trade impacted farmers and fishermen as well as the Internal Revenue Service in administering the Health Coverage Tax Credit (HCTC). The program worked with the Department of Agriculture to develop a process for certifying eligibility of farmers and fishermen by a County Extension Service representative. In addition, TAA has begun meeting with representatives from the Department of Commerce in an effort to better coordinate trade certifications with the DOC TAA for firms components.. Integration of the HCTC has required more ongoing effort, including a monthly conference call with the IRS to ensure that issues are identified and addressed in a timely manner. In addition, HCTC has required coordination with another DOL program -- National Emergency Grants in providing states with "gap filler" grants for HCTC costs to cover the 60-day lag period that can occur between trade affected separation and the start of HCTC coverage. Finally, through a co-enrollment process, the program collaborates with WIA to provide an array of wrap-around services to trade-affected workers such as assessment before training and placement after training.

Evidence: TEGL 9-04 Services Authorized by USDA Trade Adjustment Assistance for Farmers Program. TEGL 10-02 Use of National Emergency Grant Funds Under the Workforce Investment Act, as Amended, to Develop Systems for Health Insurance Coverage Assistance for Trade-Impacted Workers UIPL 2-03 Health Insurance Tax Credit for Eligible Trade Adjustment Assistance/Trade Readjustment Allowances (TAA/TRA) Recipients

Does the program use strong financial management practices?

Explanation: DOL reviews the quarterly financial reports from state agencies for accuracy and completeness. Any inconsistencies are documented and forwarded to the appropriate regional office for resolution. Auditors have not identified material internal control weaknesses in TAA financial systems. With the establishment of the training funding formula in FY 2004, DOL has significantly improved state and Federal accountability for TAA program funds. The Department has $220 million available for TAA training each year and provides states with a base allocation at the beginning of each year which enables them to better plan and manage administration of the TAA program. In addition, DOL has provided financial management training to state agency staff in an effort to ensure that individuals understand the importance of accurate participant and expenditure reporting. Prior to development of the funding formula which ensured that funds were allocated according to need, and that funding was available for unanticipated trade dislocations, National Emergency Grants (NEGs) provided training for trade affected workers when TAA funds were exhausted. NEGs were used to provide wrap around services through Dual Enrollment grants, and also provided training under WIA provisions for dislocated workers who may also have been trade impacted. Significant improvement has been made in management of training funds area and has virtually eliminated the reliance on National Emergency Grant -- Dual Enrollment (NEGs) funds to supplement TAA training funds. In addition, states are better able to plan for response to trade dislocation as they happen rather than being forced to request funds for each occurrence. Finally, states no longer need to obligate this year's funds for training in subsequent years when an individual's training approval spans more than one fiscal year because the formula factors this need into the equation. To guard against having all funds allocated when the unexpected happens, the Department also holds part of the $220 million allocation in reserve which allows states that experience large unanticipated layoffs to request additional TAA training funds.

Evidence: DOL's FY 2006 Annual Report on Performance and Accountability, which is accessible at (www.dol.gov/dol/aboutdol/main.htm).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: ETA identifies management deficiencies in various ways, including GAO and OIG reports as well as by its regular oversight activities. Oversight is primarily provided through Regional Office monitoring of the program in states. These monitoring visits have a corrective action component, and taken jointly, help identify areas where continuing guidance is needed. ETA has consistently taken steps to address management deficiencies detected. In response to issues detected surrounding performance data, the agency revised its TAA reporting system and conducted nationwide training to improve the state's understanding of the requirements (see Question 3.1). The agency is updating its core monitoring guide and will offer training and guidance on its proposed performance reporting system, which states will use to report performance outcomes. To test the effect of mandatory universal co-enrollment which would provide core and intensive wrap-around services to all TAA enrollees, the agency commissioned the six-state co-enrollment pilot. This pilot is designed to test both the positive and negative outcomes of full co-enrollment on both TAA and WIA outcomes. Results will be available in the fall of 2007, and those results will help management determine how to ensure these services for the TAA population without negatively impacting WIA performance. In response to outdated program regulations that created management problems described in Section I, the agency has developed and published for comment extensively revised TAA regulations. Finally, as indicated in 3.4 above, the efficiency measure surrounding petition processing time has been reduced from an average of 96 days in 2002 to its current average of only 31 days - significantly under the 40 day statutory requirement contained in the Trade Reform Act of 2002. This was accomplished through development of a new business processes which streamlined the movement of applications from one group of reviewers/processes to another. The new model provides: application processing standardization; central and automated data storage and retrieval; workload management capability; decision support capability; national and regional reporting capability; and standardized case correspondence. This model has allowed DTAA to reduce petition processing time from an average of 96 days to an average of 31 days. Our next step will be the development of a process for independently measuring the accuracy of TAA determinations in addition to the multi step review process already built into each certification.

Evidence: ?? ETA Common Measures Directives ?? WIA and TAA Co-Enrollment Pilot Project Draft Design Report of September 15, 2006 ?? Federal Register Vo. 71/ No 165 Friday, August 25, 2006

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The program's performance is trending upward over the last several years, indicating considerable progress in achieving its long-term goals of 74.5% for entered employment and 92.5% for employment retention in FY 2011. Between FY 2003 and FY 2006, TAA's performance in placing its participants in jobs (entered employment) increased from 62% to 72%. During the same period, the program's performance in participants keeping those jobs (employment retention) increased from 86% to 90% and earnings replacement (measure to be replaced by average earnings in FY 2007) increased from 74% to 77%. Note that because the TAA program is a fiscal year program, the performance data in DOL's Performance and Accountability Report and the Congressional Budget Justification are estimates. Final FY 2006 data, reflected above, are found on the TAA web site.

Evidence: DOL FY 2006 PAR (http://www.dol.gov/_sec/media/reports/annual2006); DOL ETA TAA Web site at (http://www.doleta.gov/tradeact/taa_stats.cfm)

Does the program (including program partners) achieve its annual performance goals?

Explanation: As noted in the previous question (4.1), TAA has made considerable progress in achieving its long-term goals, which is reflected in positive trends in entered employment, employment retention and increases in the wage replacement rate. However, performance against individual annual performance targets has been less consistent, especially for wage replacement. Additionally, some of this performance was achieved under performance targets that were not as ambitious as they could have been.

Evidence: DOL FY 2006 PAR (http://www.dol.gov/_sec/media/reports/annual2006)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: DOL does not have necessary data to determine the cost-effectiveness of TAA training or the efficiency with which states run the TAA program. There is no direct relationship between funding levels and performance outcomes, because outcomes are rates, e.g., the percent of program exiters who entered employment, not levels. As an entitlement program with a statutory training cap, Congress funds TAA training plus the estimated costs of income support and related services, which depend on the number of eligible workers and the cost and duration of weekly income support payments. DOL's efficiency measure -- cost per participant -- is based on the amount of funds appropriated divided by the number of program participants. Actual costs per participant can vary significantly, depending on the cost of training, whether or not they participate in training, and the number of weeks of income support used. DOL does not collect data on actual training costs for each participant. In other aspects of the TAA program, DOL has improved its administrative efficiency between FY 2003-06, as measured by average days to process a certification request and make determination. As noted previously, average processing time was reduced from 96 days to its current 31 day average. DOL has also realized improved cost efficiencies following its reengineering of the certification process and switch to contractors to improve staffing flexibility to deal with fluctuating workloads. Between FY 2005 and FY 2006, the total cost per determination went from $964 to $745, a 23% reduction in cost per petition reviewed.

Evidence: DOL FY 2008 CBJ - ETA Performance Chapter

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: TAA and the WIA Dislocated Worker (DW) program both serve workers affected by mass layoffs. While TAA Entered Employment and Earnings Replacement are lower than DW's, TAA program exiters who complete training have a higher Employment Retention rate than DW exiters who complete training. Information on the characteristics of each program's participants suggests that the poorer TAA outcomes are not a reflection on the effectiveness of the program's training and relocation services. Rather, they appear to be a reflection of the population served by TAA, which tends to be older and dislocated from relatively low-skill, high-wage employment. The TAA program helps workers dislocated because of foreign trade adjust to changing market conditions and shifting skill requirements. Addressing the needs of workers involved in this transformation is a particular challenge when workers are being dislocated from relatively low-skill, high-wage employment. In many cases, this is complicated by mass layoffs or plant closures that occur in single industry towns, which makes finding comparable employment in the same geographic area difficult. Furthermore, many of these jobs are lost permanently from the domestic economy, requiring the skills of affected workers to be completely retooled.

Evidence: Program Year 2005 WIA Summary Report for Dislocated Workers http://www.doleta.gov/performance/results/WIASRD/PY2005/DW-05-Summary-Rpt.pdf FY 2006 Year End Performance Highlights for TAA; population characteristics http://www.doleta.gov/Performance/results/Quarterly_report/PerformanceHighlights06.pdf TAA Performance Goals and Outcomes http://www.doleta.gov/tradeact/performance.cfm

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While TAA does not yet have results from the planned impact evaluation, two recent GAO reports indicate both petition processing time improvements from the TAA Reform Act and positive program outcomes. A September 2004 GAO report found that after the TAA Reform Act, TAA reduced the average petition processing time from 107 days to 38 days and that workers enrolled in training sooner than they had previous to the Reform Act. This study also found challenges associated with implementing TAA reforms, including an increased administrative workload from issuing training waivers to allow workers to qualify for the Health Coverage Tax Credit (HCTC). A January 2006 GAO study found that most TAA participants in four of the five sites studied either found a new job or retired by the the time of the study's survey. The fifth site, where only 37 percent had found new jobs, was the site that had closed most recently (eight months prior to the study's survey). The majority of those who found work did so between 6 and 12 months after losing their previous job. Reemployed workers at all five sites were able to replace, on average, between 79 and 94 percent of their pre-layoff wages.

Evidence: Relevant GAO/OIG studies include the following: ?? September 2004: Reforms have Accelerated Training Enrollment, but Implementation Challenges Remain (GAO-04-1012) ?? September 2004: Health Coverage Tax Credit - Simplified and More Timely Enrollment Process Could Increase Participation (GAO-04-1029) ?? January 2006: Most Workers in Five Layoffs Received Services, but Better Outreach Needed on New Benefits (GAO-06-43) ?? April 2006: Labor Should Take Action to Ensure Performance Data Are Complete, Accurate, and Accessible (GAO-06-469) ?? September 2005: GPRA Data Validation Review, Trade Adjustment Assistance Program (OIG 22-05-007-03-330)