Detailed Information on the
Bureau of Indian Affairs - Job Placement and Training Assessment

Continue to implement the new common measures and collect the baseline information.

Collect data to compare program performance with other job training programs.

Work with states and other Federal agencies to acquire data on program participant employment one year after completing the program.

Work with affected tribes to clarify the common measures and to encourage tribal participation in the collection of information in support of the new measures.

Measure: Percentage of participants that attain improved literary skills.

Explanation:

Measure: Cost per individual receiving job placement services (New measure, added August 2007).

Explanation:

Measure: Improvement in adult participant earnings.

Explanation:Modified measure is a more precise indicator of the percentage of program participants whose earnings have improved.

Measure: Cost per job achieved. (lower number is good. Cost includes transportation, tools, and maintenace until first paycheck is received.)

Explanation:

Measure: Percent of participants who report a positive exit from the Job Placement and Training Program.

Explanation:

Is the program purpose clear?

Explanation: This voluntary program is intended to demonstrate how Native American Governments can integrate similar federal programs on employment, training and related services funded by BIA, DOL, HHS and DoEd to improve the delivery and effectiveness of those services. Under this program, Native American Tribes can pool funding from all of these sources to meet individual Tribal needs. Effectiveness relates to reduced joblessness in Federally recognized Native American communities and fostering economic development on Indian lands. In addition, the program supports and promotes Native American self-determination and self-governance.

Evidence: Pub. L. 102-477, of October 23, 1992, the "Indian Employment, Training and Related Services Demonstration Act of 1992." Amended by Pub. L. 106-568, Section 1103 of December 27, 2000, the "Indian Employment, Training, and Related Services Demonstration Act Amendments of 2000.

Does the program address a specific and existing problem, interest or need?

Explanation: The program's purpose is to: 1) eliminate duplication of effort by tribes implementing many related federal employment, training, education, and related services programs; 2) address high unemployment, low educational attainment, and low wages of Native Americans; and 3) promote self-determination.

Evidence: 1) Legislative history. 2) Documentation of high unemployment and high poverty on Indian reservations from U.S. Department of Agriculture report. 3) Bureau of Indian Affairs Labor Force Report documenting high poverty and high unemployment levels. 4) One tribal chairman testimony regarding preference for Pub. L. 102-477 coordinated federal programs. 5) Washington University study on "new and innovative approaches to restructing and integrating services under Pub. L. 102-477.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: This program integrates Federal funding for the purpose of job training, tribal work experience, employment opportunities or skill development, or any program designed for the enhancement of job opportunities or employment training so all funding can be rolled into a single comprehensive resource for tribes. Current funding sources include BIA, DOL, and HHS.

Evidence: 1) Pub. L. 102-477, as amended, design of the program. 2) Pub. L. 102-477 Tribal Work group documenting comprehensive approach to services. 3) 477 Regulations documenting reduction of duplication in tribal reporting from 166 pages to 6 pages annually. 4) U.S. Department of Labor website documenting benefits of Pub. L. 102-477. 5) Programs eligible for integration into one comprehensive program.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Some Indian communities do not have sufficient jobs on Native American reservations to place all tribal members that are trained through the program and some trained individuals do not wish to work outside the reservation, which contibutes to the continued high unemployment even though the individuals are trained for jobs.

Evidence: 1) Public law 102-477 is Under-Utilized and Section 1103 allows for job creation. 2) Brookings Institution on lack of jobs at liveable wages in fight against welfare reform.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Through this program, Tribes spend less funds on administration and more directly on client services through the reduction of administrative burdens. For example, annual tribal reporting has been reduced from 166 pages a year of forms and instructions to 12 pages per year. Implementation of the program requires one set of client files and application instead of as many as 12 different application forms, eligibility documents, and other related burdens

Evidence: 1) Pub. L. 102-477, as amended stating target of program. 2) annual tribal report forms OMB approved, documenting reduction in administrative reporting burden. 3) 1998 submission to OMB for approval documenting 166 pages reduced to 6 pages annually. (now 12 pages annually).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program is part of the Job Training Common Measures Initiative. Accordingly, it has adopted outcome measures and one efficiency measure for adults and youth and lifelong learning. Beginning in 2004, BIA using these common measures, will better measure the impacts of the program than current measures and allow caparisions across similar Government programs. BIA is implementing the common measures and will establish numerical targets for 2005.

Evidence: 1) White House Initiative on Common Performance Measures for all Federal employment and training programs, nationwide. 2) OMB approved pub. L. 102-477 annual report forms. 3) strategic plan for Department representing the Pub. L. 102-477 program.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: GPRA measures include long-term goals of tribal communities reaching parity with other rural communities for unemployment rates

Evidence: 1) strategic plan for Department representing the Pub. L. 102-477 program. 2) Bureau of the Census data identifying annual average unemployment status for rural U.S. for purposes of striving for parity.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Measures on job retention, cost per job and job creation are included within the DOI Strategic Plan .

Evidence: 1) Strategic Plan. 2) 05 Budget Request. 3) newly approved OMB reporting forms capturing job retention, cost per job and job creation data from tribes.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The BIA has been measuring job retention on the basis of 90 days employment constitutes success of an individual in the program since FY 1999. The new measure within the DOI Plan requires that this goal track retention out to one year beginning in FY 2004.

Evidence: 1) Strategic Plan. 2) Annual Performance Plans. 3) Previously approved OMB 477 reporting forms collecting 90 day job retention data. 4) current OMB approved reporting forms for Pub. L . 102-477 revised per White House directive.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Reporting of program performance has received full compliance from grantees since the inception of the initiative. Federal partners continue to participate including Department of Labor (Gregg Gross representative (202) 693-3752) and Robert Shelbourne representing DHHS, (202) 401-5150 and DHHS Child Care representative Ginny Gorman, (202) 401-7260).

Evidence: Copies of grantee submitted OMB approved forms since beginning of program.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Independent reports by local universities, Department of Labor, HHS contractors , and tribal representatives have concluded that 477 programs are generally more effective than if the programs are implemented by tribes as seperate programs.

Evidence: 1) Joint report prepared by Udall Center for Studies in Public Policy of the University of Arizona and the Geroge Warren Brown School of Social Work of Washington University. 2) U.S. Department of Labor website on benefits of Pub. L. 102-477. 3) Report of the Indian and Native American Employnent and Training Coalition concerning evaluating 10 years of 477 implementation. 4) Congressional Record and statements made by Senator Stevens on effectiveness of 477 in Alaska. 5) Program comments by University of Arizona.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget requests are written in the context of what the program success rate will be and the number of jobs that will be created and individuals placed in employment.

Evidence: FY2005 Budget justifications.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: As part of the Job Training Common Measures initiative, the program has adopted long-term and annual measures and will establish numerical values for those goals in 2004. BIA's performance measures have been modified to show a greater accountability in tracking job retention for an entire year.

Evidence: 1) Revised OMB information collection requirement for tribes to track job retention for one year based upon UI data provided by states (standard requested by white House). 2) Strategic Plan.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Grantee reports are used to determine whether their programs are operating successfully. If the program is having management trouble then DOL technical assistance funding is requested to allow for oversight and assistance to correct deficiencies.

Evidence: 1) Sample copies of annual reports for calendar year 2003 2) Sample of on-site program reviews documenting DOI staff evaluation of tribal performance. 3) sample letter sent to tribe denying further participation due to non-compliance.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: If grantees are delinquent in submision of annual reports, they will cease receiving funds until reporting is complete. If audit problems are uncovered, the grantee will be placed on quarterly payment status until the problems are resolved. In addition, BIA included GPRA performance measures in the Individual Performance Plans of all program managers in FY 2004. Because this is the first year of implementation accountability for adhering to performance and cost measures cannot yet be proven. The performance measures lack measurable outcomes or outputs and because BIA has just implemented the measures to rate individuals, there is no evidence that BIA has actually held individuals accountable.

Evidence: 1) Letter sent to one tribe limiting drawdown of funds due to audit issues. 2) Copies of annual reports for 2003.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: While typical delays do occur in the Federal obligation of funds, once grant awards are made and accepted by the tribe, grantees usually have funds within 5 weeks. The program is still working to improve this timeline. In reviewing the annual reports it appears that some program participants are not obligating all the funds allocated to them in a timely manner. BIA explains that this is usually due to the varied timing of funds from the various agencies. Some DOL and HHS funds are not issued until after July 1 of each year and the TANF funding is obligated in May of each year. If a tribe begins a program on July 1 or they receive this late funding from DOL and HHS, they are still required to submit their annual report on September 30 and it may appear that they have large balances because they just recently received their funding.

Evidence: BIA reports that funding from non-BIA sources are awarded tiemely if no audit sanctions or other related issues exist. BIA TPA funds are delayed to the extent that funds located at Regional offices must be pulled int Central office for distribution. BIA is working on improvements in 2004 to shorten the obligation process.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: DOL funds support an incentive effort for those grantees that show "Best Practices". For those grantees that have good overall success rates and show innovative approaches to employment and job creation there is a small amount of funding that can be made available to implement new initiatives under the program.

Evidence: 1) Tribal 477 plans containing performance measures; 2) performance monitoring through annual tribal reports; and 3) copies of BIA on-site progam reviews.

Does the program collaborate and coordinate effectively with related programs?

Explanation: Coordination and collaboration is daily between the DOI, HHS and DOL. A MOU is in place, federal partners transfer funds, receive 100% of tribal grantee reports, approve tribal plans, provide TA to tribes and assist by serving as a resource for tribes.in the day-to-day management of 477 and meet regularly. There are Annual Federal Partners meetings and there are quarterly Tribal Workgroup Meetings.

Evidence: 1) Copy of initial MOU between all participating agencies, DOL and Interior. 2) Sample meeting agenda for the most recent, March 19, 2004 annual Federal partners Meeting.

Does the program use strong financial management practices?

Explanation: 100% of funds received from DOL and HHS are transferred to tribes. Tribes with audit issues are placed on quarterly payments systems until audit issues are resolved. System could be improved by providing additioanl resources to handle financial transactions. BIA's 2003 audited financial statement did not identify any reportable conditions for this program.

Evidence: 1) Copy of initial MOU between all participating agencies, DOL and Interior. 2) Example of denying further participation due to non-compliance. 3) Sample letter limiting drawdowns due to audit issues.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Funding and obligation practices are being reviewed to reduce the level of carryover balances. The timing of funds released from Federal partners has been adjusted to avoid transfers late in the year that only allowed 3 days for accounting and obligation practices before fiscal close out.

Evidence: Tribes and partners meet at least quarterly and federal partners meet at least annually to discuss and improve financial, management and progam operations. Reviewed document containing past ten years of implemention activities.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: 100% of grantees are monitored once every three years or more. Attention is paid to compliance with Single Audit Act or funds are withheld from grantee, as necessary. If Participant's Annual Report reviews show problems, onsite reviews are conducted.

Evidence: Sample of an on-site program review reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: All grantee reports are made available to HHS, DOL and DOI. HHS and DOL use this data in their annual reports. DOI uses the data in performance reports. Data could be more successfully distributed if staff was available. To ensure that future performance data is compatible accross government, the program is participating in the President's Job Training Common Measures initiative.E67

Evidence: 2003 tribal reports.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: BIA has adopted new long-term goals as part of the Job Training Common Measures initiative. BIA will establish numerical annual and long-term targets in 2004. Partial credit is based on the fact that BIA had a previous long-term measure, which supports achievement of new long-term goals -- lowering the unemployment rate hasn't been met.

Evidence: 1) BIA report "Ten Years of Building a New Tribal Federal Relationship, dated June 2004. 2) Brookings Institution study addressing lack of access to higher wage employers contributing to on-going welfare on Indian Reservations. 3) The Citizen Potawatomi power point presentation on their successful job creation projects.

Does the program (including program partners) achieve its annual performance goals?

Explanation: BIA has adopted new long-term goals as part of the Job Training Common Measures initiative. BIA will establish numerical annual and long-term targets in 2004. Partial credit is based on the fact that BIA had existing job retension measures that showed that BIA has been able to successfully meet its goals and maintain a 92% success rate for the program which is defined by 90 day job retention.

Evidence: Department of Labor's web site on the benefits of Pub L. 102-477 and successes experienced.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: 100% of the grantees participating in 477 report reduced administrative burdens and increased resources to assist members with job training and education. In addition, Senate Indian Affairs Committee and GAO, support the program for all the good work it has done.

Evidence: Report entitled The "477' Demonstration - Ten Years of Building a New Tribal-Federal Relationship, June 2004.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The 477 program has a higher success rate than Federal partner programs such as the DOL's Division of Indian and Native American Programs, which measures positive termination in the same manner as the 477 program. DOL's program has an 83% success rate compared to the 477 program's 93% success.

Evidence: Based on the performance measures BIA has inplace that are similar to some of the White House Initiative common measures.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent reports by local universities and HHS contractors have concluded that 477 programs are generally more effective than if the programs are implemented by tribes as seperate programs. However, until the adoption of common measures across Government for similar programs there is no way to quantify what the reviewers were reporting.

Evidence: BIA provided a number of studies conducted to evaluate the program. However, additional studies need to be accomplished once the common measures are in place.