Detailed Information on the
Bureau of Indian Affairs - Operation and Maintenance of Roads Assessment

Working with program partners to establish a clear division of responsibilities within the Indian Reservation Roads system, including performance reporting within schedules and guidelines.

Working with Congress via coordination with program partners to avoid redundancy of effort, including by reassessing current law regarding HTF funding for maintenance in the IRR system.

Preparing an annual report on the collection and use of relevant data, including use of a refined efficiency measure, to improve program performance and make management decisions.

Measure: Percent of miles of road in acceptable condition based on the Service Level Index.

Explanation:

Measure: Percent of bridges in acceptable condition based on the Service Level Index.

Explanation:

Measure: Cost per mile to maintain BIA roads in acceptable condition.

Explanation:

Is the program purpose clear?

Explanation: The Bureau of Indian Affairs (BIA) Road Maintenance Program provides maintenance services for some roads and bridges within the Indian Reservation Roads (IRR) system. It provides services such as routine pavement maintenance, snow removal, striping, and roadside mowing. The program does not undertake road improvements or construction. The BIA road system is a subset of the larger IRR system, which includes all public roads on reservations. The IRR system provides transportation and public access to, within, and through Indian reservations for Native Americans, visitors, recreational users, resource users, and others, while contributing to the health and safety and economic development of Native American communities. Improvements to the IRR system are funded from the Federal Highway Trust Fund (HTF). A 2005 policy change to the Safe, Accountable, Flexible, Efficient Transportation Equity Act -- A Legacy for Users (SAFETEA-LU) permits up to 25% of the IRR program funds to be used for the maintenance of the IRR. However, the use of these IRR funds for maintenance of roads and bridges is based on the approval of Indian Tribes.

Evidence: 23 U.S.C. 101(a) defines key terms; 25 CFR 170.5 defines terms in the regulations; 23 U.S.C. 204(a), Federal Lands Highways Program, describes policies and procedures for the HTF; SAFETEA-LU, Section 1119(i), Maintenance of Indian Reservation Roads, describes the use of HTF money for maintenance in the IRR program.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program maintains roads and bridges to protect the investment that was made in constructing them and to provide safe transportation for tribes and the general public traveling through Indian reservations. In FY 2007, the IRR consisted of 85,000 miles of public roads with multiple owners, including Indian tribes, BIA, states and counties. As of March 2008, the IRR inventory had increased to about 100,000 miles, of which approximately 28,000 miles are BIA-owned. In addition, approximately 900 bridges in the IRR system are BIA-owned. A majority of the BIA-owned roads are earth or gravel surface and susceptible to flooding or other damage if not adequately maintained. The purpose of the program is to assist in the maintenance of these assets.

Evidence: IRR inventory as maintained by the BIA Division of Transportation, Miles of Road by Organizational Responsibility and Surface Type Report.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: There is overlapping responsibility within the IRR program for various functions among various entities. SAFETEA-LU now permits up to 25% of IRR program funds to be used for road maintenance activities within the IRR system, which is composed of roads owned by multiple entities, such as states, local governments and tribes. The IRR program is a separate program, which is jointly administered by BIA and the Federal Highway Administration (FHWA) through a memorandum of agreement. The BIA road maintenance program suffers from other entities not making efficient use of funding available from the HTF, but instead relying too heavily on this program for maintenance needs. BIA is responsible for maintenance of the BIA-owned transportation infrastructure, even though it was not responsible for construction. However, there are tens of thousands of more miles in the IRR system, for which other entities have maintenance responsibility. This situation makes coordinated priority-setting difficult. In many cases, on a given section of road, maintenance responsibility may shift from one entity to another in very short distances. Other entities are not using all of their HTF funding on the BIA system to reconstruct roads and bridges that are at the end of their useful life, thereby increasing BIA maintenance costs for those deferred reconstruction projects. The reasons for taking roads into the BIA and/or IRR inventory are unclear and may create incentives for other entities to shirk their maintenance responsibilities. There is no actual duplication of maintenance efforts, but the program needs to ensure clear responsibility among the various entities to avoid program implementation that is redundant or working at cross-purposes with other programs and entities.

Evidence: 25 CFR 170, Subpart G (§170.801 - §170.813) BIA Road Maintenance Program, describes the definition, policies, funding eligibility, and roles and responsibilities of various jurisdictional authorities pertaining to the road maintenance; SAFETEA-LU Section 1119(i).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program performs its functions for basic maintenance and operations. It has never been authorized to improve or reconstruct roads, but instead is supposed to "maintain" them in a safe and effective manner. If roads cannot be maintained adequately, they are taken out of service or reconstructed under the IRR program. As discussed in 1.3, there does not appear to be clear jurisdictional responsibility under this and related programs. States, counties and local governments have constructed roads on reservations using HTF funding, but: (1) those public entities are not fully utilizing HTF funding for reconstruction projects, causing tribes to redirect their IRR HTF funding towards those projects; and (2) when those entities also do not consistently maintain their roads adequately, this lack of attention to maintaining current assets causes earlier and more expensive reconstruction to be needed. As a result, BIA road maintenance resources and tribal HTF resources are faced with higher maintenance costs and accelerated road deterioration. A reassessment of current law regarding HTF funding and the roles and responsibilities of various governmental entities is in progress. The program needs a mechanism for better coordination and priority-setting within the context of the entire IRR system. Specifically, the process for determining which roads are BIA's responsibility for maintenance should be reviewed. There are some short sections of road that currently BIA's maintenance responsibility that could possibly be maintained by other entities more efficiently with adequate incentives and legal framework. BIA must ensure that the program is implemented in a manner that is consistent with tribal sovereignty and self-determination.

Evidence: SAFETEA-LU Section 1119(i) indicates that up to 25% of IRR funding under the Highway Trust Fund can be used for maintenance. In 1973, BIA and the Federal Highway Administration (FHWA) identified certain roads as BIA's responsibility. In 1974, BIA and FHWA entered into two separate agreements which set out the joint and individual statutory responsibilities of FHWA and BIA for constructing and improving Indian reservation roads and bridges. The agencies encourage collaboration on project and maintenance activities.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Road maintenance funds are targeted for the benefit of tribal members and the traveling public. The program performs road and bridge maintenance activities on BIA assets, as well as other functions as provided by law, such as snow and ice removal. The program focuses on ongoing maintenance of paved roads (because of the lower cost of maintenance of these roads as compared to unpaved roads) and emergency maintenance and snow/ice removal to keep roads open.

Evidence: SAFETEA-LU Section 1119; Obligation tables by program activity (Program administration, routine maintenance, snow/ice removal, etc.); Distribution of funds by Region (correlating the miles of road/bridges responsibilities with resources); 25 CFR 170 Subpart G, BIA Road Maintenance, defines the eligible activities of the program.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has established goals and measures within the Department's Strategic Plan that focus on providing safe roads and bridges, for the advancement of quality communities for tribes. BIA has measures to demonstrate: 1) Percent of miles of road in acceptable condition based on the Service Level Index; and 2) Percent of bridges in acceptable condition based on the Service Level Index. The program also has established an Annual Efficiency Measure (Cost per miles of roads maintained in acceptable condition), but 2007 was the baseline year and the measure needs further refinement. The program uses a Level of Service Index based on a 5-level condition rating system, ranging from the level 1 being excellent to the level 5 failing. This rating system is similar to the one used by State highway departments, based on a simplified "windshield survey" of roads (so that any driver would perceive the same condition for roads). The program rates the condition of bridges using certified bridge inspection engineers as required by the National Bridge Inspection Standards (NBIS) established by the Federal Highway Administration (FHWA). It is unclear whether these condition assessment tools and processes adequately meet the concerns raised by the Inspector General's 2007 review of the program, and it is also unclear how these assessment tools relate to other government-wide measures, such as the Facility Condition Index method.

Evidence: Interior's Strategic Plan contains these measures, and guidance has been issued to field staff regarding the condition assessment. The program has established an efficiency measure (cost per miles of roads maintained in acceptable condition), but 2007 was the baseline year.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Long-term targets and timeframes appear to be ambitious, taking into consideration (1) the increase in the number of miles of BIA roads and bridges to be maintained and (2) funding that has remained level or is proposed for reductions. The annual and long-term outcome measure for roads is the percent of miles of road in acceptable condition based on the Service Level Index. Targets are 14% for FY 2008, 8% for FY 2009, and 8% for FY 2012, although the actual reported percentage was as high as 21% in FY 2005. The annual and long-term outcome measure for bridges is the percent of bridges in acceptable condition, and the targets are 44% for FY 2008, 36% for 2009, and 36% for FY 2012, although the actual reported percentage was 81% in 2007 (but the program reports that reporting anomolies associated with these numbers were quickly identified and corrected).

Evidence: Annual Performance Accountability Reports show what goals were achieved. According to program staff, long-term targets for percentages of roads and bridges in acceptable condition were determined based on the changing trend of road condition rates, reduced resources at the field level, and increased inventory miles in successive years. They also argue that many assets come into the inventory in low-quality condition and that costs for materials and fuel have been going up.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: BIA applies annual targets to its long-term outcome goal/measure of road and bridge service level condition (discussed above) and in so doing can monitor incremental progress toward long-term targets. However, BIA does not appear to have a count of how much of its inventory has had the condition assessments performed and on what timetable. Such review and the resulting data should be more comprehensive over time.

Evidence: Interior's Strategic Plan contains performance goals and measures for the program. According to program data, the percent of miles of road in acceptable condition in FY 07 was 15%, exceeding the 2007 target of 14%. For bridges, the 2007 actual was 81% compared to 2007 plan of 44%, because the method of collection and computation for condition criteria changed from 'good/better' (levels 1 and 2) to 'fair/better' (levels 1,2,and 3) by expanding the range of acceptable condition during the reporting year. (This is expected to result in a lower performance for out years once the range reverts to 'good/better' in FY 08.) BIA staff indicate that the agency is currently collaborating with other Bureaus within the Department using this rating system to calculate the Facility Condition Index on roads and bridges so that the Department can compare the condition of capital investments across all Bureaus.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program's targets for its annual measures appear to be ambitious, taking into consideration (1) the increase in the number of miles of BIA roads and bridges in the inventory, (2) funding that has remained level or has been proposed for reduction, (3) many assets come into the inventory in a low-quality condition, and (4) costs for materials and fuel have risen. (The program has established the same measures for both long-term and annual use.) BIA started collecting performance data for roads and bridges in 2004, and annual performance measures and the baseline were established. For the annual outcome measure of "percent of miles of roads in acceptable condition based on SLI:" in 2005, the target was 15% and the actual was 21%; in 2006, the target was 16% and the actual was 17%; and in 2007, the target was 14% and the actual was 15%. For the annual outcome measure of "percent of bridges in acceptable condition based on SLI:" in 2004, the target was the baseline year data with 47% actual; in 2005, the target was 47% with 52% actual; in 2006, the target was 49% with 62% actual; and in 2007, the target was 44% with 81% actual (but there were reporting anomolies associated with these numbers). The program has also established a long-term/annual efficiency measure (cost per miles of roads maintained in acceptable condition), but 2007 was the baseline year and this measure needs further refinement.

Evidence: Annual Performance Accountability Reports show what goals were achieved. According to program staff, targets for percentages of roads and bridges in acceptable condition were determined based on the changing trend of road condition rates, reduced resources at the field level, and increased inventory miles in successive years. They also argue that many assets come into the inventory in a low-quality condition and that costs for materials and fuel have been going up.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Because of competing priorities and unclear assignments of responsibility, various program partners are often working at cross-purposes. The complexity of the IRR system is partly to blame. The BIA and the FHWA operate on a government-to-government relationship with federally recognized tribes, and have consultation and collaboration protocol in place. While these planning efforts are useful, there still appears to be limited collaboration in priority-setting and limited effectiveness of the routine interaction among the various program partners, including states and local governments.

Evidence: 25 CFR Subpart B describes how BIA performs consultation, collaboration and coordination with partners. Meetings of the Tribal Budget Advisory Council (TBAC) include both BIA top management and tribal leaders. The IRR Program Coordinating Committee also has regular meetings.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program has undergone two reviews. An Inspector General report was a follow-up to a previous OMB PART review. A second independent study of greater scope, focusing on the main functions of the program, is almost finalized, and the program has committed to additional reviews.

Evidence: The 2007 IG review of the program only partially addressed several issues raised in the 2004 PART review. In FY 2007, FHWA arranged for an independent review and contracted with an engineering company, PBS&J of Denver, Colorado. The scope of the study included research of the program organization; its activities and engineering practices; and program laws, regulations, and policy documents. The final report was not complete as of May 2008, but a substantially complete draft was reviewed.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BIA has not provided evidence that the requested funding level is based on local or national program needs and priorities. It is unclear from program materials how different funding levels would impact program performance, other than via employment levels.

Evidence: Budget submission materials; documentation of Tribal Budget Management Advisory Council meetings. There is a lack of evidence that budget requests are explicitly tied to accomplishment of the annual and long-term performance goals.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The program has taken meaningful steps to correct strategic planning deficiencies, and problems identified in program reviews are under review. Most significantly, the program has developed new performance goals to track actual condition of roads and bridges to ensure safe and efficient travel in Indian Country, but data collection efforts are uneven. In addition, further work may be needed on measures and how the information collected is used in practice. First, it is unclear that condition assessments are complete and are being updated regularly. Second, condition assessments apparently are not being used effectively to inform decisions about setting priorities. Finally, information collected is not being used effectively to help build better partnerships and collaborations. For example, there is no evidence that the new regulation issued to help address the role of states and local governments in maintenance responsibility has resulted in any improvements in that regard. Still, even though work remains, the program has made progress in addressing these deficiencies.

Evidence: The 2007 Inspector General's review identified deficiencies in the BIA Road Maintenance program and made three recommendations. The first recommendation was to develop performance goals and measures, baseline information and targets. BIA has taken steps to collect baseline information on the condition of its roads and has developed a new performance measure to assess the condition of the roads and bridges, but there are questions about the credibility and reliability of the data. The second recommendation was to develop a process to encourage states and local governments to meet their responsibilities on reconstruction of their roads crossing reservations on a timely basis. Regulations implementing the process (25 CFR 170.811) were developed and published in the Federal Register. BIA has no authority to compel local governments to take action, but referring deficiencies to the DOT may prompt remedial action because DOT can withhold approval of projects (Title 23 USC, Section 116). However, there is no evidence that this step has been taken in any case. The third recommendation was to develop a process for and schedule of independent program evaluations. As a follow-up step, BIA arranged for FHWA to hire a contractor to perform a study, but the draft report available as of May 2008 has not resulted in tangible policy options for improvement of program performance.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program is struggling to collect timely and credible performance information from regional offices regularly. In spite of putting in place a protocol for reporting, there are unanswered questions about the quality of this system and the quality of the condition assessment data being used (as noted elsewhere). Furthermore, field staff are not consistently using it. The Deferred Maintenance Reporting (DMR) system is designed to capture and report costs for maintenance needs, maintenance accomplishments, and deferred maintenance for BIA route, structure and equipment assets, but it is not clear how well the system is working. Most importantly, there is insufficient evidence of how any of these data are being used for purposes of program management and performance improvement.

Evidence: DMR User Manual excerpts. Performance data forms and reports, which apparently are collected from some, but not all, regions regularly. A recent internal memo notes that reporting is incomplete and asks regional office directors to certify the accuracy and completeness of it. There is a lack of evidence for how data that are reported are used for purposes of program management and performance improvement.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners are held accountable for results to some extent. Program performance goals are reflected in contracts and annual funding agreements (AFAs) with tribes and in individual performance plans for BIA managers.

Evidence: Selected AFAs and individual performance plans.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Funds are made available to the Road Maintenance Program and obligated each year, and are expended for the intended maintenance program activities, ranging from program management to snow/ice removal on BIA-owned roads and bridges. In a normal fiscal year, the obligation rate of the program funds averages from 85 to 95 percent. In order to be able to address any unexpected contingencies and disasters at the beginning of new fiscal year, it is a prudent management practice to carryover a small portion of program funds from fiscal year to fiscal year.

Evidence: The financial data from the Federal Finance System and Activity Based Costing system provide budget execution reports, including status of obligation and expenditure of road maintenance funds. These reports show the status of allocation and suballocation, the status of obligation in total amount and percentages, and the status of expenditures by regions.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Procedures are in place for improving efficiency. IT improvements have been made, including advancements in the Deferred Maintenance Reporting (DMR) system. BIA has initiated competitive sourcing cost comparison studies in two regions. The program's efficiency measure is new and has not yet produced sufficient data or cost comparisons to similar programs. The efficiency measure needs further refinement to allow for more task-specific data sorting and cost analyses.

Evidence: Cost comparison data. Regarding competitive sourcing, the Navajo region began a standard study for the road maintenance program to identify the Most Effective Organization (MEO). The Rocky Mountain region performed a streamlined study, and the program claims cost-savings of $130,000 in the following year, although evidence is incomplete.

Does the program collaborate and coordinate effectively with related programs?

Explanation: BIA has in some cases pursued collaboration and coordination possibilities within program constraints. It needs to improve and expand on these efforts to gain efficiencies by entering into formal collaboration agreements with other entities (states, counties and tribes) with responsibility for roads and bridges in their areas. By necessity, the program collaborates with the IRR program within DOT, but more can be done, possibly including work on a common set of performance measures for the IRR system as a whole.

Evidence: As shown by several agreements, BIA has collaborated with tribes, states and counties on some construction and maintenance projects for leveraging other sources of funds to supplement the BIA program funds. However, this practice is not routine and has not been institutionalized across all regions. 25 CFR 170 Subpart G, BIA Road Maintenance, defines the eligible activities of the program and the roads to be maintained under the program.

Does the program use strong financial management practices?

Explanation: The program appears to have good oversight of financial practices within its regional office system. BIA uses the Federal Finance System and the Activity Based Costing system that provide for obligation and expenditure of program funds as well as for tracking the costs of activities such as program management, snow/ice removal, routine maintenance, and other financial data. The Budget Execution Reporting Tool provides a web-based tool that enables BIA managers to determine the execution status of programs and accounts on a near real-time basis, access the data at various levels (e.g. appropriation, fund, program class, and program) and download the information to Excel for local use and analysis.

Evidence: Tables generated from data in these systems show that program funds are obligated and expended for various activities ranging from program management to snow/ice removal.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: While the program has much more work to do to improve management deficiencies, it has taken small but meaningful steps to make improvements. More focus is required to follow through on addressing identified deficiencies. The program needs more consistent application of reporting requirements throughout its areas of responsibility. It also needs to take more seriously the deficiencies notes in previous reviews and make plans to address the underlying concerns more directly. In 2007, OIG published a review of the BIA road maintenance program to address three issues noted in the 2004 PART review. One of the suggestions by OIG was that BIA discontinue the practice of using road maintenance staff to perform condition assessments, and instead use independent contractors with more detailed procedures for rating road conditions as conducted by the road construction program. BIA disagrees, and futher analysis may be needed to resolve this issue. A second recommendation concerned cooperation with states and local governments. BIA promulgated a regulation defining responsibilities for states and local governments to maintain their roads within the IRR system. However, there is no evidence that this regulation has improved the situation or that any enforcement has taken place. The third major OIG finding was that the road maintenance program condition assessments using the Service Level Index are not reliable. Again, BIA disagrees, and again further analysis may be needed to clarify the concerns and try to resolve them.

Evidence: The Inspector General's report and follow-up actions were reviewed. In addition, four management and quality assurance field office reviews were recently undertaken, and four more are planned. In 2007, a Corrective Action Plan implemented access controls to prevent unauthorized data changes by regions. BIA has improved its information and data collection procedures with the implementation of the DMR system, which replaces the use of Excel spreadsheets.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Long-term performance goals have been established for percentages of roads and bridges in acceptable condition, and the program has demonstrated partial progress in achieving them. However, as explained elsewhere, the measures themselves may need further refinement, questions remain about the credibility and completeness of the data, and it is unclear how ambitious the goals are. Finally, it is not clear that program partners have committed to these goals.

Evidence: Performance data reports; Annual Performance Accountability Reports and FY 09 Goal Performance Table reflect the long-term performance goals. There is a lack of evidence regarding the commitment of program partners to performance goals.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Annual performance goals have been established for percentages of roads and bridges in acceptable condition, and the program has demonstrated partial progress in achieving them. (The long-term and annual measures are the same.) The annual performance goal for percentage of bridges in acceptable condition has been met over the past three consecutive years, and the annual performance goal for percentage of miles of roads in acceptable condition was met in 2005 and 2007. However, as explained elsewhere, the measures themselves may need further refinement, questions remain about the credibility and completeness of the data, and it is unclear how ambitious the goals are. Finally, it is not clear that program partners have committed to these goals.

Evidence: Performance data reports and Annual Performance Accountability Reports. There is a lack of evidence regarding the commitment of program partners to performance goals.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has taken steps but has not demonstrated improved efficiencies or cost-effectiveness in achieving its goals. BIA conducted two cost savings studies for the road maintenance program, but the status of implementation and actual savings is unclear. The roads program recently established a new efficiency measure of "cost per mile of BIA roads maintained in acceptable condition," but there is not good data yet for comparison purposes. The measure is intended to track the effectiveness of a strategy that directs resources towards maintenance of level 2 (Good) and level 3 (Fair) rated paved roads. The efficiency measure needs further refinement for it to a useful tool for purposes of program management and performance improvement.

Evidence: The Navajo Region study indentifed potential savings by streamlining Road Maintenance program operations, but it has not been implemented. The Rocky Mountain Region study also pointed to potential savings through a similar streamlining effort. Regarding the efficiency measure, the program gives the example that cost per mile to perform crack sealing and pot-hole patching of level 2 and level 3 paved roads is estimated at $4,000 per mile to preserve the roads in the same condition, excluding other variables. On the other hand, it could cost from $250,000 to $1,000,000 per mile to reconstruct these paved roads if they are allowed to deteriorate.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: It is difficult to compare this program to others, given the unique nature of, and divided responsibilities for, the road system throughout Indian Country. However, it appears from limited data provided by the program that similarly situated agencies and programs maintain roads and bridges at similar cost. Other entities are responsible for maintaining roads and bridges in a more concentrated geographic area, and their assets are often in better condition as originally constructed.

Evidence: Program staff indicate that they focus program resources on preventing roads in acceptable condition from degenerating into an unacceptable state. Such preventive efforts should prolong the life of roads in acceptable condition and reduce the need for additional construction. For those targeted roads, BIA appears to compare favorably with other programs in terms of dollars spent per mile of road maintained, according to anecdotal evidence compiled by program staff and information listed in the FHWA review. Unlike state and county road maintenance programs, responsible for geographical areas in relatively close proximity, most BIA miles are located in remote areas in 32 States throughout the country.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: While the program has undergone two limited reviews, they are not of sufficient scope and quality, and they do not demonstrate program effectiveness. The 2007 Inspector General review was merely a follow-up to the 2004 PART; it indicates that BIA has made some limited progress in response to the prior PART. The FHWA-contracted study is also a somewhat limited program review. The latest draft of that study (May 2008) does not indicate program effectiveness but does point out several ideas for improvement.

Evidence: In its limited review, OIG acknowledged that BIA has developed a performance measure, issued regulations encouraging state and local governments to maintain their roads on Indian reservation, and arranged for the FHWA, which has an oversight responsibility in the administration and implementation of the IRR program, to provide an evaluation of the program. In FY 2007, FHWA contracted with a consultant to perform the evaluation. The scope of work for the evaluation included four areas; 1) BIA road maintenance program management; 2) BIA organizational structure; 3) program activities, involving maintenance responsibilities, objectives, operation and engineering practices; and 4) program laws, regulations, and policy documents.