Detailed Information on the
Hazardous Materials Transportation Safety - Emergency Preparedness Grants Assessment

To develop a closer linkage between this program's performance goals and grantee's performance goals,PHMSA is compiling examples of best practices in strategic planning, where State grantees can share methods to improve their effectiveness when allocating Grant funds to local first responder agencies.

Develop a closer linkage between this program's performance goals and grantee's performance goals, either through legislative or administrative means.

Schedule a comprehensive program evaluation to be conducted in FY 2007.

Measure: Number of serious hazardous materials incidents.

Explanation:

Measure: Hazmat responders trained. This measure reflects one aspect of the universe of DOT activities conducted to achieve the DOT-wide goal of reducing serious hazardous materials incidents.

Explanation:Standard classroom training.

Measure: Emergency plans completed. This measure reflects one aspect of the universe of DOT activities conducted to achieve the DOT-wide goal of reducing serious hazardous materials incidents.

Explanation:

Measure: Number of local emergency planning committees supported.

Explanation:Committees assisted by funding and/or technical assistance.

Measure: Exercises conducted.

Explanation:Operational/field training for public sector employees to respond to accidents and incidents involving hazardous materials.

Is the program purpose clear?

Explanation: The program purpose, as stated in the Federal Hazardous Materials Transportation authorizing legislation, is: Planning Grants: 'to make grants to States and Indian tribes-(A) to develop, improve, and carry out emergency plans under the Emergency Planning and Community Right-to-Know Act of 1986 (42 U.S.C. 11001 et seq.), including ascertain if flow patterns of hazardous material on lands under the jurisdiction of the State or Indian tribe, and between lands under the jurisdiction of the State or Indian tribe and lands of another State or Indian tribe; and (B) to decide on the need for a regional hazardous material emergency response team.' Training Grants: to 'make grants to States and Indian tribes to train public sector employees to respond to accidents and incidents involving a hazardous material.'

Evidence: 49USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130; 42 USC 11001 et seq.; North American Emergency Response Guide Book; Number of hazardous materials serious incidents (http://hazmat.dot.gov/hmep/hmepcong.pdf).

Does the program address a specific and existing problem, interest, or need?

Explanation: Under Federal law, DOT has the exclusive right to regulate safety, including security, of the transportation of hazardous materials in commerce. When hazardous materials incidents occur, the responsibility to mitigate the incident falls on local first responders, many of whom are poorly funded volunteer fire fighters. The HMEP grant program funds planning and training activities to prepare first responders at the State and local level, and is the only Federal program that provides funds to assist communities in planning for and responding to hazardous materials incidents that may occur within their jurisdictions. Other programs provide more specific assistance to first responders. For instance, the recently expanded Firefighter Assistance Grant program is a discretionary program, where localities apply for grants for equipment (e.g., fire engines), and training related to operating the equipment.

Evidence: HMEP Report to Congress, 1998 (hazmat.dot.gov/HMEP/hmepcong.pdf). The number of local responders is approximately 2 million, all of whom require refresher training annually, at a minimum (www.whitehouse.gov/homeland/firstresponders).

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: RSPA's HMEP grant program is unique in that it supplements hazmat planning and training at the local level, to reduce both the occurrence and the consequences of serious hazmat incidents. The section 5116 funds are the only Federal funds available to all 50 states, Puerto Rico, DC, the territories, and Indian tribes for these purposes.

Evidence: 49USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130; GAO Report "Hazardous Materials Training: DOT and Private Sector Initiatives Generally Complement Each Other," July 2000.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Because all funding provided is allocated to grantees according to a risk-based formula, States, territories, and Indian tribes can be confident of funds availability. States may then distribute funds based on their perceived needs. RSPA believes that States are most knowledgeable in allocating Federal funds according to their needs and priorities. RSPA obligates funds during the last week of the fiscal year, so that funds are available to be reimbursed on the first day of the following fiscal year.

Evidence: 49USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130 HMEP Report to Congress, 1998 (http://hazmat.dot.gov/HMEP/hmepcong.pdf)

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Funds allocated to States and Territories are then assigned to individual jurisdictions, at the discretion of the State or Territory. States are required by law to pass through at least 75% of the funds to local jurisdictions.

Evidence: 49USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130 Grantees certify that at least 75% of all grant funds are passed on to local and regional authorities who are responsible for hazmat responder training, exercises and regional planning (outputs).

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The DOT Hazardous Materials Safety program has a specific, readily identifiable and understood, and measurable national goal which is directly focused on reducing the consequences caused by hazmat incidents on the nation's transportation system, as measured by the number of serious hazmat incidents per year. Our long-term goal is to reduce serious hazardous materials incidents to no more than 488 in FY 2008, a 15% reduction from FY 2000.

Evidence: DOT 2004 Performance Plan and RSPA FY 2004 Budget Submission to Congress. DOT FY 2005 Performance Plan will contain the long-term (2008) outcome goal contained in the 'measures' tab.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The long-term outcome goal covering DOT's entire hazmat safety program (including this grant program) is ambitious, especially since the funding for this program has been held level at $14.3 million. The target is based on historic performance data, and the FY 2008 outcome target of 488 will be a stretch for RSPA.

Evidence: RSPA FY 2004 Congressional Justification.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: RSPA's hazmat safety program, which is the Federal regulatory aspect of DOT's overall hazmat safety program, shares DOT's overall goal for 2008 of 488 incidents. The authorized purpose of these grants is to train local first responders to handle hazmat incidents safely, and thereby contribute to achieving the national serious incident goal. The HMEP grant program contributes to the serious hazmat incident goal by reducing the potential for an incident to become serious and/or reducing the severity of a serious incident. The program does not have an efficiency measure, as the small size of the program would result in few, if any, savings through such a measure.

Evidence: DOT FY 2004 Performance Plan and FY 2002 Performance and Accountability Report and all FY 2003--2005 RSPA budget submissions. RSPA performance plan is integrated with its Congressional Justification material.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: While the overall DOT hazmat program's long-term outcome measure is ambitious, the four specific output targets for the HMEP grant program are being held constant because program funding has been held level at $14.3 million. The four output targets are based on past historical performance at the $14.3 million funding level. While actual outputs have fluctuated above and below the target levels, on average they have not deviated far, implying that the output performance targets are appropriate.

Evidence: DOT FY 2004 Performance Plan and FY 2002 Performance and Accountability Report and all FY 2003--2005 RSPA budget submissions. RSPA performance plan is integrated with its Congressional Justification material.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: While RSPA's partners are committed to reducing the number and consequences of hazmat incidents on the roadways, states are not required to specifically address, nor are states required to track, their progress toward agency goals. In their application for HMEP grants, the States specify target audiences to be trained at the awareness, operations and specialist levels. Also, local emergency planning committees detail the tasks to be funded with HMEP grants. These plans are not directly tied to the agency's long-term hazmat incident goal.

Evidence: Local emergency response plans; commodity flow studies.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: No independent evaluations of sufficient scope are conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness. Currently, the agency relies on achieving their performance goals to assess performance information. The agency does not have an independent evaluation to evaluate if the program could be improved or become more effective.

Evidence: Evaluations are not scheduled by independent, unbiased parties with no conflict of interest, such as every two to five years, on a periodic basis, or on a reasonable time schedule. GAO Report "Hazardous Materials Training: DOT and Private Sector Initiatives Generally Complement Each Other," July 2000; www.lepcinfoexchange.com/docs/rc00190.pdf

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: RSPA's FY 2003-2005 budgets clearly tie and justify resource requests to the long-term outcome and output measures.

Evidence: FY 2003-2005 RSPA budget submissions.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The HMEP grant program was created by Congress in the early 1990s, as a grant program with significant grantee discretion. However, information about its performance and effectiveness, plus support of the states, has allowed RSPA to take the grants through successive planning processes in order to make the program more effective in meeting Federal, state, and local needs, in the information required through the annual application process as well as the end-of-year performance reporting.

Evidence: 49USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130; 42 USC 11001 et seq.; Hazardous Materials Emergency Preparedness Grant Program DRAFT: Application Kit

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Summary reports from states, territories, and Indian tribes communicate the progress of the programs at the local level. Information feeding the summary reports are submitted by the grantees.

Evidence: Form 269 Financial Status Reports, Form 270; Semi-annual grant workshops serve to share lessons learned and best practices among grantees.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: The HMEP program is required by authorizing legislation, annual appropriations, and anti-deficiency requirements to assure grants conform to cost limitations. Grantees are responsible for monitoring their sub-grantees, and may refuse funding to local agencies in future years for program inadequacies.

Evidence: 49 USC Chapter 51, Section 5116; 49 CFR Parts 101.1-110.130; 42 USC 11001 et seq. Form 269 Financial Status Reports, Form 270

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Federal funds are obligated just before the beginning of the fiscal year, using funds collected in past years from the hazmat registration program. Any deficiencies found by the Single Audit Act audits or any other deficiencies identified in program reviews are required to be addressed along with any correction action that was taken.

Evidence: Audit Reports

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program distributes $14.3 million, the largest proportion allocated among 70 grantees. The program does not utilize outside contractors, and automated records are confined to spreadsheets that justify disbursements in the accounting system. Thus there is little opportunity for such efficiencies.

Evidence: NA

Does the program collaborate and coordinate effectively with related programs?

Explanation: The grant program conducts two workshops per year for the benefit of grantees. At these meetings, grantees can discuss successes and challenges within their jurisdictions. Grant staff also provide assistance with grant applications and suggest improvements via online telephone support.

Evidence: The program manager participates in monthly meetings of subcommittees on training of the National Response Team. The NRT is made up of 16 federal agencies, each with responsibilities and expertise in various aspects of emergency response to pollution incidents.

Does the program use strong financial management practices?

Explanation: The grant program is included in the Single Audit of all grantees and no material internal weaknesses have been identified by the auditors. If any weaknesses are identified, they are immediately corrected.

Evidence: The program uses adequate financial management practices in executing the grants awarded to each state, territory, and Native American tribe. Each payment request is scrutinized for errors, proper matching amounts and funding available. Each payment is subtracted from a summary sheet in the grants unit and then at the accounting office, providing a strong double check. Finally, the payment request with Automated Clearing House (ACH) sheet is forwarded to the Oklahoma City accounting office for payment.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: As deficiencies are identified, they are immediately addressed and closed out when corrective measures are in place.

Evidence: Internal controls instituted at the program's inception, such as the process described in 3.6, have evolved to highlight program deficiencies quickly and mitigate them upon discovery.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: States are required to submit a HMEP grant application detailing proposed activities. Actual expenditures are tracked by headquarters staff. The HMEP staff holds regular technical assistance sessions to help grantees identify deficiencies and take corrective measures. These activities provide adequate oversight.

Evidence: HMEP Grant Program Application Kit

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Each grantee is required to submit an annual report on the results of their program. These reports are then aggregated to form an annual output measure.

Evidence: RSPA Hazardous Materials EP Grants Program Fact sheet (http://hazmat.dot.gov/hmep/hmepfact.htm).

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: The DOT-wide hazmat strategic goal of reducing the serious hazmat incident rate is close to being achieved. RSPA and the Department are continuing to work to continue to achieve that goal. The HMEP grant program supports that goal. Within limits set by legislation, the grant program has reached a steady state level of accomplishments---for example, completion of 3,700 emergency plans and 900 exercises. Accomplishment of this goal depends on the agency's partners ' states, local jurisdictions, private sector and safety organizations.

Evidence: FY 2001 actual and FY 2002 preliminary actual number of serious hazardous material incidents.

Does the program (including program partners) achieve its annual performance goals?

Explanation: While the program partners do not have to achieve annual performance goals to receive grants, the Department met or exceeded its annual performance targets for the number of serious incidents in 1999, 2000, and 2001. States and local communities are actively supporting this goal.

Evidence: Key measures, such as the number of responders trained, are measured and reported on a regular basis. FY 1999-2001 actual and FY 2002 preliminary actual outcome data.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: The grant program has continually improved its cost effectiveness since its inception. Starting with a staff of four professionals plus an administrative support contract, the program is now run by two staff members with only minor additional administrative support.

Evidence: The hours of burden in processing grant data has been significantly decreased from $525,000 to $200,000 at an approximate cost saving of $325,000 annually.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: The HMEP grant program is unique, in that it is the only Federal program that provides funds to assist communities in planning for and responding to hazardous materials incidents that may occur within their jurisdictions. Thus there is no other Federal program with similar purpose or goal. Other programs provide general assistance to first responders. For instance, the recently expanded Firefighter Assistance Grant program is a discretionary program, where localities apply for grants for equipment (e.g., fire engines), and training related to operating the equipment. The HMEP grant program deals exclusively in responding to hazmat incidents occurring during transportation.

Evidence: HMEP Report to Congress, 1998; GAO Report "Hazardous Materials Training: DOT and Private Sector Initiatives Generally Complement Each Other," July 2000.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Though independent evaluations of this program have not been conducted, RSPA completed a program review in 1998. The assessment concluded that the grants were a critical part of the national strategy to reduce the hazardous materials incident rate. The grants have assured that state and local program focused on key hazmat issues in a consistent manner. The program has achieved the intent of Congress, and has played a leadership role. The grants have not supplanted the much larger program share allocated by states, communities, and the private sector.

Evidence: HMEP Report to Congress, 1998; GAO Report" Hazardous Materials Training: DOT and Private Sector Initiatives Generally Complement Each Other," July 2000.