Detailed Information on the
Transportation Security Administration: Aviation Regulation and Enforcement Assessment

Develop a long-term outcome oriented performance measure that measures risk reduced as a result of implementing program objectives. This goal will be included in the FY 2009 Budget.

By the end of FY 2009, TSA will develop and deploy a pay-for-performance system to hold federal managers accountable for cost, schedule, and performance results.

By the end of FY2010, TSA will create an IT based system that has a capability to perform proactive and reactive analysis supporting tactical and strategic operations.

By FY 2007, develop a plan to systemmatically review current regulations to ensure consistency among all regulations in accomplishing program goals.

By FY 2007, develop baselines and ambitious targets for the annual measures.

Measure: Average number of international inspections conducted annually per inspector.

Explanation:International: Through improved automated reporting tools, increase number of inspections performed by each inspector from an average of six to nine per year.

Measure: Percent reduction in the federal share of LEO costs provided by U.S. commercial airports.

Explanation:Law Enforcement: New measure under development GOAL: Reduce the reliance on federal funds for law enforcement support through Reimbursement Agreements (RAs) at airports while increasing the number of airports with RAs.

Measure: Percent of countries with direct flights to the U.S. who are provided aviation security

Explanation:The measure reports the amount of interaction the United States has with countries providing Last-Point-of-Departure (LPD) service to the U.S. An LPD country is a country with at least one port providing direct traffic to a specific destination - usually a foreign airport with direct passenger and/or cargo flights to a U.S. destination airport. The U.S. interacts with countries providing LPD service with the goal to share aviation security policy and practices at either the national or airport level.

Measure: Average number of days of holding time before dogs enter into formal training.

Explanation:Through improved automated tracking and increased training development , reduce the average number of days (by 10% each FY) from canine procurement into entering into official training with a student (LEO handler).

Measure: Number of inspections conducted domestically by TSA.

Explanation:Compliance: A primary strategy of this program is to reduce the risk of a terrorist incident by reducing the vulnerability of the system. The program accomplishes that through inspections of established mitigation activities. The assumption is that an increase in inspection and assessment activities will reduce risk by reducing the aviaiton transportation system's vulnerabilities.

Measure: Average number of inspections conducted per domestic inspector

Explanation:Compliance: Industry compliance with security requirements is measured inspections performed by field and headquarters inspectors. The baseline number of inspections is established in FY04. Productivity estimates are based on a five per cent increase in average inspections per inspector each FY.

Measure: Number of qualified participants (aviation and mass transit) that have executed Cooperative Agreements for TSA-certified explosives detection canine teams.

Explanation:NEDCTP: This program creates an obstacle to terrorists attempting to disrupt the transportation system by providing a mobile layer of security. Resources are provided to local law enforcement agencies (aviation/mass transit) to deter the introduction of explosives devices and to effectively and efficiently restore operations when the presence of explosives are suspected. Targeted goals represent the following: 82 - Category X and I airports; 92 - Category X and I airports and top-ten mass transit systems; 102 - Category X and I airports and top-20 mass transit systems;

Measure: Percent of countries/territories with no direct flights to the U.S. who are provided aviation security assistance

Explanation:The measure reports the amount of interaction (output) the United States has with countries with no direct service to the US in the area of aviation security policy and practices at the national or airport level.

Measure: Percent of airports in compliance with leading security indicators

Explanation:The program demonstrates risk reduction as a direct result of the program's implementation of security rules and regulations and an increase in the percentage of airports compliant with the standards. In support of this risk-based approach to regulatory oversight, the data collected through the airport and air carrier compliance assessments demonstrates percent compliance over critical vulnerabilities, identified by the leading security indicators, for airports nation wide. TSA believes this compliance measure measures risk reduced as a result of implementing the Aviation Regulation program objectives of "ensuring compliance with aviation transportation security laws, regulations, directives, and agreements and conducting inspections, investigations, and assessments."

Measure: Percent of air carriers in compliance with leading security indicators

Explanation:The program demonstrates risk reduction as a direct result of the program's implementation of security rules and regulations and an increase in the percentage of air carriers compliant with the standards. In support of this risk-based approach to regulatory oversight, the data collected through the airport and air carrier compliance assessments demonstrates percent compliance over critical vulnerabilities, identified by the leading security indicators, for airports nation wide. TSA believes this compliance measure measures risk reduced as a result of implementing the Aviation Regulation program objectives of "ensuring compliance with aviation transportation security laws, regulations, directives, and agreements and conducting inspections, investigations, and assessments."

Is the program purpose clear?

Explanation: The program's purpose is to protect the nation's transportation system by: ensuring compliance with aviation transportation security laws, regulations, directives, and agreements; conducting inspections, investigations, and assessments; providing resources and coordination for law enforcement, and detection of explosives; and ensuring a transportation security focus world-wide.

Evidence: Aviation Transportation Security Act (ATSA) of 2001 (P.L. 107-71), Arming Pilots Against Terrorism Act (P.L. 107-296); Vision 100 - Century of Aviation Reauthorization Act (P.L. 108-176); NEDCTP Fact Sheet; Title 49 USC §44907 and Title 49 USC§44916; 49 CFR Part 1546.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the security threat to the aviation mode of transportation. The program strives to prevent future terrorist attacks through compliance with mitigation activities.

Evidence: P.L. 107-71; APATA; and Vision 100 - Century of Aviation Reauthorization Act (P.L. 108-176). TSA Criminal Violations (Federal Aviation Security Offenses), Title 49 and Title 18 charges; ble Stationing; 49 C.F.R. Part 1500, et seq. (67 FR 8339-8384); TSA Security Directives on threats against Rail and Mass Transit

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program serves a unique federal function either applying a nationally consistent aviation security program, providing federally managed resources to TSA partners, or ensuring US transportation security standards are maintained world-wide.

Evidence: ATSA (P.L. 107-71); APATA (P.L. 107-296); P.L. 108-176; TSA Criminal Violations (Federal Aviation Security Offenses); Title 49 and Title 18 charges; Title 49 USC §44907; ICAO Annex 17.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The organizational programs making up TSA Aviation Regulation and Enforcement were developed in a variety of ways. The overall program was recently created, therefore it has yet to return the anticipated efficiencies. The effectiveness of each program is determined independently.

Evidence: OIAPR Program Review; Sample Comprehensive Assessment; Annual Evaluation Report; reimbursement break out/funds report/summary of funds by category; Office of Compliance Functional Statement sets forth the organization mission, purpose, and structural alignment.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The organizational programs making up TSA Aviation Regulation and Enforcement allocate their resources in a variety of independent ways. The overall program has recently been formed and as such has not yet had the opportunity to oversee the allocation of resources.

Evidence: OIAPR Program Review; Sample Comprehensive Assessment; Annual Evaluation Report; reimbursement break out/funds report/summary of funds by category; Office of Compliance Functional Statement sets forth the organization mission, purpose, and structural alignment.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program is in the process of developing long-term outcome measures that focus on the reduction of risk as a result of implementing program objectives. In the interim, is developing the following measures: percent of increase in aircraft critical/leading security indicators and percent of increase in airport critical/leading security indicators. These measures will be included in the FY 2007 Congressional Justification.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The baselines and targets for the program's interim long-term goals are under development.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has a limited number of specific annual performance measures that indicate progress towards reducing the risk of a potential terrorist attack on the transportation system. The program has also developed two new efficiency measures.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications. OLE Book: The Draft Acquisition Plan shows the transition from MOA to RA and associated cost reductions achieved.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The annual goals have baselines and ambitious targets.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The partners involved in this program fall into 3 categories: signatories to international agreements, those contractually obligated (Cooperative agreements, Reimbursement Agreements), or those complying with federal government regulatory requirements. These partners commit to and work towards the accomplishment of the prgram's annual goals. This information is documented in a variety of sources, including airport security program documents.

Evidence: FY 04/05 Airports Requesting Funding for Additional Post Hours; Statement of Joint Objectives; 49 U.S.C. 44903(c), 49 U.S.C. 44903(f), 49 U.S.C. 44916(a), Annual Inspection Plan

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Many of the components of the "Aviation Regulation and Other Enforcement" program are newly re-organized and hasn't established an independent evaluation process. However, several internal audits have been conducted of the subcomponents. For example, the DHS IG and Internal Affairs has or will conduct audits of the canine and compliance programs, and the international program has been reviewed by GAO. Also, ICAO audited the US aviation transportation security system and the results of this draft are pending.

Evidence: Report on Ely Airport completed by TSA Internal Affairs OIAPR Program Review; ICAO letter to TSA dated Nov 24, 2004 (MOU); Cover sheet for "Advance List of Recommendations of the ICAO Aviation Security Audit of the United States of America, dated 7 to 20 April 2005.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program is uniquely identified in the TSA budget request and performance goals and targets are tied to the funding level.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: TSA has recently established an office that is responsible for facilitating the strategic planning process. This branch will work to develop a risk-based long-term performance outcome goal and define an independent evaluation plan to ensure assessments are high quality, of sufficient scope, and conducted regularly.

Evidence: FY 2007 TSA Congressional Justification; COO Staff Pay System report which includes Performance Branch.

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The regulations promulgated by the program are necessary to accomplish the mission and goals. Many of these regulations were carried over from FAA and have been updated to reflect the creation of TSA and the mission of DHS.

Evidence: CFR 49, Standard Operation Practices and Procedures; [Federal Register: February 22, 2002 (Volume 67, Number 36)][Rules and Regulations][Page 8339-8384].

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program regularly collects timely and credible performance data to facilitate effective management decisions. For example, information on canine and handler training is reviewed weekly to determine if additional and/or different training is required; air carriers are inspected every six months to two years to determine compliance with TSA security directives; and TSA engages regularly with airports on non-compliance issues, which can be addressed administratively before exercising its authority to levy civil penalties.

Evidence: Annual Domestic Inspection and Assessment Plan; Inspection results maintained in PARIS by location/ operator; CWS Handbook; Report Samples, International Principal Security Inspector Weekly Activities Report (IPSI) Example.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: While TSA managers are held accountable for certain program responsibilities, the program has not established clearly quantifiable performance standards for those managers. TSA is in the process of developing a pay-for-performance system.

Evidence: TSA performance management system; TSA Performance Appraisal System.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The program's resources are obligated in a timely manner and spent for the intended purpose. The program develops annual spend plans that are then used for planning purposes and then these funds are compared to actual obligations; quarterly plans will be developed starting in FY 2006. Typically, obligations occur during first quarter for enforcement programs, and spread throughout the year for international and domestic regulatory programs.

Evidence: FY 2004 Annual Financial Planning documents.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has developed an efficiency measure for each programmatic element. For example, increase the number of inspections per inspector, reduce the federal share of airport cost for law enforcement officers, reduce the processing time for foreign airport assessments, and decrease down time for canines awaiting training.

Evidence: FY 2007 TSA Congressional Justification.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Program effectively collaborates with all related programs. For example, the program coordinates with its international counterparts, including the Organization of American States to capitalize on available funding for aviation security training; TSA relies on airports to provide law enforcement for security incidents; and TSA meets with CBP monthly to share operational initiatives including establishing operations for joint cargo screening and delivery of explosive materials.

Evidence: FSD and airport partner in developing justification for Reimbursable Agreements; CPB Minutes; International agreement with Organization of American States.

Does the program use strong financial management practices?

Explanation: The FY 2004 Audit of Financial Statements revealed material weaknesses in the areas of information technology and internal control monitoring and evaluation.

Evidence: FY 2004 Audit of Financial Statements.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program, through the PART process, has developed efficiency measures for each programmatic element. Furthermore, TSA is developing an appraisal system to quantify performance measures managers will be held accountable for, and is improving its financial management practices to address material weaknesses by establishing a management control program.

Evidence: FY 2006 and 2007 Congressional Justification, Senior Manager Appraisal System, Management Control Program Guidance TSA MD3100.3.

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: The regulations that cover this program were adopted from DOT when DHS was established. There have been both substantive and technical amendments made to the regulations since TSA assumed programmatic responsibility. The substantive changes were made through the notice and comment process and incorporated input from relevant stakeholders. An Interim Final Rule was also published, with request for comments, to transfer General Aviation security in the Washington, DC area from FAA to TSA.

Evidence: Department of Homeland Security Semi Annual Regulatory Agenda of May 16, 2005; 70 FR 7150; 69 FR 65258; 67 FR 8340.

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: The program regularly prepares adequate regulatory impact analyses. For example, in the amendment published on January 24, 2003 regarding threat assessments of persons holding or applying for FAA Certificates, analyses were conducted and summaries provided in the Federal Register.

Evidence: 68 FR 3765.

Does the program systematically review its current regulations to ensure consistency among all regulations in accomplishing program goals?

Explanation: When promulgating new regulations, TSA reviews existing regulations to ensure consistency with TSA's goals and with developing regulations and standards. TSA has developed a draft guide that lists regulatory review as an example of situations that may lead to rulemaking activities. The guidelines require that all analyses conducted be systematically reviewed at various levels before the rulemaking package is forwarded. TSA is developing a plan to systemmatically review its current regulation to ensure compliance with the requirements for "610 Periodic Reviews of Rules."

Evidence: TSA Rulemaking Guidelines (Templates); Proposed Rule Milestones checklist.

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: For all regulations published according to the Administrative Procedures Act, TSA's Office of Regulatory and Economic Analysis conducts and publishes a regulatory analysis as required by OMB circular A-4. These evaluations examine costs and benefits to society. The agency's mission goals are described as well as alternatives considered. Net benefits for security are frequently stated qualitatively and every effort is made to maximize net social gain consistent with balancing all regulatory considerations such as statutes directing regulation as well as process statutes such as the Regulatory Flexibility Act. Detailed evaluations are published and available in the public docket. The regulations appear in the Federal Register with a Regulatory Evaluation Summary. For example, in the case of reopening three airports to private aircraft it was recognized that the societal cost outweighed the restrictions in place and the airports were reopened, using alternative methods (enhanced procedures) to reduce risk and minimize the compliance burden.

Evidence: Airport Security Program 49CFR1542; DOT DMS Docket number TSA-2005-20118; Document TSA-2005-20118-3, Maryland Three Airports: Enhanced Security Procedures For Operations At Certain Airports In The Washington, DC, Metropolitan Area Flight Restricted Zone; Interim Final Rule, Regulatory Evaluation, Regulatory Flexibility Determination, Trade Impact Assessment, and Unfunded Mandates Determination.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The long-term measures were just developed for the FY 2007 Congressional Justification; therefore, the program has not yet demonstrated adequate progress in achieving these goals.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Baselines for the annual goals will be established in FY 2005. Therefore, progress towards meeting these goals will not be evident until FY 2006 at the earliest.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Baselines for the efficiency goals will be established in FY 2005. Therefore, progress towards meeting these goals will not be evident until FY 2006 at the earliest.

Evidence: FY 2006 and FY 2007 TSA Congressional Justifications.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Because many of the components of the "Aviation Regulation and Other Enforcement" program are newly re-organized, they haven't been evaluated or compared to programs with similar goals.

Evidence: No evidence necessary.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: An audit of the US civil aviation security system was conducted by the International Civial Aviation Organization and results are still pending. The audit reviewed the security operations in place at JFK, which acted as a representative sample of US aviation security operations. In addition, several internal audits have revealed some findings. An internal affairs audit concluded that the compliance and canine programs have effective and efficient administrative controls in place, ensuring that information is accurately documented and processed. Further, a GAO report found that while TSA has made some improvements, systematic planning is needed to optimize resources.

Evidence: ICAO letter, MoU, and Background material; Compliance letter to OIG; GAO report 05-357T: "Systematic planning needed to optimize resources." GAO Report: "Increased federal oversight is needed."

Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: In most cases, TSA conducts analyses of costs, benefits, impacts and reasonable alternatives before issuing regulations. TSA has not yet developed guidance on the procedures used to evaluate if programmatic goals/benefits are achieved at the least incremental societal costs and maximize net benefits. However, it has done this in limited cases. For example, costs to the individual seeking FAA certification were minimized by establishing an expedited adminstrative process. The individual is afforded the right to appeal, the timeline for that appeal and for federal action on the appeal was designed to minimize cost to the individual.

Evidence: 68 FR 3762