Detailed Information on the
Transportation Security Administration: Transportation Vetting and Credentialing Assessment

Program Code 10003633
Program Title Transportation Security Administration: Transportation Vetting and Credentialing
Department Name Dept of Homeland Security
Agency/Bureau Name Department of Homeland Security
Program Type(s) Regulatory-based Program
Assessment Year 2006
Assessment Rating Results Not Demonstrated
Assessment Section Scores
Section Score
Program Purpose & Design 60%
Strategic Planning 75%
Program Management 43%
Program Results/Accountability 13%
Program Funding Level
(in millions)
FY2007 $93
FY2008 $171
FY2009 $173

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments

Fully incorporate best practices into the development of Secure Flight life-cycle cost & schedule estimates by FY2009: (update life-cycle costs & schedule estimates; demonstrate that the SF schedule has the logic in place to identify the critical path, integrates lower level activities in a logical manner, & identifies the level of confidence in meeting the desired end date).

Action taken, but not completed FY09 Fall Update: TTAC is currently analyzing the 12-step process outlined in the Cost Assessment Guide provided by the Government Accountability Office. The Secure Flight program is adjusting the Life Cycle Cost Estimate and working to demonstrate the program??s alignment with best practices detailed in the guide. The Lifecycle cost estimate will be completed and distributed by 9/30/09.

TSA will institutionalize strong program management principles and techniques to ensure accountability on behalf of contractors and staff. Furthermore, TSA will ensure staff have the proper skills to manage large capital investment programs by creating an action plan for filling skill gaps.

Action taken, but not completed This Improvement Action has been moved to Transportation Security Support.

Develop enterprise wide services as building blocks that can be combined and tailored to serve different populations: Establish a universal guidance for TTAC services by FY 2010 that will provide program offices interested in vetting with an "a la carte" menu of standardized services and costs for such things as enrollment, name based check (terrorism, criminal, and immigration), adjudication, and granting benefits (crential or access) per individual.

Action taken, but not completed FY08 Spring Update: TTAC has been developing the requirements for a universal fee rule & anticipates having a contract in FY08.This effort will involve an ABC study. It??s anticipated it will wrap-up by mid-09 & results will serve as a menu of standardized fees FY08 Q4 Update: TTAC will receive funding in 4th quarter of FY08 to further develop universal fee guidance.TTAC has developed strategies & plans on progress towards a ??1 Stop Shop?? enrollment. Plan initiating implementation by 9/30/08

TSA should take steps to promulgate a rulemaking for Alien Flight School and submit financial data supporting the new fee level. This financial data should be submitted by the end of the fiscal year.

Action taken, but not completed All financials have been completed, and draft rule is currently in TSA Office of Chief Counsel. Final rule expected by end of FY 2008. FY08 Q4: The draft rule currently remains in the TSA Office of Chief Counsel. The expected completion date to finalize the rule is during the 3rd quarter of FY10. AFSP used the completed financial data to support the new fee level for Category 4 flight training candidates in the FY09 Congressional Justification.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

By January 2007, TSA will develop a policy on the periodicity and procedures of reviewing regulations to determine if the program is maximizing the intended benefits at the least societal cost.

Completed TSA has adopted a routine of reviewing consistency in the basic security measures as each new rulemaking is undertaken. On a regular basis the Deputy Chief Counsel for regulations and staff meet with Transportation Sector Network Managements Regulatory and Economic Analysis to review progress and content of all rulemakings. Semiannually for the Regulatory Plan and Agenda a very thorough review is done by this team to see where outstanding issues may still need to be addressed.

By 2007, TSA will rebaseline Secure Flight, TWIC, and RT to ensure cost, schedule and performance goals are accurate and consistent with the program's goals.

Completed As reported in the FY 2006 Q4 update, TSA submitted the TWIC & RT rebaseline packages. TSA submitted the Secure Flight rebaseline package in December 2006. At that time, TSA received the Department??s Acquisition Decision Memorandum conveying DHS approval of the SF acquisition program baseline (APB) for cost, schedule, and performance at the FY 2008 $53M funding level. The memo was submitted to OMB as well as all of the supporting planning documents.

TSA will establish baselines and targets for its long-term, annual and efficiency measures in the FY 2008. TSA will begin demonstrating results against these performance measures in FY 2007.

Completed TSA established baselines and targets for four of the five performance measures. The remaining measure does not have established baselines and targets due to its association with a program that is not yet operational. TSA expects baselines and targets for performance measure associated with Secure Flight in FY 2008 when Secure Flight becomes an operational program.

TSA will show evidence that it is using acquisition vehicles that shift much or all of the risk away from the government, such as firm fixed price contracts. TSA will improve its contracting practices to ensure that minimum performance thresholds are documented and incentives and penalties are provides for good/poor performance. Significant improvement to shift to FFP contract will be demonstrated by January 2007. Furthermore, TSA will require the use of earned value management on most contracts no later than January 2007.

Completed TSA continues to move the risk from the Government to the contractor by moving toward fixed priced contracts and having better defined requirements. TSA implements earned value management in accordance with TSA Management Directive No. 300.11 Earned Value Reporting June 2006 and the DHS Earned Value Management Guidance Version 1.0 November 2006. New contracts issued for TTAC (that meet the appropriate earned value thresholds) in 2007 will require/incorporate earned value management reporting.

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Percentage of individuals undergoing a TTAC security threat assessment (STA)

Explanation:This measure indicates the what percentage of TTACs total defined population (656M) is receiving an STA. Estimated populations: SF - 650M (annually); CVP - 625K (annually); RT - 3M (cum); TWIC - 750K (cum); Hazmat - 1.5M (cum); AFSP - 15K (annually); and SIDA - 750K (annually)

Year Target Actual
2006 Baseline 100%
2007 100% 100%
2008 100%
2009 100%
2010 100%
2011 100%
2012 100%
2013 100%
Annual Output

Measure: Percentage of all commercial airline domestic flight passengers vetted. (Secure Flight Only)

Explanation:This is an indication of what percentage of commercial airline domestic flight passengers are being protected from potential terrorist activities.

Year Target Actual
2006 TBD Pgm Not Operational
2007 TBD No data
2008 TBD
2009 TBD
2010 TBD
2011 TBD
2012 TBD
2013 TBD
Annual Outcome

Measure: Ratio of false positives to total names processed.

Explanation:This measure assesses how well the program is restricting access to people who do (or do not) present a threat to the nation's transportation system.

Year Target Actual
2006 TBD Pgm Not Operational
2007 TBD .58%
2008 +/- 0.8%
2009 +/- 0.6%
2010 +/- 0.4%
2011 +/- 0.2%
2012 +/- 0.2%
2013 +/- 0.2%
Annual Efficiency

Measure: Security Threat Assessment (STA) cost per vetted individual.

Explanation:This measure ensures that the cost of performing security threat assessments remains within acceptable levels.

Year Target Actual
2006 Baseline $15.56
2007 $15.00 $15.00
2008 $14.50
2009 $14.00
2010 $13.75
2011 $13.50
2012 $13.25
2013 $13.00
Annual Output

Measure: Number of privacy adherence infractions

Explanation:This measure assesses the extent of production of the public's privacy by measuring the rate of compliance with privacy regulations, policies, and procedures as a function of vetting throughput.

Year Target Actual
2006 Develop Baseline 0
2007 < 5 0
2008 < 4
2009 < 3
2010 < 2
2011 < 1
2012 < 1
2013 < 1

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score

Is the program purpose clear?

Explanation: The Transportation Threat Assessment and Credentialing (TTAC) program consolidates the management of all vetting and credentialing programs within TSA. TTACs mission is to ensure that individuals engaged in various aspects of U.S. transportation do not pose a threat to national security or transportation security. The program consists of the following screening programs: Secure Flight (SF), Crew Vetting (CVP), Transportation Worker Identification Credential (TWIC), Registered Traveler (RT), Hazardous Materials Commercial Driver's License Endorsement (HAZMAT), and Alien Flight Student (AFSP).

Evidence: 1. Aviation and Transportation Security Act (ATSA) 2. Intelligence Reform and Terrorism Prevention Act 3. HAZMAT: Interim Final Rule (IFR) published by TSA on November 24, 2004 under 49 CFR part 1572. 4. AFSP: IFR published by TSA on September 20, 2004 in accordance with Section 612 of Vision 100.

YES 20%

Does the program address a specific and existing problem, interest, or need?

Explanation: Intelligence identifies a very real ongoing threat to transportation security as evidenced by the World Trade Center bombing in 1993, the Oklahoma City bombing in 1995, the USS Cole bombing in 2000, the attacks in Madrid in March 2004, in London July 2005, and the attacks on pipeline infrastructure in Saudi Arabia and Iraq in 2006. Since we cannot predict specific terrorist activity, the name-based threat assessment methodologies employed by the programs within TTAC to identify potential threats to the Nation's transportation system are a critical tool to strengthen the security of the transportation network.

Evidence: INTEL threat documents (Classified: "SECRET").

YES 20%

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Five of the six TTAC programs have distinct and unique mission requirements and no duplication exists with other government or private efforts. The Secure Flight which is being developed to replace the existing airline passenger prescreening process that is performed by U.S. air carriers themselves and some duplication of efforts will exist in the transition. However, this duplication is required to ensure the security of the aviation system until the program full operational capability.

Evidence: ATSA, Intelligence Reform and Terrorism Prevention Act, HAZMAT: Interim Final Rule (IFR) published by TSA on November 24, 2004 under 49 CFR part 1572, AFSP: IFR published by TSA on September 20, 2004 in accordance with Section 612 of Vision 100.

YES 20%

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The HAZMAT, CVP, and AFSP programs have been operational for more than a year and there is no evidence to support another design would have improved efficiency or effectiveness. However, SF, RT, and TWIC, which comprise a majority of program resources, are each in varying stages of development. The program design generally lacks an infrastructure that ensures strong program management and techniques to ensure accountability. The department has not followed a disciplined life cycle approach to develop these programs, thus major program shifts have occurred to address privacy and security issues as well as to establish an improved program management infrastructure. Because of these challenges these programs are currently updating key life cycle cost and program justification documentation, or in the case of SF undergoing a rebaseline of cost and schedule, to establish more robust program management moving forward.

Evidence: TTAC Weekly Intel Report, 9-11 Commission Report Executive Summary, TWIC Prototype Report, RT Pilot Program Initial Launch Metrics Evaluation, GAO reports on Secure Flight.

NO 0%

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: For the operational programs, TSA is effectively targeting intended beneficiaries by conducting threat assessments on nearly 2 million individuals last year. However, throughout the design and development phases for TWIC, RT and SF, they have experienced significant issues with defining and fixing cost, schedule, and performance requirements for the program. TSA has also relied heavily on high risk acquisition processes that are not performance-based in nature. Therefore, because of poor program design and a lack of sufficiently skilled staff, resources have not been targeted effectively in the design phase for these programs. TSA is actively addressing these issues by revising acquisition strategies to reach initial operational capability within acceptable cost and schedule thresholds.

Evidence: 1. SF GAO Report. 2. RT Pilot Program Initial Launch Metrics Evaluation. 3. TWIC Prototype Report and DHS-OIG Reports.

NO 0%
Section 1 - Program Purpose & Design Score 60%
Section 2 - Strategic Planning
Number Question Answer Score

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: TTAC has taken steps to establish long-term, outcome measures that would be representative of the entire program and focus on the risk reduced as a result of implementing program objectives. In the interim, the program has developed a long-term output goal to help gather data necessary to develop outcome measures. This output goal is percent of TTAC population receiving Security Threat Assessments (STA), which is intended to show the dramatic impact the program will have on the transportation sector by vetting nearly 700 million persons per year.

Evidence: FY 2007 and FY 2008 Congressional Justification and SF OMB-E300.

YES 12%

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The baseline and targets for the program's long-term measure is under development.

Evidence: FY 2007 and 2008 TSA Congressional Justification.

NO 0%

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: TTAC has developed a robust set of short-term performance measures, which directly supports the long-term measure. Current measures include average cost per security threat assessment, ratio of false-positives, and number of privacy infractions.

Evidence: FY 2007 and 2008 TSA Congressional Justification.

YES 12%

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Ambitious targets and baselines for the annual measures are under development.

Evidence: FY 2007 and FY 2008 Congressional Justification, draft measures working papers.

NO 0%

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: For the performance goals that partners are responsible for achieving, through MOUs and other agreements, the FBI, ICE, CIS, and state and local law enforcement agencies commit to specific costs and timelines for conducting threat assessments of individuals. These agencies provide data on these measures to ensure fees are established at levels appropriate to cover the costs of running the program. Furthermore, TSA is moving towards performance-based contracting and recently awarded a contract ties award fees directly program results by requiring earned value reporting and a performance evaluation plan.

Evidence: 1. HAZMAT MOA with FBI 2. AFSP/HAZMAT MOA with FBI (fingerprinting) 3. TWIC Inter Agency Agreement (IAA) with CBP 4. SF IAA with CBP 5. Performance-based contract

YES 12%

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Although relatively new programs, TWIC, RT and SF have undergone comprehensive audits by either GAO or another independent entity. These evaluations were of sufficient scope and quality to evaluate program effectiveness. Specifically, the SF report evaluated ten high-risk areas and made recommendations that need to be addressed before the program becomes operational. In addition, the RT evaluation focused heavily on the benefits provided to the traveler as well as improved TSA screening productivity as well as highlighting the success of the kiosks and biometrics.

Evidence: 1. SF - GAO Secure Flight Development and Testing Under Way, but Risks Should be Managed as System Is Further Developed (March 2005) and GAO Significant Management Challenges May Adversely Affect Implementation of Transportation Security Administration's Secure Flight Program (February 2006) 2. RT - Pilot Program Initial Launch Metrics Evaluation (August 2005). 3. TWIC - Prototype Report (August 2005) and draft DHS-OIG report.

YES 12%

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The TTAC programs annual and long-term performance measures are integrated in budget requests. The program's Congressional Justification also includes all direct and indirect costs needed to meet performance targets. Further, for the programs in development that are not fee-funded, tradeoffs in performance can be demonstrated when resource levels increase or decrease.

Evidence: 1. FYHSP 2. FY 2007 Congressional Justification 3. Program budget documents 4. DHS PMA BPI scorecard.

YES 12%

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: TSA is addressing all of the strategic planning deficiencies that have been identified for TTAC. Specifically, TSA is working to adopt develop baselines and targets for its annual and long-term performance measures.

Evidence: TSA Management Review (TTAC) template, draft performance measures.

YES 12%

Are all regulations issued by the program/agency necessary to meet the stated goals of the program, and do all regulations clearly indicate how the rules contribute to achievement of the goals?

Explanation: The operational programs (AFSP, Hazmat, and CVP) under TTAC have each issued necessary rules to clearly outline the goals of the programs and how the data being collected will be utilized to achieve the goals of the programs. The TWIC program is preparing for rollout to the nations ports and just recently released its Notice of Proposed Rule Making (NPRM) on May 10, 2006. TSA articulates the goal of the TWIC rulemaking is to meet requirements defined by the Maritime Security Act, which requires that credentialed merchant mariners and workers with unescorted access to secured areas of vessels and facilities be subject to a security threat assessment and receive a biometric credential needed to access secured areas. RT and SF are currently in the rulemaking process and plan to issue their respective regulation/guidance in the future.

Evidence: 1. Hazmat NPRM 2. TWIC NPRM 3. AFSP Interim Final Rule (IFR)

YES  %
Section 2 - Strategic Planning Score 75%
Section 3 - Program Management
Number Question Answer Score

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The timely receipt of timely performance information from critical stakeholders is very important to the success and performance of all TTAC programs. Each program has its own processing time requirements that it must comply in order to meet specific performance requirements regarding time. CVP requires data from the airlines on passengers no later than one hour before flight time. The Hazmat and AFSP each have a 30-day turnaround requirement after receiving information from the applicant to process the application. While in the planning phases, SF, TWIC, and RT will also collect data from stakeholders that will have time-based performance requirements in order to operate effectively. All of these programs do or will depend upon airlines, other government entities, or Trusted Agents to gather source data and provide it to TTAC in order to meet established processing standards. Even after the receipt of the source data all TTAC programs are still dependent upon other key government partners such as FBI, ICE, CIS, CBP, etc. to provide timely receipt of unique datasets or fingerprint processing to perform security threat assessments in a timely manner. TSA uses this performance information to set program parameters, adjust resources and/or fee levels.

Evidence: 1. Numerous agreements with other government and non-government entities. 2. AFSP Customer Satisfaction Survey

YES 14%

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: TTAC is beginning to award performance-based contracts whenever practicable. By incorporating language regarding cost, schedule and performance the program intends to make program partners more accountable for achieving program success. However, to date the program has only awarded one performance-based contract (awarded March 2006) out of 26 contracts currently issued. Since this one contract is so large it's important to note that, in dollar terms, 52 percent of the programs total contract value is associated with performance-based contracts. Furthermore, the program does not clearly define and quantify performance standards to ensure accountability for its managers.

Evidence: 1. O&M contract (performance evaluation plan). 2. Secure Flight O&M contract.

NO 0%

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Since many of the TTAC program are currently undergoing baseline reviews, several of the programs do not have a program plan that includes schedules for obligations. While in FYs 2004 and 2005 resources for SF and CVP obligated funding in excess of 92% to fund pilot program, the program is expecting large unobligated carryover balances in FY 2006 because of program delays and baseline changes. The AFSP and Hazmat programs are both operational and fee-funded each program is required to monitor fee receipts and obligations very closely to ensure these programs operate within their fee driven cash flow availability.

Evidence: 1. CAS financial reports

NO 0%

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The organization is relying more on full and open competitions, performance-based contracts, and the use of earned value management. TSA recently awarded a firm fixed price contract for SF operations and maintenance. TSA has also developed an efficiency goal to reduce the cost of each threat assessment. The program is currently developing a baseline and targets for this measure.

Evidence: TSA contract management, SF O&M contract, TWIC acquisition plans, FY 2008 Congressional Justification.

NO 0%

Does the program collaborate and coordinate effectively with related programs?

Explanation: TTACs biggest success story has been the coordination between SF and CBP. Since CBP was already communicating with all major airlines, with the exception of Southwast, to collect data for their Advanced Passenger Information System (APIS) their system became a logical starting point for SF which is required to communicate with all air carriers flying domestically in the U.S. The organizations work together to address issues such as messaging, data format, and automation challenges associated with smaller carriers to create a system that presents a single interface to the airlines to provide the Federal government the data it needs. To gain access to the government datasets utilized in performing STAs the organization had to establish agreements with other government agencies (i.e. DOS, FBI, ICE, CPB, CIS, etc.) that also require continual interaction to ensure the integrity of the data being accessed. The organization also collaborates regularly with major program stakeholders such as (TSC, CBP, FAA, airlines, etc.) to determine the best approach for addressing specific programmatic issues concerning IT development, connectivity, and changes in stakeholder business practices.

Evidence: 1. SF IAA with CBP 2. HAZMAT MOA with FBI 3. AFSP/HAZMAT MOA with FBI (fingerprinting) 4. TWIC Inter Agency Agreement (IAA) with CBP

YES 14%

Does the program use strong financial management practices?

Explanation: The draft FY 2005 Audit of Financial Statements revealed three material weaknesses in the areas of financial reporting; undelivered orders, accounts and grants payable, and disbursements; and budgetary accounting.

Evidence: FY 2005 Audit of Financial Statements

NO 0%

Has the program taken meaningful steps to address its management deficiencies?

Explanation: TTAC management representatives meet weekly to discuss program status where issues are addressed and, where possible, quick resolutions found. TTAC, and/or parts of its former self, have also participated in TSA's Management Control Objective Plan (MCOP) for both FY 2005 and FY 2006. The current MCOP addresses such management areas as information and physical security, HR personnel actions, time & attendance, acquisition procedures, and contract management. The MCOP is reviewed periodically and a year-end summary report is prepared outlining our success/failure in achieving our plan. TSA is developing a corrective action plan to address how the program intends fix the weakness and ensure that it's not repeated. Furthermore, the Secure Flight program is addressing all 10 areas of interest identified by the GAO before implementation so it should adequately address management deficiencies.

Evidence: TTAC Management Control Objective Plan (MCOP) for FY 2006.

YES 14%

Did the program seek and take into account the views of all affected parties (e.g., consumers; large and small businesses; State, local and tribal governments; beneficiaries; and the general public) when developing significant regulations?

Explanation: All TTAC program regulations have undergone the full notice and comment period required by the NPRM process before rules become final. During this phase TSA coordinates with stakeholders to gain an understanding of how the regulations/guidance will impact their operations. Furthermore, all efforts are made to try to mitigate the impact to stakeholders and economic analysis is performed to also gain an understanding of the economic impact of the rule issuance. TSA thoroughly discusses these comments in the preamble and throughout the final regulation to explain which comments have and have not been accepted and why.

Evidence: 1. Hazmat NPRM 2. TWIC NPRM 3. AFSP IFR

YES  %

Did the program prepare adequate regulatory impact analyses if required by Executive Order 12866, regulatory flexibility analyses if required by the Regulatory Flexibility Act and SBREFA, and cost-benefit analyses if required under the Unfunded Mandates Reform Act; and did those analyses comply with OMB guidelines?

Explanation: TSA prepares regulatory evaluations of all rulemakings. These regulatory evaluations cover the requirements of Executive Order 12866, regulatory flexibility act, and Unfunded Mandates act. Each evaluation was reviewed by OMB OIRA as a part of the clearance process and all recommended corrections were accomplished before publishing to the public docket.

Evidence: 1. Hazmat NPRM 2. TWIC NPRM 3. AFSP IFR

YES  %

Are the regulations designed to achieve program goals, to the extent practicable, by maximizing the net benefits of its regulatory activity?

Explanation: The regulations are designed to maximize the security benefit while giving full consideration to economic consequences. TSA does not use a cost benefit ratio but both benefits and costs are full discussed in the regulatory evaluation. TSA maximizes the benefits giving full consideration to qualitative as well as quantitative issues. NEED for example

Evidence: 1. Hazmat NPRM 2. TWIC NPRM 3. AFSP IFR

YES  %
Section 3 - Program Management Score 43%
Section 4 - Program Results/Accountability
Number Question Answer Score

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: TTAC is currently developing its baseline for the long-term goal; therefore, the program has not yet demonstrated adequate progress.

Evidence: 1. FYHSP 2. SF OMB-E300

NO 0%

Does the program (including program partners) achieve its annual performance goals?

Explanation: TTAC is currently developing its baseline for its annual goals; therefore, the program has not yet demonstrated adequate progress.

Evidence: 1. FYHSP 2. SF OMB-E300

NO 0%

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: TTAC is currently developing its baseline for its efficiency goal; therefore, the program has not yet demonstrated progress in achieving its goals.

Evidence: 1. FYHSP 2. SF OMB-E300

NO 0%

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The SF and CVP programs have worked with the FBIs Terrorist Screening Center (TSC) to compare the effectiveness of screening subjects by matching against watch-list records. Through this relationship TTAC has been able to compare the performance of its vetting engines against other name-matching systems to determine effectiveness. The RT program has worked closely with Customs & Border Protection and Canadian Immigration officials examining the performance of Canada's Nexus Air program, which is designed to simplify border crossings for pre-approved travelers between the U.S. and Canada using biometrics. In addition, the RT program has also examined the performance of the Iris Recognition Immigration System (IRIS) at Heathrow Airport in London and the Privium program at Schiphol Airport in Amsterdam. In both cases, these operations were designed to "fast track" immigration checks at these airports using iris scans as the biometric identifier. The RT program was particularly interested in the biometric results of the Privium program since it utilized a semi-fixed camera for taking the iris scans.

Evidence: 1. Vetting Analysis Summary Report 2. Passport Control Using Iris Recognition: Heathrow Passenger Operations Experience - Results and Lessons Learned.


Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The SF program has been subject to a GAO audit for the past 2-years to ensure that the program is achieving its stated goals. Both reports were supportive of program goals; however, the report was critical in noting that much work remains in the areas of redress, privacy, security, oversight, and life-cycle costs. GAO noted that these areas must be addressed before the program becomes operational. Some progress has been made to address these concerns.

Evidence: 1. Independent evaluations of TWIC and RT. 2. SF GAO Audit


Were programmatic goals (and benefits) achieved at the least incremental societal cost and did the program maximize net benefits?

Explanation: The CVP, AFSP, and Hazmat programs have only been operational for a short period of time (each less than 2 years) so no program analysis has been performed to observe the effects of these regulations. While some benefit can be demonstrated, TSA has not developed guidance on procedures used to evaluate if programmatic benefits are achieved. Since the HAZMAT rule was an IFR, TSA is reviewing public comments as part of finalizing the rule. Numerous program modifications have been made to allow for flexibility which reduces burden. TSA balanced issues of federalism raised by the states against pure economy of scale issues raised by other members of the public. The costs were minimized within these other constraints.

Evidence: The AFSP, Hazmat, and recently released TWIC rule making documents each include a reference to OMB Circular A-25 requiring a review of the fee at least every 2-years. Since Hazmat and TWIC each provide a a benefit that only covers a maximum 5-year period these regulations/procedures will be revisited automatically within that time period before indidivuals reapply for extension of the same benefit. 1. Hazmat NPRM 2. AFSP IFR 3. TWIC NPRM

Section 4 - Program Results/Accountability Score 13%

Last updated: 09062008.2006SPR