Detailed Information on the
Customs and Border Protection: Automation Modernization Assessment

Revisit the performance measure "Percent of total duties and fees paid through Periodic Monthly Statements (PMS)" in terms of increasing participation and revising out-year targets.

Focusing on performance goals that were not met in 2007, document progress for newer measures, and post existing independent evaluations that demonstrate how programs are achieving results. Implement an outreach strategy engaging industrial communities and companies to achieve future targets, measuring targeted top duty-paying companies against periodic payment participation.

Focusing on maintaining performance standards and revisiting/updating out year performance targets.

Measure: Percent of CBP workforce using ACE functionality to manage trade information.

Explanation:The number of Customs and Border Protection people using Automated Commercial Environment (ACE), compared to the targeted adoption rate shows that internal personnel have easier, timelier, access to more complete and sophisticated information than in the past.

Measure: Number of Border Patrol Agents trained in rescue and emergency medical procedures.

Explanation:This measures the extent to which ACE is made available to and used by members of the trade community (importers, brokers, carriers, etc.) to process and manage trade - related information.

Measure: Percent of total duties and fees paid through Periodic Monthly Statements (PMS)

Explanation:Periodic Monthly Statement (PMS), introduced in June 2004, makes it possible to view filings or make payments by account. This feature of ACE allows users to consolidate periodic daily statements and pay once a month. Increased usage of PMS will demonstrate the additional flexibility in working capital management for the Trade and reduced processing time from traditional transaction-based payment. Data shown for 2006 are year to date as of 6-15-2006.

Measure: Number of security-focused intensives designated for exam

Explanation:This measure includes all shipments designated for exam, regardless of their ATS score. It reflects the number of shipments designated for document review and those requiring examination. As the system becomes more sophisticated, there should be a trend to greater acceptance of shipments with higher risk scores to be examined. As of 6/15/06, the measure and associated targets are unofficial and preliminary.

Measure: Percent of cargo risk mitigated

Explanation:This outcome measure is defined as the percentage of shipments with ATS scores between 150 and 189 at date of arrival that are examined.

Measure: Percent of entry summaries for which there is no further Trade activity required (i.e., entry summary by-pass)

Explanation:Entry summary bypass is defined as the number of entry summaries bypassed (no docs required) divided by total # of entry summaries. Through better targeting and more automation, there will be fewer "document reviews" at entry and summary. The "file and go" feature will save both CBP and the trade time and money and free up resources for mission essential work. As of 6/15/06, the measure and associated targets are unofficial and preliminary. Data shown for 2006 are year to date as of 6-15-2006.

Measure: Percent reduction in truck processing time

Explanation:Introduced in 2005, the e-Manifest Truck feature of ACE replaces ACS as the system used to process shipments at land border ports of entry. Processing times using ACE are compared to ACS baseline data collected at select ports of entry. This measure reflects transaction time improvements over ACS. Data shown for 2006 are year to date as of 6-15-2006.

Measure: Percent of system-generated high risk cargo security targets held.

Explanation:This measure includes all shipments designated for exam, regardless of their ATS score. It reflects the number of shipments designated for document review and those requiring examination. As the system becomes more sophisticated, there should be a trend to greater acceptance of shipments with higher risk scores to be examined. As of 6/15/06, the measure and associated targets are unofficial and preliminary.

Measure: Percent of system-generated cargo security risk mitigated.

Explanation:This outcome measure is defined as the percentage of shipments with ATS scores between 150 and 189 at date of arrival that are examined. As of 6/15/06, the measure and associated targets are unofficial and preliminary.

Measure: Percentage of mainframe capacity that is used

Explanation:Enterprise servers (mainframes) and associated systems should not operate at or near full capacity. Doing so would lead to persistent service failures. Operating at 80% or less of capacity is highly desirable.

Measure: Percentage of network availability

Explanation:The CBP network provides the basis for linking all IT systems for communications and access to mission critical systems. High levels of system availability are needed to accomplish CBP's mission.

Measure: Number of upgraded mainframes and associated systems

Explanation:Many mission critical systems are hosted by mainframe computers and associated storage and communication systems. With workloads growing at 20-25% a year, mainframe capacity is periodically increased.

Measure: Number of supported CBP users

Explanation:This is a determinant of the workload on the COPPS infrastructure.

Measure: Telecommunications costs per user

Explanation:Average cost associated per CBP user to provide telecommunication services.

Measure: Total duties and fees paid by Periodic Monthly Statement (PMS), per month

Explanation:Periodic Monthly Statement (PMS), introduced in June 2004, makes it possible to view filings or make payments by account. This feature of ACE allows users to consolidate periodic daily statements and pay once a month. Increased usage of PMS will demonstrate the additional flexibility in working capital management for the Trade and reduced processing time from traditional transaction-based payment. Data shown for 2006 are year to date as of 6-15-2006.

Is the program purpose clear?

Explanation: U.S. Customs and Border Protection (CBP) is the guardian of America's borders. Our mission is to safeguard the homeland at and beyond our borders, balancing protection of our Nations' economic security through identifying threats to our security from terrorists and the instruments of terror while facilitating lawful international trade and travel. Automation Modernization is CBP's response to the explosive growth in trade volume and is a much-needed tool to combat illegal activities and terrorist threats to the United States. The Automation Modernization Program is composed of the Automated Commercial Environment (ACE), the Critical Operations Protection and Processing Support (COPPS) and the International Trade Data System (ITDS). The ACE cccomponent of the Automation Modernization program, is the modernized U.S. trade processing system designed to consolidate and automate border processing to significantly enhance border security and foster our Nation's economic security through lawful international trade and travel. COPPS provides the communications, hardware and software infrastructure that supports the complete array of mission critical CBP systems. The ITDS is an information system that serves as a government data collection and distribution facility, to provide the trade and public with a single window for submitting data and making payments required by all federal government agencies that regulate import and export transactions for international trade.

Evidence: The development of ACE is in alignment with Title VI of the North American Free Trade Implementation Act (NAFTA), which emphasizes electronic processing and self-compliance in the Customs Modernization Act. Title VI of the North American Free Trade Agreement Implementation Act (NAFTA) [P.L. 103-182, 107 Stat. 2057] - Subtitle B - The National Customs Automation Program (NCAP) became law on December 8, 1993. Title VI has often been referred to informally as the "Customs Modernization Act" or "Mod Act." This legislation is the primary business driver and legal foundation for the CBP Modernization efforts. The Mod Act emphasizes electronic processing and self-compliance as the preferred method for handling commercial importations and also directs CBP to pursue automation programs. The Act also provides the foundation for redesign of the trade compliance process and the strengthening of selectivity and targeting systems. Strengthening the selectivity and targeting system is a key requirement for border security and expediting legitimate trade. Additionally ACE supports the eight major CBP business areas: release processing, entry processing, finance, account relationships, legal and policy, enforcement, business intelligence risk, and is in direct alignment with the President's Management Agenda (PMA). To maintain this alignment, ACE uses e-business technologies whenever possible and is governed by citizen needs. ACE is being implemented through a series of software releases. Currently, Release 4, of eleven planned releases, is in progress. At each staged release, new ACE functionality will replace thecomparable legacy, Automated Commercial System (ACS) functionality. ACS applications will be retired as follows: Sea Automated Manifest System in 2008; Rail Automated System in 2008; Air Automated Manifest System in 2009; Cargo Selectivity System in 2009; Border Cargo Selectivity System in 2009; and the Automated Export System in 2010.

Does the program address a specific and existing problem, interest, or need?

Explanation: The U.S. has approximately 6,900 miles of border, 5,000 with Canada and 1,900 with Mexico, to protect. On a typical day, CBP personnel process 1.1 million passengers and pedestrians, approximately 720,000 aliens; 64,000 truck rail and sea containers; 2,600 aircraft; 365,000 vehicles; and 75,000 merchant entries coming into the U.S. By design, ACE is being implemented to expedite and secure the trade process, with the goal of facilitating trade movement through more effective account management and increased border security. CBP is modernizing its business processes, information technology systems, and infrastructure to support its homeland security mission and sustain its global leadership position in the years to come. ACE is CBP's response to the explosive growth in trade volume and will replace and upgrade the current capability that exists in the Automated Export System (AES) and the Automated Commercial System (ACS).

Evidence: Within the Department of Homeland Security (DHS), ACE provides a much-needed tool to combat illegal activities and terrorist threats to the United States. ACE is CBP's modernized import and export processing system. The program facilitates the movement of legitimate trade through more effective trade account management and strengthening border security by identifying import and export transactions that could pose a threat to the United States.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Automation Modernization program manages compliance and enforcement of laws related to terrorist enforcement and trade compliance at the U.S. borders and beyond. There are no private or other government entities that are able to provide the required IT infrastructure. Automation Modernization provides thetechnology infrastructure delivering mission critical systems to the first line of defense in protecting our Nation's borders, CBP's front line personnel . CBP's systems give immediate access and availability to CBP officers and Border Patrol agents providing critical components to CBP's multi-layered defense at our nation's borders.

Evidence: COPPS consolidates nearly all CBP information technology infrastructure into one program area. This infrastructure provides all CBP personnel, and representatives from other government agencies, with access to CBP's mission critical systems, facilitating enforcement of trade and law enforcement law. The functionality and processing capacity is unmatched by any other system. For example, it has the largest server environment in North America; the largest transactional database in the world; the largest volume of data processed through the middleware infrastructurein the world , and the mainframes supports over 100,000 users, both within and outside of CBP. Although additional funding is transferred, CBP's infrastructure is also the foundation for delivering significant functionality of the US VISIT program. The data network infrastructure provides connectivity from appropriate Other Government Agencies (OGA's) to a variety of data captured and maintained at CBP's National Data Center (NDC). One of the major mission critical systems that COPPS infrastructure supports is the Container Security Initiative (CSI). The program methodology is endorsed as a "best practice" for securing containers and cargo against terrorism, by the World Customs Organization's (WCO) Framework of Standards to Secure and Facilitate Global Trade. Implementing WCO standards throughout the world supports the strategic goals of preventing terrorism and protecting the country.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Automation Modernization provides CBP a cost-effective and integrated automated IT system to efficiently process people, goods and merchandise entering the United States. The program is constantly evaluating alternative ways to develop planned releases in the most cost efficient and effective way possible. The ACE Cost Analysis (CBA) was last reworked in 2005 to ensure that ACE was still the most cost efficient and viable alternative. ACE was chosen to fulfill the stated objectives outlined in NAFTA and the Mod Act. ACE seeks to provide a more effective and efficient process for cargo traffic by using a to both CBP and the trade community of importers, exporters, and carriers. ACE focuses on post-entry compliance, using a focused assessment to enforce trade laws.

Evidence: Under the Mod Act, importers are required to use "reasonable care" as established by accepted "reasonable care guidelines" to meet importation requirements. This includes using non-customs personnel such as brokers, attorneys and other consultant specialist to meet customs trade requirements. Development and deployment of ACE is being implemented in phases. CBP has a formal program that tracks recommendations in response to GAO and OIG audit recommendations. Within the Quarterly Report to Congress, the Office of Information and Technology (OIT) outlines the progress made and steps to be taken to actively address open GAO recommendations.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: ACE's program plan and expenditure plan are reviewed and updated on an annual basis. These documents are prepared each year and consider new threats, current funding availability, and requirements identified by stakeholders (through TSN meetings, and outreach by ACE ambassadors). These documents, as well as quarterly reports, are prepared and submitted to the U.S. Department of Homeland Security (DHS), the Office of Management and Budget (OMB), the General Accounting Office (GAO), and Congressional staff. The ACE program plan was significantly redesigned after September 11, 2001, ensuring that functionality and program priorities were refocused to address terrorism threats. TSN meetings are conducted on an on-going basis to ensure continued dialogue with stakeholders from the trade. In addition, there are regular meetings of Other Government Agencies (OGA) stakeholders and requirements submitted and prioritized by the ITDS board of directors. Ongoing Independent Validation and Verification (IV&V) is conducted on individual release ACE cost estimates, as well as the program, and are reported to OIT management. Requirements are reviewed annually and allocated to ensure continued operations and maintenance of CBP's information technology infrastructure, striving to maintain systems performance given increased processing demands. Priority is given to ensuring continued network and systems availability and accessibility by all domestic and international CBP personnel at all ports of entry and border stations, consistent with the CBP goal of providing "One Face at the Border." Annual operations and maintenance requirements include, among other things: system and network monitoring and trouble shooting at CBP field locations, as well as at the NDC; hardware and software licensing agreements based on anticipated capacity requirements; and necessary network and systems help desk, Local Area Network (LAN), and security support. Systems availability and performance targets for automation modernization spending are identified in the DHS FY2006 Congressional Justification. Three performance goals contain measures that are directly dependent on the continued and immediate availability of mission critical systems to screen and target passengers, trucks, cargo, boats, either prior to, or upon their arrival at our national borders. The performance goals include: Improving the targeting, screening and apprehension of high-risk international cargo and travelers; Preventing potential terrorists from crossing into the US; and Improving the threat and enforcement information available to decision makers from legacy and newly-developed systems. The measures under each of these goals are dependent on the information technology infrastructure being immediately available. Programs such as CSI, that have extended the US boundaries outward, have put further dependencies and demands to have systems available at all times, without any scheduled downtime. In addition, we continue to look for efficiencies to support a larger CBP workforce, as well as other government agencies, that access CBP systems. OIT continually monitors the network and systems to minimize downtime and optimize performance.

Evidence: The automation modernization program's investment in ACE effectively addresses its beneficiaries' needs by hosting Trade Support Network (TSN) Conferences with stakeholders from the trade. The TSN is comprised of trade officials representing the breadth of the trade community: trade associations, importers, brokers, carriers, sureties and others. The TSN is conducted to discuss specific business processes, and to develop user requirements for each stage of ACE development. Committees are formed and disbanded in parallel with stages of ACE development. Some committees currently active are Account Management, Revenue, ITDS, and Supply Chain Security. The TSN helps to assure that all end user needs are being met. In addition, CBP employees throughout the U.S. are trained as ACE ambassadors. ACE ambassadors hold local outreach events in their communities to inform and prepare CBP employees and members of the trade community for ACE and Modernization. The events can include a presentation, a portal demonstration, a question and answer session, and a group discussion to promote a better understanding of ACE. These ambassadors transmit information back to the Cargo Systems Program Office (CSPO). In addition, an ACE program plan, expenditure plan, and quarterly reviews are prepared which involve a review of current status, requirements, and costs to ensure that program priorities are addressed. Operations and maintenance requirements, funded primarily by COPPS, are submitted, prioritized and aligned annually to ensure that CBP's NDC, the extensive field data network, as well as all of CBP's mission critical trade, enforcement, financial and management systems, are available and accessible to all employees. Systems are monitored and reengineered as necessary to respond to increased demands, while also trying to ensure systems availability 24 hours a day, 365 days a year. Funding supports CBP's efforts to operate, maintain, engineer, and support nearly all of CBP's information technology hardware and software (includes local area networks, wide area networks, routers, mainframes, servers, etc.) that are deployed to, and/or accessed at, 317 ports of entry (including land borders and international airports), 20 Border Patrol Sectors, and 33 border patrol check points. COPPS strives to provide continued availability of critical analytical tools/queries that are essential to CBP officers and BP agents in meeting the daily demand of screening: nearly 1.2 billion passengers and pedestrians; 69,000 truck, rail and sea containers; 235,000 incoming international air passengers; 333,000 privately owned vehicle; as well as collecting $81 billion in fees, duties and tariffs. Systems capacities, processing, and networks are monitored constantly, and funding is adjusted as necessary to: minimize network outages, maximize systems performance, provide necessary equipment upgrades when required, ensure the security of critical CBP data, and provide necessary field engineering and operations support.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Long-term performance measures that focus on outcomes are included in the PART analysis and in the Future Years Homeland Security Program (FYHSP). Currently, the Automation Program aligns to four CBP goals: 1. Preventing Terrorism at ports of entry 2. Facilitating legitimate trade 3. Protecting America and its citizens 4. Modernizing and managing ACE performance measures are in direct alignment with DHS and CBP strategic goals. ACE software releases satisfy user requirements that are in alignment with ACE performance goals as well as higher-level organizational strategic objectives. CBP strategic goals are further broken down into strategies and objectives to ensure accurate alignment at a more granular level. Performance goals also link to CBP benefits as outlined within the current ACE Cost Benefit Analysis. Long-term performance measures are detailed in the Performance Reference Model (PRM) section of the ACE OMB 300 submission, PART, PAR and FYHSP to track actual against planned performance.

Evidence: Automation Modernization's success in meeting these goals is illustrated in measures 2, 9, and 11. These measures reflect the purpose of the program and facilitate legitimate trade by providing CBP officers with tools to manage trade information. Long-term measures are employed to examine such outcomes as access to automated trade accounts, providing the most up-to-date information to law enforcement, and ensuring that the whole of CBP has the network availability needed to do their jobs. ACE is currently developing Federal Enterprise Architecture (FEA) aligned performance measures that articulate both the effectiveness and efficiency of ACE. Measure 2, which examined the percent of trade accounts with access to ACE, set its first target in 2005. As this measure was adopted, we discovered the first year target was too ambitious and unattained. Thus far in FY 2006, ACE is on pace to meet the 6 % of trade accounts with access to ACE target in support of Goals 2 and 4. The increased rate of trade participation is a result of creating non-portal accounts which allows importers to pay duties and fees monthly through a broker without having to first establish an ACE portal account; completing an outreach campaign to the chief financial officers of top-duty paying companies; and continued general outreach efforts with the trade community. Measure 7 shows the early results of implementations of ACE. As of June 2, 2006, the effect of ACE on truck processing time at land border ports continued to vary by port. In the high-volume port of Detroit, Michigan, trucks are processed 26.7 percent faster since the introduction of ACE. In Port Huron, Michigan, and El Paso, Texas, trucks are also processed faster by 1.1 percent and 4.9 percent, respectively. In Nogales, Arizona, and Pembina, North Dakota, truck processing times have increased by 25.5 percent and 5.3 percent, respectively, since the introduction of ACE. CBP anticipates the impact of ACE on truck processing time will continue to vary by port depending on the pace of truck processing dictated by a port's traffic volume; the extent to which ACE modifies a port's workflow procedures; and the extent to which ACE further strengthens cargo screening by reducing paperwork, providing shipment information sooner to primary officers, and enabling officers to increase their focus and time spent on risk assessment. Overall, CBP expects that the deployment of ACE at land border ports, combined with the forthcoming requirement to submit time-saving ACE e-Manifests, will result in widespread efficiency and qualitative gains for CBP and the trade community. Additionally, ACE is implementing new measures (measures 4, 5, 6, and 8; baseline data in 2006), which will assess the successful use of enhanced automation by tracking the percentage of duties, taxes, and fees electronically collected from trade accounts. These new measures will encompass Entry Summary, Accounts and Revenue, Screening and Targeting, and Cargo Control Release. Each set will develop outcome/effectiveness measures for both CBP and Trade, and will be supported by lower level output and input measures, which will illustrate efficiency. Measure 5 quantifies the outcome ACE will achieve in preventing terrorism at ports of entry. Performance measure 8 identifies a major savings to the customer, benefiting the trade community. This shows the percent of paperwork reduction, which is a major benefit of the effort from ACS to ACE. As more ACE software releases are deployed, manual processes will be automated, resulting in time and cost savings. Measure 7 illustrates how filing electronic manifests will allow quicker processing time at the borders and facilitate legitimate trade.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Automation Modernization's performance goals have established targets that are scheduled to occur within the next six years, as is illustrated by measures 2, 3, 4, 7, 8, and 14. Actual results are compared against these targets to assess ACE's performance. Implementation of ACE systems programming modules began testing in December 2004. Baselines are mapped from the previous legacy system to assess modernized ACE against previous capabilities. Targets have been quantified to assess performance, while being compared against the former legacy system baseline.

Evidence: Actual results of tracking the percentage of duties, taxes, and fees processed electronically collected by ACE from trade accounts facilitates legitimate trade to date are illustrated in performance measure 3. One of ACE's most aggressive targets is to have 100% of trade accounts have access to ACE to manage trade information by 2010, as illustrated in measure 2. Additionally, ACE has expanded to over 2,300 accounts with trade partners representing 30 percent of the total annual duties paid. Measure 3, shows the increase to 25 percent of the total monthly duties/fees collected by CBP paid by current periodic monthly payments, resulting in increased collections from $85,000 per month to $513 million per month, one of the clear cost benefits of the Periodic Monthly Statements. One of the ACE program's most agressive targets is for 100 % of the trade community to access ACE to manage trade information by 2011.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ACE is included in the strategic objective to modernize critical information systems and limiting technologies. The strategic performance measures are tracked quarterly through operational plans, and measured annually to determine the effectiveness of the timely delivery of planned systems and technology deployment. ACE has recently reviewed its performance measures to insure alignment with CBP and DHS strategic plans. This effort has identified an at-large set of more than 60 measures that track new functionality and functionality subsumed by ACE. These measures are in direct alignment with applicable strategic plans and support the phased release approach of ACE. As ACE is deployed and new functionality is available, performance goals are tracked to measure the effectiveness of the release. These measures are designed to capture how effective ACE is and how well it is being utilized in meeting the programs long-term goals. The success of ACE is dependent on the success of each release, in order to create a truly functional system. The ACE Performance Reference Model (PRM) is designed to measure each release or functionality on an annual basis to ultimately measure the true success of ACE as a system.

Evidence: ACE has two annual performance measures, 1 and 6. Measure 1 reflects the percentage of applicable staff internally using ACE to manage trade. Over the next six years, CBP expects this number to rise until all applicable internal users are utilizing ACE. Measure 6, currently being collected as a baseline measure this fiscal year, will demonstrate cost savings to CBP through more efficient, automated processing. As manual processes are replaced by automation, CBP will realize greater cost savings while striving to meet the CBP goal of facilitating legitimate trade and travel. Each of these measures has a quantified baseline, planned improvements to the baseline will report actual results in the FY08 business case submission. Additionally, ACE provides 17 tracked performance measures in the ACE Quarterly Report to Congress. Annual measures illustrating the success of COPPS, the program that provides the majority of data communications, hardware and software infrastructure necessary to deliver CBP's mission critical systems, supports the long-term goal included in measures 10 and 13. Measure 10 shows COPPS success in maintaining mainframe capacity so that the continued needs of the agency are met. Measure 13, the number of CBP users supported by the COPPS infrastructure shows the increasing demand being met. The target beginning in 2003 has quickly doubled in just two years and is expected to increase significantly over the next five years. The projected growth in users reflects the increasing mission requirements that results in access to an expanding infrastructure base. Expansion of system resources is being planned and implemented to meet the expected increase in resource utilization.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Ambitious targets can be found in measures 1, 2, 3, 5, 10, 13 and 15. These measures show the expansion of Automation Modernization through ACE and COPPS.

Evidence: Measure 1 shows a very ambitious targeted goal of having 100% CBP participation by 2010, with rapid progression of targeted outcomes from 2005 -2010. In measure 10, the ambitious target may not be quite as obvious, since the target remains the same each year. However, maintaining the correct level of capacity requires constant monitoring in an organization the size of CBP, especially given the diversity of system requirements and user needs. Measure 13 shows the expected target-number of users to be supported over the next five years. Supporting a system of this size requires extensive strategic and implementation planning. Measure 5 is setting a baseline measure in 2006 of the risk for shipments through a set of selected criteria and other anomalies. Once the system generates a selection of shipments designated by the system for general exam, a CBP officer will decide whether or not to inspect that shipment's cargo. The higher the score a shipment receives, the more it warrants inspection by an officer. ACE will increase the number of examinations of high-risk cargo, resulting in a higher percentage of cargo risk mitigated. The targets for this measure become more ambitious as ACE capabilities are more fully implemented, demonstrating that ACE is performing to meet its established intent. The recently developed the PRM as the highest set of measures for ACE. A subset of these measures is in the ACE OMB 300 Submission, the ACE Desired Business Results (DBR), DBR Objectives and Performance Measures, v17.0, and the OIT Strategic Plan. The highest-level ACE measures are reported in the PART and the Future Year Homeland Security Program (FYHSP) as is illustrated by measures 1, 2, 3, 4, 5, 6, 7 and 8. The supporting process and activities and technology measures are housed within the PRM. This process will allow management to view ACE performance at a high level and then look at a more granular level to determine why ACE is achieving a specific level of performance. Using this method, managerial and budgeting decisions can be determined from performance evaluation.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: CBP is responsible for the implementation of the ACE program and works in partnership with a contractor, the ACE Support Team, for development. Both parties adhere to a common set of shared principles, and each party has specific interdependent responsibilities as is outlined in the contract language. Other contractors work in concert with CBP and the ACE Support Team to perform other essential program elements for the delivery and deployment of ACE. For example, cost estimates are compared against independent cost estimates to establish software deployment delivery dates and costs. The estimates are used by CBP management to make cost and scheduling decisions.

Evidence: A CBP Modernization Program Award Fee has been established, which is tied to contract milestones and the timeliness and quality of work performed by the contractor. Additionally, project-level incentive fees are negotiated for all delivery/task orders. These incentive fees are linked to performance measures that are tied to achievement of CBP Modernization goals at the project level. Contractor support is reviewed and scored by CBP officials using various criteria as a basis of determining if award fees and incentives fees will be disbursed.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Office of the Inspector General (OIG) and the Government Accountability Office (GAO) conducted seven audits of the ACE program, during the last three years. Twenty-three recommendations resulted from the audits. At this time, twelve recommendations have been closed and eleven are outstanding. Senior OIT management meets with program officials bimonthly to discuss various apects of the program. Each project for every program area is reviewed for key milestone progress, problems affecting timelines, and funding status. Outstanding audit issues are discussed, and plans addressed toward mitigating open issues. Additionally, oversight of the program is conducted by an external IV&V contractor. The IV&V vendor meets quarterly with OIT senior management to share evaluations and recommendations.

Evidence: Of the eleven recommendations that remain open, none are classified as "high risk." Seven outstanding recommendations are either completed or corrective action is underway; two have developed a plan to address the recommendation; and two are pending GAO review of the corrective action. For example, one recommendation to strengthen the user account and password management process, requires a sequenced series of events resulting in the migration to Windows Server 2003 in 2007. However, the majority of open recommendations should be closed this year. Reference GAO-03-406; OIG-04-35;OIG-04-01; GAO-04-587;OIG-05-39; GAO-05-267SU; and OIG-06-16. The Quarterly Report to Congress on ACE provides an update on ACE accomplishments, challenges, fiscal status, and upcoming program milestones. Most significantly, the report demonstrates how ACE is helping CBP achieve the DHS strategic objectives. The report addresses the ongoing CBP efforts to integrate ACE with other systems and engage other Government agencies to participate in ITDS. It also includes the status of efforts to resolve GAO open audit recommendations. Overall, the report provides a status of progress against program commitments.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: CBP's annual budget justifications use performance measure results to justify the budget request. Dollar amounts for each program are published in CBP's annual budget requests. ACE expenditure plans and quarterly status reports clearly describe the program's operations, goals, and measures in a series of documents, and align to resource allocation plans (RAP), OMB exhibits, and congressional justifications. COPPS funding provides the majority of funding required to ensure the network and systems are available to support CBP's "One Face at the Border," provide the critical engineering, operations, security, network monitoring, as well as provide support to all field ports of entry, border patrol sectors, field management offices, and CBP's NDC. Successful operation and immediate accessibility to CBP's information technology infrastructure (from field workstations located throughout the U.S. and oversees) significantly contribute and impact nearly all measures contained in CBP's Congressional Justification.

Evidence: Budgeted funding directly aligns to support all six strategic CBP goals by modernizing the technology infrastructure. For example, it directly relates to Managing and Modernizing, a CBP Strategic Goal, by building a new generation of innovative and secure systems, as well as providing significant funding required to ensure the continued operations, maintenance, engineering of the information technology infrastructure (from the field desktop to CBP's NDC) that is utilized and accessed by all CBP employees. Further, the CBP Strategic Goal of Balancing Legitimate Trade and Travel with Security is demonstrated by ensuring the availability of mission-critical systems that contribute to CBP's multi-layered defense to screen and identify suspect cargo and people at our nations' borders, while ensuring the continued, efficient collection of more than $81 billion in revenue. Each of the new sets of ACE performance measures identifies cost efficiencies of input/output or Technology/Processes and Activities. The performance measures are directly related to the benefits outlined in the ACE Cost Benefit Analysis.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: CBP has recently created an Office of Information and Technology Strategic Plan that details the goals and objectives for this critical program area. Focus groups have determined that the current performance measures illustrated in the OMB 300 Performance Reference Model (PRM) section, serve as initial indicators to truly evaluate the overall performance of ACE as a program. An Enterprise Performance Architecture Working Group (EPAWG) has worked to more closely align ACE performance measures to the CBP Strategic Goals in alignment with the PRM and Business Reference Model guidance provided in the Federal Enterprise Architecture.

Evidence: The Strategic Plan will be periodically reviewed and updated as new or changing requirements develop. Performance Measures will be strengthened and enhanced based on lessons learned and analysis of data. Operational plans are being developed based on the objectives of the Strategic Plan. The plans contain critical milestones and a list of actions needed to achieve the objectives within each goal. Operational plans will be reviewed quarterly to track progress and update actions to be taken to meet the objectives and ultimately the enterprise strategic goal.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Current baselines have been established for annual and long-term performance goals, which for example, are outlined in the ACE Quarterly Report to Congress. Currently, performance is tracked in an Executive-level report, and actual results are compared to planned results or improvements and progress is characterized in green, yellow or red. The performance status is updated regularly to assess program performance and the need for any improvements or changes in strategic planning. As part of DHS's Future Years Homeland Security Program (FYHSP), CBP is developing a complete set of measures including outcomes, output, and efficiency measures against which progress in meeting annual goals is measured. These measures are used in CBP's budget justification documents, and have also provided the sub-programs with valuable insights into their own performance.

Evidence: Performance information is used to assess program performance and schedule to determine if ACE is within the scheduled parameters for distribution within each of the planned releases. OIT increased its emphasis on Earned Value Management (EVM) analysis at ACE program level performance reviews. The EVM system is used to predict potential future performance, plan for and implement corrective actions, and assess whether management changes have positively affected cost and schedule performance. ACE has developed expanded and upgraded capabilities beyond those from the former legacy systems, such as ACS. Examples are multi-modal manifests, increased targeting capabilities, increased integration of systems for more efficient and timely processing at the borders, post-entry summary, wireless processing capabilities, better revenue collecting and tracking through periodic monthly statements. COPPS long-term measures, for example, measures 9, 10 and 11, focus on optimizing the performance of the infrastructure. Operationally, they are critical services provided to the users that are essential to accomplishing the mandated mission. TECS and the communications network must be available when performing security checks at the border. Downtime, unavailable and inaccessible systems are not acceptable. The optimum performance for enterprise servers and mainframes is 80%. A rate above this degrades system performance and decreases critical services to the users. The measures focus on providing the maximum system and network availability, optimization, consistency and dependability in order to provide seamless security-related transaction processing to CBP users around the world whenever they need it.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Through its Strategic Plans, CBP strives to maintain the focus created by the Government Performance and Results Act (GPRA) improving program efficiency and effectiveness, maintaining a results-oriented focus, clearly describing the goals and objectives of programs, and developing a means of measuring progress and as a budget allocation tool. Developing an integrated planning methodology that is supported by meaningful performance measures is a primary focus towards demonstrating CBP's business results. Within ACE, there are quarterly oversight meetings with the contractor to ensure they are performing according to the statement of work. Deviations from the statement of work or late performance by the contractor are documented and utilized in the quarterly evaluation of the contractor resulting in the reduction of funds from the award pool.

Evidence: Within the Automation Modernization program, all production-system projects are subject to a rigorous vetting and control process based on Software Development Life Cycle (SDLC) and Configuration Management (CM) methodologies as well as CBP's Investment Management Process (IMP). The SDLC monitors schedule, cost and performance, and the IMP monitors investments. All CBP managers and program partners associated with these projects are held to strict accountability standards. The Assistant Commissioner, Office of Information and Technology, holds bi-monthly Program Review budgets meetings with the Director, Cargo Systems Program Office (CSPO), to review program projects status (i.e., funding, accomplishments, issues, risks, planned activities, cost, schedule, measures, etc.). CSPO has implemented a more in-depth Participating Government Agency (PGA) integration process that replaced the Agency Readiness Reviews to ensure the current 28 federal agencies and future agencies have their requirements for ACE/ITDS identified and mapped to the appropriate ACE release. An outcome will be to ensure their commitment to the program, and to identify that the budgetary, financial, and technical readiness of the agency aligns to the software releases in ACE. The process begins with outreach activities to an agency, which is followed by a series of workshops to educate the agency about ACE and CBP processes and gather essential information from the agency. Using a functional assessment and business process analysis approach, the agencies' functional and data requirements are gathered and mapped to the ACE releases. One of the last documents developed is an agency concept of operations, or CONOPS, for their ACE involvement, which provides a basis for a memorandum of understanding (MOU) with CBP to formally establish the operational relationship between the agencies. There are approximately 79 federal agencies that are involved in international trade that could potentially use the functionality of ITDS, with the ultimate goal to have all remaining agencies integrated into ACE.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The program has a highly structured process for obligating funds in a timely manner and ensuring that funds are spent on the intended purpose; Automation Modernization funding has carried over in order to ensure program continuity while the annual expenditure plan is being reviewed by agency, Departmental, OMB and Congressional review bodies. ACE funding is distributed after Congressional approval of an annual expenditure plan. Once the spending plan is approved, funds for initiatives are allocated. Rigorous monitoring of expenditures is performed monthly and surplus funds identified through this process are promptly reallocated to other uses within the program.

Evidence: ACE requires Earned Value on its development and enhancement work and requires monthly performance reporting using Contract Performance Reports (CPR) Formats. CPRs are reviewed by the program office for accuracy, reliability and compliance within Earned Value Management System (EVMS) guidelines contained in the American National Standards Institute (ANSI) standard. These reports are provided to senior program management for resolution and will be tracked through completion. Program funding is not obligated until an expenditure plan meets all of the following criteria: 1) Meets the capital planning and investment control requirements by OMB; 2) Complies with the DHS information systems enterprise architecture; 3) Complies with acquisition rules, requirements, guidelines and systems acquisition management practices; 4) includes a certification that there is an IV&V agent for the project, and; 5) is reviewed by the GAO; 6) is approved through DHS, OMB, and Congress.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: As part of the DHS Investment review process and the OMB 300 process, which both the ACE and COPPS programs have completed, competitive sourcing/cost comparisons, IT improvements, and appropriate incentives are measured and cost effectiveness is evaluated.

Evidence: Risk measures are constantly tracked in the CBP Active Risk Manager (ARM) tool. There is a direct crosswalk between these risks and the 19 risk areas identified by OMB and reported in the business case submission. Risks, such as funding modifications or delays, are identified and tracked from inception to resolution. Additionally, independent cost and schedule estimates are conducted to assess project cost and schedule variances under the current baseline. In the last business case submission, both cost and schedule variances were within the established thresholds. CBP internal procedures to review the business case are conducted by the Technology Review Committee (TRC), and annually by the Investment Review Board (IRB). Some examples of program procedures can be found in the following: Process Assets, Cargo Systems Program Office, PFM 5.07, Independent Government Cost Estimate Process; Border Enforcement and Management Systems Program Office, PFM 5.01, Project Cost Estimate Checklist; and the Office of Information and Technology PFM 3.35, Needs Analysis Document (NAD) Template. (included in evidence provided)

Does the program collaborate and coordinate effectively with related programs?

Explanation: CBP's Automation Modernization supports State, local, and international governments as well as over 79 other Federal government agencies (examples include: the Department of Transportation, Department of State, Department of Commerce, Bureau of Census, Federal Communications Commission, Department of Agriculture, Food and Drug Administration, U.S. Fish and Wildlife Service, Federal Bureau of Investigation, International Trade Commission, and components of DHS, such as the Coast Guard and Secret Service) who use CBP data for trade, law enforcement and anti-terrorism purposes.

Evidence: The Automation Modernization program continues to coordinate with other homeland security systems programs in four areas. First, the overall DHS cargo screening information technology architecture will be coordinated by a DHS CIO Council Committee. Second, ACE Selectivity and Targeting (S&T) capabilities are being integrated with the Automated Targeting System (ATS). Third, OIT is coordinating ACE implementation with the existing support for the Container Security Initiative (CSI) via existing production systems, including the Automated Manifest System and the ATS. Fourth, CBP continues to coordinate with other Federal agencies through the ITDS and the US-VISIT program, in particular.

Does the program use strong financial management practices?

Explanation: The program implements financial management as specified in the CBP Budget Manual, and the Systems Application and Processes (SAP) accounting system, a state-of-the-art resources management and accountability system that integrates budget, procurement and asset management. Investments are managed according to OMB Circular A-11 requirements, and in particular, the capital investment requirements of Part 7 and Exhibit 300. All the financial management practices of the program are directed at: - Distributing available resources to functional activities of the program in a manner that will achieve the maximum accomplishment of agency program objectives with a minimum expenditure of funds, and within legal and administrative limitations. - Controlling the allotment, commitment, obligation and expenditure of funds within the limits permitted by anti-deficiency legislation and supplementing regulations and identifying responsibility for violations of such legislation and regulations. - Identifying and correcting out-of-line operating situations and/or conditions; and - Achieving the purposes and objectives for which the budget is approved.

Evidence: The CBP Performance and Annual Report includes last year's financial statements, material weaknesses and repairable conditions, the CBP Budget Manual, SAP-related directives and manuals, System Development Life Cycle (SDLC) and Investment Management documentation. (provided with reference documents) The program's budget personnel perform monthly analyses throughout the fiscal year and review them with senior managers to determine whether sufficient funds exist to meet the objectives of the program. Immediate action is taken to realign requirements when spending is not in line with available funds, or when spending is not in line with highest priorities. In addition, available funds are reallocated to more higher priority uses, especially in cases where those funds will address some of the program's unmet needs and further its overall objectives. As an example, for ACE releases, progress toward completion is reviewed on an on-going basis. If anticipated shortfalls exist, a change request is submitted through an official change request process for consideration of funding from an established management reserve. When approved, funds will be made available. In addition, as savings are identified, funds are deobligated and reinvested to meet funding shortfalls.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: In response to OIG and GAO audits, an internal OIT Strategic Plan was recently completed to clarify management direction to efficiently develop and deliver IT solutions to meet the CBP mission. The OIT organization reflects the six major program areas such as Cargo Systems and Border Enforcement and Management Systems, also includes three other significant support activities, Program Integration, Technology Operations and Laboratories and Scientific Services. In addition, OIT has an ongoing schedule of Project Management Education and Certification Program Courses to ensure a workforce trained in project management.

Evidence: A recent reorganization of the OIT was implemented to align with the six major program areas of the enterprise to better focus managerial attention and resources on the highest priority program objectives. Performance measures are tracked monthly in various performance metrics to assess program operational effectiveness. Areas that are found not to be in alignment with the baselines and proposed improvements are reviewed by management for potential changes. In addition, the OIT reorganization includes a Program Integration Division to guide integration of activities and architecture across program areas. The Technology Operations Division is responsible for providing ongoing operations and maintenance support to technologies that are in a steady state.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Actual results are compared against ambitious baselines to see how well Automation Modernization programs are performing. Measures 9,11,12, 14 and 15 provide examples of progress towards achieving long-term performance goals. In all instances, the actuals are in close correlation to their associated targets in reaching their long-term performance goals. ACE fell slightly short in reaching its established targets in goals 1, 2, 3. Per the March 31, 2006, Report to Congress (RTC), the following actions have been taken to promote targets being met on time in the future: "Total monthly statement collections since the first monthly payments were made in July 2004 now total $3.2 billion. While reaching the $1 billion mark in collections took 13 months, cumulative collections totaled $2 billion in only four months, and reached $3 billion in just four months. Contributing to this growth was the agency's October 2005 creation of non-portal accounts, allowing importers to pay duties and fees monthly through a broker without having to first establish an ACE portal account. The outreach campaign to the chief financial officers of top-duty paying companies (completed during the first quarter of FY06) also contributed to the growth in periodic payment participation. Of the 72 companies targeted by this campaign, 28 have either increased their usage of the monthly statement capability, gained approval to begin using this capability, or established an ACE account." The main strategy to achieve future targets was more outreach to engage the applicable community to participate. Additionally the ACE program is reviewing its long-term performance goals to ensure alignment with Federal Enterprise Architecture (FEA) guidance. The process includes cross walking the DHS and CBP strategic goals to focus the planned effect of ACE. This cross walk defines the highest level, OUTCOME, in support of the goals. The existing processes, activities and technologies are being aligned to support the high level measure. Multiple sets of performance measures will be in place to support ACE releases to align with the FEA. This process will develop effective and efficient measures to assess the true performance of ACE.

Evidence: Automation Modernization uses actual performance results as a management tool to guide and shape the development and deployment of its programs. Actual results are closely monitored and compared against targets to insure that the needs of the users are being delivered as planned, in an effective and efficient manner. Measures 9 and 10 relate to the availability and capacity of the mainframe servers to meet the needs of the CBP community and law enforcement needs especially. In both measures, ambitious targets that maintain availability and dependable service and accessible information, are defined through 2010. Actuals are tracked closely against these targets to insure that the system requirements are being met for users both at headquarters and in the field in a law enforcement setting. Currently, actuals are in close alignment with targets and no strategic change is anticipated. If the actuals did not closely track to their established targets, strategic changes would resolve the disparity.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The ACE program tracks its annual targets in an executive reporting tool that compares current baselines and goals against actual performance. Progress was made toward reaching the established goals. The tool tracks areas where progress is made and highlights areas where progress is not meeting performance goals. If performance goals are not met, management creates a plan to reach the intended level of progress. This is especially important because ACE performance goals measure the true outcome of ACE in its effort to support the CBP and DHS missions while achieving its intended long-term result. There are 18 agencies currently participating in ITDS with requirements mapping to ACE releases. ACE expanded its program partners to include federal agencies with missions tied to international trade and transportation security, regulation, and analysis. The International Trade Data System (ITDS) supports PGAs as they prepare for, integrate their business requirements into, deploy and sustain ACE. This is being done by creating a single window for the trade, using ACE as the vehicle of delivery.

Evidence: Automation Modernization is on target to meet its annual performance goals as is illustrated by performance measures 1-3, 9-12, and 14-15. Evidence includes data from the GPRA performance report, Performance and Accountability Report (PAR), strategic plan, or other administrative goals and objectives documents. Many federal agencies are engaged in ITDS and ACE software release level activities. As more agencies participate, so will the quantity and quality of critical security, public health, public safety, and environmental protection information centralized in the ACE electronic platform, eliminating the need for multiple and redundant government data filing requirements and inefficient paper-based systems. The PGA integration into ACE increases the accuracy and completeness of trade data and encourages interagency information sharing, providing greater efficiencies in data collection and intelligence gathering allowing for more coordinated risk management. Thus, the collaboration of CBP and PGA at the border facilitates faster and more efficient cargo release.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Automated Commercial Environment (ACE) portion of the Automation Modernization program supports the key goal of Facilitating Legitimate Trade and Travel, through the increasing deployment of e-Manifest. The Strategic program goals are defined through linked measures to DHS and CBP Strategic goals. Efficiency goals are also defined in support of the high-level outcome measure. Measure 3 provides an example of increased automation of duties and fees being paid by the trade community that has yielded a savings in both processing time and in fees collected.

Evidence: Time saved from improved processes directly result in cost savings to CBP as illustrated in measure 3. The increased time efficiency resulting from automating previously manual processes, expedites team review. This time efficiency results in labor cost savings to CBP and in more efficient collections processing. A recent ACE Cost Benefit Analysis resulted in a Net Present Value (NPV) of 22.5 billion and a Benefit Cost Ratio of 8.29. That is, for every dollar spent, CBP realizes $8.29 in benefits. The ACE program has begun to demonstrate cost effectiveness by increased participation in the program, garnering 8% of the total trade traffic for the last fiscal year. This illustrates an increase from the base year of FY 2004 when the baseline value was zero. The program has facilitated improved efficiencies as evidenced by the reduction in truck processing time at the Ambassador Bridge in Detroit from 105 seconds per truck to 80 seconds. Also, costly manual input of manifests into a tracking system continues to decrease due to the growing use of submitting manifests electronically, e-Manifests, by shippers. For example, ACE has recently received 309 e-Manifests representing over 340,000 shipments.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: CBP is charged with the management, control, and protection of our Nation's borders at and between the official Point of Entries (POE) and with keeping terrorists and terrorist weapons out of the country while enforcing hundreds of U.S. laws. Through the implementation of Automated Commercial Environment (ACE), CBP is striving for increased border protection and the more efficient facilitation of legitimate trade across the borders, measurable through the annual and long-term performance goals of ACE. No other federal program is comparable to ACE as it is unique to CBP. ITDS stands out as the only repository of data for all government agencies with missions that include trade and trade-related activities. As part of ACE, ITDS continues to expand with more participating agencies providing data to and using data from just one major source. The program provides a comprehensive, reliable, and standard database of trade data, not only improving legitimate trade but also providing invaluable trade information to assist in protecting our borders. The COPPS supports CBP's IT infrastructure, facilitatates "One Face at the Border" and enables more efficient and effective sharing of information and images among trade and law enforcement agencies. The systems are vital tools used to defend our borders. The major mission critical systems and applications supported by the COPPS include, but are not limited to: Automated Commercial System (ACS), Treasury Enforcement Communications System (TECS), Automated Export System (AES), Advanced Passenger Information System (APIS), Seized Asset and Case Tracking System (SEACATS), and CBP's Financial and Human Resources Management Systems. Major components of COPPS include -Communications services such as: Blackberry servers, internet and extranet connections, network bandwidth and fiber optics; and -Hardware such as: scanners, network operations center, LANs, Wide Area Networks, mainframe computers, and workstations; and -Software such as: operating systems, and program and configuration management tools.

Evidence: There are no other programs that offer a similar function for the purpose of comparison. ACE coordinates with other homeland security programs to achieve maximum effectiveness and efficiency. Additionally, the most recent Cost Benefit Analysis conducted in August of 2005, identified the current ACE implementation plan to be the best alternative. CBP's technology infrastructure is unique and one-of-a-kind in that it includes the largest Datacom transactional database in the world and the largest volume of data processed through the middleware infrastructure (which provides efficient transfer of data among systems and governmental and private organizations), and the largest server environment in North America. The mainframes support over 100,000 users. In addition, the infrastructure currently supports over 47,000 CBP staff, more than 100 Federal, State, local, international government agencies, and more than 3,000 commercial organizations. CBP's infrastructure consists of 47,000 total workstations/PCs, 20,000 laptops, 11,000 network printers, 5,000 host printers, 1,500 domain controllers, 225 application servers, 850 Novell Servers, four mainframes, and 2,200 switches. Support is also provided to wireless technologies, such as Blackberry devices.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Under the Mod Act, CBP and the importer share responsibility for compliance with trade laws and regulations. Importers are responsible for declaring the value, classification, and rate of duty applicable to entered merchandise, and CBP is responsible for informing the importers of their rights and responsibilities under the law. The Act resulted in the development of automation programs for handling trade and compliance. The Automation Modernization program is a major effort to modernize the trade processing system and to enhance border security while continuing to facilitate lawful international trade and travel. The OIG and GAO have conducted seven audits over the last three years. One of the primary focuses of these audits is the ACE Expenditures Plans that are reviewed prior to implementation and expenditure of funds. The audits serve to strengthen management accountability and to ensure that planned software releases deliver promised capabilities and benefits within budget and on time. Based upon a GAO recommendation, CBP established a permanent IV&V function. A Special Assistant for Audit Issues and Quality was appointed and acts as the government representative to oversee the IV&V effort and to help ensure the independence of the reporting.

Evidence: DHS has a solid program management foundation of acquisition processes, program analysis and reporting mechanisms, and management systems to effectively manage the program. This is complemented by strong stakeholder relationships that support the development of ACE requirements, and provide feedback on ACE capabilities. Specific criteria are established for all ACE development milestone reviews. The process requires verification that all problems have been resolved or have viable resolution plans before the milestone is considered successfully accomplished. Problem areas are prioritized and assessed to determine whether deferring closure to post-milestone review resolution is an acceptable risk. The resolution plans are implemented and tracked closely until the problem is resolved. This process reflects careful consideration and deliberate decisions by DHS officials as they seek to balance program objectives. In the GAO report delivered in March 2005, GAO-05-267, the following statement was made. "The fiscal year 2005 ACE expenditure plan, including related program documentation and program officials' statements, largely satisfies the legislative conditions imposed by the Congress." In addition, some of the recommendations that GAO has previously made to strengthen ACE management have been addressed, and DHS has committed to addressing those that remain. Of the seven audits of the ACE program conducted during the last three years, 23 recommendations were made by either the OIG or GAO. As of June 2006, 12 recommendations are closed and 11 are outstanding. Of the 11 outstanding measures, two are currently pending GAO review of the corrective action; seven have either completed the recommended corrective action or corrective action is underway; and two have developed plans to address the recommendation. None of the 11 measures are classified as "high risk." The majority of all open recommendations should be closed by the end of this fiscal year. For the past two years,the ACE program has contracted with an IV&V vendor, to observe ACE deployment in areas of concern. This year, the contractor felt the deployment process was not well organized to ensure effectiveness throughout the enterprise. Specifically they were concerned that major issues are still being addressed after the fourth ACE software deployment. In response, CBP developed the following process documents critical to facilitate successful deployments: 1) Deployment Plan approved by the Assistant Commissioner, 2) Detailed Cluster Implementation Plan (CIP) coordinated with all personnel associated with deployment, training, and security, along with personnel from the Office of Field Operations and port staff, 3) Technical site survey documentation which details required infrastructure upgrades, 4) Workflow site survey, outlining the operation procedures for each port, and 5) Port Leadership Information Briefings. The newly developed processes were prototyped to selected sites to test and validate the new processes before full deployment to all sites. As ACE has been deployed, additional processes were created and existing processes were updated to minimize disruption at the ports, decreasing deployment time at each port, and reducing "last minute" changes at the port.