Detailed Information on the
Airlift Program Assessment

The DoD evaluate the specific requirements for airlift capabilities in order to support the 2006 Quadrennial Defense Report goals and Secretary of Defense Priorities.

Proposes that DoD continue to develop methods for assessing the efficiency and effectiveness (or otherwise) of the overall airlift program in light of the needs of the 2001 QDR defense strategy and the global war on terrorism.

Measure: Percent change in acquisition costs for individual programs from the total cost estimate. For example, actuals show changes for the C-17 program. Data taken from DoD's annual Selected Acquisition Reports. The December 2001 report represents a two-year reporting period (1999-2001) due to the absence of a December 2000 report.

Explanation:

Measure: Provide 100 percent of strategic airlift capacity (54.5 million ton miles/day), a requirement established by DoD in its Mobility Requirements Study 2005

Explanation:

Measure: When normalized to constant year dollars, there is no change from the previous year's Average Procurement Unit Cost (APUC) for the C-17A, C-130J and C-5 RERP programs.

Explanation:The APUC is the average cost for procuring a specific major acquisition item and is reported quarterly for all major acquisition programs in the Department of Defense Selected Acquisition Report. This measure's goal is for no annual increase in the cost of producing the following programs: C-17A, C-130J and C-5 RERP. This goal is achieved if the APUC for the end of the fiscal year does not change from year to year when adjusted for inflation.

Is the program purpose clear?

Explanation: DoD's airlift program provides for the rapid deployment of troops and materiel to overseas operating locations.

Evidence: This capability is one of the Air Force's core competencies. The annual Government Performance and Results Act (GPRA) Performance goal for DoD is to maintain a mobility capability to move military forces from the US to any location in the world using airlift if necessary.

Does the program address a specific interest, problem or need?

Explanation: As shown in the war on terrorism, the United States has a continuing need for airlift to meet the requirements of various contingencies.

Evidence: Requirements for airlift aircraft are outlined in the Mission Needs Statements prepared by DoD that establish the capabilities that are needed, GPRA Performance Report, and statements before Congress. 99% of the airlift missions to Afganistan have been provided by the DoD airlift fleet.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: The individual acquisition programs within the Airlift Program all contribute to US airlift capabilities.

Evidence: DoD devotes over $4.7 billion to investment in airlift programs in FY 2003. Specific program detail can be obtained from budget documents.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: The program is not redundant or duplicative with any other Federal or non-federal efforts.

Evidence: No other funding is being expended to acquire this type of military airlift capability.

Is the program optimally designed to address the interest, problem or need?

Explanation: The Mobility Requirements Study 2005 completed by the Office of the Secretary of Defense in 2000, recommended quantities and types of specific airlift aircraft. However, due to other factors, such as industrial base capacity and budgetary pressures, the DoD is not always able to procure aircraft in the quantities required. The Department, in developing its Future Years Defense Program, assesses requirements in all mission areas and allocates resources in order to best ensure accomplishment of its National Security missions.

Evidence: The Mobility Requirements Study 2005, and analyses of program alternatives conducted by DoD prior to the start of development for each major defense acquisition program determine how and whether acquisition programs like the C-17 will meet the required capability.

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has a long-term goal of providing a strategic airlift capacity of 54.5 million ton miles/day.

Evidence: The Mobility Requirements Study 05 (MRS 05) established the overall goal for DoD's airlift fleet.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: DoD establishes performance goals for each aircraft acquisition program. These include cost, shedule, and system capabilities goals that DoD tracks on an annual basis.

Evidence: The President's Budget documentation provides details on program estimated costs, contract award dates, delivery schedules, construction dates. Comparisons of one fiscal year budget to the next reveals variances. Selected Acquisition Reports, which are provided by DoD to Congress at least annually, describe the program and show deviations from the program baselines.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: There are no true "partners" for this program, given its unique governmental nature. Contractors are fully committed to the program through their contractural obligations.

Evidence:

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: This program does not share a common purpose or goal with any other program.

Evidence:

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: By statute and regulation, DoD conducts evaluations of individual programs on a regular basis. These evaluations monitor program progress against cost, schedule, and performance criteria.

Evidence: Prior to establishing a new program, or approving the continuation of an aircraft program, DoD develops an acquisition program baseline (APB) that sets out the cost, schedule and performance goals for the program. This process requires an extensive analysis of each program and its potential contribution to the overall mission. Two examples of reports that include this information are the Annual Selected Acquisition Reports, which are sent to Congress, and the quarterly Defense Acquisition Executive summary reports.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: The DoD program planning and budgeting system requires budgeting based on a determination of the resources needed to achieve the acquisition goals of each aircraft within the overall airlift program.

Evidence: DoD budget documents lay out the required funding and programmatic objectives to be achieved for each year.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: The goal of the program is to meet the airlift target of 54.5 MTM/D. DoD is attempting to achieve that goal through the construction of airlift aircraft -- primarily the Air Force's C-17.

Evidence: DoD's FY2003 budget documents for the C-17, C-130J and C-5 aircraft upgrade programs describe how each will meet its goals and thereby meet the overall airlift program objective.

Are acquisition program plans adjusted in response to performance data and changing conditions?

Explanation: DoD conducts regular reviews of acquisition programs at which changes to programs can be, and sometimes are, made. For example, the C-17 program was recently extended beyond its original goal of producing 120 aircraft.

Evidence: DoD Defense Acquisition Directives describe milestones when DoD should undertake reviews of the program, and when goals should be achieved.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule and performance goals?

Explanation: The Mobility Requirements Study 2005 reviewed airlift requirements and recommended quantities and types of specific aircraft required to meet mobility needs. In addition, an DoD conducts an Analysis of Alternatives (AoA) before the start of development for each major defense acquisition program. These AoAs compare the cost and capabilities of various alternatives and help DoD choose the appropriate system to meet mission requirements.

Evidence: The Mobility Requirements Study 2005 was a comprehensive assessment of airlift needs and capabilities. DoD also completed an AoA for upgrades to the C-5 aircraft in November 2001. In addition, each year, during the program or budget review processes, programs are subjected to additional review.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: DoD regularly collects performance information on the individual airlift aircraft programs and uses the data to inform senior leadership and to make program decisions. Program progress reflecting performance in cost, schedule, and attainment of system performance parameters by both the program office and the contractor(s) is regularly collected and reported to the senior leadership.

Evidence: DoD reports performance information for each program in annual Selected Acquisition Reports, AoA analyses, and Defense Acquisition Executive Summaries.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: Acquisition directives assign accountability to program managers for cost, schedule, and performance. Contractual requirements are stated in performance terms.

Evidence: An Oct 30, 2002 memorandum from the Deputy Secretary of Defense entitled "The Defense Acquisition System" provides guidance on program manager (PM) accountability. PMs receive charters upon assignment, giving authority, responsibility, & accountability.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: DoD budget reviews include a review of obligation and expediture data to ensure that funds are obligated in accordance with DoD standards.

Evidence: Expenditure data are reported monthly on standard form 1002 by the Defense Finance and Accounting Service.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While there are no efficiency targets, before programs are initiated, analyses of alternatives are conducted to ensure selection of the best solution for the requirement. Program performance plans contain performance metrics that focus on cost schedule, and performance. Application of Earned Value Management also helps to ensure efficiency and effectiveness in program execution. Selected Acquisition Reports monitor total and unit costs.

Evidence: Periodic management reports, such as the Selected Acquisition Reports, Defense Acquisition Executive Summary, and program performance plans are developed and evaluated periodically to ensure adherence to cost, schedule, and performance. When deviations occur, DoD management makes adjustments to ensure the most effective use of resources.

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: The DoD budget documents include estimates for all programatic costs associated with developing and procuring airlift aircraft.

Evidence: DoD budget documents lay out the costs for each program on an annual basis. For example, the FY2003 President's Budget included $3,946.6 million to develop and procure the C-17 aircraft.

Does the program use strong financial management practices?

Explanation: DoD's financial management system is being improved, but slowly. No audit reports showing that the airlift program is free of internal control weaknesses have been provided.

Evidence:

Has the program taken meaningful steps to address its management deficiencies?

Explanation: DoD uses an earned value management system to track program performance. Currently, this system is showing that the major airlift programs are meeting their cost and schedule performance targets.

Evidence: DoD management receives quarterly reports showing how airlift programs are progressing.

Does the program define the required quality, capability, and performance objectives for deliverables?

Explanation: All elements of the airlift program have well defined key performance parameters.

Evidence: Selected Acquisition Reports, and program requirements documents lay out the performance objectives of each program in detail.

Has the program established appropriate, credible, cost and schedule goals?

Explanation: All DoD budget estimates include unit costs, annual costs and overall program acquisition costs. The C-17 is a stable procurement program proceeding within its cost and schedule targets, the C-130J program's budget estimates appear to be reasonable.

Evidence: Selected Acquisition Reports and DoD budget documents, describe the cost and schedule goals for each program .

Has the program conducted a recent, credible, cost-benefit analysis that shows a net benefit?

Explanation: A cost benefit analysis is not normally done for DoD capital programs.

Evidence:

Does the program have a comprehensive strategy for risk management that appropriately shares risk between the government and contractor?

Explanation: DoD uses an earned value system to track program performance, and each acquisition element within the airlift program normally would include a risk mitigation plan. In addition, cost plus type contracts are used by DoD for highly risky programs.

Evidence: Quarterly management reports and specific contract language describe the extent of risk and how it is shared between the government and contractor.

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: The airlift investment program is nearing completion of the first phase of the C-17 program which has increased airlift capabilities. However, the program has still not met its target capacity. Attainment of the inter-theater airlift capability is dependent on fielding new C-17s, retiring the aging C-141 fleet, and eventual fielding of C-5 Reliability Enhancement & Reengining Program (RERP) aircraft. Deliveries of the C-130J will increase intra-theater capabilities.

Evidence: Increasing deliveries of C-17 aircraft to replace the C-141 has resulted in rising inter-theater delivery capabilities. C-5 RERP program is planned to become operational in 2010 increasing inter-theater capabilities even more. C-130J deliveries are in the early stages of the program and will contribute to increased intra-theater airlift capabilities.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Annual performance goals are program-management related and include accomplishing planned deliveries, minimizing variances from program baselines, and making incremental progress in airlift capabilities.

Evidence: Deliveries accomplished are reflected in the President's Budget; the annual Selected Acquisition Reports show variances from program baselines, other working documents reflect progress in airlift capabilities.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The C-5 RERP First Flight and the planned date of operational capability have been accelerated 3 months from the APB schedule; the C-130J APB schedule has been maintained, with no change in Program Acquisition Unit Cost; C-17 deliveries have been at an average 124 days ahead of schedule.

Evidence: These data are described in the Selected Acquisition Reports.

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: This program does not share a common purpose or gaol with any other program.

Evidence:

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: Recent General Accounting Office (GAO) reports have reviewed the C-130 aircraft, overall air transport capability, and the readiness of air transport capability. These reports evaluate the overall airlift capability; while they reported deficiencies, they also note that DoD is taking steps to address/implement some of the GAO recommendations. The purpose of the programs is to provide a specific airlift capability. The programs are progressing towards providing this capability. Besides these GAO reports, DoD conducts regular independent reviews of each program to assess progress against specific cost and performance goals. Issues raised during these reviews must be addressed in order for the programs to proceed. These reviews include input from both functional and financial representatives, to ensure that all appropriate issues are examined. Also, the Mobility Requirements Study 2005 evaluated requirements and current/projected capabilities.

Evidence: Recent GAO reports including [NSIAD-00-135] Military Readiness: Air Transport Capability Falls Short of Requirements, [NSIAD-98-108] Intratheater Airlift: Information on the Air Force's C-130 Aircraft, [GAO-01-495R] Military Readiness: Updated Readiness Status of U.S. Air Transport Capability, Mobility Requirements Study 2005, and regularly scheduled DoD independent reviews such as the Defense Acquisition Executive Summary.

Were program goals achieved within budgeted costs and established schedules?

Explanation: The largest element of the airlift program, the C-17 program, has been delivering aircraft ahead of schedule. Other elements, such as the C-5 upgrade program have experienced delays.

Evidence: FY 2003 budget documents for the C-17 and C-5 programs lay out the cost and schedule goals and aircraft deliveries for these programs.