Detailed Information on the
FAA Research, Engineering & Development Assessment

In coordination with NASA, develop a follow-on plan to the National Aeronautics Research and Development Policy. In addition FAA will work with NASA to identify challenges and develop R&D solutions to safety, environment, and human factors issues affecting NextGen.

Include efficiency measures and targets in the FY 2005 President's Budget.

Measure: Turbulence forecast products developed that allow pilots to avoid hazardous flight conditions while improving safety and ensuring efficient airspace use.

Explanation:FAA conducts research and develops products on turbulence-forecast to improve aviation safety. On average, turbulence caused 44 accidents, 95 injuries, and 41 fatalities with costs exceeding $135 million per year between the years 1990 and 2000. 2002 Target--Develop clear-air turbulence product above 20,000 ft in experimental use. 2003 Target--Develop clear-air turbulence product above 20,000 ft fully operational. 2004 Target--Develop clear-air turbulence product above 10,000 ft for test. 2006 Target--Develop convective induced turbulence product for test. 2007 Target--Develop mountain wave turbulence product for test. 2008 Target--Develop probabilistic turbulence product for test. 2009 Target - Obtain FAA and National Weather Service (NWS) approval of turbulence forecast with text generation for implementation on operational Aviation Digital Data Service (ADDS). 2010 Target - Implement probabilistic and mountain wave turbulence forecast capability on operational ADDS. 2011 Target - Implement convectively-induced turbulence forecast capability on operational ADDS. 2011 Target - Develop clear-air turbulence product for all altitudes for test. 2012 Target - Test global turbulence forecast. 2013 Target - Obtain FAA and NWS approval to implement turbulence forecast capability for all flight levels on operational ADDS. 2014 Target - Obtain FAA and NWS approval to implement probabilistic turbulence nowcast capability on operational ADDS.

Measure: In-flight icing and freezing precipitation aloft forecast products developed that allow pilots to avoid hazardous flight conditions while improving airspace use.

Explanation:FAA conducts research and develops products on in-flight icing and freezing precipitation to improve aviation safety. Icing is a major safety issue for general aviation and small commuter aircraft. On average, icing caused 31 accidents and 43 fatalities per year between the years 1982 and 2000. 2002 Target--Implement year round product guidance and icing type nowcasts. 2004 Target--Develop forecast icing potential product for operational use. 2006 Target--Develop current icing potential severity product for operational use. 2007 Target--Develop forecast icing potential with severity for experimental use. 2008 Target--Develop forecast icing product with super cooled large drops for experimental use. 2009 Target - Test in-flight icing forecast in Alaska. 2010 Target - Test in-flight icing forecast for North America. 2011 Target - Obtain FAA and National Weather Service (NWS) approval for Alaska icing forecast for operational Aviation Digital Data Service (ADDS) implementation. 2012 Target - Implement Alaska icing forecast capability onto operational ADDS.

Measure: New technologies, procedures, test methods, and criteria developed for preventing accidents that result from hidden in-flight fires and fuel tank explosions.

Explanation:Fire continues to be an FAA priority for a number of reasons. Fire and fuel tank explosions were responsible for 24 percent of the 1470 fatalities in scheduled U.S. Air Carrier accidents over the 20 year period ending in 2007. In 2006, there were over 800 reports of smoke in the cockpit or cabin. FAA also is being proactive by addressing fire safety concerns associated with new all-composite aircraft (e.g., Boeing 787) and lithium batteries (passenger items, bulk cargo, and aircraft emergency power). 2002 Targets--Fabricate and install nitrogen inerting system in 747SP for ground-based inerting of the center wing; Test a seat fabric fiber to determine if it provides a 50% reduction in heat; Propose a technical standard order for flammability test on airline blankets; Finish research on full-scale test evaluation of cargo compartment fire supression system. 2003 Targets-- Develop and demonstrate thermoset resin for cabin panels with factor of 10 reduction in heat release rate and draft technical report describing results; Conduct 40 hours of flight tests on FAA inerting system using an Airbus A320 and Boeing 747; Draft 2 technical reports describing fuel tank inerting research progress. 2004 Targets--Draft report comparing fuel tank inerting concentrations during A320 flight tests with model predictions; Conduct 30 hours of flight tests on FAA inerting system in NASA 747 (Edwards AFB). 2005 Targets--Draft report describing FAA/NASA inerting system flight tests and fuel vapor measurements; Develop and demonstrate a thermoplastic for cabin molded components with factor of 10 reduction in heat release rate and draft a report describing results. 2006 Target--Develop and evaluate the improved fire resistance of structural composite epoxy resins containing phosphorus. 2007 Target--Develop and demonstrate a seat fabric fiber with a factor of ten reduction in heat release rate and draft a report decribing the results. 2008 Target - Develop safe, acute exposure limits for gaseous halocarbon extinguishing agents in ventilated aircraft. 2009 Target - Develop fire test criteria to limit the emission of hazardous gases during post-crash fire exposure of burn-through resistant fuselage, including composite construction. 2010 Target - Develop and validate a methodology for predicting flammability of wing fuel tanks of aluminum or composite construction. 2011 Target - Provide comprehensive fire safety guidance for high energy density lithium batteries in passenger carry-on items and aircraft power systems. 2012 Target - Define composite fuselage fire safety design criteria. 2013 Target - Demonstrate the improvements in post-crash fire survivability, provided by ultra-fire resistant materials under full-scale test conditions. Note: The Target column in the table below indicates the number of milestones which were planned for a given year, and the Actual column indicates the number of planned milestones which were completed in a given year.

Measure: Sustain the System Planning and Resource Management Budget at 2% or less of the total R,E&D budget.

Explanation:The System Planning and Resource Management budget line item provides contract dollars for the overall management of the FAA R&D program. The goal is to cap growth in this area relative to the overall R,E&D budget. This measurement provides a ratio of the total line item to the total R,E&D budget. No more than two cents per dollar of R,E&D budget will be used for management costs. Report measure as actual dollar amount of management costs and percent of management cost compared to total R,E&D budget. Annual Products from the System Planning and Resource Management activity include: Develop and deliver the National Aviation Research Plan (NARP); Manage and support the R,E&D Advisory Committee (REDAC); Develop and deliver the R,E&D budget portfolio; Support the FAA field offices at NASA Langley and NASA Ames.

Measure: Maintain an R,E & D management workforce comprising no more than 10% of the overall R, E & D workforce.

Explanation:The R,E&D management workforce provides overall management of the R,E&D program and portfolio development. The goal is to maintain this overhead workforce at 10 percent or less of the total number of R,E&D positions. This measurement provides a ratio of the total number of overhead positions to the total number of R,E&D positions. No more than 10 out of every 100 positions are overhead positions. Report measure as actual number of overhead positions and percent of overhead positions compared to total number of R,E&D positions. Annual Products of the management workforce include: Develop and deliver the National Aviation Research Plan (NARP); Manage and support the R,E&D Advisory Committee (REDAC); Develop and deliver the R,E&D budget portfolio; Support the FAA field offices at NASA Langley and NASA Ames.

Is the program purpose clear?

Explanation: FAA's Research, Engineering & Development (R,E&D) program conducts aviation safety research on: (1) fire and smoke resistance technologies; (2) aircraft maintenance and structural technologies; (3) the relationship between human factors and aviation accidents; and (4) air traffic control.

Evidence: Federal Aviation Act (P.L. 85-726); Title 49, Subtitle VII, Aviation Safety Research Act (P.L. 100-591); Omnibus Reconciliation Act of 1990 (P.L. 101-508).

Does the program address a specific and existing problem, interest, or need?

Explanation: The FAA is the sole certification authority for the United States aviation community; R,E&D provides the research and information necessary for the FAA to regulate and create standards for industry, which leads to a reduction of the aviation fatal accident rate.

Evidence: Federal Aviation Act (P.L. 85-726), Title 49, Subtitle VII; Aviation Safety Research Act (P.L. 100-591); FAA Strategic Plan; R&D Strategy.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: The program is solely a unique FAA function, and no other organization duplicates this program; FAA does research to establish standards in the aviation community. The FAA is the sole certification authority for the industry. If the program did not exist, no other public or private organization could take its place.

Evidence: Federal Aviation Act (P.L. 85-726), Title 49, Subtitle VII, Aviation Safety Research Act (P.L. 100-591); R&D Strategy; National Aviation Research Plan; FAA Operational Evolution Plan; National Aviation Weather Initiatives.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program engages both internal and external stakeholders to provide input and assessment of the program on a regular basis. R,E&D also leverages its external partners for people, skills and resources. For example, its Centers of Excellence partners from academia and industry provide R,E&D with matching resources for aviation-related R&D.

Evidence: National Aviation Research Plan/REDAC Recommendations; R&D Portfolio Development Process, Guidance/Reference Document; FAA Joint Resource Council Process; The Product Development Team for In-Flight Icing, 2001 Plan.

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: R,E&D supports the FAA's operational, regulatory, and oversight functions, which, in turn, directly support the flying public. Unlike other federal research programs, each research project focuses on a particular high-priority regulatory activity.

Evidence: National Aviation Research Plan/REDAC Recommendations; R&D Strategy; Research, Engineering and Development Advisory Committee and Subcommittee Recommendations and Reports; R&D Portfolio Development Process, Guidance/Reference Document; FAA Joint Resource Council Process; The Product Development Team for In-Flight Icing, 2001 Plan

Does the program effectively articulate potential public benefits?

Explanation:  

Evidence:  

If an industry-related problem, can the program explain how the market fails to motivate private investment?

Explanation:  

Evidence:  

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The R,E&D program has specific long-term performance measures, tied to specific research programs/projects that support accomplishment of long-term national and agency goals. As one example, as part of the FAA goal to reduce the fatal accident rate, the Weather Research Program has a performance measure to develop 5 turbulence forecast products that allow pilots to avoid hazardous flight conditions improving safety and ensuring efficient airspace use by 2008.

Evidence: R,E&D Budget Linkage sheet; National Research Plan for Aviation Safety, Security, Efficiency, and Environmental Compatibility; FAA Operational Evolution Plan; National Aviation Research Plan; ARA Annual Performance Plan (faa.gov/ara/perform/); ARA Quarterly and Annual Performance Plan Goal Reports

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The R,E&D program's annual goals are ambitious. The long-term research goals are mapped to multi-year objectives, which help track the progress of the research through the establishment of annual milestones.

Evidence: FAA Strategic Plan; R&D Strategy; FAA Operational Evolution Plan; ARA Annual Performance Plan; ARA SES Short-Term Incentives; Communications, Navigation, and Surveillance Business Management Handbook; FAA In-flight Icing Plan; In-Flight Icing PDT (#4) Technical Direction, FY 2002

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: The R,E&D program has annual performance measures that can demonstrate progress toward achieving the program's long-term goals. Progress toward these goals are measured quarterly.

Evidence: ARA Annual Performance Plan; ARA Quarterly and Annual Performance Goal Reports; ARA SES Short-term Incentives, DOT and FAA Strategic Plans, DOT Performance Plan, FAA Strategic Plan Supplement.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: The program develops baselines and ambitious targets, in conjunction with sponsors and partners, for all of its annual measures.

Evidence: ARA Annual Performance Plan; ARA Quarterly and Annual Performance Goal Reports; ARA SES Short-term Incentives; National Aviation Research Plan

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All partners commit to the annual and long-term program goals through a variety of means, such as MOU's, SOW's, Joint Councils, and management plans. Regularly scheduled reviews are conducted to ensure compliance and progress.

Evidence: Memorandums of Understanding; FAA Grants Order; ARA Goal 2 (Human Factors) Integration in Research and Acquisition, Project Deliverables and Status Report; FAA/NASA Management Plan; Communications, Navigation, and Surveillance Business Management Handbook; Statement of Work: Fracture Mechanics Properties Standards; FAA Joint Resource Council Process.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: There are many regular and ad-hoc reviews of the program, including: 1) annually, research sponsors review program outcomes and outputs, prioritize and plan research efforts, and make decisions about the program; 2) the R,E&D Advisory Committee (REDAC), comprised of representatives from industry, universities, other agencies, users, and associations, reviews research and makes recommendations about budget and program priorities and merit; 3) external groups, such as the National Academy of Science, review program and results; 4) research is presented at international conferences and in Technical Reports available to the external research community.

Evidence: National Aviation Research Plan/REDAC Recommendations; National Academy of Science and Transportation Research Board Publications; The AVR R&D Requirements Process; Program Planning Team Documents; and R&D Portfolio Development Process, Guidance/Reference Document.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Last year, R,E&D's Budget request related FAA Strategic goals to resource requests (all direct and indirect costs). Unfortunately, the FAA draft document needed more work and we expect that it will submit a performance-based request this year.

Evidence: FY 2004 Budget, FY 2004 FAA Congressional Justification.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Realizing the need for a R&D strategy to guide program investments, in FY 2002, FAA published its first five-year R&D Strategy. R,E&D is now examining the programs/projects in the National Aviation Research Plan and mapping them to the R&D Strategy to identify potential gaps in the strategic planning process and to evaluate any gaps to determine the appropriate corrective action, i.e., revision to the Strategy or revision of the Research Plan.

Evidence: R&D Strategy; Draft R&D Strategy Assessment; R&D Portfolio Development Process: Lessons Learned; R&D Executive Board Portfolio Development Process Project Management Plan

If applicable, does the program assess and compare the potential benefits of efforts within the program to other efforts that have similar goals?

Explanation: In addition to continued reviews by FAA management, research sponsors, and the REDAC subcommittees, each research area works closely with customers and other agencies to ensure continuing relevance of the work. In addition, the program receives continuous external review to ensure that it is meeting customer needs by: meeting with the users; seeking feedback; presenting progress reports at public forums and science reviews; publishing and presenting technical papers; obtaining formal peer validation of science; training specific users on product usage; and maintaining and sharing lessons learned.

Evidence: Research, Engineering and Development Advisory Committee and Subcommittee Recommendations and Reports; Interagency Agreement, Number DTFA01-98-Z-020024 Between the FAA and NOAA; Integrated Icing Forecast Algorithm 9IIFA) Assessment at Regional Airlines -- Final Report; FAA Current Icing Potential (CIP) and Forecast Icing Potential (FIP) Regional Airlines Benefit Analysis; Commercial Aviation Safety Team (CAST) documents.

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: Program priorities are determined in concert with internal/external reviews conducted by the REDAC, internal sponsors, and national and Departmental guidance, such as from OMB. Priorities are also outlined in the DOT Research Development, and Technology Plan, as well as in the FAA Strategic Plan and the ARA Performance Plan. Using external input, the R&D Executive Board, through a documented process and working through program planning teams, provides budget guidance for budget planning and allocation.

Evidence: R,E&D Budget Linkage sheet; R&D Executive Board Portfolio Development Process Project Management Plan; National Aviation Research Plan; R&D Strategy; DOT Research, Development, and Technology Plan; ARA Annual Performance Goals; Joint Resource Council Process; Research, Engineering and Development Advisory Committee and Subcommittee Recommendations and Reports; The AVR R&D Requirements Process; Decision-Based Weather Needs for the Air Route Traffic Control Center Management Unit.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The goals in the annual ARA Performance Plan are tracked and reported on quarterly. In addition, projects within the program regularly collect performance information from partners and use it to manage the program and improve performance.

Evidence: ARA Annual Performance Plan; ARA Quarterly and Annual Performance Plan Goal Reports; ARA Goal 2 (Human Factors)--Project Deliverables & FY 02 Status Report; FAA/NASA Wake Turbulence Research Management Plan; FAA/NASA Joint University Program reviews; COE Program reviews.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: Program managers are responsible for achieving results and performance measures are included into performance evaluations and annual work plans. In addition, through program and business plans, as well as contractual arrangements and grant language, and through formal agreements, such as Memorandums of Understanding/Agreement, program managers hold partners accountable for cost, schedule, and performance results.

Evidence: ARA Short-Term Incentives; ARA Goal 2 (Human Factors): Project Deliverables & FY 02 Status Report; FAA/NASA Wake Turbulence Research Management Plan.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The FAA Budget Office ensures that all program funds are obligated in a timely manner in accordance with the program plan. The R,E&D program traditionally obligates 95% of it funds in the first year and unobligated funds are carried forward. Obligations are reviewed monthly, quarterly, and at the end of each fiscal year; corrective action is taken as necessary.

Evidence: ARA Monthly Financial and Personnel Reports

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: R,E&D's performance plan does not include efficiency measures and targets. However, there are many efficiencies realized through assessment of program performance. The program also gains tremendous cost effectiveness through its Centers of Excellence Program, which provide matching funds from non-federal sources, enabling the program to leverage industry sources to help finance critical safety research. Furthermore, the program uses a labor distribution reporting system, which tracks the personnel hours and costs an employee is working on a project.

Evidence: Core Compensation Program information is on-line at www1.faa.gov/corecomp/plans_policies.cfm; COE brochure; COE program reviews

Does the program collaborate and coordinate effectively with related programs?

Explanation: The R,E&D program actively collaborates with its external partners on related programs and to leverage program funding. For example, the FAA/NASA Interagency Air Traffic Management Integrated Product Team and the FAA/NASA Aviation Safety Program share resources and conduct joint planning to achieve common aviation goals.

Evidence: FAA/NASA Interagency Air Traffic Management Integrated Product Team Integrated Plan; FAA/NASA Integrated Safety Research Plan; FAA Operational Evolution Plan; FAA/NASA MOUs and MOAs; The National Plan for Civil Aviation Human Factors: An Initiative for Research and Application (FAA, NASA, DOD); FAA/NASA Wake Turbulence Management Plan; FAA/NASA Roadmaps

Does the program use strong financial management practices?

Explanation: The program is free of material internal control weaknesses. Monthly accounting reports monitor fiscal status, the financial system has on-line, real-time inquiry capability for reporting and monitoring obligations. In FY 2004, FAA will implement a new cost accounting system that will strengthen financial management by allowing R,E&D to view plans versus and actual at various reporting levels, directly within the accounting system, and on a real-time basis; drill-down to the individual accounting transaction level; and reduce accounting errors.

Evidence: OIG report on FAA's financial statements for FY 2001-2002; FAA Performance and Accountability Report, Independent Audit Report, Financial Statements; all docs can be found on-line www1.faa.gov/aba/html_fm/finst.html

Has the program taken meaningful steps to address its management deficiencies?

Explanation: In response to R,E&D's human capital issues, FAA conducted an organizational assessment and found skill gaps in the areas of project/program management, leadership, and systems engineering. As a result, core curricula and training were instituted in these areas. Human Factors (HF) specialists were needed in integrated program teams, and, subsequently, HF specialists were hired and placed on those teams.

Evidence: R&D Executive Board Portfolio Development Process Project Management Plan; Draft R&D Strategy Assessment; ICIP documentation.

Does the program allocate funds through a competitive, merit-based process, or, if not, does it justify funding methods and document how quality is maintained?

Explanation: The R,E&D program allocates 100 percent of its funding (less some congressional direction) using a competitive process that ensures good science, proper use of public funds, and is consistent with Circular A-11. Once awarded, contract and grant progresses are regularly evaluated against scientific and technical criteria to ensure quality. Criteria are defined and reviewed internally and with external partners.

Evidence: Circular A-11; FAA's Acquisition Management System; FAA Grant Order; Research, Engineering and Development Advisory Committee and Subcommittee Recommendations and Reports; Center of Excellence Program documents; Small Business and Innovative Research Program.

Does competition encourage the participation of new/first-time performers through a fair and open application process?

Explanation:  

Evidence:  

Does the program adequately define appropriate termination points and other decision points?

Explanation:  

Evidence:  

If the program includes technology development or construction or operation of a facility, does the program clearly define deliverables and required capability/performance characteristics and appropriate, credible cost and schedule goals?

Explanation:  

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: The R,E&D Program is making significant progress in achieving its long-term goals. For FY 2002, the annual accomplishments will allow the R,E&D Program to meet or surpass its long-term goals.

Evidence: R,E&D Budget Linkage sheet; National Research Plan for Aviation Safety, Security, Efficiency, and Environmental Compatibility; FAA Operational Evolution Plan; National Aviation Research Plan; ARA Annual Performance Plan (faa.gov/ara/perform/); ARA Quarterly and Annual Performance Plan Goal Reports

Does the program (including program partners) achieve its annual performance goals?

Explanation: The R,E&D Program met or exceeded its annual performance goals.

Evidence: R,E&D Budget Linkage sheet; National Research Plan for Aviation Safety, Security, Efficiency, and Environmental Compatibility; FAA Operational Evolution Plan; National Aviation Research Plan; ARA Annual Performance Plan; ARA Quarterly and Annual Performance Plan Goal Reports

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: Although R,E&D's performance plans do not include efficiency measures or targets, the FAA has succeeded in a number of efforts that make conducting its research more efficient by reducing costs and increasing outputs. For example, R,E&D reduced overhead FTE from 27% to 13% in four years; reduced correspondence processing time from 9,050 minutes to 170 minutes per grant award; and expanded outputs of research per unit cost through Centers of Excellence, which require 50/50 cost sharing.

Evidence: FAA/NASA Interagency Air Traffic Management Integrated Product Team Integrated Plan; FAA/NASA Integrated Safety Research Plan; R&D Portfolio Development Process: Lessons Learned; R&D Portfolio Development Process, Guidance/Reference Document; R&D Executive Board Portfolio Development Process Project Management Plan; R&D Strategy Assessment (draft); FY 1999 Hammer Award; FTE data years FY 2000-2003.

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: There are no evaluations comparing R,E&D to other research programs.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The program is reviewed by an external advisory committee, the congressionally-mandated Research, Development and Engineering Advisory Committee, as well as external bodies, such as the National Academy of Science. These groups believe the Program is effective and achieving good results. The Advisory Committee also meets with NASA's research advisory committee once a year to ensure their is no duplication of effort and that resources are focused on high priority national research goals.

Evidence: Research, Development and Engineering Advisory Committee recommendations and subcommittee reports; National Academy of Science and Transportation Research Board Publications; R&D conference proceedings; FAA Technical Reports.

If the program includes construction of a facility, were program goals achieved within budgeted costs and established schedules?

Explanation:  

Evidence: