Detailed Information on the
NASA Advanced Capabilities in Space Exploration Assessment

Under the leadership of PA&E, benchmark R&T practices in performance & budget integration and performance measurement (i.e. efficiencies & evaluations) with other government agencies.

Enter into an arrangement with the National Research Council for an independent assessment of NASA's restructured Exploration Technology Development Program (ETDP) to determine how well the program is aligned with the stated objectives of the Vision for Space Exploration and assess the quality of the research. ETDP will report, and incorporate NRC recommendations into the Exploration Technology Development Program.

Establish an ongoing process to perform independent retrospective evaluations of the quality of Human Research Program (HRP). Conduct an independent evaluation for HRP to demonstrate the new process.

Establish means to maximize return on available resources, metrics to measure efficiencies gained, and demonstrate improved efficiencies for Space Radiation Research Facility.

Measure: Provide the knowledge and technology necessary to enable the Constellation program to meet its major milestones to develop the Orion Spacecraft by 2014 and return humans to the Moon by 2020.

Explanation:The Constellation program is the main customer for the investment in the Advanced Capabilities Division. To enable the Constellation Program to develop the Orion Spacecraft by 2014 and return humans to the Moon by 2020. The Advanced Capabilities Division will a) develop biomedical countermeasures to protect humans during space travel; b) develop topographical maps of the Moon; and c) develop technologies such as life support and habitation and surface power systems.

Measure: Reduce uncertainty in estimating the risk of fatal cancer due to radiation exposure during human space expeditions.

Explanation:Base-lined in 2004, HRP's best assessment of predicted risk of cancer due to radiation exposure in space has a 5-fold uncertainty. Reducing this risk uncertainty would allow for an improved estimate in the number of days astronauts can be safely exposed to a radiation environment. HRP has developed a research strategy to reduce this uncertainty by conducting experiments that will characterize and eliminate current sources of uncertainty. The challenge for HRP is to reduce the uncertainty for lunar (<2-fold) and Mars (<0.5-fold) architecture decisions on a timeline to support agency goals in vehicle, mission, and crew selections.

Measure: Develop an operational countermeasure to lower the risk of renal stone formation due to increased bone loss during long duration missions in microgravity to below 1%.

Explanation:Exposure to space flight results in increased bone loss and therefore potentially causes increased risk of renal stone. The HRP is seeking an operational countermeasure to lower the risk of renal stone formation impacting long duration missions in microgravity to below 1%. The successful achievement of these targeted milestones is key for extended duration human spaceflight. The target is to achieve the following set of deliverables that indicate progress in multiple years toward delivery of the operational countermeasure. In fiscal year 2006, although researchers made progress toward achieving this goal, the renal stone experiment was not completed. Additional data is planned for collection on one more subject. NASA expects to complete the study in FY 2007. In fiscal year 2007: Test proposed countermeasures via on-orbit ISS research experiment(s). In fiscal year 2009: Develop an operational protocol that meets the standards of the Office of the Chief Health and Medical Officer (OCHMO). In fiscal year 2010: Complete and provide an operational protocol for use on human spaceflight missions.

Measure: Characterize the level of risk of bone fracture in the long-duration microgravity environment, improve the predictability of a fracture event by 25%, and subsequently develop appropriate operational countermeasure(s) for human missions to Mars.

Explanation:Exposure to microgravity results in bone mineral unloading of the musculoskeletal system and loss of bone mineral density (BMD). Although the range of bone loss for ISS missions of 6 months can be estimated, it is unknown if the rate of loss will continue unabated during longer missions. It is also unknown if fractional gravity, such as that present on the moon and Mars would mitigate the loss. The HRP challenge is to assess the likelihood of the risk through modeling, ground-based testing, and ISS research, and to develop appropriate countermeasures in time to support lunar habitat assembly tasks. The results of this research will be a set of mechanical and pharmaceutical countermeasures for long-term human presence in space. The target is to achieve the following set of deliverables that indicate progress in multiple years toward delivery of the operational countermeasure. The Foot-Ground Reaction Forces experiment, concluded in April 2006, focused on understanding the mechanics of bone mineral loss. This was a key deliverable toward the output encapsulated in this measure. In fiscal year 2007: Complete initial bone fracture risk computational model. This model allows calculation of the uncertainty due to lack of knowledge (epistemic uncertainty) and uncertainty due to individual variability. The model will help drive down the epistemic uncertainty from 50% to 25%. In fiscal year 2009: Complete a validated ground analog for fractional gravity impact on bone loss. In fiscal year 2010: Initiate fractional gravity bone loss studies in the ground analog. In fiscal year 2016: Complete ISS research to quantify the risk of bone fracture; In fiscal year 2017: Identify Lunar countermeasures and habitat design considerations for further development; In fiscal year 2025: Perform an initial evaluation of lunar countermeasure suite during fractional gravity operations on the moon.

Measure: Number of biomedical risks for which NASA can reduce the probability and consequences to the health of humans in long-term space environment exposure.

Explanation:Thirty-one biomedical risks due to long-term exposure to the environment of space were described in the February 2005 Bioastronautics Roadmap (NASA/SP-2004-6113 available at http://bioastroroadmap.nasa.gov). ACD is developing countermeasures to mitigate these risks. The Countermeasure Readiness Level (CRL) scale (1 to 9) will be used to measure this progress. The measure tracks the number of research deliverables achieving a CRL of greater than 6 (i.e., countermeasure validation on the ground or in space).

Measure: Map and measure the topography of the lunar surface in preparation for future missions to the moon at a granularity of 50 meters horizontally and 1 meter or less radially.

Explanation:The LRO and LCROSS missions will carry out extensive measurements of lunar topography, resources, and thermal and radiation environments. This data about the lunar surface is critical for site selection, safe landing, and astronaut safety of the first human lunar sortie. The mission will: a) precisely measure lunar topography and rock abundance ( 25 times higher resolution than previous missions to the moon, which will provide images that can resolve features that are less than 1 meter in scale); b) assess meter-scale surface features to enable safety analysis of potential lunar landing sites; c) characterize illumination and thermal environments; d) characterize lunar mineralogy, and investigate the presence of water and other volatiles in permanently shadowed regions; e) measure the radiation environment to assess the potential hazards to human explorers; and f) demonstrate the technical performance of a synthetic aperture radar for scientific investigation of the lunar surface. For 2005 the Formulation Agreement Document (FAD); for 2006 Preliminary Design Reviews (PDR); for 2007 Critical Design Review leading to the non advocate review and authority to proceed to development; for 2009 Launch and 2010 data delivery to Planetary Data System for use by the Constellation mission planners and scientific community.

Measure: Cumulative number of technologies that are transitioned from Advanced Capabilities' Exploration Technology Development Program (ETDP) to either ISS or the Constellation Program.

Explanation:Technologies transitioned from ETDP to Constellation will impact the design of the Orion Crew Exploration Vehicle, its Ares 1 launch vehicle, the Ares V heavy lift launch vehicle, the crewed lunar lander, and lunar surface operations. Technologies transitioned to ISS will fill critical needs for ISS and also provide benefit to the Constellation System Program in the future. There will be a cumulative number of technology products transferred with an aggressive target of two every two years. A technology product is defined as a capability that mitigates a technical risk on a system or sub-system-level. A technology product may consist of multiple component technologies or a single key-technology.

Measure: Successfully identify, test and/or deliver technology products that enable the capability to put humans in Earth orbit, to launch and then land humans on the lunar surface.

Explanation:There are some key technologies that have to be developed to enable the Orion crew exploration vehicle to conduct flights to the ISS by 2014 and to launch and then land humans on the Moon with a target of 2020. Specifically, these technologies will be in support of the Orion Crew Exploration Vehicle, its, Ares I launch vehicle, the Ares V Launch Vehicle and will provide a future basis for landing on the lunar surface. The target is to achieve the following set of deliverables that indicate progress in multiple years toward the needed capabilities. In fiscal year 2006, the robotic systems ATHLETE Rover, K10, and Centaur were demonstrated for the first time, providing mobility and capabilities for planetary surface construction and inspection. Further, in fiscal year 2006, the RL10 engine 11 to 1 deep throttling was demonstrated; this is an essential technology for a lunar descent system. In fiscal year 2007: Deliver a prototype inflatable structure with an airlock, demonstrating capabilities for the construction of an inflatable lunar habitat. This will offer systems level benefit in terms of reduced mass and increased flexibility for a future lunar base. Demonstrate the on orbit operation of the Lab-on-a-Chip Application Development-Portable Test System (LOCAD PTS), which is an instrument for in-situ bacteriological measurement. In fiscal year 2008: Deliver test automated rendezvous and docking sensors for Orion to the Constellation Systems Program. In fiscal year 2009: Deliver a prototype 5-meter diameter ablative heat shield for Orion to the Constellation Systems Program. In fiscal year 2010: Test candidate prototype propulsion systems for the lunar lander. In fiscal year 2012: Provide concepts for lightweight composite structures to reduce the Ares vehicle mass. Test a prototype zero boil- off cryogenic propellant storage system to enable long-duration space missions.

Measure: Successfully identify, test and/or deliver technology products that enable sustainable life support systems for a future lunar outpost.

Explanation:There are some key life support systems technology products that have to be developed to enable increased resource loop closure (recover resources from wastes products (air, water and solid wastes) while reducing mass, power, volume and expendable requirements) for a sustainable human lunar outpost by fiscal year 2020. The technology development effort will entail demonstration of a series of prototypes in relevant environments (International Space Station, Orion Crew Exploration Vehicle, Ground Test-beds). Technology products will support waste management, air revitalization, water purification, atmospheric contaminant detection and efficient lighting, all with a goal of reducing resource utilization in a space habitat environment. These experiments will be occurring through time and their successful achievement is key to the ultimate outcome of human presence on the Moon and then in the solar system. The target is to achieve the following set of deliverables that indicate progress in multiple years toward the needed capabilities. In fiscal year 2006, a prototype Sorbent Based Air Revitalization (SBAR) system was tested. In fiscal year 2007: Test a prototype Carbon Dioxide and Moisture Removal Amine System (CAMRAS) for Orion. Test the Lab-On-Chip Application Development Portable Test System (LOCAD-PTS) for detecting bacteria on ISS. In fiscal year 2008: Establish an integrated life support laboratory test bed that incorporates prototype air revitalization, atmospheric monitoring, and fire detection systems. In fiscal year 2009: Deliver two instruments to detect atmospheric contaminants on the ISS: the Electronic Nose (E-Nose), and the Vehicle Cabin Air Monitor (VCAM). Deliver a Solid State Lighting Module to Space Operations Mission Directorate for validation and verification on the ISS. In fiscal year 2010: Deliver the Smoke Aerosol Measurement Experiment (SAME) for flight on the ISS. Demonstrate the capability to produce oxygen from lunar regolith to reduce the mass of consumables that must be resupplied from Earth. In fiscal year 2011: Demonstrate in a ground test bed a water recycling system the lunar outpost that reduces water needs by 95 percent.

Measure: Average percentage baseline cost overrun for space flight hardware under development.

Explanation:ACD projects estimated cost at completion will not exceed their baseline cost by more than 10% on average. The baseline cost is as provided at the official project confirmation and authority to proceed into implementation, per the key decision point encapsulated in the NASA Procedural Requirements 7120.5 "Space Flight Program and Project Management Requirements". For research tasks, not formally controlled under that NASA policy, the initial baseline per the signed research task plan will be the baseline used. The projects to be included in this measurement are LRO and LCROSS, and the research & technology tasks to be included are the prototype ablative heat shield for Orion, the Electronic Nose (E-Nose), and the Vehicle Cabin Air Monitor (VCAM).

Measure: Average percentage baseline schedule slip for space flight hardware under development.

Explanation:ACD projects estimated schedule at completion will not exceed their baseline schedules by more than 10%, on average. The baseline schedule is as provided at the official project confirmation and authority to proceed into implementation, per the key decision point encapsulated in the NASA Procedural Requirements 7120.5 "Space Flight Program and Project Management Requirements". For research tasks, not formally controlled under that NASA policy, the initial baseline per the signed research task plan will be the baseline used. The projects to be included in this measurement are LRO and LCROSS, and the research & technology tasks to be included are the prototype ablative heat shield for Orion, the Electronic Nose (E-Nose), and the Vehicle Cabin Air Monitor (VCAM).

Measure: Given an annual constant dollar technology funding, demonstrate improvements in the EVA Work Efficiency Index for humans and robots working cooperatively to deploy the power system infrastructure for the lunar outpost. The Work Efficiency Index (WEI) is defined as: WEI = (Time to complete a task using humans and robots) / (Time to complete a task using humans only)

Explanation:Work efficiency measurements will be conducted in annual desert field tests using humans wearing EVA suits and autonomous and teleoperated robotic assistants. Advanced robotics, EVA, and mission operations technology will reduce the time that humans need to complete the task.

Measure: Increase the amount of research beam time for space radiation experiments at NSRL, hence science data collection, by reducing the non-science overhead to 20% for set up, tuning and maintenance, assuming a fixed annual investment (in constant year dollars) and adoption of new operational procedures.

Explanation:NASA will increase the research beam time for space radiation experiments at the National Space Radiation Laboratory (NSRL) by implementing more efficient operational procedures. NASA currently purchases 950 hours of beam time per year from the Department of Energy, the facility owner, and 33 % of this time is overhead need for beam set-up, tuning, and maintenance. The new operational procedures will allow a reduction in the overhead times to allow more time for collecting research data, with a future target of reducing the overhead for set-up, tuning, and maintenance down to 20% by fiscal year 2011, while keeping the investment funds fixed.

Is the program purpose clear?

Explanation: The Advanced Capabilities Division (ACD) helps realize the President's Vision for Space Exploration by engaging in research and development (R&D) to provide new tools and knowledge to further space exploration missions. This set of programs does this by providing key scientific knowledge and technology development for NASA, and specifically the Constellation Program set of missions. ACD comprises three programs: the Exploration Technology Development Program (ETDP), the Human Research Program (HRP), and the Lunar Precursor Robotic Program (LPRP). ETDP matures and validates technologies for spaceflight systems, and transitions them to those systems. HRP provides new knowledge on the effects of spaceflight on humans, and matures and validates technologies to safeguard crew health and improve crew performance. LPRP executes robotic missions to the Moon to gain early information about the lunar environment and enable future human missions to the Moon.

Evidence: The purposes of ACD, ETDP, HRP, and LPRP and how they relate to the Vision are documented on NASA's website at: http://www.exploration.nasa.gov/programs/, http://www.exploration.nasa.gov/programs/human.html, http://www.exploration.nasa.gov/programs/lunar.html, and http://www.exploration.nasa.gov/programs/systems.html. The purpose of ACD programs are also provided in NASA's Integrated Budget and Performance Document (http://www.nasa.gov/about/budget/index.html) and are further detailed in NASA program authorization and implementation documents (e.g., HRP Program Commitment Agreement & HRP Program Plan, HRP-47051).

Does the program address a specific and existing problem, interest, or need?

Explanation: The President's Vision for Space Exploration calls for extended space exploration missions that cannot be accomplished without advances in spaceflight technology, human health and performance measures, and knowledge of the lunar environment. ACD engages in research and development to achieve these advances through a direct alignment with the Constellation Program set of missions. These programs are designed to provide technology, lunar surface knowledge and human scientific knowledge to conduct the short and long-term Constellation missions, which is the mainstay of the achievement of the Vision. The full required set of scientific knowledge and technologies does not exist today to meet Constellation's needs. Further, HRP is enabling the Office of the Chief Health and Medical Office to develop its standards and recognized protocols for the health and safety of NASA's astronauts as they support long-duration space missions. Each of these individual ACD programs has requirements linked to the requirements of Constellation OCHMO (for HRP) as outlined in their individual requirements documents. ETDP develops technologies in areas such as propulsion, energy production and storage, extra-vehicular activity, environmental control and life support, and in-situ resource utilization, all essential to reduce the technical risk for missions. HRP provides needed new knowledge and capabilities to protect crews exposed to microgravity, radiation, and isolation in spacecraft during long-duration exploration missions, thereby keeping crews physically and mentally healthy and productive. LPRP is currently developing robotic spacecraft to obtain additional lunar surface data, in areas such topography, temperature and lighting profiles, and available resources, to enable future human landing and extended presence. The ETDP portfolio was originally shaped by the Exploration Systems Architecture Study (ESAS) to include 12 technology discipline areas. With the maturation of the Constellation program and definition of its requirements, the ETDP portfolio has been further focused to map its technology portfolio to Constellation program prioritized needs, supplemented by needs developed by the Lunar Architecture team's technology assessment group. This is encapsulated in the Constellation Technology Insertion plan. The HRP portfolio is based on mapping astronaut health standards to the potential risks to meeting these standards based on the environmental factors to be seen in the Constellation set of missions. Spaceflight risks to human health, safety and performance have been documented internal to NASA and by external parties such as the National Academies' National Research Council. The understanding of these risks has been developed through years of scientific research by both NASA and other space faring organizations. The LPRP portfolio of missions derived their relevant lunar requirements flowed-down from the Exploration Technical Requirements Document and original Robotic Lunar Exploration Program requirements, in support of Constellation.

Evidence: The lunar exploration strategy, robotic and human collaboration, launch systems and operations, science, opportunities for international participation, commercial opportunities, and education and public outreach were discussed and made public at the 2nd Exploration Conference in December 2006 which is available at: http://www.nasa.gov/mission_pages/exploration/main/2nd_exploration_conf.html. Specifically, these proceedings describe the direction that NASA is heading in, the component program pieces, their program content and how they relate to each other. ESAS study is available at http://www.nasa.gov/mission_pages/exploration/news/ESAS_report.html. The Constellation Technology Insertion Plan is not yet available for public access but has been provided to OMB under separate cover. The National Research Council's report Safe Passage, which provides information on the risks to a human associated with spaceflight is available at: http://books.nap.edu/openbook.php?record_id=10218&page=R1 and NASA's Bioastronautics Roadmap is available at http://bioastroroadmap.nasa.gov/index.jsp). The LPRP portfolio of missions derived their relevant lunar requirements flowed-down from the Exploration Technical Requirements Document ESMD-RQ-0010, 2/22/05 and the original Robotic Lunar Exploration Program requirements document ESMD-RQ-0014 (2/22/05) have been made available to OMB. All ACD products are described on public web sites for the use of the U.S. general scientific and educational communities. These products provide general information on these programs and how they are aligned to exploration: http://www.exploration.nasa.gov/programs/human.html, http://www.exploration.nasa.gov/programs/lunar.html, http://www.exploration.nasa.gov/programs/systems.html, In addition, HRP documents its activities and plans at: http://hrp.jsc.nasa.gov/, http://taskbook.nasaprs.com/peer_review/index.cfm, and http://www.nsbri.org/ Information on LPRP activities for LRO and LCROSS can be found at: http://lro.gsfc.nasa.gov/, http://lcross.arc.nasa.gov/, and http://www.lprp.msfc.nasa.gov.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: NASA is the only US government agency chartered to send Americans into space to explore the Solar System, and ACD is the only program focused on providing knowledge and technology contributions to enable this activity. The program is specifically focused on NASA's needs for realizing this charter. Other entities develop knowledge and technologies for spacecraft such as the Department of Defense and the private sector. Other entities participate in health research that may be relevant to crew safety, such as NIH and NSF and academia. To assure that duplication is not seen with these other federal agencies, industry and academia, the ACD coordinates its research programs with these entities through joint working groups and joints projects where the parties define their individual roles through an agreement. The ETDP coordinates its research programs with DoD by participating in interagency working groups in common technology areas, and by partnering with other government agencies such as the DARPA, USAF, and DOE in technology development projects where each party has a defined role through a documented agreement. The HRP has a long history of issuing joint solicitations with NIH over a number of years in the area of Biomedical research and also participates in common research focus area working groups. With respect to lunar mapping, none of the current known foreign missions (e.g., Selene http://www.isas.ac.jp/e/enterp/missions/selene/index.shtml or Chandrayaan http://www.isro.org/chandrayaan/htmls/home.htm) have the ability to produce spatial resolutions close to or meeting all of NASA's LRO goals. NASA has made efforts and instituted mechanisms to share the data from the Selene and the Chandrayaan missions. Further, NASA assures that there is minimal redundancy in the research and technology development efforts with its international partners by participating in joint working groups in space life sciences, life support systems for spacecraft, lunar planning, and mars planning activities. State and local governments do not have any programs focused on the same mission as ACD. NASA also maintains international partnerships and working groups to share scientific data and knowledge world-wide and to help better leverage federal funds.

Evidence: Some direct examples of interagency working groups and partnerships for joint technology development for spacecraft related technologies: ACD's ETDP Program Executive co-chairs the NASA DOD propulsion working group. NASA is cooperating with DoD on payloads through the Space Experiment Review Board. NASA also works with DoD to make sure that it's portfolio is not redundant in solicitation calls for SBIR and STTR. ACD is leveraging work being done in Canada for In Situ Resource Utilization (Resolve project) and in low pressure plant growth. NASA is working with the DOE on its Affordable Nuclear Fission Surface Power Program. NASA has 33 currently active Memorandums of Understanding (MOU) with various federal agencies, which include NIH (through its various organizations such as the NCRR, NIDCD, NIA), DOD, DOE, NOAA, NSF, ONR, CD, USGS, and USDA. These are the areas of common research and these MOUs are used to clearly define the different parties roles and responsibilities and assure no duplication occurs and that each party can leverage the work of the other. Areas of collaboration include various aspects of Biomedical research such as Sensory and Vestibular function, Cardiovascular, Pulmonary and Hematologic Studies, Aging, Neurology, Radiation Health, Musculoskeletal and Exercise Physiology, Food Safety, Water Reclamation, Plant Research and use of Testbeds. Within NASA, ACD collaborates with the other mission directorates (Science and Space Operation Mission Directorates) to validate technologies (Environmental monitors and Lighting Technologies on ISS with SOMD) and seek synergism in Exploration missions (LRO mission collaboration with SMD). Recently HRP formed a joint working group with NIH, NSF and other related institutions to identify issues related to human health. This is being done under the guidance of the NASA Advisory Council. As an example the draft MOU has been provided to OMB.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The ACD programs are well designed for what they are intended to do. The three programs are designed to focus their resources on the areas of scientific knowledge and technology development that enable the achievement of NASA's goals for exploration, as enabled through the Constellation Program set of missions. Processes have been set and implemented to assure that the priorities of ETDP, LPRP and HRP reflect the priorities of Constellation. In addition to the program portfolios mapped to priorities, the implementation of the activities are set forth in plans and heavily reviewed to assure that the plans reflect these priorities and can meet the technical requirements. The programs are held to NASA standards and requirements for the conduct of programs and projects per NASA Procedural Requirements (NPR) 7120.5 "NASA Spaceflight Program and Project Management Requirements" and NASA Procedural Requirements (NPR) 7120.8 "NASA Research and Technology Program and Project Management Requirements". This set of requirements provides minimum standards for oversight, project/program conduct and reviews, and the ACD programs follow the processes for program management prescribed in NPR 7120.5 and its sister document NPR 7120.8 for R&T activities. Each review results in actions to address any deficiencies before the program or project is allowed to proceed further. For example, the LPRP's LRO and LCROSS projects have both conducted System Requirements Reviews (SRR), Preliminary Design Reviews (PDR) and Critical Design Reviews (CDR) that assess design maturity and adequacy of requirements. There are other reviews that have been conducted as well. Independent Review Boards review the technical requirements and designs to ensure the program effectiveness. There is no strong evidence that another approach would be more efficient or effective.

Evidence: Evidence of the design and prioritization processes are provided in the evidence section of questions 1.1, 1.2 and 1.5 and on further questions in sections 2.0 and 3.0.. The evidence provided here demonstrates how the program and its projects mitigate identified design flaws. Design flaws have been mitigated for LPRP missions through the conduct of SRR (8/16/05), PDR (2/10/06), Confirmation Review (5/17/06), Integrated Baseline Review 6/28/06), and CDR (11/9/06) for LRO and conduct of SRR (7/11/06), PDR (9/6/06), Confirmation Review (1/19/07) and CDR (2/22/07) for LCROSS. The detailed results of these are captured in the minutes of the meeting as conducted at the Agency Program Management Council available to the public at: NASA's Procedural Requirement for the conduct of space flight development programs and projects is available at: http://nodis3.gsfc.nasa.gov/. The NASA Procedural Requirement for the conduct of research and technology programs and projects is currently under revision and undergoing senior management review.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The three programs are designed to focus their resources on the areas of scientific knowledge and technology development that enable the achievement of NASA's goals for exploration, as enabled through the Constellation Systems Program (CxP) set of missions. Processes have been set and implemented to assure that the priorities of ETDP, LPRP and HRP reflect the priorities of Constellation, the intended beneficiary of the ACD products. HRP meets the requirements of the Office of the Chief Health and Medical Officer (OCHMO). ACD's constituent programs regularly conduct technical interchange meetings with the customer to assess the progress so that products (research knowledge/technologies) are synchronized with the Preliminary Design Reviews (PDRs) for particular missions. ETDP receives technology development priorities from the CxP, and uses this to guide program investment decisions in future space technology. Priorities established by Constellation Systems Program are documented in Constellation Program Technology Insertion Strategy Document (CxP 70079). ETDP is completing the process of developing agreements that will document insertion points of these technical efforts to assure that the CxP Projects are ready to receive the ETDP products at the needed timeframe. The ETDP matures high-priority technologies and demonstrates them in relevant environments so that they are ready to be used in the preliminary design of flight systems. The CxP assumes management responsibility for technologies that have advanced to TRL 6. Projects were selected through a Broad Agency Announcement/ Internal call for proposals and awarded to the "best" entity in support of the ETDP requirements. HRP has aligned it's research and development tasks to the standards for human health during spaceflight provided by OCHMO, mapped against the environment that astronauts supporting the CxP missions are expected to survive in and then return to Earth from. A series of risks to not meeting these standards due to the long-duration flight and space outpost environments were identified. These are contained in the HRP Bioastronautics Roadmap, which provides a research plan that lays out the key areas for investment. Further, characterization of the risks is performed and the highest risk to humans for the environments the CxP expects are mitigated through the work contained in the HRP portfolio. Funding is applied to the priority risk areas. HRP supports CxP design reviews and aligns tasks to address identified program requirements for human health maintenance and human factors design. The HRP portfolio is mostly solicited through the NRA mechanism and awarded based on relevance to the program requirements. Directed research projects that are lead through the NASA Centers, are peer reviewed through Non-Advocate Review Panels, to assure that the resources are employed on the best proposal including relevancy to the requirements. The LRO/LCROSS mission provides information about the Lunar terrain, to resolve the many unknowns about the Lunar Poles thus enabling the CxP to better design its spacecraft and to reduce its risks for Lunar landing systems. The LRO mission's instrumentation was selected to satisfy the Requirements detailed in the Robotic Lunar Exploration Program (RLEP) Requirements ESMD-RQ-0014. Level 1 Requirements for the Lunar CRater Observation and sensing Satellite (LCROSS) are documented in LPRP-RQMT-1003 baselined in October 2006. The instruments for these missions were chosen through Announcements of Opportunity and selected to best support the needs of CxP. Because ACD's activities are focused on the needs of CxP, itself a NASA program, ACD does not provide any inappropriate subsidies.

Evidence: Technology prioritization methodology is described in Constellation Program Technology Insertion Strategy Document (CxP 70079). Priority flow down and resource alignment to these priorities for ETDP is found in NASA's Integrated Budget and Performance Document (IBPD) found at http://www.nasa.gov/about/budget/index.html which outlines the objectives, long-range performance goals mapped to the resource allocations to the projects. Each project has a Project Plan that defines the organization, technical approach, milestones, and resource allocation to performing organizations. The following document describes the health standards that are planned to when setting the HRP portfolio: NASA NPD 8900.5A, "NASA Health and Medical Policy for Human Space Exploration," found at http://nodis.hq.nasa.gov/displayDir.cfm?t=NPD&c=8900&s=5A; and NASA NPD 8900.1F, "Medical Operations Responsibilities in Support of Human Space Flight Programs," found at http://nodis.hq.nasa.gov/displayDir.cfm?t=NPD&c=8900&s=1F. Additionally, there are some pre-existing documents for Health and Medical Standards And Requirements that are also being used: JSC 26882, "Space Flight Health Requirements", NASA STD 3000, "Man- System Integration Standards" http://msis.jsc.nasa.gov/Volume1.htm , and the Astronaut Medical Evaluation Requirements Document (AMERD)). The HRP Bioastronautics Roadmap can be found at: http://bioastroroadmap.nasa.gov/index.jsp The NASA Announcement of Opportunity for the Lunar Precursor missions can be found at: http://research.hq.nasa.gov/code_s/nra/current/NNH04ZSS003O/index.html

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The measures are designed to ensure the successful development of the critical technologies needed to achieve the objectives of the January 14, 2004 Vision for Space Exploration, http://www.whitehouse.gov/space/renewed_spirit.html and http://exploration.nasa.gov/documents/vision.html, and the NASA Authorization Act of 2005 (http://thomas.loc.gov/cgi-bin/query/z?c109:s.1281:) including meeting the timeline for the initial missions to the ISS and the lunar surface. ACD has a long-term outcome measure realted to the Vision, and several key output measures in support of the outcome. ACD's mission is to provide the knowledge and technology necessary to enable the CxP to meet its major exploration milestones to develop the Orion Spacecraft by 2014 and enable the human return to the Moon by 2020. ETDP provides key technologies to enable the Orion crew exploration vehicle and living on the lunar surface and will be held to some significant output measures in support of this. HRP will work towards the reduction of uncertainty in estimating the risk of fatal cancer due to radiation exposure and the delivery of key countermeasures including the reduction of risk of bone fractures so that exploration mission physical tasks can be safely performed.

Evidence: The long-term performance goals and associated outcomes are documented in the PART performance measures tab. Additional measures employed by the program are in NASA's Annual Performance Plan which can be found at: http://www.nasa.gov/about/budget/index.html

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ACD's long term goals have specific, amibitious targets and timeframes that are consistent with Constellation mission planning. These timeframes extend to 2020, when NASA plans to return humans to the Moon. The goals of Constellation and its mission set are ambitious given that it has been over 30 years since the nation has developed a new human space transportation system and delivered them to the Moon. Annual technical milestones for assessing progress are defined in Program and Project Plans. Technical milestones are tied to long-range performance outcomes and outputs. Project Plans show a schedule for maturing major technology products to specific Technology Readiness Levels (TRLs). The goal for all three programs is to deliver mature technology products at TRL 6 (prototype demonstrated in a relevant environment) to the Constellation Systems Program in time to support program requirements.

Evidence: The targets are documented in the PART performance measures tab. Constellation's ambitious goals are documented in the Constellation Architecture Requirements Document which was provided to OMB under separate cover. For example, the timeline is extremely aggressive to estimate the risk of fatal cancer due to radiation exposure during human space expeditions, the research challenge is fourfold: 1) humans have not traveled prior into these environments so there is no previous body of data, hence it must be created in complex ways to not harm humans by exposing them to this environment; 2) there are multiple types of radiation that have a complex interaction that must also be simulated; 3) is a human-system level problem that must be extrapolated to a cellular level; and 4) the interaction of this radiation with gravity as testing is done on Earth complicates understanding of the absence of gravity and how it interactions in space. To date there has been a 5-fold reduction in this uncertainty and NASA is pushing this to a 2-fold reduction by 2010.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The ACD programs have specific annual performance measures that demonstrate progress toward achieving the program's long-term goals and outcomes. These annual measures serve as performance indicators that ACD is on track to deliver the key scientific knowledge and technologies in support of the Constellation missions. The annual measures demonstrate continued progress in key research areas in both ETDP and HRP that are directly linked to both the Orion spacecraft design and that of the various launch vehicles. For example, ETDP plans to deliver, in fiscal year 2009, a prototype 5-meter diameter ablative heat shield for the Orion spacecraft. There is almost yearly progress planned for LPRP, demonstrated through a series of annual performance goals (targets of performance), that will be necessary to provide the lunar topography data for the 2020 return to the Moon by humans.

Evidence: The annual performance goals and their relevance to the associated outcomes are documented in the PART performance measures tab. Additional measures employed by the program are in NASA's Annual Performance Plan which can be found at: http://www.nasa.gov/about/budget/index.html

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The ACD programs have baselines and ambitious targets for their annual measures. Each measure includes annual targets for performance assessment at the year-end. As stated previously, each of the annual targets demonstrate an indication that the programs are on track for meeting their science and technology deliverables toward he goals of Constellation, which are ambitious given that it has been over 30 years since the nation has developed a new human space transportation system and delivered them to the Moon. Combining this ambitious goal, with the current state of knowledge and technology supports that these are ambitious targets. To add stringency there are two annual measures, held on top of these science and research targets, which hold all of these deliverables to cost and schedule goals. Examples of this are provided in the Evidence section below. Each project has near-term, annual technical milestones that demonstrate progress toward achieving long-range program objectives, which the targets reflect. These annual technical milestones (targets) for assessing progress are also defined in Program and Project Plans. Technical milestones are tied to long-range performance outcomes and outputs.

Evidence: The annual performance targets and their relevance to the associated annual measures are documented in the PART performance measures tab. Additional measures employed by the program are in NASA's Annual Performance Plan which can be found at: http://www.nasa.gov/about/budget/index.html For example the delivery of a prototype 5-meter diameter ablative heat shield used to assure the safe reentry into an atmosphere, in fiscal year 2009, in support of the development of the Orion Crew Exploration Vehicle, is an extremely aggressive target. There has never been a heat shield of this size used in space, nor with the multiple requirements for this one. It is being developed in a similar timeframe to past heat shields. The Apollo heat shield was 3.9-meters and had to only support a single mission, to take the spacecraft to the Moon and then back to Earth. The Orion heat shield must be able to survive missions to the International Space System, the Moon and beyond, a much more complex set of requirements. This project is also being asked to do this within a certain cost and schedule, which will be measured annually.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ACD program activities are conducted/executed by personnel working at NASA Centers, as well as in industry, universities, and other government agencies. Each of these personnel and participating organizations have specific, documented roles in achieving the program goals, and they participate in program reviews, technical interchange meetings, and annual planning. The programs use all instruments that are available to federal agencies to enter agreements with other entities to obtain commitments to working toward and reporting on progress in achieving the annual and/or long-term goals of the programs. For example, HRP research solicitations are designed to solicit work that will fill specific research gaps associated with human health and performance space flight risks. During the process of assessing research merit in the evaluation of proposals HRP also assesses whether that research is aligned with its goals and timeframes. The various contracts that are employed on LPRP and ETDP outline what these parties are to deliver in support of the program/project goals. Further, many commitments between cost-sharing partners, other federal agencies and other government partners are captured in a formal memorandum of understanding/agreement. Roles of performing organizations are documented in program and project Plans. Partner activities are planned and integrated into a master schedule. Program and Project Plans indicate how each project contributes to addressing the annual and long-term program goals. Programs awarding grants adhere to NASA Grant and Cooperative Agreement Handbook, NPR 5800.1 and Human Research Program Science Management Plan (HRP-47053), which provide a basis for how the grantees will be held to the NASA goals.

Evidence: A representative project plan was provided to OMB under separate cover. The NASA Procedural Requirement 5800.1 "Grant and Cooperative Agreement Handbook", NPR 5800.1 can be found at: http://nodis3.gsfc.nasa.gov and the Human Research Program Science Management Plan (HRP-47053) can be found at: http://hrp.jsc.nasa.gov/ and has also been provided to OMB under separate cover. The Exploration Systems Mission Directorate Aquisition Policy Document. The contract for the delivery of the LCROSS, awarded to Northrup-Grumman has been provided as evidence to OMB under separate cover. Representative memorandum of understanding and a list of all those being employed by ACD has been provided to OMB under separate cover.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The ACD programs have cerain processes in place to conduct independent evaluations for the relevance, quality of research and the performance toward planned activities in support of the research goals, but these processes are not complete. HRP has not demonstrated adequate independent retrospective analysis of the quality of its research portfolio. Each of the three programs, ETDP, HRP and LPRP, has a process that is tailored to the type of work it conducts. Reviews are tailored for basic research versus applied research and technology development, and research activities versus the design and build of hardware. All projects and program elements follow the NASA standards for the conduct of program/project management and grants management which require independent review both of which call for a series of independent reviews. Customers (OCHMO and CxP) conduct relevancy reviews of the ACD portfolio. The relevance of the research portfolio of HRP was validated by independent external parties, specifically, the National Academies Institute of Medicine (IOM). The IOM convened a committee, on behalf of NASA, to address the need and methods for getting answers to questions about astronaut health. The committee identified the clinical and health services research that would be required before and during long-duration missions. This study was used to guide the investments that HRP undertakes at a strategic level. HRP also developed the Bioastronautics Roadmap to guide at investments at a more tactical level, also reviewed by IOM. HRP mainly awards grants and a comprehensive set of independent evaluations are regularly conducted to both award through independent peer-review and then track progress. If progress is not demonstrated or lacks the grantees are redirected or the grant is cancelled. These evaluations ensure HRP funded research is of high quality, sufficient scope to achieve desired outcomes, and unbiased. These reviews assure that the research remains relevant to the NASA customer. Annually, HRP's grants are reviewed for performance to task by processes guided by NASA's grants management standards. ETDP's portfolio was shaped by the Exploration Systems Architecture Study which identified the key research and technology areas . The ESAS was conducted through the use of an external, independent chair, supplemented by NASA and non-NASA personnel. Most recently, ACD chartered the Space Studies Board to convene a Panel of Experts meeting that evaluated the process and content of the ETDP non-exploration research portfolio. ETDP is planning a review of it the relevance and quality of its current research portfolio later this year by the National Academies. The specific research tasks were selected by a peer-review process just prior to ESAS and then down-selected to the current set when the requirements of ESAS were overlaid. For LPRP, the major review milestones were completed this year including LRO Systems Requirement Review (SRR), Preliminary Design Review (PDR), Confirmation (authority-to-proceed to development) Review (CR), Integrated Baseline Review (IBR), and the Critical Design Review (CDR); and LCROSS SRR, PDR, CR and CDR. Teams of experts and non-advocate reviewers conducted all reviews. All of the ACD programs are subjected to an Agency-level quarterly performance review, entitled the "State of the Agency" process, which independently evaluates programs and projects. On a monthly basis, an independent assessment team (consisting of the Offices of Program Analysis & Evaluation, the Chief Engineer and Safety and Mission Assurance) provides the Agency with a performance assessment and rating for all key programs and projects. The results of the independent assessment (technical, schedule, cost, & programmatic) are reported at the Agency Program Management Council (PMC) and inform decisions at the Agency and the Mission Directorate (program) level on the program and project directions.

Evidence: NASA's Procedural Requirement for the conduct of space flight development programs and projects is available at: http://nodis3.gsfc.nasa.gov/ The NASA Procedural Requirement for the conduct of research and technology programs and projects is currently under revision and undergoing senior management review. The National Research Council's report Safe Passage is available at: http://books.nap.edu/openbook.php?record_id=10218&page=R1 and NASA's Bioastronautics Roadmap is available at http://bioastroroadmap.nasa.gov/index.jsp). The specific objectives and frequencies of all HRP independent reviews are baselined the Human Research Program Science Management Plan (HRP-47053) which has been provided to OMB under separate cover. The National Space Biomedicine Research Institute (NSBRI), one of HRP's grants management processes, 5 year review was completed in December 2005, the results of which has been provided to OMB under separate cover. Annually a solicitation for research proposals is made, through the NSBRI. These proposals are evaluated by independent review panels for technical merit and quality, as a first gate., then are further reviewed by NASA to assure that they are delivering a product that is aligned with its research needs and for a cost that reflects best value. The specific objectives and planned frequency of all ETDP independent reviews are baselined in the program and project plans. State of the Agency process- independently evaluate programs; the monthly Agency PMC report; and the Mission Directorate (program) quarterly independent assessment response are internal documents that have been provided to OMB under separate cover.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NASA justifies its budget request to Congress in its annual Integrated Budget and Performance Document (IBPD). This document describes each program, provides the budget request for each program, and presents the annual performance goals of each program. The performance goals as presented in the IBPD are the same as those reported for PART. The IBPD presents the full cost of each program, including all direct and indirect costs. At program or project inception the linkage of budget and performance is set. The programs' ability to accomplish annual and/or long-term goals is factored into how NASA and the mission directorate determine the direction and priorities. Through the annual budget formulation process, the linkage of the performance commitments to past appropriations, the current request and any past performance issues are linked into the Congressional justification. The effects of budget changes versus what was originally planned, on the achievement of the performance goals can thus be identified and is done so yearly. Issues are dealt with during the budget formulation process, with an eye on the impact to the planned performance commitment.

Evidence: The FY 2008 IBPD is available at http://www.nasa.gov/pdf/168652main_NASA_FY08_Budget_Request.pdf

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ACD and its component programs regularly review strategic plans and measure/adjust them against programmatic resources. NASA uses a series of strategic architecture studies and roadmapping activities to outline the required strategic direction to achieve key Agency goals. Programs are aligned to this strategy. HRP and EDTP both realigned their portfolios after the ESAS activity to better their customer's directions. LPRP projects participated in Lunar Architecture Team (LAT) tasks and identified goals, objectives and requirements for the LPRP program in support of the Constellation missions. When this activity was completed, LPRP assured that its projects aligned to the strategic direction. When appropriate, outside entities such as the NAC and the NRC are engaged in answering questions about strategic planning and portfolio alignment. For example, the National Research Council's Aeronautics and Science Evaluation Board (ASEB) is currently reviewing NASA's radiation shielding plans. The study will recommend a strategic plan for developing the necessary radiation mitigation capabilities to enable the planned lunar architecture. The results of this will be factored back into NASA's current strategy.

Evidence: The IBPD document describes the realignment of ETDP content after ESAS and LAT activities. HRP has supported the OCHMO to re-baseline the Spaceflight Human Systems Standards Volume I and is currently in the process of working with OCHMO on the baselining of Volume II environmental health and human-system interfaces during human spaceflight, and define the standards for space facilities and related equipment which directly interface with crewmembers during spaceflight and in lunar and planetary habitats (e.g., moon and Mars), and the procedures to operate and maintain the facilities and equipment. These documents had been focused on ISS and Space Shuttle missions. LPRP submitted its goals and objectives in the Formulation and Authorization Document (FAD) and Program Plan consistent with the LAT results. Details on the realignment of these programs after the key architecture studies can be found in NASA's budget justification: http://www.nasa.gov/about/budget/index.html Details on the current study on radiation shielding and its relevance for the strategic direction of these programs, can be found at: http://www8.nationalacademies.org/cp/projectview.aspx?key=48696. The Evaluation of Radiation Shielding for Lunar Exploration, study number ASEB-J-06-01-A.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: All ACD programs conduct analyses of alternatives at the portfolio, program/project and research task-levels. The ACD portfolio was mainly driven by the analysis of alternatives with cost, schedule and risk implications, that occurred with the Exploration Systems Architecture Study (ESAS) and Lunar Architecture Team (LAT) architecture studies. The results of this analysis were further refined as the Constellation Program has matured and their portfolios were refined. To support decisions within the portfolio, each program performs a gap analysis that takes into account the current state of the art and the required capabilities for future missions, and performs trade-offs with respect to cost, schedule and performance enhancements that an investment will offer. Once an investment is decided on and encapsulated in a research and/or technology development project, the method of research and the specifics are set by reviewing the alternatives proposed through a NASA Research Announcement (NRA)/Announcement of Opportunity (AO) proposal award process, the Non-Advocate Reviews (NARs) process for directed research or at an Integrated Baseline Review (IBR)/Confirmation Review (authority-to-proceed into development) or NAR of a development project. Recently, all of these mechanisms were employed for analyzing the alternatives and trade-offs for cost, schedule, risk and performance goals. ESAS and LAT occurred in fiscal years 2006 and 2007, respectively which resulted in the portfolio of ACD. ETDP was formulated to address the technology priorities for lunar exploration identified by ESAS and LAT. ESAS conducted an analysis of alternatives, cost, and risks for its projects and research portfolio in support of this. HRP aligned its portfolio post the ESAS activity to focus on research and technology development tasks that focus on the standards of medical care provided by the OCHMO and maximize use of the ISS. HRP has used multiple NRAs and NARS to down select to the current set of research tasks. Further HRP is currently undergoing an NRA process in support of its research on radiation effects due to long-duration space flight. LPRP conducted an AO to down-select for the LRO mission in fiscal year 2005 and the LCROSS instruments in fiscal year 2007. For each LPRP selected project, independent review boards provided a technical, cost and schedule assessment to support the Confirmation Reviews.

Evidence: There have been 4 NARs since HRP was formed in October 2005: two in-flight studies ("Nutritional Assessment" and Nutrient Stability") were passed: one ground-based study ("Exercise Countermeasure Baseline Capabilities") was withdrawn for revision; and one flight study ("Dysrhythmia") is still under review. The results of the HRP current NRA process will not be available until later in June but can be provided at that time. The LRO Confirmation Review was completed in May 2006 and the LCROSS Confirmation Review completed in January 2007.

If applicable, does the program assess and compare the potential benefits of efforts within the program and (if relevant) to other efforts in other programs that have similar goals?

Explanation: This question is non-applicable. There are no other programs that share the goals of ACD for providing key scientific knowledge and technology for delivery and then long-term duration of humans in space. No other U.S. entity has this in its charter. Where appropriate, ACD programs such as ETDP evaluate a wide range of alternative technologies and approaches, and compares these to other government and industry programs with overlapping technology portfolios.

Evidence: N/A

Does the program use a prioritization process to guide budget requests and funding decisions?

Explanation: The Exploration Technology Development Program (ETDP) receives technology development priorities from the Constellation System Program, and uses these priorities to guide program investment decisions in future space technology. Technology priorities established by Constellation Systems Program are documented in Constellation Program Technology Insertion Strategy Document (CxP 70079). The program investments are guided through the Exploration Systems Mission Directorate (ESMD) Research & Technology Prioritization Process. The Human Research Program (HRP) receives science and technology development priorities from the Constellation System Program and from the priority risks that have been identified for human health during spaceflight encapsulated in the Bioastronautics Roadmap, and uses these priorities to guide program investment decisions. HRP then further prioritizes its work based on which appropriate platforms are available for that research conduct including ground-based facilities, ISS and various spaceflight analogs (for example, bed rest) to appropriately retire these risks. The program investments are guided through the Exploration Systems Mission Directorate (ESMD) Research & Technology Prioritization Process. The Lunar Reconnaissance Orbiter (LRO) mission's instrumentation was selected to satisfy the Requirements detailed in the Robotic Lunar Exploration Program (RLEP) Requirements ESMD-RQ-0014. Level 1 Requirements for the Lunar CRater Observation and sensing Satellite (LCROSS) are documented in LPRP-RQMT-1003 baselined in October 2006. These priorities were developed as a response to Constellation's needs. Each of these projects at the time of their Confirmation Review set a requested budget profile that is revisited annually in the budget formulation process. The elements of the Exploration Systems Mission Directorate (ESMD) Research & Technology Prioritization Process are as follows: 1) Identification of R&T development requirements by operational programs in a time-phased manner, based on the reference architectures, current state of the art, and other analyses; 2) R & T Programs use these requirements to develop proposed research and technology development tasks; 3) Both the operational and developmental programs through technical interchange meetings come to an agreement on the proposed R&T portfolio; 4) The R&T portfolio is mapped to requirements and presented at the Directorate level; 5) Customer-Supplier agreements are developed to align budgets and milestones to meet the insertion schedule needs of operational programs. Annually, all priorities are reviewed at an Agency-level as budget formulation occurs. NASA employs the Planning, Programming, Budgeting and Execution (PPBE) process, which reviews past performance and planned budgets when determining the next budget year request.

Evidence: The Exploration Systems Mission Directorate Research & Technology Prioritization Process Implementation Plan (ESMD-R&TPIP 1.0) and the Technology Prioritization Process Plan (ESMD-TIP-12.06-Revision-1) have been provided to OMB under separate cover. These documents describe the set of procedures, processes and tools used to determine the priorities for the ACD research and technology portfolio and flow it to budget requests. The Constellation Program Technology Insertion Strategy Document (CxP 70079) has also been provided to OMB under separate cover. NASA NPD 8900.1F (Medical Operations Responsibilities in Support of Human Space Flight Programs) describes the medical standards that HRP must meet. Additionally, there are pre-existing documents for Health and Medical Standards And Requirements that are also being used (JSC 26882, Space Flight Health Requirements, NASA STD 3000, Man- System Integration Standards, and the Astronaut Medical Evaluation Requirements Document (AMERD)). The Bioastronautics Roadmap may be found at: The NASA PPBE process be viewed at: http://www.nasa.gov/offices/ocfo/references/ocfo_fmr_detail.html

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: All ACD Programs and Projects collect timely and credible performance information. Performance data is collected on cost, schedule, risk mitigation and technical/research performance. Data is collected on a monthly, quarterly and annual basis. This data is then reviewed at the appropriate management level and redirection, if warranted, occurs. HRP collects monthly technical, performance and cost information from contractors and cooperative agreement partners, as well as on some period from grantees. Grantees submit an annual report on technical progress. Data are reviewed monthly by HRP responsible organizations and quarterly by the HRP Program Manager. The responsible organizations review actual costs against the plan on a monthly basis, and the HRP Program Manager and ESMD management also conduct Quarterly Program Reviews to monitor costs and scientific/technical performance. ETDP collects technical accomplishments, schedule status, earned value data and financial status every month from all projects. The program uses this information to develop risk mitigation strategies, adjust priorities, make resource allocations, or take other appropriate management actions. ETDP projects report status monthly to the Program Office. LPRP requires that LRO and LCROSS projects report monthly against baseline schedules, baseline cost and performance requirements that were approved at Confirmation Review. Also, LRO project uses the Earned Value Management System and actively manages project risks, and provides this data on a monthly basis to program management. Quarterly, all three programs report to the Exploration Systems Mission Directorate Management at a Mission Directorate Program Management Council (MDPMC) meeting. The ESMD Windchill database is used to track monthly status reports from all projects. Further, the Agency collects cost, schedule, technical and risk data on each of the ACD programs and specific key projects for the State of the Agency review, held at the Agency Program Management Council (APMC) which is managed through the Office of the Chief Engineer (OCE). Each of the PMCs is a decision-making body, per NASA's governance processes, that acts on the performance information and relevant program/project information brought forward. One example of this is provided below in the evidence section below that also turned out to be a management deficiency.

Evidence: SAP and Business Warehouse, are NASA's systems of record for financial data. ESMD's Resources Management Reporting System (RMRS) documents the official current year phased plan and approved budget changes. Windchill records are available on demand as evidence but the system is password protected, hence, no link is provided here. Performance review presentations and minutes of the Program Management Council Meetings have been provided to OMB under separate cover. An example of how these meetings are used at the Mission Directorate level is that during a MDPMC meeting it become apparent during the LCROSS portion of the review that the project was encountering a breakdown in communication with the launch vehicle provider. Many data items that were needed for the spacecraft design, were not scheduled to be delivered on time while other were not even in the projects plans. There were schedule and deliverable disconnects that became visible due to the review. ESMD management assigned an action to the program to get to the bottom of the root cause of the issue, which resulted in an assignment of responsibility to an individual and creation of a new project role.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Explanation: All ACD program and project managers and partners (NASA Centers, contractors, subcontractors, universities, other government agencies) are held accountable for their performance. Each contract in HRP and ETDP is a performance-based contract. The majority of the contracted work in LRO is of a performance-based cost plus award fee contracts nature. These are primarily support service contracts. Contractors are reviewed every 6 months, or on some periodic frequency, and award fee determination or continuation is made based on established criteria, including cost, schedule and technical performance. For LPRP, all deviations from the baseline plan are evaluated by objective standards for their impact on LPRP projects, and impacts are highlighted to the program office, ESMD, and to the agency for action. All ETDP projects are monitored the Earned Value Management System that takes into account the cost, schedule and performance. This reporting is performed for all intramural and extramural projects. Internally, action is taken to remove or change project or program management if results cannot be seen. The program and project managers are held accountable for the success of their respective programs and projects. The Program Managers directly report to ESMD Associate Administrator. Their performance metrics are documented in the individual performance plans for each Program Manager. These include among other aspects the execution of programs with time-phased milestones. Any deviation from the planned execution goals have to be explained to the AA to his/her satisfaction. Parallel to this Project Managers report to their respective Center Director's and through their individual performance plans they are held accountable in the same way. More specifics on grantees and how they are held accountable is found in the response to question 3.CO2, later in this section.

Evidence: Monthly presentations are provided by LRO and LCROSS to the respective center management, ESMD program and HQ personnel, and to the agency chief engineer. The Exploration Technology Development Program (ETDP) is located at Langley Research Center. Technology project managers at the NASA Centers report to the ETDP. Technology project plans, signed by the program manager, the project manager, and the Director of the performing NASA Center are required every year. ETDP conducts annual Project Continuation Reviews and evaluates Earned Value Management (EVM) indicators. For the Human Research Program (HRP), managers evaluate cost, schedule, and performance to determine whether projects are continued, modified, or terminated. In case of poor performance, HRP Program management will agree on corrective actions with the project counterparts. Grantees will face early termination in the case of non-performance. Information on the NASA Employee Performance Communication System can be found at: http://nasapeople.nasa.gov/perform/pm.htm

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: All ACD Programs obligate funding in a timely manner, and spend it for the purpose as appropriated by Congress. Since, annual budgets are planned to correspond with the activities and key deliverables per each project or task plan, their timely use keeps the programs to plan. It is a time phased budget plan. Added to this, the Agency holds metrics on the obligation and accrual of funds from any appropriation. ACD's annual appropriation is available for obligation for a two year time period during which they are fully obligated. At the beginning of the fiscal year, obligation and cost monthly phasing plans are developed and used as the basis for tracking actual obligations and cost. During the Exploration Systems Mission Directorate monthly reviews, each program reviews actual obligations and cost against the plans and discussed. Financial data are reviewed monthly and quarterly with the Program Managers. HRP managers review quarterly there progress internal to the program. HRP grants are obligated in a timely fashion, however there is often a considerable lag in costing due to grantee organization internal procedures as well as HHS processing mechanisms. Unobligated balances in HRP performing organization accounts are reviewed annually and reprogrammed by HRP Program Manager based on priority research needs. ETDP has financial metrics imposed and enforced by ESMD, and which it routinely meets. These metrics are 100% Obligation by June of the second fiscal year. Programs in non-compliance are adjusted downward during the next Fiscal Year to compensate. 100% of the ESMD budget appropriated and authorized by the U. S. Congress is spent for its intended purpose. Agency-wide controls ensure that funds are spent for the intended purpose. Monthly, LPRP project offices review actual obligations and cost are reported against the plans and discussed, and funds not obligated are subject to reprogramming at the Associate Administrator's discretion. Exceptions are granted for problem procurement actions and minor funds cleanup. Unobligated balances are also considered when determining where to make reductions in order to fund Agency or Mission Directorate contingencies.

Evidence: ESMD expects all PY 2006 funds to be obligated by the end of August 2007. As of the end of February 2007 91.2% of ACD's FY 2006 funding was obligated. The breakdown by program follows: LPRP 96.8%; HRP 92.6%; Prom 79.2%; ETDP 90.0%.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ACD has not demonstrated that efforts to improve efficiency are an established part of program management. The program has, however, devised and baselined an efficiency measure for the use of one of its facilities.

Evidence: The efficiency measure can be found in the PART measures Tab.

Does the program collaborate and coordinate effectively with related programs?

Explanation: In general, all ACD Programs and projects collaborate with ESMD's Constellation Program, other NASA Programs and Mission Support Offices. For example, due to long lead-time of research programs, HRP investments are needed now to ensure human health and performance risk prevention and mitigation capabilities are available when Constellation Program operations need them. HRP deliverables are needed in FY2007 and 2008 to support Constellation development of the new space suit. ETDP initiates development of high-payoff technologies, and matures them to the point where they can be validated in major system demonstrations involving ground-based analogs. Technologies are then transitioned into Constellation Program for mission insertion. ETDP works closely with the Constellation Systems Program to insure that new technologies will be incorporated into its projects. LPRP collaborates actively within the Agency, specifically with ETDP, Constellation, HRP and SMD. Further information regarding ACD collaboration and coordination with other government and private efforts can be found in the response to Question 1.3.

Evidence: ACD Programs are in regular daily contact through organizational meetings and communications, since ACD and Constellation are managed in the same parent organization, Exploration Systems Mission Directorate. In this manner, HRP and Constellation have agreed on day-to-day coordination mechanisms to achieve program requirements. ETDP and Constellation have developed lists of technologies required to satisfy mission needs, they are in the process of developing formal agreements. ETDP and ISS have worked successfully to facilitate the incorporation of environmental sensors for the ISS. LRO and LCROSS data will be used to refine Constellation Lunar hardware and mission requirements and will be provided to the scientific community for study. Directorate Integration Panel Meetings and minutes.

Does the program use strong financial management practices?

Explanation: The most recent Independent Auditor report for NASA identified two (2) material weaknesses, all of which are repeats, as well as noncompliance with the Federal Financial Management Improvement Act.

Evidence: NASA's FY 2006 Performance and Accountability Report (www.nasa.gov/about/budget/index.html) includes the communication from the NASA Inspector General and the report of the Independent Auditor. In addition, the GAO has published numerous reports identifying shortcoming in NASA's new financial management system as well as its financial management processes (example is GAO-04-754T released on May 19, 2004).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: There were several strategic and management deficiencies that were identified in the previous 2005 PART review on the Human Research Program. These included: 1) a lack of a risk mitigation process for the Bioastronautics Roadmap deliverables for Human Space Exploration. and 2) insufficient measures to ensure directed research is fully peer reviewed using the Non-Advocate Review Process. HRP has addressed these deficiencies, with very little work remaining to retire them. Albeit the ETD nor LPR Programs were not reviewed prior through the PART process, deficiencies recognized internally are dealt with and addressed same as the recorded ones for HRP. In all cases, as management deficiencies are found through performance reviews or external review entities, they are addressed. One of the major management challenges for an Advanced R&T Program is to convince the customer to invest in long lead-time deliverables. In the absence of a customer buy-in, advanced programs are threatened. ACD has addressed this problem and mitigated it to a great extent by mapping R&T products to match NASA/ESMD long-lead technology and solution requirements.

Evidence: Specifically, HRP was asked to develop a critical path analyses for each deliverable including schedule and resource requirements. Much progress has been taken to address this deficiency and as of earlier this year, the risk mitigation process had three aspects under development. a) Refining risk probability and consequences for Exploration, b) Developing Spaceflight Human Health Standards that define acceptable risk, and c) Risk Mitigation through vehicle design, monitoring, diagnosis, and treatment. The Human Research Program is developing an integrated research plan that identifies schedules, priorities, and resources required to mitigate high-priority risks. This plan will be completed in mid-fiscal year 2007. In response to the second documented deficiency for HRP on insufficient measurement to ensure directed research is fully peer reviewed using the Non-Advocate Review Process. HRP adopted a metric that 100% of all directed research projects will undergo the NAR process. This process has been documented formally by HRP. The metric has been met and every directed research task is peer reviewed by specially formulated NAR panels constituted and managed by NASA headquarters personnel. In fiscal year 2006 proposals were received for 6 directed research projects for NAR review. Four of the 6 were completed, and the remaining 2 reviews are to be completed in fiscal year FY07. An example of an internally diagnosed management deficiency on LPRP, that was addressed was provided in response to question 3.1 and will be included here as well. At a Mission Directorate-level performance review meeting it become apparent during the LCROSS portion of the review that the project was encountering a breakdown in communication with the launch vehicle provider, co-manifested spacecraft with LRO and managed by LRO, Many data items that were needed for the spacecraft design, were not scheduled to be delivered on time while other were not even in the projects plans. There were schedule and deliverable disconnects that became visible due to the review. ESMD management assigned an action to the program to get to the bottom of the root cause of the issue, which was found to be that the launch vehicle integration responsibility was not clearly assigned to any individual. The resolution was the assignment of a liaison from the LCROSS project (from the ARC) to the LRO project (at GSFC). Further, a mission integration manager position was identified, to be located within the program office, that oversee both projects (LRO and LCROSS) to assure that disconnects are not seen in the future.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: Yes, for all the ACD programs deliverables are identified in terms of products/capabilities required by the Customers described in 2.RD2. Briefly, the Constellation Program (CxP) is the customer of all three component programs, HRP, ETDP and LPRP which have mapped their activities towards reducing CxP risks. In addition, the HRP provides products that support the standards for crew care provided by OCHMO. The R&T portfolio is mapped in consonance with the need dates and the available resources. The progress of the tasks is measured on a periodic basis to assess their progress to meet the delivery dates. The ETDP requires all projects to have an approved project plan that establishes baseline deliverable, performance metrics, cost, schedule, and a work breakdown structure. For HRP, projects report against cumulative key milestones, schedules, cost and performance requirements baselined at ESMD quarterly reviews. Specifically, LPRP, the LRO and LCROSS projects report monthly against monthly and cumulative key milestones, baseline schedules, baseline cost and performance requirements that were approved at Confirmation Review. Per NASA policy the LPRP projects are under a formal baseline control that has a series of required reviews that assess the deliverables, capability/performance characteristics and credible cost and schedule goals. Cost estimates are independently reviewed. Several required reviews were completed this year including LRO Systems Requirement Review (SRR), Preliminary Design Review (PDR), Confirmation (authority-to-proceed to development) Review (CR), Integrated Baseline Review (IBR), and the Critical Design Review (CDR); and LCROSS SRR, PDR, CR and CDR. Teams of experts and non-advocate reviewers conducted all reviews.

Evidence: For LPRP, Monthly presentations are provided by LRO and LCROSS that contain budget performance, schedule performance and technical performance metrics. Program and project objectives, technical performance goals, deliverables, budget, and schedule are baselined in project plans that comply with NASA standard 7120.5. Earned Value Management is used to track cost and schedule performance against the baseline plans. For HRP, program objectives, technical performance goals, deliverables, budget, and schedule are in the process of being formally baselined in project plans that comply with NASA standard 7120.5. HRP also conducts quarterly reviews to evaluate budget, schedule, and technical performance against these plans. NASA Procedural Requirements (NPR) 7120.5 "NASA Spaceflight Program and Project Management Requirements".

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: For ACD programs using grants, awardees are selected through a competitive peer-review process in response to NASA Research Announcements. There are both Agency and program-level standards that must be met for this process. The majority of grants within ACD occur in HRP. For ETDP, research grants in the ISS Research project have been peer reviewed. This area has not had any solicitations recently due to budgetary reductions, but anticipates some small solicitations in future years. The peer review panels, comprised of experts internal and external to the Agency, assess scientific merit and provide their recommendations and ratings to NASA. These research solicitations are per existing ACD policies for peer review. The proposals relevance to the program is determined by ACD management who does a specific mapping to customer priorities. For ACD programs using grants, peer review comments including assessment of scientific merit are included in packages that are forwarded to procurement at the time that grants are awarded. These comments, as well as grantee responses to the comments are on file with procurement, and with respective grant Contracting Officer Technical Representatives (COTRs) ACD provides outreach to encourage the participation of new grantees. The Program Excecutives and Scientists attend Professional Society meetings where they give talks about the program content and also appraise the participants about potential new opportunities for research. Additionally, NASA advertises its research opportunities through the web pages of professional socities and organizations. Procedures for grant renewals are explicitly stated in the NASA FAR Supplement. This is a companion document to each NASA Research Announcement.

Evidence: NASA Procedural Requirement 5800.1 "Grants & Cooperative Agreement Handbook", NASA Policy Document 1080.1A, "NASA Science Policy" and NASA Procedural Requirements 1080.1 "NASA Science Management may all be found at: http://nodis3.gsfc.nasa.gov/. These documents govern the NASA internal policies, standards and procedures for the conduct of grants award, performance monitoring and oversight. Internal standards are also held at the program level within the Human Research Program Science Management Plan which is found at:http://hrp.jsc.nasa.gov As an example of outreach, the calls for Graduate Student Research and NASA Reserach Announcements was posted on the American College of Sports Medicine Website: http://forms.acsm.org/GRANTS/grants.htm. For evidence regarding renewals, refer to NASA FAR Supplement Provision, Instructions for Responding to NASA Research Announcements (Section d). This can be found as Appendix B in the Radiation NRA. http://nspires.nasaprs.com/external/viewrepositorydocument/71615/NNJ07ZSA001N.pdf

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Program has oversight practices as defined in Agency and program level grants management standards and within each NASA Research Announcements (NRA). Through this practice the Program is able to identify grantee critical questions that the proposed research may contribute to answering. Grant progress towards the scientific goals stated in the original proposals is reviewed on an annual basis via an annual report required from each grantee. The annual report outline that is available to public can be found at the following website: http://peer1.nasaprs.com/publication/index.cfm. Annual reports are submitted by grantees and are forwarded to relevant disciplines for evaluation of continued need, specifically, a more detailed report comprising of the scientific/technical progress is submitted to the program management. This report also gives the cost and performance data as stated in the proposal and agreed by the program management at the time of the grant award. Review and summarization of annual reports received from grantees is routinely done. Reviews are made available to science management personnel and to contracting officers. Additionally all grantees are required to attend an annual Investigator meeting where they disseminate the research results to the wider community. These meetings provide additional performance data to program management as to how the grantee is faring.

Evidence: NASA Procedural Requirement 5800.1 "Grants & Cooperative Agreement Handbook", NASA Policy Document 1080.1A, "NASA Science Policy" and NASA Procedural Requirements 1080.1 "NASA Science Management may all be found at: http://nodis3.gsfc.nasa.gov/. These documents govern the NASA internal policies, standards and procedures for the conduct of grants award, performance monitoring and oversight. Internal standards are also held at the program level within the Human Research Program Science Management Plan which is found at:http://hrp.jsc.nasa.gov The annual grants result report, available to public can be found at the following website: http://peer1.nasaprs.com/publication/index.cfm.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Grantees in support of HRP update annually to a document called "The Taskbook". The Taskbook contains summaries of life sciences research accomplishments and demonstrates NASA program relevance. The Taskbook is intended to serve as a conduit of information to the general public. The ETDP does not maintain a Taskbook as most of their projects are developed through contracts which uses internal to NASA monitoring of technical, cost and performance review by the ACD project and program management, which is not made available to the public.

Evidence: The Human Systems Research and Technology Taskbook can be accessed by the public at: http://taskbook.nasaprs.com/peer_review/index.cfm

For R&D programs other than competitive grants programs, does the program allocate funds and use management processes that maintain program quality?

Explanation: ACD generally uses Broad Area Annoucenemtns to solicit proposals for research and technology acquisitions. Responses to these solicitations are merit-reviewed. In some cases NASA implements focused directed research and technology development through its research centers. This is done to either expeditiously achieve the results, take advantage of an available flight opportunity or to use the unique expertise that resides within NASA. ACD's internally directed research has been awarded via one of the processes discussed below. The Human Research Program (HRP) uses a process called a Non Advocate Review (NAR). Research proposals are required by the internal performers, which are subjected to a panel of extramural and intramural experts that review them for their technical/scientific merit. This panel may recommend modifications to the plans or alternative methods for obtaining the scientific knowledge. This is provided to program management who makes the final determination of how and what work will be conducted. The Exploration Technology Development Program (ETDP) selected its internal technology development tasks through an internal proposal call at the same time as it was employing a Broad Agency Announcement for extramural work. These internal proposals were peer-reviewed using both internal and external panelists who provided recommendations to program management. Further, ETDP plans to use the services of the NRC to review the program portfolio for quality and "best performer" later this year.

Evidence: Criteria for the use of different science acquisition methods can be found in the Human Research Progam Science Management Plan (HRP-47053). All the NAR activity is documented and presented to the projects and program management. Process and an example report have been provided to OMB under separate cover.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: ACD made progress towards several of its long-term output goals. ACD achieved annual milestones towards characterizing the risk of bone fracture in the microgravity environment and developing technologies for human exploration of the Moon. All of these achievements illustrate progress towards the program's long-term outcome goal of enabling Constellation to fly a new human spaceflight vehicle in 2014 and return humans to the Moon in 2020. ACD failed, however, to achieve a milestone towards developing an operational countermeasure to lower the risk of renal stone formaiton in the microgravity environment.

Evidence: ACD conducted the Foot Ground Reaction Forces experiment, demonstrated robotic systems for lunar mobility, and tested life support systems. ACD failed to complete a renal stone experiment. See the PART measures section.

Does the program (including program partners) achieve its annual performance goals?

Explanation: ACD made progress on most but not all of its long-term/annual goals, as explained in Question 4.1. In addition, ACD made its annual goals towards mapping the lunar surface and managing cost and schedule.

Evidence: ACD completed Preliminary Design Review for LRO, and kept average cost and schedule slippage under 10%. See aslo Question 4.1.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: ACD has not demonstrated an improved efficiency over last year. It has devised and baselined an efficiency measure, but has yet to show progress beyond this baseline.

Evidence: Reference the PART measures section for demonstrated progress toward efficiencies and cost/schedule baseline exceedance. Also reference the NASA FY2005 and FY2006 Performance and Accountability Reports found at:http://www.nasa.gov/about/budget/index.html.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: ACD is one of several technology development programs in the Federal government. ACD has not provided any evaluations or data to demonstrate that it compares favorably with those programs.

Evidence: Technology development programs with in-house customers are sponsored by DOD, FAA, and others.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: There have been reviews conducted in relevance, quality and performance for LPRP. For LPRP, Independent review boards provide technical, cost and schedule assessment for each project during major reviews. These reviews are conducted per the NASA policy that governs the conduct of spaceflight programs and projects. HRP has not demonstrated adequate independent retrospective analysis of the quality of its research portfolio. ETDP has not yet conducted its full suite of reviews but does have plans to do so in the near future. ETDP is planning a review of it the relevance and quality of its current research portfolio later this year by the National Academies.

Evidence: The Aeronautics and Space Engineering Board of the NRC is conducting a review "Evaluation of Radiation Shielding for Lunar Exploration" to "evaluate the radiation shielding requirements for lunar missions and recommend a strategic plan for developing the necessary radiation mitigation capabilities to enable the planned lunar architecture." http://www.8.nationalacademies.org/cp/projectview.aspx?key=48696. For LPRP, LRO Systems Requirement Review (SRR) on August 16, 2005, the Preliminary Design Review (PDR) on February 10, 2006, the Confirmation Review on May 17, 2006, the Integrated Baseline Review (IBR) on June 28, 2006, and the Critical Design Review (CDR) on November 9, 2006. LCROSS project completed the SRR on July 11, 2006, the PDR on September 6, 2006, and the Confirmation Review on January 19, 2007.

Were program goals achieved within budgeted costs and established schedules?

Explanation: LRO Annual Performance Goal was "successful completion of Critical Design Review (CDR). For LPRP, goal was successfully completed on November 9, 2006. LRO maintains positive schedule and cost reserves against baseline metrics. Project Estimate At Completion (EAC) has not changed since the Confirmation Review. The ETDP has been successful in demonstrating a throttleable rocket engine for the lunar lander, a life support system for Orion, and environmental monitoring instruments for the ISS. HRP has been supporting OCHMO baseline Spaceflight Human Systems Standards Volume I and is currently in the process of working with OCHMO on the baselining of Volume II. HRP's cost and schedule performance are reviewed quarterly, and for FY 2006 HRP ended the year within program plan.

Evidence: Reference the NASA FY2005 and FY2006 Performance and Accountability Reports found at http://www.nasa.gov/about/budget/index.html.