Detailed Information on the
International Space Station Assessment

Develop alternatives to the Space Shuttle for resupplying the International Space Station.

Measure: ISS percent complete and available, in order to provide an orbiting facility for research to be utilized by NASA, its International Partners, and the National Laboratory users

Explanation:The ISS Program revised the assembly sequence at the February 2006 Heads of Agency meeting with all the International Partners. The agreed to assembly sequence documents all the components necessary to meet the U.S. space exploration goals and U.S. commitments to the International Partners for delivery of their components. With a completed assembly configuration and uninterrupted operations after Shuttle retirement, the ISS will be fully capable of providing a platform for NASA to gain operational and technical experience for exploration, and for the International Partners and National Laboratory participants to conduct long term research and related technology development consistent with international agreements and memorandum of understanding.

Measure: Percent of cargo requirements satisfied by U.S. commercial services

Explanation:In order to facilitate program viability and reduce operations costs after Shuttle retirement, commercial services should provide all cargo supply to the ISS minus any cargo requirements previously negotiated through barter agreements with International Partners.

Measure: Number of Type-A or Type B mishaps

Explanation:Type A mishap = damage to property of at least $1M or death; Type B mishap = damage to property of at least $250K, or permanent disability or hospitalization involving 3 or more persons

Measure: Percent of optimal on-orbit resources and accommodations (including power, data, crew time, logistics and accommodations) available to support research

Explanation:Leading up to and beyond ISS assembly completion in 2010, the resources provided throughout the life of ISS will be key to conducting research. This measure addresses those capabilities and resources needed to expand and validate future space flight applications and conduct research onboard the ISS: Six-crew capability in 2009 and beyond; 1500 total available crew hours per year for research in U.S. On-Orbit Segment; 20 research rack sites; 30 kilowatts of power. These targets require comprehensive coordination with our International Partners, the Space Shuttle Program, National Lab users, and commercial transportation providers to integrate, test, and validate capabilities and plans. These targets are an unprecedented amount of capability to perform research in low earth orbit over several years that has never been achieved in any prior program including SpaceLab, SpaceHab and Skylab missions. Furthermore, this does not reflect the seven days a week, 24 hours a day operational capabilities of the ground support system to conduct research remotely. The activities further incorporate the annual performance goal to accomplish a minimum of 90% of the research objectives as established one month prior to a given mission increment.

Measure: Activities executed and milestones to ensure that U.S. commercial cargo capabilities are adequately integrated into the ISS program.

Explanation:This measure tracks the ISS safety, technical and operational activities related to integrating commercial cargo capabilities with the ISS. In order for commercial suppliers to deliver cargo to the ISS, certain milestones must be met. This measure includes key milestones between the ISS program and the commercial supplier that enables cargo delivery to the ISS. This measure will become effective in 2009 once awards are made for the ISS Commercial Resupply Services contract, which is expected at the end of 2008. It replaces "Develop alternatives to the Space Shuttle for resupplying the ISS" improvement plan of 2005. The targets capture what is the first effort by the U.S. government, or any other government, to acquire commercial space transportation capabilities that today do not exist and are not expected to exist when the contract(s) are awarded. Year: 2009, Target: Manifest delivered to contractor; Year: 2010, Target: Joint integrated testing, Operation simulation, Cargo delivered to contractor

Measure: Annual Cost Performance Index (CPI)

Explanation:The ISS program measures its cost and schedule performance through earned value metrics. Earned value is the measure of work accomplished against work planned. Each work element within the program tracks its cost and schedule to accomplish the work planned and is integrated into an overall program performance index. There are two basic elements of earned value metrics: the Cost Performance Index and the Schedule Performance Index. The Cost Performance Index is the ratio of the value of the work accomplished versus the actual cost of the work accomplished. A ratio of 1 indicates the cumulative value of work accomplished within the fiscal year matches the cost accrued in the performance of the work during the same period. The current ISS plan assumes a reduction in the operational costs of ISS through the life of ISS after assembly is complete. Since the value of the work accomplished is tied to these planning assumptions, a CPI close to 1 is desirable because it shows efficiency in performance versus planning. A CPI =1 represents the ideal condition and is an ambitious target to measure performance.

Is the program purpose clear?

Explanation: The International Space Station (ISS) is a permanently crewed facility in space that enables NASA to develop, test and validate the next generation of technologies and operational processes needed to continue space exploration. The ISS also demonstrates the utility of working with international partners, and provides a platform for other government entities, commercial industry, and academic institutions to conduct research in association with the ISS National Laboratory.

Evidence: U.S. Space Exploration Policy: http://www.nasa.gov/news/budget/index.html, http://www.nasa.gov/mission_pages/station/structure/iss_manifest.html

Does the program address a specific and existing problem, interest, or need?

Explanation: The ISS Program directly addresses the nation's future needs for human space exploration. In 2004, the Bush Administration released "A Renewed Spirit of Discovery: The President's Vision for U.S. Space Exploration" (reprinted in the Vision for Space Exploration) which outlined plans for U.S. space exploration. One integral part of these plans is the need to conduct research on human biomedical performance in microgravity, in order to enable U.S. crews to conduct extended duration space missions in the future. A closely related aspect is the need to develop next generation spacecraft technologies and test those technologies in the space operating environment. In addition, the NASA Authorization Act of 2005 further established the need for ISS to be operated as a national laboratory that could host space-based research by other U.S. government agencies, academic institutions, and private firms. The ISS Program enables NASA's Exploration Systems Mission Directorate to develop, test, and validate the next generation of human biomedical and space craft system technologies necessary to support space exploration. The ISS Program also meets the needs of public and private organizations engaged in fields of science and technology for which they are responsible.

Evidence: A Risk Reduction Strategy for Human Exploration of Space: A Review of NASA's Bioastronautics Roadmap (2006) Report of the Committee on Review of NASA's Bioastronautics Roadmap, National Research Council http://www.nap.edu/catalog.php?record_id=11467 Review of NASA Plans for the International Space Station (2006) Review of NASA Strategic Roadmaps: Space Station Panel. Space Studies Board http://www.nap.edu/catalog.php?record_id=11512#toc 2006 NASA Strategic Plan: www.nasa.gov/pdf/142302main_2006_NASA_Strategic_Plan.pdf The Vision for Space Exploration: www.nasa.gov/pdf/55583main_vision_space_exploration2.pdf NASA Authorization Act of 2005 (S.1281): http://thomas.loc.gov/cgi-bin/query/D?c109:5:./temp/~c109I3ryp Memorandum of Understanding Between the National Institutes of Health and NASA for Cooperation in Space Related Health Research: http://www.nasa.gov/mission_pages/station/science/nlab/index.html

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Once complete, the ISS will be the largest and most complex orbiting space laboratory ever constructed. It is unique in its capability for ground controllers and onboard crew members to conduct research in the micro-gravity environment of low-Earth orbit. The ISS is also the only existing Federal or private facility for supporting prolonged human research activities in space.

Evidence: As of April 3, 2008, 70% of the ISS is deployed on orbit. There is no other human-rated, multi-mission space station in low-Earth orbit and none are currently planned. http://www.nasa.gov/mission_pages/station/structure/iss_manifest.html www.nasa.gov/pdf/142302main_2006_NASA_Strategic_Plan.pdf The Vision for Space Exploration: www.nasa.gov/pdf/55583main_vision_space_exploration2.pdf

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The ISS Program is on track to provide the capabilities and resources necessary to conduct on-orbit research. The Program is structured both on-orbit and on the ground to support research integration, testing, operations and maintenance. Vital to the Program's ability to conduct on-orbit research is cargo and crew transportation to and from the ISS. NASA has negotiated balance of agreement terms with its international partners, the European Space Agency (ESA) and the Japanese Aerospace Exploration Agency (JAXA), to provide cargo supply capabilities. ESA has developed the Automated Transfer Vehicle (ATV) that has been demonstrated on orbit in 2008. JAXA is developing the H-II Transfer Vehicle (HTV) that is expected to be operational in 2010. NASA has contractually purchased Russian cargo transportation services though 2011 to help ensure the viability of the ISS in the post-Shuttle timeframe. A key component of NASA's plans for providing cargo to the ISS is the development and procurement of U.S. commercial providers. Within the Commercial On-orbit Transportation Services (COTS) program, NASA is investing in commercial partners to develop the capability to provide upmass and downmass capabilities that are expected to be available in the 2010 timeframe. At the same time, NASA is pursuing the acquisition of such services in anticipation of their availability in 2010. In the event that U.S. commercial providers are not available when expected, NASA is developing plans to either purchase cargo transportation services from its international partners, or possibly maintain the ISS in a limited operational capability until domestic providers are available. Another possibility is to deliver components and spares on one or two contingency Shuttle flights, which could be flown if schedule allows before the end 2010. For crew transportation and rescue capabilities in the post-Shuttle timeframe, NASA is pursuing relief from the Iran, North Korea, and Syria Non-proliferation Act (INKSNA) in order to purchase Russian Soyuz capabilities until either U.S. commercial crew transportation and rescue services or the NASA-developed Crew Exploration Vehicle is available. This crew capability will ensure that NASA meets obligations under the International Partner Memoranda of Understanding to provide crew transportation and rescue services for the non-Russian partners and rescue services for all onboard crew members for the duration of the ISS lifetime.

Evidence: Space Act Agreements with COTS providers http://prod.nais.nasa.gov/eps/eps_data/126269-OTHER-001-002.pdf Commercial Resupply Services RFP http://procurement.jsc.nasa.gov/issresupply/ International Partner Memorandum of Understand provide the framework used to negotiate crew and cargo services.http://www.nasa.gov/pdf/189189main_NASA_NIH_mou_20070912.pdf

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The ISS program design and resources are effectively targeted for the successful assembly, operation and utilization of the facility. The ISS Intergovernmental Agreement (IGA) and Memoranda of Understanding (MOU) create a framework for integrated program implementation that align with the NASA ISS Program goals and validate the International Partners' commitment to strategic and tactical planning. In addition, NASA maintains a Consolidated Utilization Plan which provides the multilateral governing approach to research and utilization between the ISS partners. NASA also has developed the "Human Research Program Utilization Plan for the ISS" that addresses the NASA portion of research onboard the ISS. This plan is maintained by the Exploration Systems Mission Directorate. Additionally as part of the ISS National Laboratory, NASA is cooperating with the multiple government agencies to fully utilize the ISS research capabilities. NASA recently negotiated a Memorandum of Understanding with the National Institute of Health.

Evidence: IGA, MOU, Contracts lists: http://www.nasa.gov/mission_pages/station/structure/elements/nasa_japan.html http://www.nasa.gov/mission_pages/station/structure/elements/nasa_rsa.html http://www.nasa.gov/mission_pages/station/structure/elements/nasa_esa.html http://www.nasa.gov/mission_pages/station/structure/elements/nasa_csa.html http://nasa-mis.nasa.gov/nasa_mis/main_frame.cfm?CFID=15205&CFTOKEN=70093033 http://gsd.msfc.nasa.gov/GSRT/NPRDRevJ.pdf http://nasa-mis.nasa.gov/nasa_mis/main_frame.cfm?CFID=15297&CFTOKEN=27035224 ( go to ISS view and then choose desired item from business management ) http://iss-www.jsc.nasa.gov/nwo/ext_int/home/web/key_agree.shtml Memorandum of Understanding between the National Institutes of Health and NASA for Cooperation in Space Related Health Research: http://www.nasa.gov/pdf/189189main_NASA_NIH_mou_20070912.pdf http://www.nasa.gov/mission_pages/station/science/nlab/index.html NASA Research and Utilization Plans for ISS Report delivered to Congress March 2008. http://www.nasa.gov/pdf/180615main_ISS_National_Lab%20Final_Report.pdf http://www.nasa.gov/mission_pages/station/structure/iss_manifest.html

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program has two long-term performance measures, one of which focuses on outcome. The first measure demonstrates the percentage of ISS assembly completed by the 2010 and services available through 2016. The second measure drives the program to seek out commercial services for its cargo requirements after the Space Shuttle retires in 2010, consistent with the Vision for Space Exploration.

Evidence: Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html; President's FY 2009 Budget Submit to Congress http://www.nasa.gov/news/budget/FY2008.html Detailed Program-level performance indicators are updated at least monthly and available on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: It is the intent of the ISS program to have all the planned resources for research available at assembly complete in 2010 through 2016. Maintaining all planned services and meeting 100% of our operational plans is ambitious given the nature of on-orbit planning, requiring many years or months of advance attention to detail. NASA has put forward the ISS Program as the leading program for on orbit research for the international human space flight community and is committed to providing the unprecedented capability to conduct research on orbit. Based on the uniqueness and long term habitation of the ISS, there is no other program, past or present, which compares to the ISS research platform capabilities. Sustaining operations after Space Shuttle retirement will be challenging, and relying more on commercial cargo post-2010, when it has yet to be demonstrated, represents an ambitious plan. Both long-term measures enable an unprecedented amount of crew hours to be spent operating in space, learning how to adapt to space, and conducting research for future exploration missions.

Evidence: The ISS is 70% complete as of April 2008 and resides in low-Earth orbit. The ISS has been crewed since November 2000 and will accumulate approximately 3700 crewed days by the end of 2015. By comparison, during the 1970's the NASA Skylab program was crewed for only 172 days. In terms of commercial cargo supply, NASA is engaged in funded Space Act Agreements with two companies that have never attempted to send spacecraft to the ISS. ISS Reference Guide: http://www.nasa.gov/mission_pages/station/news/ISS_Reference_Guide.html Research performance is regularly updated (crew hours available, power available, etc.) on NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program has three annual output measures and one efficiency measure to track how well the ISS meets its long-term goal of providing an orbiting research facility to NASA and other ISS stakeholders. Specifically, the ISS annual measures demonstrate progress toward successful assembly, reliable cargo supply after Shuttle retirement, and efficient and safe operation throughout its life. Further supporting these measures, the ISS program has identified six annual performance goals as described in the NASA performance plan and each supports the program's long term goals.

Evidence: Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html Detailed Program-level performance indicators are updated at least monthly and available on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ISS has baselines and ambitious targets for its annual performance measures. Several of ISS performance targets reflect the maximum achievable objective. To support research objectives, the target of 100% functional availability for all ISS subsystems that support on-orbit research operations has been established. Further implicit in this target is a set goal of 90% completion of all planned research investigations. ISS has also set a target of zero Type A or Type B mishaps. In addition, to ensure that good program management and earned value is achieved, ISS has set its target Cost Performance Index equal to one, which demonstrates efficiency in performance versus planning. Finally, a measure exists that contains target milestones to ensure U.S. commercial cargo capabilities are adequately integrated into the ISS program. These measures are even more ambitious considering the required continuous U.S. crew presence on ISS during the time between Shuttle retirement and the Orion Crew Exploration Vehicle's full operational capability.

Evidence: Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html; President's FY 2009 Budget Submit to Congress http://www.nasa.gov/news/budget/FY2008.html Detailed Program-level performance indicators are updated at least monthly and available on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ISS Intergovernmental Agreement (IGA) and Memoranda of Understanding (MOU) create a framework for integrated program implementation that align with the NASA ISS Program goals and validate the International Partners' commitment to strategic and tactical planning. Multilateral control boards, working groups, and technical interchange meetings provide forums for measuring progress and addressing cost, schedule, and/or technical concerns. For contracted U.S requirements, Contract Performance/Award Fee Evaluation processes are in place to encourage and reward the contractor for safe, high quality, cost effective performance in fulfilling the contract requirements in alignment with ISS Program goals. The evaluation process provides objective and subjective assessments by the Government, which allows percentages of the potential fee to be based on the contractor's performance measured against performance criteria in areas of safety, technical, management, customer satisfaction, cost control, and socioeconomic considerations. This process allows the Government to award or penalize the contractor's performance.

Evidence: The ISS is 70% complete as of April 2008 and resides in low-Earth orbit. http://www.nasa.gov/mission_pages/station/main/index.html ISS Performance Evaluation Board, One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request. Space Station Program Control Board agendas, presentations, minutes and actions. One NASA MIS http://nasa-mis.nasa.gov Coordination with partners is facilitated by announcements of opportunities, such as the resupply RFP issued in draft, then followed by final release including industry comments. http://prod.nais.nasa.gov/cgi-bin/eps/sol.cgi?acqid=128918#Draft%20Document

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Currently, multiple independent/quality evaluations are conducted by external entities on a mission, daily, monthly, quarterly, semi-annually, annually, and 'as-required' basis depending on their charter. The ISS uses these high quality advisory groups to obtain external input to its strategies and performance planning and evaluation activities and to improve the performance and effectiveness of the program. NASA ADVISORY COUNCIL; The NASA Advisory Council (NAC, 6 committees) provides the NASA administrator with counsel and advise on program and issues of importance to the Agency. The ISS Program is under the Space Operations Committee. Each committee conducts fact-finding sessions throughout the year in an effort to gain a broad understanding of current NASA issues and future mission implementation plans. The committees then bring this information to the full Council in order that specific recommendations may be prepared. The Council deliberates on topics raised by each committee in public sessions and presents any findings and recommendations to the NASA administrator on a quarterly basis. The NASA administrator has the discretion to act on any findings or recommendations. The NAC has addressed several key activities of the ISS program such as cargo requirements, transportation plans, on-orbit performance and risk items. GENERAL ACCOUNTABILITY OFFICE: The General Accountability Office (GAO) recently conducted a study in 2007 on the challenges in completing and sustaining the International Space Station. (Report GAO-07-1121T July 24, 2007) This report discusses the challenges faced by the NASA on the International Space Station and the Space Shuttle. ISS INDEPENDENT SAFETY TASK FORCE: As required by the NASA Authorization Act of 2005 (Public law 109-155), this report of the International Space Station Independent Safety Task Force (IISTF) to NASA and the United States Congress assesses vulnerabilities of the ISS that could lead to its destruction, compromise the health of its crew, or necessitate its premature abandonment. The IISTF completed its assessment and presented its report in February 2007. PROGRAM IMPLEMENTATION REVIEW: As required by the NASA Procedural Requirements 7120.5, NASA Space Flight Program and Project Management Requirements, each program is required to conduct an independent life-cycle review by a Standing Review Board (SRB). The independent life-cycle review, the Program Implementation Review (PIR), is conducted under Terms and References (ToR) that are negotiated between the SRB and the ISS program and is conducted ever two years. The SRB's role is advisory to the program/project and the convening authorities and does not have authority over any program/project content. Its review provides expert assessment of the technical and programmatic approach, risk posture, and progress against the program/project baseline. Its outputs are briefed to the program/project under review prior to being reported to the next higher level of management. The ToR for the 2008 PIR is based on the following evaluation approach: Alignment with and contributing to Agency needs, goals, and objectives; adequacy of requirements flow-down; adequacy of technical approach as defined by NPR 7123.1 entrance and success criteria; adequacy of schedule; adequacy of estimated costs (total and by fiscal year), including Independent Cost Analyses (ICAs) and Independent Cost Estimates (ICEs) against approved budget resources; adequacy/availability of resources other than budget; adequacy of risk management approach and risk identification/migration, and adequacy of management approach. At the time of this PART submittal, the PIR is schedule to complete its review and submit its finding in August 2008.

Evidence: NASA Advisory Council http://www.hq.nasa.gov/office/oer/nac/NACPresentations/Feb-2008.htm General Accountability Office report, NASA: Challenges in Completing and Sustaining the International Space Station http://www.gao.gov/new.items/d071121t.pdf Final report of the International Space Station Independent Safety Task Force, February 2007 - http://www.scribd.com/doc/350417/NASA-170368main-IISTF-Final-Report-508?ga_related_doc=1

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program's goals and budget are clearly linked in the FY 2009 Budget. The budget requests required to execute the assembly activities, completion of integration activities, and for ongoing operations and maintenance, which are all in support of developing and maintaining the long term performance goals, are explicitly defined throughout the program. The program cost, schedule and technical baselines are captured in the ISS Program Operations Description (IPOD) document. A corresponding cost breakdown structure is captured in the Agency Integrated Enterprise Management Program (IEMP). Annual Program Operating Plan submissions can be traced to the work breakdown structure reflected in the IPOD. From the Integrated Budget and Performance Document program budget shows Operations and ISS Crew Cargo Services. The section titled, "Project Descriptions and Explanation of Changes" provides detailed description of plans, some of which are represented in the performance measures, and how the budget was changed in the requesting year to achieve these plans.

Evidence: President's FY 2008 Budget Submit to Congress http://www.nasa.gov/news/budget/FY2008.html Performance data informs the budget planning process regarding sufficiency of plan and adequacy of implementation, as updated at least monthly on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request. The ISS Program Commitment Agreement is updated periodically to agree with Agency policy, resources and Administration guidance. ISS Program Commitment Agreement, signed August 3, 2003 (update in progress).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Past problems with cost overruns are now under control as evidenced by the Annual Cost Performance Index where the program has performed at the highest rating for three years. Also, NASA instituted mechanisms between Headquarters and the Field Centers to strengthen communication and management of the ISS program, such as the Directorate Program Management Council which meets quarterly. Strategic planning to prepare for the changing conditions driven by Shuttle retirement are addressed in the FY 2008 budget appropriations, providing more than $200 million for a new logistics sparing philosophy to pre-position spares on ISS prior to Shuttle retirement, and in the FY 2009 President's budget request additional funds are deployed to address uncertainties in cost and availability of commercial services after Shuttle Retirement.

Evidence: The funding and availability of transportation of logistics and crew to ISS is significantly altered after Shuttle retirement with mitigation strategies described in FY 2009 Budget Estimates (see International Space Station Theme, ISS Program, Project Description and Explanation of Changes for ISS Cargo Crew Services; available at www.nasa.gov/budget) http://www.nasa.gov/news/budget/index.htm

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: none

Evidence: none

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Program cost, schedule, and technical performance information is routinely analyzed through the One NASA Management Information System, and the Agency Integrated Enterprise Management (IEMP) System. This information is collected at all levels of the program and is reported to management on a monthly basis. Significant issues that require a timely resolution are brought to management's attention immediately. On a quarterly basis, the Program reviews that state of risks across the program, including those associated with the participation of International Partners, which have an effect of the performance of the program. A performance measurement system is in place using both traditional earned-value techniques for industrial contracts and modified earned-value techniques for operations activities. Earned value information is reported on a monthly basis by each element of the program. The ISS Program monitors contractor and International Partner progress, technical performance, actions, risk, cost, and schedule through regularly-scheduled unilateral and multilateral reviews, audits, technical interchange meetings, boards, and panels. If the program's performance is not meeting needs or requirements, actions are assigned and tracked at the appropriate level board or panel. NASA also conducts an agency wide review [the Baseline Program Review (BPR)] on a quarterly basis where each program's cost, schedule, and technical performance, risks, and partner performance is reviewed. Cross-cutting issues that affect multiple program are discussed and actions assigned to address weaknesses and issues across the agency. The agency also conducts Program Implementation Reviews (PIR) by Standing Review Boards (SRBs) that reviews the performance of the program against the agency's established requirements, procedures and guidelines.

Evidence: Standing Review Board findings, http://www.hq.nasa.gov/office/oer/nac/NACPresentations/Feb-2008.htm International Partner Technical Interchange Meetings agendas, presentations, minutes and actions, http://www.nasa.gov/pdf/221035main_Zvedre-international.pdf Space Station Control Board agendas, presentations, minutes and actions. One NASA MIS http://nasa-mis.nasa.gov Directorate Program Management Council agendas, presentations, minutes and actions http://www.hq.nasa.gov/office/oer/nac/NACPresentations/Feb-2008.htm Multi-lateral Control Board agendas, presentations, minutes, and actions Heads of Agency agendas, presentations, minutes and actions

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers are held accountable for ISS Program performance as defined by the Agency's governing structure. Performance measures are included in key management position performance criteria. Contractors are held accountable for their performance through various contract incentives, such as performance based awards. The ISS program has instituted management discipline incorporating leading industry and government practices such as earned value management. Program management is reviewed quarterly by the Associate Administrator for Space Operations. The ISS program monitors contractor and International Partner progress, including technical performance, actions, risk, cost, and schedule through regularly scheduled reviews, audits, technical interchange meetings, boards and panels. If it is determined that a contractor is not performing at expected levels, the ISS program has multiple incentives to rectify contractor performance such as award fee and mile stone based payments. If International Partners are not meeting their performance obligations, NASA has multiple forums, such as bilateral meetings, to negotiate alternative implementation approaches with technical and schedule implications.

Evidence: Contract performance evaluations, http://prod.nais.nasa.gov/cgi-bin/eps/synopsis.cgi?acqid=126425 International Partner Technical Interchange Meetings agendas, presentations, minutes and actions Space Station Control Board agendas, presentations, minutes and actions, One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request. Directorate Program Management Council agendas, presentations, minutes and actions http://www.hq.nasa.gov/office/oer/nac/NACPresentations/Feb-2008.htm Multi-lateral Control Board agendas, presentations, minutes, and actions Heads of Agency agendas, presentations, minutes and actions Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Annual NASA R&D funds are available for obligation for two years, and fully obligated by the end of the period. Operating plans for the program year are submitted to Congress and revised as needed over the two year time period. Internally, obligation and cost plans are developed by the Program, compared to actual spending as well as work performed, and reviewed monthly by all levels of the program. NASA's Integrated Enterprise Management Program (IEMP) provides the primary system used to report monthly all obligations and costs. These are tracked against Station-unique project numbers traceable to contractor source documents, such as contractor monthly 533 reports. Contractors also provide Cost Performance Reports that provide further evidence that funds were spent for the intended purposes. Contractor and government accounting systems are audited monthly by NASA internal personnel and annually by external auditors to ensure compliance with government standards. As an example, 97% of ISS PY 2007 funds were obligated by 9/30/07, 100% by 01/31/08.

Evidence: NASA IEMP data and Contractor monthly reports (533s), CPRs, SF133 Report on Budget Execution and Budgetary Resources, FMS2108 Year-End Closing Statement, and Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Contract Performance/Award Fee Evaluation processes are in place to encourage and reward the contractor for safe, high quality, cost effective performance in fulfilling the contract requirements in alignment with ISS Program goals. The ISS Program competitive sourcing strategy approved in December 2002 reduced ISS direct contracts from 28 to six. Nine of the 28 contracts were competitively sourced and 4 of the 6 new consolidation contracts were competitively sourced. The contract for sustaining engineering was delayed in favor of an extension of the Prime contract through the Space Shuttle return to flight period. The new contracts effectively minimize duplication of specialized expertise and redundant infrastructure in multiple contracts; provide for focused accountability for a deliverable to the ISS; minimize formal product development, management and deliveries between contracts; maximize competition; require performance measures; and encourage cost savings through requirements management and infrastructure reduction. The ISS has one annual efficiency measure to track the cost-performance index.

Evidence: Contract Program Evaluation Board data, ISS Program Plan: Acquisition Strategy, Section 10.0. Contract Strategy: http://prod.nais.nasa.gov/cgi-bin/eps/synopsis.cgi?acqid=126425 NGO website http://SpaceResearch.nasa.gov/research_projects/ngo.html NGO Congressional Report web site http://SpaceResearch.nasa.gov/research_projects/ngocdt.html

Does the program collaborate and coordinate effectively with related programs?

Explanation: The cooperating agencies in the ISS Partnership (hereinafter referred to as the International Partners) are NASA, Canadian Space Agency (CSA), European Space Agency (ESA), Russian Federal Space Agency (Roskosmos), and the Japanese Aerospace Exploration Agency (JAXA). This partnership significantly enhances the capabilities of the ISS and serves as a model of international cooperation in space with important lessons for human exploration of the Moon and beyond. The ISS Program is also cooperating with several government agencies, including NIH, NSF, DOE, and DOD. The ISS Program has clear planning and operational links to other NASA programs - the Space Shuttle program for launch services, and to the Exploration Systems Mission Directorate for developing transportation, science payload manifesting, ground processing, and on-orbit resource scheduling. NASA is also developing collaborative agreements with government, industry and academic institutions to utilize the ISS resources to conduct research within the ISS National Laboratory.

Evidence: http://www.nasa.gov/mission_pages/station/science/nlab/index.html Memorandum of Understanding between the National Institutes of Health and NASA for Cooperation in Space Related Health Research:

Does the program use strong financial management practices?

Explanation: NASA and particularly the ISS program have made improvements in financial management practices. However, NASA's auditors continue to find agency-wide material weaknesses. The ISS Program now has procedures in place to ensure that payments are made properly, financial information is accurate and timely, and integrated financial and performance systems support day-to-day operations. Program performance and budget reserve status information is updated by the Program on a monthly basis in the One NASA MIS system. Weekly Space Station Program Control Board (SSPCB) meetings review and disposition content change requests and associated budget impacts. ISS Monthly Program Reviews (IMPRs) examine reserve metrics and programmatic threats against the program's cost plan for the five year budget window. A Program Risk Advisory Board (PRAB) is held approximately every six weeks to review cost threats for base lined content that are in excess of the current budget baseline. A quantitative risk assessment (QRA) is performed using Monte Carlo analytical/ statistical techniques that project the value of threats that will most likely be realized. The Program also monitors actual performance on a monthly and quarterly basis for opportunities to improve its financial posture by re-phasing or reducing content and recovering cost efficiencies. NASA also conducts an agency wide review, the Baseline Program Review (BPR), on a quarterly basis where each program's cost, schedule, and technical performance, risks, and partner performance is reviewed. Cross-cutting issues that affect multiple programs are discussed and actions assigned to address weaknesses and issues across the agency. The agency also conducts Program Implementation Reviews (PIR) by Standing Review Boards (SRBs) that reviews the performance of the program against the agency's established requirements, procedures and guidelines.

Evidence: Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: There are no outstanding management deficiencies for the ISS Program. Deficiencies identified by the ISS Management and Cost Evaluation (IMCE) in 2001 were addressed and resolved in 2003 when OMB approved the expansion of the ISS crew complement from 3 to 6 crew members. The ISS Program has a defined final configuration as agreed to by the Heads of Agency for all International Participants in March 2006. Additionally, there are a limited number of Space Shuttle flights that are required to complete the assembly of ISS. The ISS Program uses several systems to identify and correct technical and cost challenges. The ISS program has instituted management discipline incorporating leading industry and government practices such as earned value management. Program management is reviewed quarterly by the Space Operations Mission Directorate Associate Administrator. The ISS program monitors contractor and International Partner progress, technical performance, actions, risk, cost, and schedule through regularly scheduled reviews, audits, technical interchange meetings, boards and panels.

Evidence: Space Shuttle manifests and launch, http://www.nasa.gov/mission_pages/shuttle/main/index.html schedule. ISS assembly missions, http://www.nasa.gov/mission_pages/station/expeditions/index.html IMCE, http://history.nasa.gov/youngrep.pdf

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: Deliverables to support the ISS mission are clearly defined through the flight preparation process and are reviewed by the program through control boards. Capability and performance characteristics are defined by the flight and mission preparation process and are captured in flight/mission products. Additionally, the ISS is currently meeting its assembly capabilities and performance characteristics in support of the utilization operations. The program develops a yearly cost plan that enables execution of the schedule milestones in support of mission activities. The program employs earned value management. Performance goals for cost and schedule are set at >0.95 for Cost Performance Index and Schedule Performance Index to measure overall program performance. A CPI and SPI between 0.95 and 1.05 are considered optimum.

Evidence: Program and contract deliverables list and cost/schedule plans. Space Station Control Board agendas, presentations, minutes and actions. Performance data on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request. Directorate Program Management Council agendas, presentations, minutes and actions, http://www.hq.nasa.gov/office/oer/nac/NACPresentations/Feb-2008.htm

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The ISS program has demonstrated significant progress toward completion of the ISS by 2010. With the addition of the European Columbus laboratory and the Japanese Kibo Laboratory Pressurized Module and Experiment Logistics Module Pressurized Section, the ISS is at 72% complete by mass. Approximately 81% of the pressurized volume is on orbit. The U.S. Oxygen Generation System has been installed and activated on the ISS, and is operating nominally. Waste water management systems, waste collection and hygiene compartment, crew quarters, exercise and other capabilities to accommodate six crew members will be delivered to Station this year. These habitability modifications prepare the way for crew size increase in 2009 allowing more crew time available for research on ISS. The ISS Program is on track to complete integration activities to assemble and operate the remaining modules, truss assemblies, power generation systems, and research facilities in support of human exploration objectives. NASA is also investing in the aerospace community to develop the capabilities needed to deliver cargo to the ISS. Under the Commercial Orbital Transportation Services Program, NASA is investing $500M to demonstrate the capability to launch and deliver cargo to the ISS. Demonstrations are planned by the end of 2010. In addition, NASA released an ISS Commercial Resupply Services (CRS) Request for Proposals (RFP) on April 14, 2008, for resupply and return of ISS and utilization cargo. Proposals were due back to NASA at the end of June 2008, with an award expected at the end of calendar year 2008. NASA also entered into a similar contractual arrangement for privately developed water production services to be available on the ISS. The availability of on-orbit water production services would reduce water resupply upmass requirements.

Evidence: Annual NASA Performance and Accountability Report http://www.nasa.gov/news/budget/index.html; President's FY 2009 Budget Submit to Congress http://www.nasa.gov/news/budget/FY2008.html ISS reference guide: http://www.nasa.gov/externalflash/ISSRG/index.html One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Does the program (including program partners) achieve its annual performance goals?

Explanation: ISS has achieved its annual performance goal requiring achievement of a Cost Performance Index nearly equal to one (1.012 as of February 2008), the result of tight programmatic controls on cost and judicious distribution of limited (1-2%) reserves. It has also achieved the annual performance goal that all products needed for flight readiness meet requirements and schedule. Examples of flight products include hardware and software certifications for flight, operational timelines and procedures, flight rules, off-nominal procedures, and software updates to onboard and ground systems. Other products for integration across the Program include bi-lateral agreements for joint operations such as command and telemetry timelines, interface agreements and crew time allocations between the Partners. The only annual performance goal that was not achieved in 2007 was the zero number of Type A and Type B mishaps. ISS incurred one Type B mishap in FY 2007.

Evidence: President's FY 2009 Budget Submit to Congress http://www.nasa.gov/news/budget/FY2008.html; One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program continues to meet its efficiency target of a Cost Performance Index greater than one and continues to effectively manage with very limited reserves. Additionally, the program demonstrated improved cost effectiveness over the prior year by funding the $89 million ISS Cargo Crew Services shortfall in FY 2008. It did this through a combination of positive performance and content management. While the program's ability to achieve significant operational efficiencies during assembly is limited, the program has achieved many operational efficiencies which have been offset by increased content and costs resulting from the Shuttle retirement in FY 2010. Previous ISS operations used a return and repair philosophy for ISS hardware. Due to Shuttle retirement, the program has had to change to a disposal philosophy since it won't have the capability to return hardware to Earth for repair. In addition, cargo and crew transportation previously performed by and funded through Shuttle has been transferred to ISS after Shuttle retirement. There continues to be uncertainty and risk in the availability and cost of alternative transportation vehicles, but the ISS program is aggressively managing program content and budget to provide maximum flexibility in addressing potential impacts to cost and efficiency. In the midst of these uncertainties, the ISS program has completed assembly of the core U.S. on-orbit segment, program safety has been excellent and revised up-mass, volume and crew time agreements have been satisfied. The ability of the ISS program to determine its fiscal health has steadily improved over recent years. Program assessments are completed at the beginning of each fiscal year to recommend the most likely financial outcome by year's end.

Evidence: The track record of the ISS Program Office assessments compared to actuals has been steadily improving. In FY 2002, the End of Year (EOY) Assessment was overstated (costs were lower than projected) by $144 million (9%). The overstated assessment was nearly the same in the following year and improved to 7% by FY 2004. In FY 2005, a philosophy revision was completed with the goal of being less conservative, resulting in an understated assessment of $15 million (1%). These estimates are based in part on analysis of the Earned Value Management (EVM) performance measures. The EVM estimates are monitored for accuracy and they, too, are measurably accurate. Performance data on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request. Draft Request for Proposal on ISS Commercial Resupply Services, http://prod.nais.nasa.gov/cgi-bin/eps/sol.cgi?

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There is no comparable program with similar purpose or goals

Evidence: Not applicable

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations of ISS are extensive and while most studies offer suggestions, they agree that the ISS program is a unique national facility with unprecedented international cooperation. It is a highly technical program which has operated for ten years in a very hostile environment. Recent studies include a GAO Report, 2007, and the IISTF Final Report, 2007. The General Accountability Office (GA0) recently conducted a study in 2007 on the challenges in completing and sustaining the International Space Station. (Report GAO-07-1121T July 24, 2007). This document discusses the challenges faced by NASA on the International Space Station and the Space Shuttle. The GAO released written testimony on July 2007 which included no recommendations, just observations, generally confirming the program is effective and achieving results. The IISTF report indicated, p. 1-2, "Station assembly to date has gone exceptionally well and is a tribute to the ISS and Shuttle teams??the ISS program has excellent processes and mechanisms in place on multiple fronts to ensure proper Program execution. The Principal Observations section (p. 3) also builds on this finding.

Evidence: NASA Advisory Council http://www.hq.nasa.gov/office/oer/nac/ General Accountability Office testimony, NASA: Challenges in Completing and Sustaining the International Space Station http://www.gao.gov/new.items/d071121t.pdf International Space Station Independent Safety Task Force - http://www.scribd.com/doc/350417/NASA-170368main-IISTF-Final-Report-508?ga_related_doc=1

Were program goals achieved within budgeted costs and established schedules?

Explanation: ISS goals were achieved within budget and established schedules. The ISS Program has successfully achieved over seven years of continuous crewed operations and completed assembly of the core U.S. on-orbit segment. ISS assembly is on schedule and ISS continues to accomplish all annual performance goals within budget including meeting a minimum of 90% of scheduled operation hours for all operations. ISS is on track to achieve planned six crew capability in FY2009 and assembly complete in FY2010. ISS deployed Earned Value Management (EVM) practices at lower levels to improve cost and schedule monitoring and control with the same success as demonstrated when using EVM at higher levels. Its annual cost performance index (CPI), the ratio of the value of the work accomplished based on contract performance parameters and deliverables to the actual cost of the work accomplished, is targeted to equal the optimal value of one.

Evidence: Program and contractor cost and schedule performance metrics, cost and performance metrics data on One NASA Management Information System (NMIS) http://nasa-mis.nasa.gov. The NMIS system is an internal repository that is utilized to house ISS performance activities. NMIS is not available to the public however; information from the system can be obtained upon written request.