Detailed Information on the
National Archives and Records Administration: Electronic Records Services Assessment

Implementing and utilizing earned value management and other strategies to minimize risk during acquisition of the Electronic Records Archives.

Refine its records management policies and strategies and engage with Federal agencies to continue methods of improving records management across the Federal government.

Work on resolving the basis for its material weakness in IT security.

Measure: Percentage of archival electronic records accessioned by NARA at the scheduled time.

Explanation:Measures NARA's ability to accept permanent electronic records from agencies for archival preservation.

Measure: Median time in days to process archival electronic records upon receipt by NARA.

Explanation:Measures NARA's timeliness in providing public access to archival electronic records.

Measure: Per megabyte cost of managing archival electronic records through the Electronic Records Archives will decrease each year. (Targets pending development of ERA.)

Explanation:Measures the cost-effectiveness of preserving electronic records.

Measure: Percent of electronic records in ERA managed at the appropriate level of service. (Targets pending development of ERA.)

Explanation:Measures NARA's ability to manage, preserve and provide access to electronic records at a level required by the nature of the record (not all records may require an optimal level of access and preservation).

Measure: Percent of electronic records held by NARA stabilized in preparation for transfer to ERA.

Explanation:Measures NARA's efforts to take into custody and copy electronic records prior to building ERA- this does not indicate preservation of electronic records in a persistent format in a manner that guarantees accessibility over time.

Measure: Milestone measures for development of the Electronic Records Archives.

Explanation:In 2005, milestones included completing design reviews and selecting a final contractor for the system. In 2006, milestones include completion of design reviews for the first increment of the system.

Is the program purpose clear?

Explanation: NARA's Electronic Records Services Program provides guidance and assistance to Federal officials on the management of electronic records, determines the retention and disposition of Federal electronic records, and preserves for public and historical use electronic records determined by the Archivist of the United States to have sufficient historical or other value to warrant their continued preservation by the U.S. Government.

Evidence: Title 44 USC 3101 and 3301, NARA Strategic Plan

Does the program address a specific and existing problem, interest or need?

Explanation: Technology has led to a decentralized records management environment, as an increased number of Federal workers have their own desktop computers and create and manage their own records (such as e-mail), resulting in a proliferation of electronic records. In this new environment, traditional paper-based records management control techniques and procedures are often no longer appropriate, resulting in Federal records management that is not well-integrated into agency business processes, systems development, information technology infrastructure, and knowledge management. This undermines the authenticity, reliability, integrity, and usability of Federal records and information essential for Government business and public use. In addition to records management challenges, electronic records also pose unique preservation and access challenges, as technology is only now being developed to preserve electronic records in a manner that will make them available for future access.

Evidence: General Accounting Office, National Archives: Preserving Electronic Records in an Era of Rapidly Changing Technology, July 1999. SRA International, Report on Current Recordkeeping Practices within the Federal Government, December 2001. General Accounting Office, Information Management: Challenges in Managing and Preserving Electronic Records, June 2002

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: All Federal records, including electronic records, have a lifecycle that includes creation, scheduling and appraisal, maintenance, destruction or archival accessioning, preservation, and continuing use. NARA and Federal agencies are responsible for separate aspects of the records lifecycle, so the amount of duplication between efforts is limited. Particular aspects of the program, such as the Electronic Records Management (ERM) Initiative, are specifically designed to reduce redundancies and inefficiencies. The exception is that NARA's Records Centers provide records management services to agencies (such as storage of temporary records), that are also available from the private sector and other Federal agencies.

Evidence: NARA Strategic Plan; Disposition of Federal Records: A Records Management Handbook; 36 CFR 1220 While Federal agencies are responsible for creating and maintaining records that adequately document the organization, functions, policies, decisions, and other essential transactions of the agency, NARA is responsible for all other aspects of the records lifecycle, including: appraisal of agency records, approval of records schedules, and preservation and provision of access to those records taken into legal custody by NARA. www.whitehouse.gov/omb/egov/internal/records.htm. The ERM Initiative, one of the President's e-gov initiatives, is intended to provide standardized electronic records management procedures across agencies to optimize expenditures, eliminate duplicate electronic records efforts, and enhance service delivery to citizens.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program design has addressed the need to redesign federal records management given the new decentralized records management environment. It also reflects NARA's need to have a system in place to manage, preserve, and access electronic records over time, something NARA currently does not have the ability to do on the scale required. However, at this early stage of the project, and given that technologies for managing and preserving electronic records are still unproven, NARA is unable to demonstrate that its planned design for an Electronic Records Archive (ERA) is optimal in terms of whether the cost and scope of the project are such that risk to the government is minimized.

Evidence: ERA Exhibit 300, NARA's Strategic Directions for Federal Records Management July 31, 2003.

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: Activities under this program are intended to address government-wide needs. NARA has therefore requested and received input from other federal agencies to ensure that its resources are targeted to address areas of greatest need and urgency across the Federal government.

Evidence: For instance, as part of the ERM Initiative, NARA consulted with agencies prior to issuing guidance on transfer of electronic records to determine which formats to transfer and in what order guidance should be released. (Memo to Agency Records Management Officers, April 26, 2002). Per request by DOD, NARA has planned that one of the first formats to be managed and preserved by ERA will be Official Military Personnel Files (OMPFs). NARA conducted a load survey in 2004 that more broadly surveyed agency needs related to ERA.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: NARA's long-term performance goal is that electronic records are controlled, preserved, and made accessible for as long as needed. NARA has long-range performance measures that address the three aspects of the program goal (records management, preservation and accessibility) and that largely focus on outcomes. For the most part, these long-term measures are contingent upon building ERA.

Evidence: NARA Strategic Plan; FY 2006 Budget. NARA's long-term measure that addresses access to electronic records is an output (processing time to provide public access to electronic records upon receipt by NARA), but sufficiently reflects progress toward the intended outcome (provision of public access to federal archival electronic records).

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets and timeframes of the long-term measures for the program are ambitious, given that technology for long-term management and preservation of electronic records is still unproven.

Evidence: NARA Strategic Plan; FY 2006 Budget

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Since NARA's long-term measures are contingent upon a working ERA, NARA for the most part uses milestones for each fiscal year specific to its long-term performance measures to indicate progress towards achieving its goals. NARA also has an annual efficiency measure related to the processing time to provide public access to electronic records upon receipt by NARA, as well as a measure indicating its ability to preserve electronic records prior to building ERA.

Evidence: NARA Strategic Plan; FY 2006 Budget.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: NARA's annual milestones, primarily connected to building and preparing for the build of ERA, are ambitious. For NARA's measures that are quantifiable, targets have baselines and are also ambitious.

Evidence: NARA Strategic Plan; FY 2006 Budget. NARA rebaselined its measure related to preservation of electronic records prior to transfer of ERA in its FY 2005 Performance Plan after prior year performance indicated targets were set too low.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Ultimately, all Federal agencies are partners with NARA in the management of electronic Federal records because of their responsibility in creating and maintaining federal electronic records that adequately document the organization, functions, policies, decisions, and other essential transactions of the agency. If other federal agencies do not commit to sufficient electronic records management practices, records may be at risk, which would prevent NARA from successfully accomplishing the long-term goals of its program. Evidence indicates that with certain exceptions, agencies for the most part do not commit to sufficient records management practices.

Evidence: Report on Current Recordkeeping Practices within the Federal Government, SRA International, December 10, 2001.The report cites the following factors as evidence that several agencies view records management and recordkeeping as a low priority: lack of staff and budget resources, absence of up-to-date policies and procedures, lack of training and lack of accountability. Report to the Interagency Committee on Government Information: Barriers to the Effective Management of Government Information on the Internet and Other Electronic Records, June 2004. The report cites the problem that records management is not viewed as critical to agency missions as one barrier to effective management of electronic records.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: NARA's planning for ERA has undergone numerous independent evaluations. The Government Accountability Office (GAO) has performed reviews of the development of ERA annually since 2002 and has periodically reviewed other aspects of NARA's Electronic Records Services program. The National Research Council evaluated the initial development of ERA and is currently preparing a second report that will address long-term and broad-scope electronic records issues. NARA has released a request for quote for an independent contractor to provide independent verification and validation (IV and V) services related to software development for the project as well as capability evaluations of the ERA contractor. In addition, NARA's OIG received funding for the first time in FY 2005 to perform regular, ongoing independent evaluations of the ERA program.

Evidence: General Accounting Office, GAO-03-880, Records Management: National Archives and Records Administration's Acquisition of Major System Faces Risks, August 22, 2003.General Accounting Office, GAO-02-586, "Information Management: Challenges in Managing and Preserving Electronic Records," June 25, 2002. NAS. Computer Science and Telecommunications Board. Building an Electronic Records Archive at the National Archives and Records Administration: Recommendations for Initial Development (National Academies Press, 2003). Request for Quote for Independent Verification and Validation Services, July 8, 2004.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NARA's budget request is clearly aligned to each of NARA's strategic goals, and NARA includes information on performance costs by linking goals and activities to dollars from each of its budget accounts. Further improvements could be made by more explicitly presenting base resource needs for the ERA program office.

Evidence: NARA's FY 2005 Budget, Congressional Justification.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: In response to the challenge posed by NARA to gain more buy-in from federal agencies on the need for records management programs, including electronic records, NARA has revamped its records management program to better enable agencies to see the value to their business processes of incorporating sufficient records management practices. NARA has also acted to address strategic planning deficiencies for ERA as identified by GAO. NARA has fully addressed 3 of GAO's recommendations from a previous evaluation of ERA, and has partially addressed the other 6 recommendations, which for the most part have required more time to fully implement or relate to ongoing evolving processes.

Evidence: GAO-99-94, GAO-02-586, GAO Findings Summary Report (April 19, 2004); NARA response to GAO April 19, 2004 report (May 26, 2004). Among the items that continue to be addressed are: hiring remaining critical staff positions in the ERA program office, full incorporation of earned value management, full compliance with industry standards on ERA policies, plans and procedures, completing NARA's enterprise architecture and improving information security. Although not fully implemented, GAO has acknowledged that NARA has made progress in addressing remaining recommendations.

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals and used the results to guide the resulting activity?

Explanation: NARA conducted an analysis of alternatives for acquisition of ERA. This analysis will be expanded to include the proposed design for an ERA system after final determination of the contractor. In addition, NARA continues to annually update its ERA business case to reflect new information and to determine the most likely costs and benefits of these alternatives.

Evidence: ERA Exhibit 300, ERA Analysis of Alternatives, ERA Business Case Analysis

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: NARA collects and reports regular monthly performance data that it uses to manage all programs and improve performance via the Performance Measurement and Reporting System (PMRS), including electronic records measures that are quantifiable. NARA has also recently implemented an earned value management system for ERA which will enable it to assess government and contractor cost and schedule performance.

Evidence: Performance Measurement and Reporting System (PMRS); Periodic Performance Reports to the Archivist (March, August, October); OIG Reports: Evaluation of the Accuracy of the Performance Measurement and Reporting System. NARA's Inspector General assists in determining the credibility of data collected by PMRS via yearly evaluations to assess data accuracy and validity of a portion of NARA's performance measures. OIG reports over the the last 3 years indicate that the majority of performance measures it has reviewed are supported by credible data. The OIG evaluated one electronic records measure in its 2003 review and made a recommendation to revalidate its source data. That action has been completed.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: NARA holds its managers accountable for performance by tying performance plans to annual performance targets and measuring performance against these results. No development contract has been awarded yet for acquisition of ERA, but NARA has identified contractor performance measures in its Request for Proposal. Currently design contractors are expected to report on and meet specific quantitative thresholds annually for each performance objective- the same standard will be applied for the development contract.

Evidence: ERA Request for Proposal; Performance Measurement and Reporting System (PMRS); Periodic Performance Reports to the Archivist (March, August, October); NH periodic performance report.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds appropriated for NARA's Electronic Records Archive are exclusively used towards this project. Funds requested for ERA through FY 2004 are multi-year, and in cases where unobligated balances remain at the end of the year they are carried over to be used in following years.

Evidence: SF 133s and SF 132s

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: NARA has recently implemented an earned value management system to analyze monthly contractor cost and schedule performance to assist in identifying problems, impacts, and areas of cost and schedule risk to development and acquisition of the Electronic Records Archive. Program control functions are compliant with ANSI-748 standards. NARA also has efficiency measures that indicate NARA's timeliness in providing public access to electronic records and in completing processing of electronic records to be permanently accessioned to NARA. Finally, NARA also has a cost-efficiency measure for storage of electronic records.

Evidence: NARA Strategic Plan; FY 2006 Budget

Does the program collaborate and coordinate effectively with related programs?

Explanation: NARA's electronic records services program has been developed in coordination with agency partners and other stakeholder groups. By incorporating NARA's technical specifications for permanent electronic records into their work processes, agencies will have a more uniform body of records to manage while the records are being used for agency business and the transfer of records to NARA for accession will be simplified. The benefit for NARA of such cooperation is that preservation of electronic records will cost less and researcher access will be facilitated (fewer disparate formats to migrate, fewer software viewers required, etc.).

Evidence: ERA RFP, ERA Acquisition Strategy, NWM 10.2002 (April 26, 2002), Memorandum to Agency Records Officers, Chief Information Officers, and Information Resources Managers: Request for comment on Proposed NARA guidance on new transfer standards for electronic records. "FBI Puts Records to Work," Federal Computer Week, June 21, 2004. Some specific examples of coordination include the following: 1). The ERM initiative work plan was developed with the records management stakeholder community in April 2002 and the specific transfer formats were identified and prioritized by agency records officers. 2). The Nuclear Regulatory Commission (NRC) built into its rule governing submission of documents to NRC the technical specifications from the NARA transfer guidance for scanned images of textual records. 3). The State Department incorporated into its requirements for its State Messaging and Archive Retrieval Toolset (SMART) system a requirement to meet the DoD 5015.2 STD requirements, as NARA Bulletin 2003-03 advises.

Does the program use strong financial management practices?

Explanation: NARA reported one material internal control weakness in its FY '04 Financial Manager's Financial Integrity Act (FMFIA) report that directly relates to its electronic records program- IT security (a material weakness since FY '00).

Evidence: FY 2004 FMFIA report. FY 2004 Performance and Accountability Report. NARA produced audited statements on its appropriated funding for the first time for FY 2004. As a result of the audit process, NARA is in the process of improving its financial management policies and practices. According to its FY 2004 PAR, NARA expects to resolve the basis for its material weakness in IT security during FY 2005.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: NARA has taken meaningful steps to address its management deficiencies related to the program. NARA responds to management deficiencies identified both internally and externally. Within NARA, managers prepare annual assurance statements, which internally identify management deficiencies and steps for remediation, including deficiencies related to its electronic records services program. NARA's IT review board holds weekly meetings to review the status of ERA, and prepares quarterly reviews of the project. To address externally identified management deficiencies, NARA managers prepare 3 reports during each fiscal year for the Archivist, which address progress on implementing recommendations from audits and reviews. Examples of steps NARA has taken to improve overall management of the electronic records program include continuation of corrective actions to address its IT security material weakness as well as strengthening of its enterprise architecture.

Evidence: GAO Findings Summary Report, April 19, 2004; NARA response to GAO April 19, 2004 report (May 26, 2004) Annual Assurance Statements; periodic performance reports to the Archivist; Monthly Strategic Schedule Reviews.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: NARA has identified and defined the required capability and performance characteristics expected for the initial operating capability for ERA in its related requirements document and request for proposal. However, since the contract has yet to be awarded, it is too early in the acquisition process for NARA to demonstrate that it makes management decisions based on whether contractor milestones are being met.

Evidence: ERA Request for Proposal

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: NARA's long-term measures for electronic records are for the most part new, and many will not be applicable until deployment of the Electronic Records Archive. Until deployment of its initial operating capacity, NARA will be unable to demonstrate adequate progess in achieving its long-term performance goals.

Evidence: FY 2004 Performance and Accountability Report

Does the program (including program partners) achieve its annual performance goals?

Explanation: NARA has met many of its milestones related to electronic records services, although slippage in the acquisition schedule for ERA has occurred. NARA achieved its annual goal related to preservation of electronic records in preparation for transfer to ERA.

Evidence: FY 2004 Performance and Accountability Report

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: NARA has an EVM system in place for its development and acquisition of the Electronic Records Archive, as well as efficiency measures for its electronic records program generally. However, the EVM system is too new to demonstrate whether improved efficiencies have occurred related to its deployment, and the efficiency measures NARA has in place will be unable to demonstrate results until deployment of ERA.

Evidence: FY 2004 Performance and Accountability Report

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Due to the scope of what NARA envisions achieving with ERA in terms of the volume of electronic records to be preserved, the number of formats requiring access and preservation, and the need to provide certain standards of authenticity of electronic records, no other Federal, state, or local government entity provides a viable comparison, particularly since NARA is further along than other government entities (Federal agencies and other National Archives) in its work on preserving electronic records. Private sector entities also do not have the extensive archival requirements of NARA, particularly regarding public access issues (the need to provide public access while simultaneously dealing with a variety of restrictions on access, including all levels of national security, FOIA statutes and the Presidential Records Act).

Evidence: The Library of Congress has just announced its first set of research projects related to digital preservation, an area in which NARA has been engaged in research since 1996 (in the InterPARES project). The National Archives of the U.K. established its first archival system for electronic records in 2003, a small system which provides on-site access to electronic records. NARA's Access to Archival Databases application, also established in 2003, provides web access to nearly 50 million historic electronic records created by more than 20 federal agencies. The government of Australia has recently announced a project to develop standards for preserving electronic records, but the National Archives of Australia is using a small prototype that implements a preservation strategy articulated by researchers working under NARA sponsorship at the San Diego Supercomputer Center. While the National Archives of Australia holds a larger volume of digital data than NARA, its operational capabilities are largely limited to storing and copying digital media.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Evaluations of ERA have focused on planning and development of the system. Evaluations by GAO on planning for ERA indicate that NARA has made progress with implementing recommendations GAO believes are necessary to ensure that ERA is delivered on cost and on schedule. However, not all recommendations have yet been fully implemented, which GAO believes continues to put acquisition of ERA at risk.

Evidence: GAO-99-94, GAO-02-586, GAO Findings Summary Report (April 19, 2004); NARA response to GAO April 19, 2004 report (May 26, 2004). General Accounting Office, GAO-03-880, Records Management: National Archives and Records Administration's Acquisition of Major System Faces Risks, August 22, 2003.General Accounting Office, GAO-02-586, "Information Management: Challenges in Managing and Preserving Electronic Records," June 25, 2002.

Were program goals achieved within budgeted costs and established schedules?

Explanation: Work related to the development and planning of the Electronic Records Archives remained within budget in FY 2004. Some schedule slippage has occurred while the ERA program office made adjustments to planning documents to incorporate recommendations from GAO and in awarding the contract for ERA. However, long-term program goals are contingent upon deployment of ERA, and therefore it is too early in the process to make a definitive assessment of whether long-term program goals are being achieved within budgeted costs and established schedules.

Evidence: FY 04 monthly schedule reporting, Monthly Strategic Schedule, Quarterly reports to Congress on ERA