Detailed Information on the
Office of the National Coordinator for Health Information Technology Assessment

Develop outcome-oriented measure for network exchange related activities, including milestones/targets. Measure should reflect desired results and with ambitious targets.

Complete draft operational plan that will incorporate steps needed to achieve goals outlined in the Federal HIT Strategic Plan.

Develop additional performance measures that will link progress with achieving the goals of the Federal HIT Strategic Plan.

Incorporate the Federal HIT Strategic Plan into the ONC performance budget.

Identify program/office priorities to achieve over the shorter-term (two-years). These priorities will have outcome-oriented results and show clear links to the program's overall mission.

Develop an efficiency measure.

Measure: Increase physician adoption of Electronic Health Records (EHRs).

Explanation:ONC's overarching goal, as stated by the President, is that by 2014, most Americans will have access to EHRs. At least 50% of physicians will have EHRs for their patients and consumers as measured by standardized, industry-accepted methodology. The targets listed for this measure refer to adoption within physician offices only. ONC's current estimated baseline and subsequent target and performance data is based on two different survey sources. Actual baseline and subsequent target data should be derived from the same source, for an apples to apples comparison.

Measure: Increase the percentage of small practices with Electronic Health Records (EHRs).

Explanation:This measure strives to address the gap in adoption between large physician practices (20 or more physicians) and small physician practices (those with 5 or less physicians) through strategies that help small physician offices have access to and adopt health IT at an appropriately proportional rate. Currently, the adoption rate for small physician practices is significantly lower than the national average. The targets for adoption among small practices reflect a lower starting point and the expected adoption rates associated with that baseline. Note that small physician practices will also be more likely to have practices in underserved settings. ONC's currently estimated baseline and subsequent target data is based on two different survey sources. Actual baseline and subsequent target and performance data should be derived from the same source, for an apples to apples comparison.

Measure: Develop a mature Nationwide Health Information Network (NHIN) architecture that will support broad health information exchange.

Explanation:The Nationwide Health Information Network (NHIN) will be a "network of networks" to support the exchange of health information by connecting disparate systems. Moving data securely and confidentially, the NHIN will improve the quality of care by allowing health information to follow the patient.The NHIN will also help stimulate increased rates at which EHRs are adopted by assuring that systems can share data reliably, securely, and efficiently among each other. Development of baseline and target data is underway, which will provide clear linkages to the NHIN outcomes, as well as ONC's longer-term outcomes and goals.

Measure: Increase the implementation of recognized standards in Federal and commercial systems.

Explanation:Data for this measure will be collected through Federal measuring tool and measured for the current year and prior year divided by the prior year number of standards implemented. For commercial systems, data will be determined by the number of standards included in certification criteria for the current year as reported by the Certification Commission for HIT (CCHIT) (or similar organizations) and the prior year divided by the prior year number of standards recognized. The outcome of this measure will be increased use and implementation of standards, thus leading to increased interoperability and adoption of health IT.

Measure: Percent of physician offices adopting ambulatory EHRs in the past twelve months that meet certification criteria.

Explanation:Increased EHR adoption is also dependent on a process that guarantees that the purchasers of EHRs are implementing systems and products that meet their needs for functionality, security, and interoperability. The certification process guarantees that the EHRs being purchased incorporate the appropriate standards for these three areas. The percent of adoption of certified health IT will be measured as the percentage of EHRs adopted in a given year that are certified.

Measure: Cost per physician for adopting certified EHRs.

Explanation:This measure will monitor the cost of certification, as more physicians adopt electronic health records. The calculation will take into account the total of ONC funding for Standards, Privacy and Security, Architecture and Adoption and Governance plus the charge to vendors for product certification, as reported by the CCHIT. This sum will be divided by the number of physicians in the US, as reported by the American Medical Association, multipled by the results of the annual adoption survey that reports the percent of physicians adopting general ambulatory certified electronic health records.

Is the program purpose clear?

Explanation: The mission statement states: "The Office of the National Coordinator for Health Information Technology provides leadership for the development and nationwide implementation of an interoperable health information technology infrastructure to improve the quality and efficiency of health care and the ability of consumers to manage their care and safety. Further, Executive Order 13335 estabished the position of the National Coordinator. On April 27, 2004, President Bush stated the goals of the Office of the National Coordinator are to "provide leadership for the development and nationwide implementation of an interoperable health information technology infrastructure to improve the quality and efficiency of health care??" President Bush further stated that his goal was for most Americans to have electronic health records (EHRs) within 10 years (2014).

Evidence: The Office of the National Coordinator (ONC) mission statement and executive order can be found at: http://www.hhs.gov/healthit/missionorder

Does the program address a specific and existing problem, interest, or need?

Explanation: There is a demonstratable need for interoperable electronic health records (EHRs), as well as the need for leadership to facilitate the establishment of common health IT standards and a framework for EHR interoperability throughout the government and private health care sectors. This program exists because of the fragmentation of the healthcare system today - with more than 500,000 office-based physicians, approximately 5,000 community hospitals, more than 16,000 certified nursing facilities, and many other providers in many other care settings. An HHS contracter convened a Leadership Panel which identified as a key imperative that the Federal government should act as leader, catalyst, and convener of the nation's health information technology effort, since private sector investment in health IT has not yet yielded significant adoption, and the efforts to date have largely been non-interoperable health records. In the Federal sector, health IT programs have been developed in VA, DoD, and IHS among others, but these programs are not widely interoperable.

Evidence: A 2005 study in the Journal of the American Medical Association (JAMA) showed that clinical information is frequently missing at the point of care, and that this missing information can be harmful to patients. That study also showed that clinical information was less likely to be missing in practices that had full electronic records systems. Five years ago, the Institute of Medicine (IOM) estimated that as many as 44,000 to 98,000 deaths occur each year as the result of medical errors. Other studies have demonstrated that health IT, when properly implemented, improves quality of care and patient safety. A 2004 survey by the Agency for Health Care Research and Quality (AHRQ) with the Kaiser Family Foundation and the Harvard School of Public Health found that nearly 1 in 3 people say that they or a family member have created their own set of medical records to ensure that their health care providers have all of their medical information. The Framework for Strategic Action (http://www.hhs.gov/healthit/readiness.html) highlights various studies demonstrating the need for an interoperative HIT system. Fragmentation of the U.S. Health System is described in:Physician stats: American Medical Association. Table taken from Health Industry Distributors Physician Market Report; and, Hospital Stats: American Hospital Association, "Fast Facts on U.S. Hospitals," 2003, www.hospitalconnect.com/aha/resource_center/fastfacts/fast_facts_US_hospitals.html (11 January 2005); JAMA study: http://jama.ama-assn.org/cgi/content/abstract/293/5/565; AHRQ survey information referenced above found at Chart 25 from National Survey on Consumers Experiences With Patient Safety and Quality Information - Kaiser Family Foundation, AHRQ and Harvard School of Public Health, released November 17, 2004.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Prior to the appointment of a National Coordinator, no coordinating body existed in the private or public sector to ensure interoperable health information exchange. The National Coordinator is directed to coordinate programs and policies regarding HIT adoption across the Federal government. In addition, the National Coordinator is charged with coordinating outreach and consultation between the Federal Government and the private sector. The American Health Information Community (AHIC) provides such consultation. This Federally-chartered commission provides input and recommendations to the Secretary of HHS (and, therefore, also to ONC) on how to make health records digital and interoperable. It is made up of public and private sector health care leaders and stakeholders. Other Federal programs such as DOD, VA, and OPM collaborate with ONC to encourage the widespread adoption of HIT thorughout the health care system.

Evidence: The website (http://www.hhs.gov/healthit/federalprojectlist.html#intitiativestable) describes the ONC responsibilities and its oversight with Federal HIT programs. Information on the AHIC can be found at: www.hhs.gov/healthit/ahic. An HHS contracted Leadership Panel identified as a key imperative that the Federal government should act as leader, catalyst, and convener of the nation's health information technology effort: The Leadership Panel Report Press Release: http://www.hhs.gov/news/press/2005pres/20050511.html. Reports from DOD, VA, and OPM examine their lessons learned, respectfully, and how they can contribute to a nationwide health information infrastructure. These are found in the "Framework for Strategic Action."

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: ONC strategies were developed with significant stakeholder input. To develop the long-term infrastructure for interoperable health IT, RFPs were issued in June of 2005, and contracts were awarded between September 2005 and November 2006. These contracts have been put in place to develop processes for standards harmonization, health IT certification, and a National Health Information Network (NHIN) architecture that will be able to evolve and be self-sustaining in the future. Consistent with the Executive Order for ongoing stakeholder input and collaboration, the Secretary of HHS has formed the American Health Information Community to: 1) advise the Secretary and recommend specific actions to achieve a common interoperability framework for health IT; and 2) serve as a forum for participation from a broad range of stakeholders to provide input on achieving interoperability of health IT.

Evidence: Documents including: 1. Request for Information (RFI) executive summary describes responses (over 500) to questions regarding policy standards, legal and regulatory issues, and management and organizations considerations, among others. This RFI was published in November, 2004, and solicited public input about whether and how a NHIN could be developed. 2. AHIC charter which describes the purpose and participants of the AHIC. 3. Ways and Means Testimony - David Brailer April 6, 2006. Testimony describes ONC actions to date and progress/process for further developing EHRs and a nationwide interoperable HIT network. 4. Congressional Justification Funding Requests in detail where/what projects FY 2007 funds are to be directed.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The purpose of the program is to coordinate and develop health information exchange so that most Americans will have EHRs by 2014 that can share and exchange health information among entities. Because the technical capacity for health information exchange does not exist today, ONC has put in place programs, through consultation with beneficiaries and stakeholders: to harmonize standards, which invoves solidifying consistency with interactive IT communication processes; to certify health IT products as they evolve; and, to foster the development of a Nationwide Health Information Network (NHIN) architecture that will serve as the blueprint for how a NHIN will connect interoperable EHRs. Additionally, ONC continues to develop and refine testing and processes in order to facilitate certification of the functional requirements for interoperable EHRs. This technical capacity will enable health information exchange for stakeholders (both government and private sector) of the healthcare system. The program has set the foundation and is designed to bring a broad array of stakeholders to the table, thus ensuring beneficiary input and collaboration.

Evidence: Award contracts are designed to create and evaluate processes for harmonizing health information standards; develop criteria to certify and evaluate health IT products; develop NHIN prototypes; and develop solutions to address variations in business policies and state laws that affect privacy and security practices that may pose challenges to the secure communication of health information for beneficiaries: (http://www.hhs.gov/news/press/2005pres/20051006a.html) and (http://www.hhs.gov/news/press/2005pres/20051110.html). Congressional Justification Funding Requests: Contracts with strategic partners for these and other related programs represent over 75% of ONC's budget for FY2006, and over 88% planned for FY2007. Press release (of RFI response described in 1.4) describes overarching concepts suggested by the respondents including the need for the Federal government to play a role in advancing national HIT network: http://www.hhs.gov/news/press/2005pres/20050603.html.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ONC has developed specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program which is to have electronic health records for most Amercians by 2014. These long-term measures include increasing physician adoption of electronic health records and increasing the percentage of small practices with electronic health records.

Evidence: The Executive Order and the Congressional Justification describe ONC's mission and overarching goal. To measure progress, ONC has contracted for an annual report on health IT adoption which builds on existing survey efforts.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The overarching goal is to have electronic health records for most Americans by 2014. ONC's current estimated baseline and subsequent target data is based on two different survey sources. Determination of the ambitiousness of targets and timeframes cannot be made until actual baseline and subsequent target data are derived from the same data source, for an apples to apples comparison.

Evidence: To measure progress, ONC has contracted for an annual report on health IT adoption which builds on existing survey efforts; however, current estimated baselines are based on accepted industry expectations derived from survey data conducted by Harvard University. It is unclear whether the assumptions and methodology used in this survey are the same as those in the CDC survey tool, which will be used for measuring progress. Actual baseline and target data, from the same source, will be available in FY 2007. Baseline Data: Derived from published survey data conducted by Harvard University (under subcontract with George Washington University). Target Data: Derived from newly standardized CDC survey tool (the National Ambulatory Medical Care Survey and the National Hospital Ambulatory Medical Care Survey).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: ONC's annual measures will move the health IT initiative closer to the ultimate goal of most Americans having electronic health records by 2014 through a series of steps taken to increase the rate at which EHRs are adopted within the clinical community. For example, assuring that EHRs and systems are interoperable -- can share data reliably, securely, and efficiently among each other -- is one of the key components to encourage the adoption of EHRs. Interoperability is dependent on the data standards developed through the Standards Harmonization process. It is also dependent on the development of the National Health Information Network (NHIN), if patients require care remotely, or at significant distances from their home. A second driver of increased adoption is a process which guarantees the purchasers of EHRs that they are implementing systems and products that meet their needs for functionality, security, and interoperability. The certification process does just that - it guarantees that EHRs being purchased incorporate the appropriate standards in these three areas.

Evidence: Measures include: the development of a NHIN architecture in order to support broad health information exchange; development of a unified set of standards to support the requirements for broad health information exchange; and, the adoption of certified ambulatory EHRs by physician offices.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: For the NHIN and harmonization measures, the targets are not well-defined, nor clearly linked to long-term outcomes. Clearer demonstrated target data for these two measures should link to the longer-term outcome measures. Individual targets should reflect the ultimate outcome expected - the development of an interoperable health IT system which will bring closer the ultimate goal of EHR usability and adoption by 2014. There is one measure (certified EHR adoption) where baselines and targets have been established.

Evidence: Well-defined and clear links to long-term outcome measures should be evident. For example, the expected outcome of prototype development and how each step (milestone deliverable) is used in succession should provide evidence of achievement towards the long-term goals. For the harmonization measure, demonstrated usefulness of the milestone deliverables each year should clearly demonstrate progress. The certification measure presents baseline and target data.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ONC is facilitating the development of a long-term health IT infrastructure through partnerships that have been identified with contracts. These contracts specify milestones and deliverables which contribute to the overarching goal of most Americans having an electronic health record by 2014.

Evidence: The Executive Order and the Congressional Justification describe ONC's mission and overarching goal. Contracts reference the Executive Order and the goals of the President. Such language includes: "On April 27, 2004, the President issued an executive order (EO) announcing his commitment to the use of health information technology (health IT) in order to reduce medical errors, lower costs and provide better information for consumers and physicians. In particular, the President called for the widespread adoption of electronic health records (EHRs) and for health information to follow patients throughout their care in a seamless and secure manner..."

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ONC has contracted for an independent evaluation of its long-term health IT adoption goals. This review is aimed at determining the effectiveness of policies designed to accelerate adoption of EHRs and interoperability, and is scheduled to be completed in October, 2006. In the May 2005 report, the GAO commended ONC's progress in its initial steps toward the development of a national strategy for health IT, but recommended more detailed milestones and project planning be developed. In the follow-up March 2006 report, the GAO commended ONC on its progress in addressing the above; at the time of this updated follow-up report, ONC had issued contracts with defined milestones and outcomes.

Evidence: ONC has partnered with the George Washington University and Massachusetts General Hospital/Harvard Institute for Health Policy for the HIT adoption initiative contract. May, 2005, GAO report, "Health Information Technology: HHS Is Taking Steps to Develop a National Strategy."(ID number: GAO-05-628); March, 2006 GAO report, "Health Information Technology: HHS is Continuing Efforts to Define a National Strategy." (ID number: GAO-06-346T). The 2005 report indicated a need for detailed milestones for its project planning; the later report reflects the progress ONC had made to date.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ONC is currently developing its measures and future budget justifications will reflect these efforts. The current budget does not reflect use of performance measures as justification for its request. Future requests will include the use of performance measures, as well as discussion on direct, indirect, and marginal costs.

Evidence: FY 2007 Congressional Budget justification request reflects ONC's requested resource needs only, and does not provide performance information.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: An initial GAO inquiry indicated the need for ONC to have detailed milestones for its project planning. Since that time, ONC has put in place detailed milestones for the development of its long-term infrastructure projects. In addition to these milestones, the program is working to identify and adopt a limited number of specific, ambitious long-term performance goals and a limited number of annual performance goals that demonstrate progress toward achieving the long term goals.

Evidence: Evidence includes: March, 2006 GAO report, "Health Information Technology: HHS is Continuing Efforts to Define a National Strategy." (ID number: GAO-06-346T). Report describes ONC's progress to date with using and implementing milestones and planning information. For instance, in the Standards Harmonization contract, milestones and due dates are clearly identified for "identifying gaps in current standards landscape to meet the relevant use-cases." This particular milestone was met successfully. Currently, existing performance measures are limited, and the baselines for these are in the process of being established.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Program partners have been established through contracts, and progress is measured through the specific milestones and deliverables that are defined in these contracts. ONC project directors regularly obtain project updates and deliverables according to schedules defined in each contract. ONC has in place requirements for regular and timely collection of performance information related to the measurement of health IT adoption as well as the progress toward milestones in the contracts. All contracts are reported at least monthly. ONC is still in the first year of contracts, but actions are taken as necessary to ensure all projects are aligned. For instance, ONC leverages project status reports and updates to better manage the program and improve overall performance. For those instances where performance may slip, ONC meets with project managers to ensure risks are mitigated. When needed, ONC notifies the necessary managers of lapses in schedule, cost, and delivery. ONC leadership will leverage performance best practices, encouraging best practice implementation across all ONC activities wherever possible.

Evidence: Vendors are each required to submit monthly reports on contract performance that include information about earned value, scheduled delivery status, financial information to track actual expenditures to plan, and identification of issues for management to be aware. Task 1 from the Standards Harmonization contract is an example of the performance management requirement. To date, there has been very little slippage in contracts except minor changes in contract deliverable dates based on updating requirements.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Strategic program partners are held accountable for cost, schedule, and performance results through the specific milestones and deliverables that are defined in these contracts. These deliverables are reported on monthly and ONC managers ensure that milestones are being met according to the agreed upon schedules. Additionally, by serving as project officers, the Directors are responsible for the success of their respective projects through technical monitoring that is required in ONC contracts. As a result, performance of ONC managers and their strategic partners are jointly held accountable for overall program performance which is predominantly being executed through strategic contracts. In the event that work is deemed unsatisfactory, the project officer (in consultation with the contracting officer) must determine what further actions are required, asking advice of legal council if necessary.

Evidence: Performance plans/contracts have been developed for each of the Directors, which tie to HHS' strategic plan include detailed performance expectations. There are times when the Secretary of HHS has assigned ad-hoc tasks that are not necessarily a part of of an ONC manager's performance plan. On these occasions, the tasks are often a matter of public record (evidence: Transcript of January 06 AHIC meeting), and thus incure public scrutiny if not accomplished satisfactorily and timely. Federal partners have agreed through Interagency Agreements (IAA) with HHS to support ONC goals. These agreements clearly identify expected performance and deliverables. For example, the IAA with VA outlines deliverables and due dates that VA agreed to comply with in support of the Federal Health Architecture (FHA) program through staffing and funding. VA is integrally involved with all decisions that are made in moving FHA forward and in supporting AHIC efforts. VA has subject matter experts that are assigned on an as-needed basis depending on what the upcoming work entails. Due dates for deliverables are agreed upon and assigned, one staff member is given responsibility to ensure that products are delivered timely, and, in the spirit of working together, these deadlines are given top priority.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: ONC has developed a financial operating plan, in close cooperation with the Office Directors. ONC uses an internal financial management system which is reconciled regularly with the Department's CORE accounting system to track obligation amounts on a quarterly basis. To ensure that progress is made toward achieving the plan goals, ONC continuously monitors commitments and obligations to ensure that contracts are awarded according to schedule. In October, 2006, ONC will be transferring to UFMS, a Department-wide financial management system.

Evidence: Budget briefing spreadsheets describe delienated spending plan. The ONC Operating Plan describes contract financing by quarter. FMS (the financial management system) is an internal, real-time tracking system used by ONC and OSEO. Although CORE is the system of record, there is a lag in the time required to award contracts, so CORE and FMS are reconciled regularly to ensure up-to-date status on fund obligations. Contract delivery schedules specify timing deadlines of contract submissions in order to be considered for awards. Also includes plans for UFMS (HHS financial management system) transition. Additionally, Earned Value Management (EVM) data is required of all major contracts. The language reads as follows: "The Contractor shall...Develop a comprehensive work plan building upon the plan included in the original proposal submission. The work plan shall include a written description of proposed process/strategy to execute all tasks and apply Earned Value Management (EVM - as referenced in section 300.4 of OMB Circular A-11) to the extent practicable. The work plan shall also provide the Project Officer with project activities; task prioritization; resource requirements, including person hours by task; interim milestones to achieve deliverables; interdependencies and intersections with other activities and risk mitigation strategies." Current contracts are being executed within budget.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ONC does not have a measure of efficiency and is currently developing one. At this time, however, many of the programmatic goals for ONC are being achieved through competitive contracts with private sector organizations. These contracts require awardees to meet defined milestones, and the Nationwide Health Information Network (NHIN) architecture contracts include clauses for Earned Value Management.

Evidence: Evidence includes performance-based competitive contracts describing costs and expectations for the development of a prototype for a NHIN. Additionally, EVM reports provide data on planned and actual expenditures, in addition to work scheduled and performed.

Does the program collaborate and coordinate effectively with related programs?

Explanation: By its very design, the program has established effective mechanisms for collaboration. The American Health Information Community (AHIC) brings together representation from both the public and private sectors. Federal agencies (i.e., DOD, VA, OPM, and other HHS components) are represented by senior leadership at these meetings and commit their organizations and resources in support of common goals related to EHRs. At the technical and policy level, the Federal Health Architecture and the Health IT Policy Council have been structured to coordinate efforts that will align programs and policies across agencies. Through these meetings, Federal agencies work together to develop components of ONC efforts in areas such as standards and certification requirements. Each agency commits funds and staff resources in these collaborative efforts and works across agency lines to successfully move the President's agenda forward. In addition, staff from other components across the Federal government have been detailed to support ONC while permanent staff are being recruited.

Evidence: AHIC Charter which states, "...recognizing the need for public and private sector collaboration to achieve (these goals)...". AHIC recommendations help move the initiative forward. The Charter also describes the pubic and private sector membership. (http://www.hhs.gov/healthit/ahic.html). "Aligning the Federal Health Architecture: Strategic Overview" describes a Federal health IT environmment that is interoperable with the private sector and supports the President's health IT plan through its efforts to coordinate Fereral health architecture contributions. Interagency Health Information Technology Policy Council Founding Document which describes the make-up of the Federal entity and participation responsibilities as members. The Council provides a forum for sharing agency activities and endorsing and coordinating HIT efforts as addressed. Additionally, MOUs describe the detailing of staff from other entities to ONC to perform specified responsibilities.

Does the program use strong financial management practices?

Explanation: As an office in the Office of the Secretary, ONC financial oversight is carried out by the Office of the Secretary Executive Office (OSEO). ONC is currently putting in place systems to augment its financial management information. In FY2007, the Unified Financial Management System is projected to be completed for HHS, and these tools will be available to operations managers for additional controls around financial and operational management. Greater then 75 percent of the program's budget during its 2 years of existence has been contract-oriented. Contract activity is managed by certified Project Officers to ensure performance and financial programs are achieved. Performance data is reviewed centrally by ONC management on a monthly basis. ONC budget level data is tracked in the HHS Financial Management System (FMS) and all obligations are certified by HHS budget personnel. ONC execution and invoice data is tracked at the transaction level in the HHS PSC Core Accounting System (CORE), which meets US Standard General Ledger requirements. In regards to reimbursable funds for IAAs and MOUs (such as with FHA), ONC is currently working from a paper-based system. However, the organization is preparing to track these items through the PRISM procurement system and reconcile them to FMS.

Evidence: Departwide-wide notification describes the move to UFMS. Listing of ONC contract awards describes the cost, duration and descriptions of contract activity. CORE and FMS are reconciled regularly to ensure up-to-date status on fund obligations. Monthly funds status reports describe period and total expended funds. ONC's Operating Plan describes its quarterly spending. Documentation details move to PRISM procurement system.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: ONC was formally established by the Secretary of HHS in August 2005, and while no major program or management deficiencies have been identified, opportunities for improvement have been conveyed. ONC is proactive in improving its management practices, and as a new entity within HHS, ONC is establishing policies and procedures to ensure that day-to-day operations meet all applicable Federal requirements. Hiring permanent staff is potentially a management concern. ONC has an aggressive hiring plan and expects to have a full complement of permanent staff by the end of the year. ONC uses a web-based health IT listserv to advertise positions and ensure that the most qualified people are advised of the opportunities.

Evidence: ONC organizational chart and recruitment activities describe the organization and supporting staff needed/recruited for particular positions. ONC has filled 13 or 35% of vacancies (3 were filled prior to 1/1) over the past 6 months. Seven additional positions have closed and interviews are ongoing or ready to begin; six additional positions will be fulled with Summer 2006 announcements. May, 2005, GAO report, "Health Information Technology: HHS Is Taking Steps to Develop a National Strategy."(ID number: GAO-05-628); March, 2006 GAO report, "Health Information Technology: HHS is Continuing Efforts to Define a National Strategy." (ID number: GAO-06-346T). These reports demonstrate before and after scenarios of responsiveness for the need for detailed milestones for its project planning.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: HHS does not yet have actual performance data to measure progress.

Evidence: In progress

Does the program (including program partners) achieve its annual performance goals?

Explanation: HHS has not yet developed outcome-based targets to measure progress in its annual performance goals.

Evidence: In progress

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Efficiency measure is currently under development and achievement toward this measure will be measured accordingly.

Evidence: In progress

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: No comparative programs have been identified for performance comparison purposes. Prior to the appointment of a National Coordinator, no body existed in the private or public sector that took on the leadership and coordinating role in creating a nationwide interoperable health IT infrastructure. This program is unique in its goals and its approach by the nature of the programs and institutions that it touches.

Evidence: ONC is charged with providing leadership for the development and nationwide implementation of an interoperable health information technology infrustructure. As such, it draws from Federal, State, local, and private entities to encourage the adoption and develop the standards for which such a system will exist. The AHIC, a public/private partnership, allows for open discussions to support the development of this interoperable system; a unique entity unto itself to provide for this purpose. Because of ONC's uniqueness, there are no comparisons to be made. The previously mentioned RFI (Q 1.5) re-iterates this point in that the concepts suggested by the repondents include the need for the Federal government to play a role in advancing a national HIT network. However, other entities striving to implement HIT and EHRs, in particular, will benefit from the mission and work of ONC and supporting organizations, such as the AHIC.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: ONC has contracted for the independent evaluation of the long-term health IT adoption goals through a survey that will measure this progress. These results are due in October, 2006 and will serve as an indicator of progress. Additionally, the GAO reports, as mentioned in question 2.6 and others, report on progress made with ONC achieving its purpose and overarching goal.

Evidence: The Office of the National Coordinator for Health Information Technology (ONC) will partner with the George Washington University & Massachusetts General Hospital / Harvard Institute for Health Policy through a contract on the Health IT (HIT) Adoption Initiative. The new initiative is aimed at better characterizing and measuring the state of EHR adoption and determining the effectiveness of policies aimed at accelerating adoption of EHRs and interoperability. May, 2005, GAO report, "Health Information Technology: HHS Is Taking Steps to Develop a National Strategy."(ID number: GAO-05-628); March, 2006 GAO report, "Health Information Technology: HHS is Continuing Efforts to Define a National Strategy." (ID number: GAO-06-346T). The 2005 report indicated a need for detailed milestones for its project planning; the later report reflects the progress ONC had made to date.