ExpectMore.gov


Detailed Information on the
Bioterrorism: Biosurveillance Assessment

Program Code 10003541
Program Title Bioterrorism: Biosurveillance
Department Name Dept of Health & Human Service
Agency/Bureau Name Centers for Disease Control and Prevention
Program Type(s) Direct Federal Program
Assessment Year 2006
Assessment Rating Results Not Demonstrated
Assessment Section Scores
Section Score
Program Purpose & Design 100%
Strategic Planning 62%
Program Management 100%
Program Results/Accountability 25%
Program Funding Level
(in millions)
FY2007 $71
FY2008 $53
FY2009 $101

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2006

Establish baselines and targets for the program's annual and long-term performance measures.

Action taken, but not completed Baselines and targets for the BioSense outcome measure and the LIMS/Results Messenger outcome and output measures are under revision or replacement consideration. In addition, In March, 2008, DGMQ underwent a PART reassessment by the Office of Management and Budget (OMB). As part of this review, DGMQ has proposed new performance measures for its activities. After final approval of the new measures by OMB, DGMQ will retire both the output and outcome Biosurveillance measures.
2006

Explicitly tie budget requests to the accomplishment of annual and long-term goals, and present resource needs in a complete and transparent manner.

Action taken, but not completed Improvements to CDC??s budget and performance planning tool include streamlining processes, better aligning project planning across the agency, restructuring project classification variables, and enhancing IT system performance. The system provides for execution and management of projects by giving users the ability to update progress against milestones, provide evidence of accomplishments and results, monitor spending versus budget, and identify risks and develop mitigation strategies.
2006

Demonstrate adequate progress in achieving the programs annual and long-term performance goals.

Action taken, but not completed Biosurveillance output measures have been collecting/reporting annual data since FY2005. Processes for annual data collection/reporting for the biosurveillance outcome measures remain under development as justification for target replacement is under consideration. The 2 Biosurveillance Quarantine-related measures have undergone a recent PART assessment; therefore these Biosurveillance measures will be replaced upon OMB final approval of measures).
2006

Independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results.

Action taken, but not completed The first COTPER BSC meeting is scheduled for August 5-6, 2008. The agenda for the first board meeting will be to focused on orientation of the board to the roles and responsibilities of COTPER and discuss the specifics of how the board will proceed with external peer review of COTPER programs.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments

Program Performance Measures

Term Type  
Long-term/Annual Output

Measure: Number of top 50 metropolitan areas using BioSense


Explanation:

Year Target Actual
2005 10 10
2006 40 38 (Unmet)
2007 50 57 (Exceeded)
2008 50 12/2008
2009 50 12/2009
2010 50+ 12/2010
2013 N/A
Long-term/Annual Output

Measure: Number of quarantine stations that are fully staffed with public health professionals who are preparednes to respond appropriately when needed.


Explanation:

Year Target Actual
2010 10 10
2009 20 18 (Unmet)
2008 Up to 25 20 (Met)
2007 Up to 25 12/2008
2006 Up to 25 12/2009
2005 Up to 25 12/2010
2013 N/A (To be retired)
Long-term/Annual Outcome

Measure: Reduce the time needed for a Laboratory Response Network (LRN) laboratory to enter and message LRN-related standardized results to the CDC.


Explanation:The ability to exchange laboratory data, both within the Laboratory Response Network (LRN) as well as between the LRN and the CDC, is critical to initiate event-specific standard operating procedures (e.g., aggregation of data at a national level) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Reducing the time needed to for a LRN laboratory to enter and message LRN-related standardized results to the CDC is one aspect of CDC efforts to minimize the time required to initiate event-specific standard operating procedures.

Year Target Actual
2007 baseline Chem/Bio 30/37 min.
2008 Chem/Bio 23/27 min. 12/2008
2009 Chem/Bio 17/16 min. 12/2009
2010 Chem/Bio 10/5 min. 12/2010
Long-term/Annual Outcome

Measure: By 2010, the Quarantine Stations will reduce the time needed from a triggering biosurveillance event (notification of an international or interstate traveler who traveled while infectious with a quarantinable disease or other infectious disease of public health importance) to initiate event-specific standard operating procedures (e.g., isolation, quarantine, contact notification) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance.


Explanation:

Year Target Actual
2010 TBD TBD
2009 TBD TBD
2008 TBD TBD
Long-term/Annual Outcome

Measure: By 2010, the Laboratory Response Network Results Messenger will reduce the time needed from a triggering biosurveillance event (i.e., transmission of data regarding the identification of any Category A or B agent) to initiate event-specific standard operating procedures (e.g., aggregation of data at a national level) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Baseline and targets to be determined by June 2007.


Explanation:

Year Target Actual
2010 TBD TBD
2009 TBD TBD
2008 TBD TBD
Long-term/Annual Output

Measure: Number of Laboratory Response Network member laboratories able to use their current Laboratory Information Management System (LIMS) for LRN-specific electronic data exchange.


Explanation:This measure reflects CDC's efforts in working with LRN member laboratories to migrate away from their current use of the LRN Results Messenger to their own LIMS to exchange LRN-specific results. Transitioning from the LRN Results Messenger to LIMS will improve the speed and accuracy of results messaging to CDC, thus decreasing the time required to initiate a public health response. The targets proposed for the revised measure are considered ambitious in light of the availability of resources and competiting priorities. Many LRN labs either do not have, or are currently implementing, a LIMS. Approximately 30 percent of LRN labs now receive direct funding through the Public Health Emergency Preparedness Cooperative Agreement, enabling them to purchase and maintain a LIMS. Another 30 percent may receive funds indirectly, but these funds are limited and are often spent on other priority items. Of the LRN labs that currently have functional LIMS in place, many are still working to develop this capacity for their primary lab functions and later add LRN-Specific data exchange fundtionality to their LIMS for biological and chemical terrorism reporting. The use of a lab's LIMS to electronically exchange LRN-specific data is CDC's ultimate goal.

Year Target Actual
2007 baseline 0
2008 3 12/2008
2009 7 12/2009
2010 15 12/2010
Annual Efficiency

Measure: Decrease annual costs for personnel and materials development with the development and continuous improvement of the budget and performance integration information system tools.


Explanation:As information technology improves, the need for staff time and resources will accordingly decrease. Benefits of improved systems technology are expected to be realized over the next three years.

Year Target Actual
2010 $0/BPI & HI 12/2010
2009 $0/BPI & HI 12/2009
2008 $0/BPI & HI 12/2008
2007 $50,000/BPI & HI $8,685.20/BPI & HI
2006 N/A $86,800/BPI & HI
2005 N/A $101,000/BPI system
2004 Baseline $125,000/Excel

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: CDC's biosurveillance program includes three major program activities, BioSense, Quarantine Stations, and the Laboratory Response Network Results Messenger. The purpose of the biosurveillance program is to improve the Federal government's capability to rapidly identify and characterize a potential bioterrorist attack or other public health emergency. Taken together, these three biosurveillance program activities ultimately provide ongoing collection of data that is critical to develop and implement control measures for many types of public health threats. The BioSense program receives, analyzes, and evaluates health data from numerous data sources such as emergency rooms, ambulatory care clinics, pharmacies, poison control centers, and clinical laboratories. The purpose of the BioSense program is to improve the nation's capabilities for real-time biosurveillance and situational awareness; provide immediate, constant, and comparable information needed to inform local, state, and national public health and to support national preparedness; and chart incoming health data about current patient symptoms from hospitals nationwide to identify spikes of activity that might signal a disease cluster outbreak or bioterrorism attack. The Quarantine Stations monitor public health at ports of entry. The purpose of the Quarantine Stations is to prevent the introduction, transmission, and spread of communicable diseases from foreign countries into the United States. The Laboratory Response Network Results Messenger supports early event detection and case confirmation by reporting results of diagnostic tests related to disease surveillance and research. In supporting the Laboratory Response Network, Results Messenger focuses on the Category A and B bioagent threats and nearly 150 chemical toxins and radiation. The Results Messenger enables rapid identification of these agents and threats and ensures timely receipt of electronic test reports by the laboratory to support a quicker return of results.

Evidence: ?? PUBLIC LAW 107-188, Public Health Security and Bioterrorism Preparedness and Response Act of 2002. ?? BioSense Frequently Asked Questions: http://www.cdc.gov/phin/component-initiatives/biosense/FAQ_BioSense_App.pdf ?? LRN Results Messenger Overview: http://www.cdc.gov/phin/software-solutions/LRN_Brochure_v3.ppt ?? LRN Fact Sheet: http://www.bt.cdc.gov/lrn/factsheet.asp ?? Quarantine Mission: http://www.cdc.gov/ncidod/dq/mission.htm ?? Quarantine Laws and Regulations: http://www.cdc.gov/ncidod/dq/lawsand.htm ?? Quarantine Fact Sheet: http://www.cdc.gov/ncidod/dq/sars_facts/quarantineqa.pdf ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx

YES 20%
1.2

Does the program address a specific and existing problem, interest, or need?

Explanation: Current national systems to identify public health trends are antiquated and slow, increasing the likelihood of illness and death in the event of a bioterrorism attack or naturally occurring infectious disease epidemic. Recognizing this weakness in the public health infrastructure, Congress provided funds to improve electronic biosurveillance, quarantine capabilities, and laboratory connectivity. The three program activities of the CDC Biosurveillance program combine to address these needs. BioSense addresses the lack of common data standards and the lack of connectivity between health information systems and public health systems. When fully deployed, BioSense will provide rich and timely data for early event detection and situational awareness. Expansion of quarantine stations to cover a greater proportion of entrants to the United States improves the capacity to prevent the introduction of infectious agents from foreign countries. The Laboratory Response Network Results Messenger is designed to address the identified need for a system that can manage the reporting, aggregation, and analysis of laboratory test results from over 100 labs conducting millions of tests.

Evidence: ?? Public Health Security And Bioterrorism Preparedness And Response Act Of 2002: http://www.eh.doe.gov/biosafety/library/PL107-188.pdf ?? CDC. Framework for evaluating public health surveillance systems for early detection of outbreaks. MMWR. May 7, 2004. http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5305a1.htm ?? GAO report GAO-05-308. June 2005: http://www.gao.gov/new.items/d05308.pdf ?? GAO report GAO-04-877. September 2004: http://www.gao.gov/new.items/d04877.pdf ?? GAO report GAO-03-769. May 2003. http://www.gao.gov/new.items/d03769t.pdf ?? IOM interim letter regarding expansion of the quarantine program: http://darwin.nap.edu/html/hr-usports/NI000579.pdf ?? FirstWatch: http://www.stoutsolutions.com/mediacontent/FirstWatchUpClose_1314.pdf ?? Trust for America's Health. Ready or not? Protecting the Public's Health from Diseases, Disasters and Bioterrorism. http://www.rwjf.org/files/newsroom/TFAHBioTerrorReport.pdf (full report) http://www.rwjf.org/files/newsroom/TFAHBioTerrorSummary.pdf (summary) ?? USC42.264. Quarantine: http://www.cdc.gov/ncidod/dq/42USC264.htm ?? CDC's Division of Global Migration and Quarantine website: http://www.cdc.gov/ncidod/dq/index.htm ?? LRN Results Messenger: http://www.cdc.gov/phin/software-solutions/257,2,Slide2 ?? PHIN Preparedness: http://www.cdc.gov/phin/preparedness/CLS_RSv1.0.pdf ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx

YES 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: CDC's biosurveillance program has two unique dimensions: it seeks to monitor timely public health information and does so at the national level. For example, while all states, localities and hospitals collect information pertinent to public health, a national system is required to identify multi-state outbreaks and to monitor public health issues of national scope or of importance to identifying and quickly intervening on threats to homeland security. CDC's portfolio of biosurveillance activities brings this public health data to a national level. Additionally, while other federal agencies also perform surveillance activities to monitor for acts of terrorism, CDC's program is the only one that uses human health data, rather than environmental indicators or animal health, to monitor possible terrorist attacks or other health emergencies.

Evidence: ?? Department of Homeland Security website. http://www.dhs.gov/dhspublic/ ?? CDC. Overview, Policy, and Systems, Federal Role in Early Detection Preparedness Systems. MMWR August 26, 2005. http://www.cdc.gov/mmwr/preview/mmwrhtml/su5401a3.htm ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx

YES 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The biosurveillance program is designed to achieve the goals of three major components of public health monitoring: rapid identification of trends in clinical data (BioSense), prevention of the introduction of infectious diseases from foreign countries (Quarantine Stations), and rapid identification of trends in terrorism-related laboratory data (Laboratory Response Network Results Messenger). Each of these program activities is designed around established public health, information technology, legal, or laboratory science parameters. In addition, each is managed according to established public health and scientific principles. There is no evidence that suggests a different program design would be more effective.

Evidence: ?? Gartner Group Contract and final report ?? Integrated Baseline Review ?? Federal Register. Proposed Rules for Quarantine/Control of Communicable Diseases. November 30, 2005. http://www.cdc.gov/ncidod/dq/nprm/docs/42CFR70_71.pdf. page 71893.

YES 20%
1.5

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: CDC's biosurveillance program is designed so that each of the three program activities ensures that resources are used directly and effectively to meet the program's purpose. BioSense resources are used to develop and implement a surveillance network to connect laboratories, hospitals and other health care facilities to public health to aid in surveillance and investigations. The network is built through recruitment of new hospitals and other data sources, implementation of data feeds from hospitals, maintenance of the data warehouse, development of tools for visualization and interpretation of data, and assurance of data quality and analysis. Quarantine resources are used directly to establish, maintain, and improve Federal quarantine stations at major points of entry throughout the U.S. Resources are used to build and improve physical field stations, hire and train quarantine public health officers, conduct inspections, and provide advanced emergency response capabilities such as isolation and quarantine. Laboratory Response Network Results Messenger resources are used directly to build the information technology architecture to support two-way secure data exchange between laboratories in the network. Further, CDC's biosurveillance program is structured so that resources and outcomes will adequately reach intended beneficiaries. Ultimately, the beneficiary of each of the program activities is the American public. All three program activities are implemented nationwide. BioSense already incorporates data from public and private hospitals in 10 U.S. cities, emergency departments and outpatient clinics from Veterans Affairs and the Department of Defense nationwide, and the largest national commercial laboratory. Eighteen quarantine stations are established, 16 in major airports and two at major land/border points of entry to the United States and cover all regions of the country. And, 90% of the US public lives within 100 miles of a member laboratory of the Laboratory Response Network, thus assuring rapid transportation of samples to a CDC-certified facility. While the American public is the ultimate beneficiary, more immediately, these activities directly provide upgraded capacity for CDC and its state and local public health and health care partners to identify and respond to urgent public health threats. Thus, resources for development of biosurveillance infrastructure also affect the development, maintenance and strengthening of the right relationships to assure the achievement of program goals. The direct establishment of BioSense data connections with health care delivery partners (e.g., hospitals) and the concomitant development of data sharing agreements that allow for data use by state and local health departments assure the availability of this important data to all levels of public health. Federal quarantine stations establish facilities, staffing and relationships within state and local jurisdictions, requiring the concomitant development of relationships among Federal, state and local officials, and between government and the public. The Laboratory Response Network Results Messenger enhances relationships among public health laboratories that are members of the Laboratory Response Network throughout the U.S. through the development of enhanced communication opportunities. As these programs continue to be implemented and evaluated against established performance measures, the structure and scope of implementation of the programs, identification of appropriate beneficiaries to be targeted, and the public reach required to achieve their objectives will continue to be refined.

Evidence: ?? Constella Contract, Project Plan and Recruitment Plan ?? SAIC Contracts, Project Plans and Implementation Plan ?? Northrop Grumman Contract and Project Plan ?? NCPHI Division of Emergency Preparedness and Response (DEPR) budget ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. Pages 37-40. http://www.iom.edu/CMS/3783/22845/29602.aspx ?? American Health Information Community (AHIC) Biosurveillance Work Group Notes ?? Policy Statement on Data Messaging of Testing Results for Biological Threat Agents by Members of the LRN to the CDC ?? LRN Result Messenger Business Rules for Biological Warning and Incident Characterization (BWIC) Pilot Program

YES 20%
Section 1 - Program Purpose & Design Score 100%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The CDC biosurveillance program aligns with CDC's overarching preparedness goal: People in all communities will be protected from infectious, occupational, environmental and terrorist threats. Subordinate to this strategic goal, CDC has developed nine specific goals in the areas of prevention, detection and reporting, investigation, control, and improvement related to emergency preparedness and response. Four of these goals are directly influenced by CDC's biosurveillance capacity: ?? Decrease the time needed to classify health events as terrorism or naturally occurring in partnership with other agencies. ?? Decrease the time needed to detect and report chemical, biological, radiological agents in tissue, food or environmental samples that cause threats to the public's health. ?? Decrease the time to identify causes, risk factors, and appropriate interventions for those affected by threats to the public's health. ?? Decrease the time needed to provide countermeasures and health guidance to those affected by threats to the public's health. Thus, the long-term performance measure for CDC capacity in the area of biosurveillance must reflect these outcome goals and the purpose of the biosurveillance program: By 2010, CDC's biosurveillance system will reduce the time needed from a triggering surveillance event to initiate event-specific standard operating procedures for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. When accomplished, the time reductions stipulated by this performance measure will directly affect the ability of public health and law enforcement personnel to decrease the time to classify health issues as terrorism or naturally occurring, to decrease the time needed to detect aberrations. Indirectly, these time savings will decrease the time needed to communicate with the public about important health issues, and to identify and provide countermeasures

Evidence: ?? CDC Health Protection goals and Preparedness Goals http://www.cdc.gov/about/goals/default.htm

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: : Achieving this long-term performance objective will require substantial improvements to information technology infrastructure, workforce recruitment and training, and availability of comprehensive and tested standard operating procedures in each of the three program activities that make up the biosurveillance program. Because the three program activities collect, analyze and report on different types of data, have different triggering events, will result in the initiation of different types of standard operating procedures, and have variable baselines, targets are specified for each program. By 2010, the BioSense program will reduce the time needed from a triggering biosurveillance event (the identification of a potential disease event or public health emergency event) to initiate event-specific standard operating procedures (the initiation of a public health investigation and, if needed, subsequent public health intervention) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Baseline and targets to be determined by June 2007. By 2010, the Quarantine Stations will reduce the time needed from a triggering biosurveillance event (notification of an international or interstate traveler who traveled while infectious with a quarantinable disease or other infectious disease of public health importance) to initiate event-specific standard operating procedures (e.g., isolation, quarantine, contact notification) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Baseline and targets to be determined by June 2007. By 2010, the Laboratory Response Network Results Messenger will reduce the time needed from a triggering biosurveillance event (i.e., transmission of data regarding the identification of any Category A or B agent) to initiate event-specific standard operating procedures (e.g., aggregation of data at a national level) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Baseline and targets to be determined by June 2007.

Evidence:

NO 0%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: : Each program activity of CDC's biosurveillance program will demonstrate progress toward the long-term goals by showing progressive improvements on the same measure each year. Because decreasing the time required to accomplish these tasks directly impacts CDC's other major preparedness goals, measuring performance on this objective should begin immediately and be monitored over time. Baseline and targets to be determined by June 2007. Annually, CDC's biosurveillance system will reduce the time needed from a triggering biosurveillance event to initiate event-specific standard operating procedures for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance.

Evidence: see measures

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: : Annual improvements in information technology infrastructure, workforce recruitment and training, and the development of comprehensive and tested standard operating procedures in each of the three program activities that make up the biosurveillance program will be required to achieve these annual targets. As for the long-term measure, annual measures are specified for each of the biosurveillance program activities. Annually, the BioSense program will reduce the time needed from a triggering biosurveillance event (the identification of a potential disease event or public health emergency event) to initiate event-specific standard operating procedures (the initiation of a public health investigation and, if needed, subsequent public health intervention) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Number of hours from triggering event to initiation of SOP 2006 Baseline 2007 2008 2009 2010 To be determined Annually, the Quarantine Stations will reduce the time needed from a triggering surveillance event (notification of an international or interstate traveler who traveled while infectious with a quarantinable disease or other infectious disease of public health importance) to initiate event-specific standard operating procedures (e.g., isolation, quarantine, contact notification) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Number of hours from triggering event to initiation of SOP 2006 Baseline 2007 2008 2009 2010 To be determined Annually, the Laboratory Response Network Results Messenger will reduce the time needed from a triggering biosurveillance event (i.e., transmission of data regarding the identification of any Category A or B agent) to initiate event-specific standard operating procedures (e.g., aggregation of data at a national level) for all infectious, occupational or environmental (whether man-made or naturally occurring) threats of national importance. Number of hours from triggering event to initiation of SOP 2006 Baseline 2007 2008 2009 2010 To be determined

Evidence:

NO 0%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All partners commit to and work toward the annual and long-term goals of the program through the COTPER Performance Measurement and Evaluation process. All three program activities in the biosurveillance program are funded through a robust competitive process and are required to establish and monitor performance standards as a condition of funding. Furthermore, continuing funding is predicated on proven past performance against these measures. Each program activity (i.e., BioSense, Quarantine, and Laboratory Response Network Results Messenger) has as one of its performance standards the relevant annual measure reported above. In addition, data from these programs is collected to contribute to the long-term performance measure. In addition, biosurveillance program activities establish commitments to annual and long-term goals from their respective public and private, state and local partners. For example, as part of their Bioterrorism preparedness goals many state health departments commit to assisting CDC with connecting hospitals in their state to the BioSense system. In addition, quarantine stations establish written agreements with local airport authorities, public health departments, in addition to memoranda of agreement with tier-1 hospitals to provide isolation and treatment for referred ill passengers.

Evidence: ?? PMET documentation ?? FY2006 Spend Plan Guidance ?? Constella Contract, Project Plan and Recruitment Plan ?? SAIC Contracts, Project Plans and Implementation Plan ?? Northrop Grumman Contract and Project Plan ?? State BT Cooperative Agreement Performance Measures ?? LRN Partners: http://www.bt.cdc.gov/lrn/partners.asp ?? LRN website/overview: http://www.bt.cdc.gov/lrn/ ?? National Center for Public Health Informatics Support of LRN Informatics: Memorandum of Agreement with Bioterrorism Preparedness and Response Program ?? Memorandum of Understanding between LRN Results Messenger and BWLIMS ?? Hospital MOA template for quarantine stations ?? Template for Quarantine Station written agreement with local airport authorities

YES 12%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: : Each program in the biosurveillance program is or has been subjected to high-quality, comprehensive, and independent evaluation. CDC is currently conducting a multi-faceted independent evaluation of the BioSense program. First, state and local public health and hospital end users of the system provide input and feedback on the BioSense program and application on an ongoing basis through a number of communication forums. Second, an independent evaluation is currently ongoing by an experienced information technology contractor to ensure the chosen architecture and implementation approach is in alignment with industry best practices. Third, approximately 25 nationally recognized experts in informatics and biosurveillance will provide individual input to CDC in a forum on June 27-28, 2006, focusing on the science of the BioSense system including appropriate algorithms, analysis and visualization techniques, and data streams of interest. In 2005, the Institute of Medicine provided a comprehensive evaluation of the nation's quarantine system and CDC's plans to expand the system, using a multitude of methods. The IOM committee gathered information between October 2004 and June 2005 from journal articles, reports, and news articles; presentations and commentary by constituencies relevant to the study; facts provided by CDC at the committee's request; visits by select committee and staff members to five quarantine stations; congressional testimony; and commissioned papers. The recommendations provided by the IOM are currently being used to strategically develop and upgrade the CDC quarantine program. Finally, the Department of Homeland Security has contracted with LMI Government Consulting to evaluate current BioWatch information technology architecture, including the Laboratory Response Network Results Messenger, and to make recommendations to improve overall efficiencies. To date the results of the evaluation have not been published. In addition, once deployed, the Results Messenger will be regularly evaluated in its entirety by the Association of Public Health Laboratories.

Evidence: ?? Gartner Group statement of work and contract ?? Public Health/Hospital Users Meeting Plan ?? Public Health/Hospital Users Meeting final report ?? Science Group Meeting Plan ?? Science Group final report ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx

YES 12%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Internally, COTPER is leading CDC in the development and use of a performance budget. Since 2004, COTPER has tied funding of program activities in all of its budget lines, including the Biosurveillance Program, to performance. To do this, it led the creation of HealthImpact.net and the IRIS Budget and Performance Integration (B&PI) module that transparently link program data and budget information. Today, COTPER continues to lead the agency by using these tools to conduct annual project submission and subject matter expert review panels to identify strategic projects, evaluate past performance, and conduct standards-based scientific reviews to determine budget execution. Past performance is based on triannual reporting of project performance against established milestones and performance measures. Currently, though, the agency is in the process of identifying long term and annual targets to assign to its performance measures. Once these targets are identified, the agency will be able to link its budget requests to the performance of the programs. Additionally, Congressional Justifications do not link funding levels with performance for this program.

Evidence: ?? PMET and Spend plan documentation Congressional Justifications

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: CDC has developed four goal areas and twenty-four strategic goals, which will drive the formation of measurable agency strategic and core objectives. These objectives will drive budget requests for agency programs that will accomplish them. Twenty-one goal team leaders have been selected to develop the goal action plans that will identify the agency's strategic priorities for accelerating health impact. Goal action plans will be the foundation of the agency's strategic plan and will drive the strategic plans for its national centers. The goal action plans will include 1) an assessment of current agency activities, funding and performance measures, 2) a review of the burden of disease and the known effective interventions, 3) analytic work on the most effective areas for research and intervention, 4) a gap analysis, and 5) the strategic objectives and actions that have the greatest potential for accelerating health impact. The goal action plans are being developed in collaboration with CDC's centers and divisions and with CDC's partners and stakeholders. Upon completion, these plans will be reviewed by CDC's Executive Leadership Board, Center Director Council, and Division Director Council and the Annual National Partners Meeting. Once they have been approved, these strategic plans will guide the annual implementation plans and budgets of all CDC organizational units. COTPER has embraced the goal action planning process. An overarching preparedness goal action plan is under development, as are six scenario-specific action plans representing health issues of particular interest to the agency. COTPER will use these action plans to drive budget formulation, spend plan priorities, its overall evaluation plan (incorporating existing and planned PART and GPRA measures), and its overall communications plan. Included in these preparedness goal action plans will be specific "PART-ready" long-term performance measures to which every program funded through the CDC's Terrorism Preparedness and Emergency Response allocation will be required to contribute.

Evidence: ?? CDC Health Protection goals and Preparedness Goals http://www.cdc.gov/about/goals/default.htm ?? Roles of Strategic Innovations Officers and Scenario Team Leaders http://www.cdc.gov/about/goals/goals.pdf

YES 12%
Section 2 - Strategic Planning Score 62%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Performance of the three program activities supported by the biosurveillance program is monitored by the COTPER Performance Measurement and Evaluation Team using the performance budget tools, HealthImpact.net and IRIS Budget and Performance Integration module. Funding of projects is predicated on the establishment and ability to monitor and report on high-quality, relevant short (i.e., milestones) and long-term (i.e., performance) measures. Once funded, program activities are required to report data to support these measures on a tri-annual basis. Project performance is summarized for COTPER and other CDC leadership at the end of each performance period. In addition, performance is summarized at the end of fiscal year in an annual report. Programs are provided feedback and technical assistance regarding their performance to manage and improve the programs. In addition, past performance is used as a major criterion in determining continuation funding for projects. In addition to managing individual projects, performance data is used to establish and determine overall accomplishment of objectives for the entire program (e.g., Biosurveillance) and for COTPER overall (e.g., progress toward the nine preparedness goals). This information contributes to the development, implementation, and monitoring of the preparedness goal action plan, budget formulation, and annual budget execution priorities. Currently, though, the agency is in the process of identifying long term and annual targets to assign to its performance measures. Once these targets are identified and baseline information is available, CDC will be able to be able to use the information it collects to improve the program performance toward achieving its performance targets.

Evidence: ?? http://healthimpactnet.cdc.gov/ (internal website) ?? PMET and spend plan documentation

YES 14%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Within the performance management tool, HealthImpact.net, each funded program activity within the Biosurveillance program has an identified program manager. In addition, all program partners, including grantees and contractors, are identified. All Federal managers are required to have a clearly defined and quantifiable performance standard that cascades from the Agency Director and is related to the CDC strategic goals. BioSense, Quarantine and Laboratory Response Network Results Messenger program managers are held responsible for cost and scheduled outcomes and performance results. Performance ratings for management officials are directly linked to organizational and leadership performance. In a cascading fashion, the performance plans of all employees in these three programs are aligned with the plans of management officials and work collectively to achieve the specific program objectives. In addition, BioSense and Laboratory Response Network Results Messenger contractors provide monthly invoice tracking. Management accountability for schedule performance results is reviewed weekly with the BioSense managers.

Evidence: PMET

YES 14%
3.3

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: COTPER allocates resources according to its annual spend plan process. Funds are obligated immediately when made available to COTPER to projects that have been previously reviewed and selected via a standards-based system. Limited amounts of unobligated funds remain with COTPER at the end of the fiscal year. CDC's Financial Management Office (FMO) ensures that appropriated funds are properly obligated in a timely manner and that mechanisms are in place to ensure that funds are spent for the purpose for which they are intended. This is demonstrated by efforts in the areas of budget execution consolidation, spending plan execution, cancelled year appropriations, and erroneous payments.

Evidence: standard operating procedures of the budget execution branch Spending plans developed at the program level (used to certify and monitor the status of funds at the program and agency level) Risk assessments were completed to determine whether they were susceptible to improper payments exceeding $10 million and a 2.5 percent error rate and required to estimate improper payments under the Improper Payments Information Act of 2002 (IPIA) and the related OMB Guidance.

YES 14%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: CDC continues to examine agency operations to identify areas where efficiencies may be realized. In addition to competitive sourcing studies to meet the requirements of the President's Management Agenda, CDC has reviewed and reorganized its organizational and reporting structure. CDC has completed several competitive sourcing studies over the past three years, covering such services as Animal Care, Facilities Planning and Management, Library Services, Statistical Support, and Writer/Editor functions. CDC has won 13 of the 14 studies completed. Savings realized from competitive sourcing are reinvested in mission-direct public health activities. FY 2006 efforts to increase efficiencies include two restructuring initiatives, covering administrative and business service functions. Goals of restructuring these functions include alignment with and support of CDC's new organizational structure, as well as targeting greater efficiencies through process improvements and standardization across the agency. The FY 2005 budget execution consolidation within the Financial Management Office will be evaluated for process reengineering, service improvement, and cost savings. Also, the Public Health Integrated Business Services (PHIBS) initiative will result in restructuring of business functions currently carried out in CDC's national centers and coordinating centers/offices. The results of these two initiatives will inform restructuring proposals as alternatives to competitive sourcing competitions. CDC has completed delayering the agency to no more than four management layers. With the elimination of over 200 "sections", a 33% decrease in the official number of organizational units since 2001 has been achieved. Additionally, CDC's supervisory ratio has more than doubled from 1:5.5 in 2002 to over 1:12.6 in January 2006. COTPER has been at the forefront of development of two information technology tools (i.e., HealthImpact.net and IRIS B&PI) for budget and performance integration. These tools are now widely used by a variety of staff and for a variety of purposes, gaining efficiencies in the consolidation of information systems, and reducing the time required to find, collate, and use data. A representative measure of the efficiency gained by this system is the continuous reduction of costs to conduct COTPER's annual spend plan execution process. With all staff using the system, the reduced time required to collect and use data for decision making translates directly into dollars saved. Thus, COTPER can report on the following efficiency measure for all of its budget lines, including the biosurveillance program: Decrease annual costs for personnel and materials development with the development and continuous improvement to the budget and performance integration information system tools.

Evidence: ?? FY03 - FY05 Competitive Sourcing Studies ?? HHS Restructuring & Hiring Memo ?? CDC Timeline for Restructuring and Delayering ?? CDC Delayering Results ?? FY 2006 Restructuring Initiatives ?? Public Health Integrated Business Services ?? The web-based application for Agency-wide Performance Planning for the upcoming fiscal year: http://healthimpactnet.cdc.gov/ ?? HHS' Office of the National Coordinator for Health Information Technology (ONCHIT): http://www.hhs.gov/healthit/ ?? Table outlines HHS IT Initiatives: http://www.hhs.gov/healthit/federalprojectlist.html#intitiativestable (link may prove informative for Sections II & IV--PHIN) ?? Executive Order of the President, 13335 (04-27-2004): http://www.whitehouse.gov/news/releases/2004/04/20040427-4.html ?? Consolidated Health Informatics Initiative (e-gov initiative of the President's Management Agenda: http://reform.house.gov/UploadedFiles/Evans.pdf ?? Mar. 15 2006 GAO report, GAO-06-346T http://www.gao.gov/new.items/d06346t.pdf ?? Public Health Information Network: http://www.cdc.gov/phin/ ?? Standardized data: http://publishing.cdc.gov/wp/drl/objectId/0b00f3eb800b941c ?? BioSense Data Volume Reports

YES 14%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: : Each program activity within CDC's Biosurveillance program collaborates and coordinates with relevant public and private partners. BioSense Federal partners are largely located at the Department of Homeland Security. CDC has significantly enhanced vital quarantine stations at key points of entry which provide first line defense to detect and evaluate potentially infectious diseases arriving in the U.S. For example, quarantine stations work in concert with State and local health departments and other federal partners to mitigate community outbreaks due to highly contagious diseases and to prepare for pandemics using 21st century approaches to traditional non-pharmaceutical interventions such as voluntary isolation and quarantine, social distancing, and infection control strategies. These partnerships are essential to preventing importation and interstate spread of communicable diseases through our ports of entry, and effecting a coordinated and effective response emerging disease threats. The Laboratory Response Networks Results Messenger is closely coordinated with federal, state and local laboratory programs. Federal collaborations occur with subject specific labs within the CDC, Department of Homeland Security, Food and Drug Administration, United States Department of Agriculture, and Department of Defense. State and local collaborations are with public health laboratories and the Association of Public Health Laboratories.

Evidence: ?? Data sharing agreement with the American Associate of Poison Control Centers ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx ?? National Center for Public Health Informatics Support of LRN Informatics: Memorandum of Agreement with Bioterrorism Preparedness and Response Program ?? Memorandum of Understanding between LRN Results Messenger and BWLIMS

YES 14%
3.6

Does the program use strong financial management practices?

Explanation: In April 2005, CDC implemented a new Unified Financial Management System (UFMS). UFMS is an integrated, Department-wide financial system that consistently produces relevant, reliable, and timely financial information to support decision-making and cost-effective business operations at all levels throughout the Department. UFMS replaced the legacy mainframe-based financial system, which was over 15 years old. UFMS provides the program with more real-time financial data, improved financial reports that allow managers to make timely decisions, and streamlined financial processes. UFMS will help the Department continue to achieve unqualified audit opinions. The HHS FY 2005 Performance and Accountability Report noted a material weakness related to the transition to UFMS, but full implementation will eliminate this material weakness.

Evidence: HHS Performance and Accountability Report CDC has implemented reviews, reconciliations, fluctuation analysis, and checks to ensure the accuracy and completeness of the financial statements. CDC has also streamlined the statement preparation outside the central financial system by using automated tools to expedite consolidating financial data.

YES 14%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: All COTPER programs are reviewed quarterly and annually through program reviews with COTPER leadership to assure that priority activities are clearly stated, specified and monitored. Management of the budget execution and performance monitoring for the biosurveillance program is subjected to this review as are all other budget lines within the Terrorism Preparedness and Emergency Response allocation. Annual activities, such as conducting PART reviews, the budget execution process, and the budget formulation process are reviewed annually by senior COTPER leadership to assure sound, scientifically-based approaches are being used. In addition, performance of all funded programs from the TPER allocation is monitored through the use of HealthImpact.net. The biosurveillance program activities also have established management improvement systems. The BioSense contractors Statements of Work have a mechanism for addressing management deficiencies. Each contract requires quality assurance plans and reviews to identify process deficiencies and insure continuous improvement. The Quarantine Program has identified management deficiencies and is working with an independent consultant to develop and implement strategic personnel, operational, science and program imperatives by June 2006. The Laboratory Response Network Results Messenger utilizes the President's Management Agenda review process established within the National Center for Public Health Informatics to identify management deficiencies. Deficiencies are addressed utilizing mechanism established in the Statement of Work for each related contract. In this manner strategic personnel, operational and program corrections can be rectified within a mutually agreeable timeframe.

Evidence: ?? FY2007 Spend Plan review process documentation

YES 14%
Section 3 - Program Management Score 100%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Each biosurveillance program activity is working on the development of ambitious targets and timeframes for its long term outcome goals. Specific baselines will be established within the next 12 months. However, each program has developed measurement systems to monitor achievement of milestones. BioSense program officials have determined that, by 2010, all levels of public health with jurisdiction over the top 50 U.S. metropolitan areas will use BioSense for biosurveillance and local health situational awareness as needed by accessing timely (<24 hours old) healthcare data from a statistically representative population. To achieve this, BioSense must: ?? Connect 3500 hospitals sending clinical data in real-time ?? Connect 3 of the top 5 emergency departments by patient volume in the each of the top 50 U.S. metropolitan areas ?? Obtain lab orders and results from 3 major commercial laboratories ?? Incorporate clinical data from VA facilities to include 150 hospitals and 1000 ambulatory care clinics ?? Incorporate clinical data from DoD sites, expected to include all 800 clinics and 45 hospitals by 2011 ?? Include problems reported to the American Association of Poison Control Centers (AAPCC), a nation-wide organization with all 62 centers which maintains data related to poison exposures and other health complaints The Quarantine Stations are making progress toward reducing the time from notification of ill passengers to appropriate action. The Quarantine Stations represent human surveillance for sentinel health events among the traveling public. Thus, the major requirement for this program to accomplish the long term objective is to have fully staffed stations at the highest volume points of entry to increase the likelihood of immediate identification of persons entering the country with health problems that pose a threat to the U.S. population. This requires creative sourcing and hiring, maintenance of support by Congress and state and local partners, implementation and testing of training programs for staff, and realization of efficiencies. Achieving the long-term objective for the Laboratory Response Network Results Messenger reflects the need to develop, maintain, and expand information technology solutions for rapid exchange of laboratory results to continuously reduce the amount of time required for critical results to be identified and reported. This requires continuous program improvement, establishment of new partnerships and data sharing agreements, information technology improvements, and realization of efficiencies. Currently, the best marker for network laboratories to indicate their readiness to improve timely reporting for action is their ability to utilize their current Laboratory Information Management System for electronic data exchange to 136 (the current number of member laboratories).

Evidence: AAPCC Data Sharing Agreement BioSense Hospitals Data Sharing Agreements BioSense Target City Healthcare Organization Analysis Data Provisioning Implementation Subcontracts (with the data source) BioSense User Logs (demonstrating active use) BioSense Data Volume Reports (demonstrating connected organizations) LRN Results Messenger BioWatch attachment Quarantine: http://www.cdc.gov/ncidod/dq/sars_facts/isolationquarantine.pdf

NO 0%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: Although the program has established long-term and annual measures, data collection to establish baselines is still underway. Once baselines and targets are established, the program can begin reporting on actual data to demonstrate achievement of annual targets

Evidence:

NO 0%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: COTPER has been at the forefront of development of two information technology tools (i.e., HealthImpact.net and IRIS B&PI) for budget and performance integration. These tools are now widely used by a variety of staff and for a variety of purposes, gaining efficiencies in the consolidation of information systems, and reducing the time required to find, collate, and use data. A representative measure of the efficiency gained by this system is the continuous reduction of costs to conduct COTPER's annual budget execution process. With all staff using the system, the reduced time required to collect and use data for decision making translates directly into dollars saved. See measures section.

Evidence: ?? PMET data ?? BearingPoint contractor data

YES 25%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Programs with similar purposes and goals to that of the CDC biosurveillance program include BioWatch and state-based quarantine efforts. However, these programs are intertwined with BioSense, the Laboratory Response Network and the federal Quarantine Programs making comparisons impossible

Evidence:

NA 0%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations of the BioSense and Laboratory Response Network Results Messenger programs are currently underway. The 2005 comprehensive evaluation of the nation's quarantine system conducted by the Institute of Medicine found that the system as it existed in 2004 was effective at achieving its limited objectives given limited funding. The IOM made recommendations supporting the ongoing expansion to 25 stations and enhancement of the system based on current quarantine authorities provided to CDC.

Evidence: ?? PMET documentation ?? BioSense contracts ?? Institute of Medicine. Quarantine Stations at Ports of Entry: Protecting the Public's Health. National Academies Press 2005. http://www.iom.edu/CMS/3783/22845/29602.aspx

NO 0%
Section 4 - Program Results/Accountability Score 25%


Last updated: 09062008.2006SPR