Detailed Information on the
FBI Intelligence Assessment

Improving the measurement of the value that customers receive from FBI intelligence products.

Identifying realistic linkages between budget requests and program performance.

Developing systems for improved management and validation of human sources, the intelligence they produce, and the ability to identify source gaps in our domain.

Measure: Responsiveness: % of FBI headquarters finished intelligence reports that are responsive to National Intelligence Priority Framework topics (Internally disseminated)

Explanation:This measure illustrates the Intelligence Program's responsiveness to Intelligence Community collection requirements (i.e., whether or not the intelligence produced by the FBI is filling important, high priority collection needs). Because the FBI has some regional or local priorities to fulfill, there will always be some intelligence reports filed that are of interest to the Bureau and its law enforcement colleagues but are not responsive to national-level NIPF topics. Accordingly, the maximum attainable percentage for this measure is estimated at 90%.

Measure: Responsiveness: % of FBI field office finished intelligence reports that are responsive to National Intelligence Priority Framework topics. (Internally disseminated)

Explanation:This measure illustrates the Intelligence Program's responsiveness to Intelligence Community collection requirements (i.e., whether or not the intelligence produced by the FBI is filling important, high priority collection needs). Because the FBI has some regional or local priorities to fulfill, there will always be some intelligence reports filed that are of interest to the Bureau and its law enforcement colleagues but are not responsive to national-level NIPF topics. Accordingly, the maximum attainable percentage for this measure is estimated at 80% (less than the maximum for headquarters reports because Field Offices are more likely to be involved in regional or local events than the national headquarters office).

Measure: Responsiveness: % of FBI finished intelligence reports that are responsive to National Intelligence Priority Framework topics. (Disseminated to Intelligence Community)

Explanation:This measure illustrates the Intelligence Program's responsiveness to Intelligence Community collection requirements (i.e., whether or not the intelligence produced by the FBI is filling important, high priority collection needs). Because the FBI has some regional or local priorities to fulfill, there will always be some intelligence reports filed that are of interest to the Bureau and its law enforcement colleagues but are not responsive to national-level NIPF topics. Accordingly, the maximum attainable percentage for this measure is estimated at 95%.

Measure: Accuracy: Number of high priority source packages put through the validation process

Explanation:This measure demonstrates the Intelligence Program's commitment to ensuring that sources are validated for reliability and accuracy. This ensures that case agents and policy makers have access to the best possible intelligence before using that information to make decisions. The targets for this measure are classified because of their relation to FBI sources and methods information.

Measure: Customer Satisfaction: % of users who visit the Law Enforcement Online (LEO) service (which provides intelligence dissemination) more than one month out of each year.

Explanation:This measure serves as a proxy for customer satisfaction. Repeated use of LEO is a strong indication that customers (other intelligence agencies, state and local law enforcement, etc.) find the information they are obtaining on the site useful. Targets for this measure are increased at a conservative rate (2% per year) because of the many factors that influence this number. Note that FY 2005 actual performance is considered uncharacteristically high--usage numbers were artificially driven up by the occurrence of the London subway bombings and a domestic emergency response training exercise in 2005. Future targets were set without assuming any repeat event-based surges in user levels, but, if future performance continues to trend high, the targets will be adjusted at that time.

Measure: Efficiency: Staff time (in workyears) saved on asset management activities through changes in the human source business process (via the new "Delta' system).

Explanation:The FBI is preparing to implement a major change in its asset management business process by introducing a new system, the "Delta' system, to automate asset management tasks and activities. The new technology will provide a centralized database of human sources and allow new functions, such as text searches, geographical displays, and data reporting, that are currently done manually. This is expected to generate savings in staff time each year, particularly as the number of human sources continues to grow. The first savings are expected in FY 2007, after 'Delta' is launched.

Is the program purpose clear?

Explanation: The mission of the Intelligence Program is to optimally position the FBI to meet current and emerging national security and criminal threats. This is accomplished by aiming core investigative work proactively against threats to U.S. interests; building and sustaining enterprise-wide intelligence policies and capabilities; and providing useful, appropriate, and timely information and analysis to the national security, homeland security, and law enforcement communities. The FBI's Intelligence Decision Unit spans ten FBI divisions and encompasses $1.1 billion (FY 2006). The Directorate of Intelligence (DI) is responsible for managing all projects and activities that encompass the FBI's Intelligence Program and for prioritizing those functions through the formulation of budgetary requirements. The Directorate carries out its functions through embedded intelligence elements at FBI Headquarters (HQ) and in each field division.

Evidence: Sec. 2002 of the Intelligence Reform and Terrorism Prevention Act provided statutory codification of the Directorate of Intelligence. Other documents, including the FBI Intelligence Program Concept of Operations (CONOPS); Report to the President of the United States, June 30, 2005; and Electronic Communication (EC) Establishing the FBI Directorate of Intelligence, outline the mission and responsibilities of the FBI intelligence program.

Does the program address a specific and existing problem, interest, or need?

Explanation: The FBI Intelligence Program addresses the problem of criminal and terrorist activity aimed at the United States or its citizens, allies, and interests around the globe. By providing timely and accurate national intelligence, regardless of point of collection, the program counters current and emerging national security and criminal threats. The existence and relevance of these threats is well documented by the FBI, which conducts regular analyses of the number and variety of terrorist and criminal activities being planned or carried out against the U.S. These analyses are supported by similar efforts among other elements of the intelligence and law enforcement communities.

Evidence: Threats and priorities are described in the ODNI National Intelligence Priorities Framework, as well as FBI Drivers and Threats documents. In addition, numerous documents describing the establishment of the FBI Directorate of Intelligence also briefly touch on the broad threat environment and the need for the FBI Intelligence Program. This includes the Report to the President of the United States (June 30, 2005) and the FBI Intelligence Program Conops.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The FBI Intelligence Program was created by Congressional and Presidential mandate to provide centralized management of the nation's domestic intelligence efforts; no other federal, state or local program shares the FBI's specific authorities and responsibilities for domestic intelligence collection. With respect to broader intelligence collection and analysis authorities, including foreign intelligence and counterintelligence, Executive Order 12333 governs the division of responsibility between FBI and other Intelligence Community members in order to ensure coordination and prevent duplication of effort. Managers of the Intelligence Program also work extensively with external partners to ensure that the FBI's program is not duplicative of other initiatives, both public and private. In some instances, this involves the active co-location of groups so that activities and policies can be better coordinated. For example, many of the FBI's Field Intelligence Groups (FIGs), which manage the FBI's intelligence functions in each Field Office, include members of state and local law enforcement and other intelligence agencies. In 22 of the FBI's Field Offices, personnel assigned to the FIGs are members of regional Fusion Centers and work alongside members of state and local law enforcement and other intelligence community personnel. In addition, special inter-agency committees have been created to allow senior leaders to monitor and minimize any redundancy between programs. The FBI Director or other senior managers sit on the Justice Intelligence Coordinating Council (JICC), GLOBAL Intelligence Working Group, and the National Intelligence Analysis and Production Board (NIAPB), just to name a few.

Evidence: Executive Order 12333 is the primary document describing the division of responsibilities between members of the Intelligence Community. Coordination activities intended to prevent duplication or waste are covered by the Field Operations CONOPs, various Committee charters, and the Fusion Center Survey (May 10, 2006). The responsibility to share and coordinate with outside groups is outlined in the EC establishing the Directorate of Intelligence.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The FBI Intelligence Program came into existence as the Office of Intelligence (OI) shortly after September 11, 2001. Expansion and refinement of the OI concept were endorsed by FBI stakeholders in Congress and the 9/11 Commission, which offered additional suggestions for strengthening the program's design. In May 2004, Director Mueller responded to those suggestions by proposing the creation of the Directorate of Intelligence as a dedicated and integrated intelligence service within the FBI. This proposal was approved and codified in statute and presidential memorandum. As it currently exists, the Intelligence Program encompasses all intelligence-related functions in the FBI, including requirements, collection, analysis, and dissemination. Those functions are managed, prioritized and coordinated by the Directorate of Intelligence through a staff element at FBI Headquarters, embedded intelligence elements in each FBI division, and FIGs in each of the Bureau's 56 field offices. Because national security and criminal intelligence are public goods, the program is necessarily carried out as a direct federal program. There is no existing evidence that a different design would be more effective. To the contrary, past experience, particularly with respect to the terrorist attacks of September 11, 2001, indicates that previous designs, such as a more informal decentralized approach, were ineffective and potentially very harmful. The FBI will continue to assess and update the design of its Intelligence Program to ensure that it remains optimally designed and fully compliant with guidance from the FBI National Security Branch and the Office of the Director of National Intelligence.

Evidence: The 9/11 Commission Report gives an accounting of the problems that existed in previous FBI intelligence designs. The history of the current Directorate of Intelligence and its design is provided in the EC establishing the Directorate of Intelligence, the FBI Intelligence Program Conops, and the Report to the President of the United States (June 30, 2005). No evidence exists of alternate design that would be more effective or efficient.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The Intelligence Decision Unit (IDU), a discrete budget program category for intelligence activities, was designed to maximize the effectiveness of the resources of the intelligence program. The IDU, adopted in 2005, places intelligence resources under a single FBI budget category in order to avoid the redirection of resources to other activities, which had been a problem in the past. Financial resources within the IDU are managed according to a threat-based planning cycle. This orientation requires a comprehensive understanding of the current and near-future threat environment, which is accomplished through the production of Drivers and Threats reports every year. Understanding the threat environment helps to facilitate the identification of fiscal priorities and the need for reallocation of resources. By this method, DI managers can ensure that the program is targeting its resources toward the most critical needs, even if that means reducing resources directed toward less crucial activities. Personnel resources will be handled in much the same way. DI staff has developed a pilot resource allocation model to analyze the placement of intelligence analysts across FBI field divisions. The model helps to predict where analysts are most needed while ensuring that divisions with lower levels of activity are not overstaffed. The resource allocation model is updated annually; however, the system is flexible enough to allow for surge capabilities, should an urgent, temporary need be required in a field office or division. Due to the nature of the program and the structure of its activities, no other activities, agencies or initiatives are unintentionally subsidized with FBI funding.

Evidence: The 2005 Consolidated Appropriations Act set up the new budget structure for the Intelligence Decision Unit. The threat-based planning cycle is described in the Intelligence Program Conops, and the FBI Forecast: Drivers and Threats (December 2005, March 2005) provides a sample of recent threat assessments. The Resource Allocation Model is described by Toffler Associates in the "IA Allocation Pilot Overview" from April 2006.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: It is difficult to define outcomes for a program that produces intelligence. In some cases, good intelligence analysis will lead to a physical outcome, such as a terrorist attack that is averted or a foreign intelligence penetration that is avoided, but this is not always the case. Productive and useful analysis may merely serve to enhance the government's body of knowledge on a particular topic. While this is not an easily measurable concept, it is the program's major goal, and it precludes the use of traditional, event-based outcome measures similar to those used by law enforcement programs. As an alternative means of gauging the FBI's ability to contribute quality information to the knowledge base of the government, the Intelligence Program's long-term measures look at the program's ability to produce intelligence that is: 1) accurate; 2) useful; and 3) responsive to intelligence needs. Accuracy will be measured by looking at the number of high priority source packages that are put through the Intelligence Program's source validation process. This ensures that high priority sources are considered for accuracy and reliability before decisions are made based on information obtained from that source. A proxy for usefulness will look at the percent of users who visit the FBI's Law Enforcement Online (LEO) service more than one month out of each year. This assumes that customers returning to LEO are doing so because they find the intelligence information they obtain through the service to be valuable; a high percentage of repeat users will indicate a high percentage of customers who find the information useful. Responsiveness will be measured by looking at the percent of FBI headquarters and field office intelligence reports that are produced in fulfillment of National Intelligence Priority Framework requirements. The responsiveness measures will look at intelligence reports that are both internally disseminated (FBI only) and externally disseminated (throughout the Intelligence Community). Achieving a high percentage on a responsiveness measure will indicate that most of FBI's intelligence work is appropriately focused on nationally-identified areas of greatest need and interest.

Evidence: See measures, descriptions, and data that are provided in the Measures Section.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Intelligence Program has established baselines for its measures, as well as targets that constitute ambitious improvement over time. Targets for the accuracy measure, because they deal closely with FBI sources and methods, are classified and may be reviewed by individuals with the appropriate clearances and 'need to know.' The long term target for the usefulness measure is to increase the percentage of repeat visitors to LEO (an FBI intelligence dissemination service) to 46% by FY 2011. Although this target tracks closely with the program's FY 2005 performance level, the target is still believed to be ambitious because repeat usage of LEO was uncharacteristically high in 2005. This was most likely due to the occurrence of the London subway bombings and a domestic emergency response training exercise that drove up user numbers. The targets for this measure, therefore, were chosen conservatively (i.e., without assuming any such event-based surge of users) despite high FY 2005 performance. The 46% goal for FY 2011 represents a growth of about 2% each year; if future data shows performance continually trending higher, the targets will be adjusted. The responsiveness measures look at the percentage of intelligence reports that are responsive to national intelligence priorities. Because the FBI and its fellow law enforcement agencies have some responsibilities to monitor regional or local events in addition to national issues, there will always be some intelligence reports that are responsive to those regional or local needs rather than the national priorities; for this reason, the target level will never reach a full 100%. The targets are highest for those reports that will be externally disseminated to the Intelligence Community (95% by FY 2011) because they will have the greatest need for priority information of a national scope. The targets are lowest for internally disseminated field reports (80% by FY 2011), which will have the greatest amount of regional/local information to deal with. Internally disseminated headquarters reports have targets in the middle (90% by FY 2011). Targets were judged ambitious despite the slowing of improvement after FY 2005. The program expects its rate of progress to slow over time as easier improvements are made and more difficult ones remain to be tackled. FY 2006 is therefore a better baseline for future performance than FY 2005.

Evidence: See data provided in the Measures Section.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: It is difficult to define outcomes for a program that produces intelligence. In some cases, good intelligence analysis will lead to a physical outcome, such as a terrorist attack that is averted or a foreign intelligence penetration that is avoided, but this is not always the case. Productive and useful analysis may merely serve to enhance the government's body of knowledge on a particular topic. While this is not an easily measurable concept, it is the program's major goal, and it precludes the use of traditional, event-based outcome measures similar to those used by law enforcement programs. As an alternative means of gauging the FBI's ability to contribute quality information to the knowledge base of the government, the Intelligence Program's annual measures look at the program's ability to produce intelligence that is: 1) accurate; 2) useful; and 3) responsive to intelligence needs. These are the same measures that the program uses for its long term goals; tracking annual achievements against these measures will allow the program to demonstrate its progress toward its long term goals. Accuracy will be measured by looking at the number of high priority source packages that are put through the Intelligence Program's source validation process. This ensures that high priority sources are considered for accuracy and reliability before decisions are made based on information obtained from that source. A proxy for usefulness will look at the percent of users who visit the FBI's Law Enforcement Online (LEO) service more than one month out of each year. This assumes that customers returning to LEO are doing so because they find the intelligence information they obtain through the service to be valuable; a high percentage of repeat users will indicate a high percentage of customers who find the information useful. Responsiveness will be measured by looking at the percent of FBI headquarters and field office intelligence reports that are produced in fulfillment of National Intelligence Priority Framework requirements. The responsiveness measures will look at intelligence reports that are both internally disseminated (FBI only) and externally disseminated (throughout the Intelligence Community). Achieving a high percentage on a responsiveness measure will indicate that most of FBI's intelligence work is appropriately focused on nationally-identified areas of greatest need and interest. The Intelligence Program also has an efficiency measures with baseline and targets. This measure is discussed fully in question 3.4.

Evidence: See measures, descriptions and data provided in the Measures Section.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The Intelligence Program has established baselines for each of its annual measures, as well as targets that constitute ambitious improvement over time. Targets for the accuracy measure, because they deal closely with FBI sources and methods, are classified and may be reviewed by individuals with the appropriate clearances and 'need to know.' The annual targets for the usefulness measure are to increase the percentage of repeat visitors to LEO (an FBI intelligence dissemination service) by 2% each year, which is fairly ambitious given the many other factors that can affect LEO usage either positively or negatively. If future data shows performance continually trending higher, the targets will be adjusted at that time. The responsiveness measures look at the percentage of intelligence reports that are responsive to national intelligence priorities. Because the FBI and its fellow law enforcement agencies have some responsibilities to monitor regional or local events in addition to national priorities, there will always be some intelligence reports that are responsive to those regional or local needs rather than the national priorities; for this reason, the target level will never reach a full 100%. For internally disseminated intelligence reports, the annual targets are to increase responsiveness by about 2% each year until the estimated maximum attainable percentage is reached. Given the increasing number of intelligence products created and disseminated by the Intelligence Program each year, 2% is a reasonable yearly goal. Targets for externally disseminated intelligence reports increase at a faster rate, reflecting the need to maximize responsiveness in those reports that are shared with the Intelligence Community. Targets were judged ambitious despite the slowing of improvement after FY 2005. The program expects its rate of progress to slow over time as easier improvements are made and more difficult ones remain to be tackled. FY 2006 is therefore a better baseline for future performance than FY 2005.

Evidence: See targets provided in the Measures Section.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The FBI has a unique role within the Federal Government and the Intelligence Community as the agency responsible for the collection of domestic intelligence; this is codified in Executive Order 12333, among other places. This precludes the Intelligence Program from having traditional program "partners" working toward the same goals. The Intelligence Program does share information and coordinate with other federal, state and local organizations, and there are a number of governmental agencies whose efforts have enabled greater success in the intelligence mission. For instance, other agencies on FBI Joint Terrorism Task Forces routinely contribute information that becomes part of the FBI's raw intelligence dissemination process. Question 3.5 provides a further discussion of this type of collaboration and coordination with other agencies and organizations within and outside of the Intelligence Community.

Evidence: Not applicable

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Evaluations of FBI programs, including the Intelligence Program, are conducted by the Organizational Program Evaluation and Analysis Unit (OPEAU). While OPEAU is a component of the FBI, it is accepted as an independent entity because OPEAU's leadership is distinct from the leadership of the Intelligence Program and there are no lines of authority for Intelligence Program managers over OPEAU's activities. The Intelligence Program is being covered in a series of evaluations conducted by OPEAU over a 6 year period that started in FY 2004. Each evaluation is looking at a distinct portion of the overall program. For example, the Special Technologies and Application Section (STAS) was reviewed along with the Cyber Division; the Dedicated Technical Program will be reviewed along with the Counterintelligence Division; and the Field Intelligence Groups are currently being reviewed as their own unit of analysis. Taken together, the set of evaluations will cover the full scope of the program. Once the 6 year period is complete and the entire program has been evaluated, the cycle will repeat itself and the reviews will continue on an ongoing basis. The structure and design of the evaluation was closely coordinated with OMB to ensure that challenging and appropriate subjects were reviewed. The evaluations are conducted as a series of interviews (with both Bureau and non-Bureau subjects) and include a rigorous review of documentation and evidence. Topics that are covered include the program's strategic planning processes, the adequacy of its data collection and performance measurement plans, management deficiencies, and the overall results produced by the program. Evaluation results and findings are structured in a way to promote program improvements (e.g., through the issuance of improvement recommendations).

Evidence: The FBI organizational chart shows the independent lines of authority for leaders of the Intelligence Program and the OPEAU. Some portions of the Intelligence Program have already completed reviews, including the Special Technologies and Applications Section, which is covered in the Cyber Division review. Further program reviews are being conducted according to the internally published OPEAU schedule. A draft of the Field Intelligence Group evaluation will become available in July 2006.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The FBI's Intelligence Decision Unit encompasses all direct and indirect costs associated with the program, including applicable agency overhead and retirement costs. In addition, budget presentations for the Intelligence Program successfully align resources with strategic goals and provide, at a high level, an indication of targeted performance levels for the budget year. However, the presentation materials lack the detail and specificity needed to adequately define the relationship between resources and individual performance goals, including the expected impact of funding, policy or legislative decisions on expected performance. For FY 2008, the FBI plans to use a zero-based budgeting approach that would make clearer the relationship between a program's full resources and the performance that can be expected. If fully applied, this would constitute substantial and commendable improvement.

Evidence: The 2007 Congressional Budget Submission illustrates the FBI's current approach to the integration of budget and performance information in resource requests. The FBI Finance Division distributed 2008 Budget Guidance on March 2, 2006 that outlines the zero-based approach that will be employed for FY 08.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Strategic planning deficiencies within the DI are identified and addressed at multiple levels within the FBI. At the highest level, the FBI has established a strategic planning office within the Deputy Director's office and an Executive Council for Strategic Planning, which includes participation from outside experts. These efforts help to ensure that senior leadership is aware of strategic planning initiatives that will impact the Intelligence Program. Within the program itself, the DI has implemented an Intelligence Program Plan that focuses on execution and against which results are continuously monitored. The program plan is fully linked to DI strategic goals and objectives. As the vision from ODNI, Congress and the President evolves, the FBI will adjust these strategic goals and objectives accordingly. Additionally, the DI produces an annual threat forecast that scans the environment for long term changes and challenges. The Intelligence Program (and, more broadly, the entire FBI) uses this environmental scan as part of the planning and budgeting processes by making adjustments for changes in the environment. In terms of strategic issues identified in this section, the Intelligence Program has taken concrete steps to address problems. For example, the FBI Finance Division distributed 2008 Budget Guidance that outlines a zero-based budget approach that will improve the program's ability to define the relationship between resources and performance. This should enable the Intelligence Program to better integrate budget and performance information in its upcoming budget submissions for FY 08 and beyond.

Evidence: The Directorate of Intelligence Performance Plan and Placemat show how strategic goals and objectives are monitored by program managers. In addition, the December 2005 FBI Forecast: Drivers and Threats document is an example of threat information that could impact planning and resource decisions. The Finance Division's guidance memo from March 2, 2006 on zero-based budgeting outlines the steps the FBI is taking to improve its budget and performance integration in upcoming budgets.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The DI, as the entity responsible for the coordination of the Intelligence Program, is managed using a detailed program plan, complete with performance measures, to stimulate and evaluate progress. These initiatives are codified in a master schedule, and information is collected and progress is monitored through monthly program reviews. When managers discover that performance is lower than targeted in a given area, the DI will make appropriate adjustments in resources, management actions, or priorities. For example, the monthly program review monitors "customer feedback" on the training that is provided to FBI Intelligence Analysts, and managers discovered that the training was not meeting customer standards. As a result, changes were made to the trio of intelligence analysis training courses: ACES 1.0, ACES 1.5, and Cohort. The Divisions within the Intelligence Program also collect performance information that is used to manage, evaluate, and improve performance, as well as ensure that cost and schedule parameters are met. In addition, the Intelligence Program must submit budget, staffing, and performance information to the Department of Justice on a quarterly basis for inclusion in the Quarterly Status Review process; progress toward quarterly and annual goals is assessed and reviewed by senior managers at the Department level. As addressed in section 2, the Intelligence Program has collected the appropriate baseline data needed to set ambitious performance targets for its measures. More information on baseline data is provided in responses 2.2 and 2.4.

Evidence: The Directorate of Intelligence Program Plan lists goals, schedules, and responsible managers for a variety of DI program activities. It is reviewed monthly and each goal is given a red/yellow/green color to represent the degree of progress. DOJ Quarterly Status Review materials also outline goals and schedules and are evaluated on a quarterly basis to assess progress. Baseline performance data can be viewed in the Measures Tab.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Department of Justice as a whole has been rated "green" for Strategic Management of Human Capital according to the President's Management Agenda. On June 30, 2004, the Attorney General transmitted decisions on the Senior Executive Service (SES) Performance-based Pay System to the Office of Personnel Management and OMB with the Human Capital report. The report included a generic work plan for all Departmental SES members, with accompanying Performance "contract" that must explicitly relate to the Department's, the President's or the Attorney General's defined goals. Consistent with the DOJ guidance, FBI Field Office Special-Agents-in-Charge (SACs) and DI Program Executives are held accountable for performance results via a series of cascading program objectives established in accordance with the Pay for Performance System. FBI HQ executive plans document scoring by their supervisors; SACs have evaluations documented by HQ divisions. Under Pay for Performance, DI Senior Executives are held accountable based on their performance results. This requires them to develop and implement workforce strategy and management directives that contribute measurable results to the DI mission and improve financial performance. The Intelligence Program in particular is managed using a detailed program master schedule. The schedule is fully linked to our strategic goals and objectives. These cascading goals and objectives form the performance elements for each FBI Directorate of Intelligence Senior Executive and are the basis of all DI performance reviews. The program does not have established mechanisms for holding "partners" accountable because the program does not have "partners" in the traditional sense--for more information, see question 2.5.

Evidence: The 2005 Department of Justice Performance and Accountability Report (p. IV-5) describes the Department's progress in PMA Human Capital initiatives and the Department's green status for Human Capital, including performance accountability. FBI Special Agent in Charge/Headquarters executive scoring criteria and sample evaluations from 2005 show the application of performance accountability within the DI program.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The FBI monitors the discretionary spending of its divisions using a spending plan process developed by its Finance Division (FD). Each FBI Headquarters division, including the DI, submits spending plans to FD on a quarterly basis that detail the division's available funding and a schedule for obligation of any remaining funds. FD reviews these plans to ensure the inclusion of all available funding and to look at actual obligations (for previous quarters) against planned obligations. Following FD's review, feedback is given to each division and office along with questions. If unintended or inappropriate obligations are identified, the program has procedures in place to make necessary corrections. For example, if an unauthorized procurement is made by an employee who lacks or exceeds delegated authority to enter into an agreement on behalf of the FBI, the Finance Division takes corrective action through the ratification process. Ratifications are also reported to the Department of Justice and considered as part of Senior Executive Service performance reviews. In FY 2005, 99.3% of the DI's allotted discretionary funds were obligated, leaving no significant unobligated balance at the end of the year.

Evidence: Sample FBI Spend Plans show the detailed information that is reviewed by the Finance Division and other senior managers for the Intelligence Program. The ratification process is described in Justice Acquisition Regulation 2801.602_3.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The Intelligence Program measures efficiency by tracking the staff time saved through business process improvements in the management of intelligence assets. By automating asset management capabilities and improving analysts' ability to work with large sets of asset data, the Intelligence Program can reduce staff time dedicated to those tasks and improve the productivity of FBI personnel. This efficiency measure has established baselines and targets, which are detailed in the measures tab. The program also has regular procedures in place to achieve efficiencies and improve cost effectiveness. For example, the FBI prepares an annual FAIR Act inventory list of positions for potential private sector competition. The FBI's last competition, which was finished in FY 2004, focused on automotive maintenance services, many of which support agents and analysts working in surveillance and other Intelligence Program activities. An additional competition is currently scheduled for FY 2006 for the provision of custodial services, a large portion of which services space utilized by the Directorate of Intelligence. In addition to competitive sourcing, the FBI's Information Technology Investment Management (ITIM) process oversees all FBI IT investments within the enterprise architecture, assesses performance, and baselines a funding priority list based upon an existing IT strategic plan. This ensures that the FBI, including the Intelligence Program, is carefully planning and executing IT improvements that lead to greater productivity and achievements of cost savings.

Evidence: The STAS OMB 300 provides detailed information on asset management business process changes (the "Delta" program) and how they will impact efficiency and productivity. The efficiency measure itself, as well as the baseline and targets, are available in the Measures Section. The procedures to achieve efficiencies and cost effectiveness are illustrated by the FBI FAIR Act inventory list (for positions competed) and the Information Technology Strategic Plan for FY 2007-2011 (for IT investment strategy).

Does the program collaborate and coordinate effectively with related programs?

Explanation: The FBI has extensive relationships with related programs at the federal, state, local, and international levels. The depth of the relationships range from occasional meetings of working groups and committees to sharing of information and even the full integration of case work and analysis. One good example of a coordination effort within the intelligence program is the FBI's participation in the activities of the National Counterterrorism Center (NCTC). NCTC operates as an interagency partnership including representatives from the FBI, the Central Intelligence Agency, the Departments of State, Defense and Homeland Security, and other entities such as the US Capitol Police. Among other responsibilities, NCTC produces integrated interagency assessments on terrorism issues and publishes warnings, alerts, and advisories as appropriate; manages a joint Operations Center to provide insight and situational awareness of developing terrorism-related worldwide issues and events; and operates a secure website, NCTC Online, which serves as the primary dissemination system for terrorism information produced by NCTC and other mission partners. FBI intelligence analysts and agents play a key role in these activities together with the many other NCTC agencies.

Evidence: Congressional testimony of Executive Assistant Director Gary Bald (9/21/05) provides an overview of many of the Intelligence Program's collaboration and coordination efforts. Details on the FBI's collaborative role at the National Counterterrorism Center, as well as information on policies and activities undertaken through the NCTC, are described in the "NCTC Overview" presentation.

Does the program use strong financial management practices?

Explanation: Although the FBI has received unqualified audit opinions, there are identified material weaknesses relating to information systems controls and internal controls over financial reporting. Because these weaknesses were identified in Bureau-wide systems, they impact the financial management of every FBI program, including the Intelligence Program, even though the systems themselves are not under the direct control of DI managers. The FBI has increased the scope of its review of its financial controls considerably in the past few years, instituting biweekly meetings with independent auditors to address material weaknesses and reportable conditions. For more information on the FBI's efforts to improve its financial management, see question 3.7.

Evidence: The 2005 DOJ Performance and Accountability Report outlines Department-wide performance in financial management. Details of the FBI audits and financial statements can also be found in OIG Audit Report 06-06, which covers the FBI annual financial statements for 2005.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Intelligence Program is managed using a detailed program master schedule, which is fully linked to strategic goals and objectives. Progress against these goals and objectives is monitored through monthly program reviews so that deficiencies can be identified early and corrected swiftly. The institution of the performance appraisal system described in question 3.2 is an example of a past action taken to rectify an identified management deficiency. The FBI, working together with OPM and the Department of Justice, restructured its performance appraisal system to bring it in line with established criteria from the President's Management Agenda. Going forward, the FBI will continue to expend effort on improving its internal financial management in order to correct existing material weaknesses and reportable conditions that have been identified in annual audits. In addition to undergoing frequent reviews by independent auditors, the FBI hopes to improve its financial management by migrating to the Department of Justice Unified Financial Management System (UFMS). This should allow FBI managers to better address and prevent weaknesses. Full migration to UFMS is scheduled for FY 2008, but steps are already underway to implement the change.

Evidence: The Directorate of Intelligence Program Plan is a major mechanism to identify and track management deficiencies. Evidence on the FBI performance appraisal system is available under question 3.2. With respect to improving financial management controls, the FBI has created a detailed matrix of audit weaknesses and findings, along with target completion dates for corrective actions. Summaries and minutes from bi-weekly status meetings track the Bureau's progress in achieving those corrective actions.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Intelligence Program has data for its performance measures from FY 2005 and FY 2006. Generally speaking, performance for FY 2006 met or exceeded performance for FY 2005, including several instances where the degree of improvement is quite dramatic. This seems to indicate that the program is making progress toward its long term goals. However, a 'small extent' is awarded due to the limited availability of data from which to discern a performance trend.

Evidence: Data provided in Measures Section.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Intelligence Program has data for its performance measures from FY 2005 and FY 2006. Generally speaking, performance for FY 2006 met or exceeded performance for FY 2005, including several instances where the degree of improvement is quite dramatic. This seems to indicate that the program is making progress toward its goals, although there is limited data available from which to discern a performance trend (because of the relative newness of the measure). Until such time when more data is available, a 'small extent' is awarded.

Evidence: Data provided in Measures Section.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Intelligence Program's efficiency measure was created this year, and, as a result, the program does not have sufficient data collected to assess whether or not it is making progress against its annual efficiency goals. This is not due to a deficiency in the program; it merely reflects the recent establishment of the efficiency measure and the need for adequate time to collect data. In the absence of data from the efficiency measure, a 'small extent' is awarded based on the FBI's use of procedures that promote efficiency and cost effectiveness, particularly competitive sourcing and information technology investment. The FBI is able to promote cost effectiveness by competing certain Bureau services to find the most economical service provider and by planning and executing IT investments that improve productivity and allow more work to be done with the same level of resources. The 'small extent', rather than a 'large extent,' is justified due to the fact that many of the FBI's competitions and IT investments are in areas that impact the Intelligence Program but are not directly related to its core activities.

Evidence: The program's efficiency measure, as well as the baseline and targets, are available in the Measures Tab. The procedures to achieve efficiencies and cost effectiveness are illustrated by the FBI FAIR Act inventory list (for positions competed) and the Information Technology Strategic Plan for FY 2007-2011 (for IT investment strategy).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: While the FBI's Intelligence Program works extensively with federal, state and local agencies, a comparison of these entities would be too difficult to perform at this time. The vast differences between programs in terms of their scopes of responsibility make a one-to-one comparison between the FBI and other programs very challenging, and the challenges are compounded by classification issues at all levels that impede access to data. In addition, the relative newness of a number of intelligence programs in the government (e.g., the DEA Office of National Security Intelligence) and the major changes that have taken place in the FBI Intelligence Program over the last 5 years have further reduced the amount of data that is available to compare this program to other intelligence entities.

Evidence: Not applicable

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Only a small piece of the Intelligence Program has been through a complete Organizational Program Evaluation and Analysis Unit (OPEAU) review to date. An evaluation of a significant portion of the program, the Field Intelligence Groups, will be completed in 2006, and other program elements will not be reviewed until later in OPEAU's 6 year schedule. As a result, there is no existing independent evaluation (or group of independent evaluations) to indicate whether or not the program is effective and achieving results. In the absence of evidence from a comprehensive, outcome-oriented evaluation of the Intelligence Program, a 'small extent' is awarded on the strength of conclusions from several smaller, more targeted evaluations conducted by the Department of Justice Office of the Inspector General (OIG). The OIG has looked at a variety of aspects of the Intelligence Program since 2001 and has generally concluded that the FBI is making progress in advancing its intelligence mission, although additional work still remains. For example, a May, 2005 evaluation of the FBI's efforts to improve its intelligence analyst workforce concluded that the FBI "has made and continues to make progress toward bolstering its intelligence analysis capabilities," but the need for improvement still exists.

Evidence: The OPEAU schedule outlines when evaluations of significant portions of the intelligence program will be available. In terms of smaller, more targeted evaluations, the DOJ OIG has produced several, including the May 2005 report on "The FBI's Efforts to Hire, Train and Retain Intelligence Analysts" and the July 2005 follow-up report on the FBI's foreign language translation program (which is part of the Intelligence Program).