Detailed Information on the
Multipurpose Law Enforcement Grants Assessment

Defining long-term and annual goals for the program, as well as performance targets, and planning to collect performance data from grantees.

Planning evaluation work to determine the impact of the program.

Recommending replacing the program because it is unfocused and cannot demonstrate results.

Measure: Application Processing Time (Days) - JAG Grant

Explanation:Processing time through final grant award.

Measure: Application Processing Time (Days) - Byrne Formula Grant

Explanation:

Measure: Percent of funds being used for federal/state multi-jurisdictional task forces focused on violent crime, gangs and drugs (New measure, added February 2008)

Explanation:Violent crime, gang and drug task force funding divided by total program funding.

Measure: Application Processing Time (Days) - Byrne Discretionary Grant

Explanation:

Measure: Application Processing Time (Days) - Local Law Enforcement Block Grant

Explanation:

Is the program purpose clear?

Explanation: The Byrne Grants are intended to allow states, tribes, and local governments to support a broad range of activities to prevent and control crime based on their local needs and conditions. The majority of these funds are awarded through the Byrne Justice Assistance Grant (JAG) program on a formula basis based on a combination of population and UCR PART I violent crime statistics. Funds can be used to address any of six broadly defined "purpose" areas: (1) Law enforcement programs. (2) Prosecution and court programs. (3) Prevention and education programs. (4) Corrections and community corrections programs. (5) Drug treatment programs. (6) Planning, evaluation, and technology improvement programs. While the bulk of the funding is awarded through the JAG program, a portion of funding also is awarded through a discretionary component that is intended to permit funding of grants for corrections alternatives. (In the last several years, the discretionary grants have been earmarked or targeted by the Congress for various law enforcement and criminal justice projects around the country that often are not related to the authorized purpose of the funds.)

Evidence: "Background on the Purpose of the JAG Grant: The Byrne JAG Web site: www.ojp.usdoj.gov/BJA/grant/jag.html. The Byrne JAG Brochure: www.ojp.usdoj.gov/BJA/grant/JAGbrochure.pdf. The Byrne JAG 05 Application Kits: www.ojp.usdoj.gov/BJA/grant/05JAGStateSol.pdf and www.ojp.usdoj.gov/BJA/grant/05JAGLocalSol.pdf. Background on the Purpose of the Discretionary Grants: 42 U.S.C. 3762a Laney, Garrine P., ""Crime Control Assistance Through the Byrne Programs,"" CRS Report for Congress, May 20, 1998."

Does the program address a specific and existing problem, interest, or need?

Explanation: The JAG Program allows states, tribes, and local governments to support a broad range of activities to prevent and control crime based on their own local needs and conditions. Activities eligible for funding include six very broad purpose areas as well as any activities previously funded under earlier Byrne Formula Grant or Local Law Enforcement Block Grant programs. As such, there is no clearly defined specific or existing problem, interest, or need that is addressed by the program. Rather, the program is intended to serve a variety of potential local law enforcement needs. While the authorization of the Byrne discretionary grants addresses a more specific issue, the discretionary grants have for several years been earmarked or targeted for a wide variety of local law enforcement and criminal justice-related projects. The grants do not address a specific issue or need.

Evidence: See, for example, the Byrne JAG Web site: www.ojp.usdoj.gov/BJA/grant/jag.html. The Byrne JAG Brochure: www.ojp.usdoj.gov/BJA/grant/JAGbrochure.pdf and the Byrne JAG 05 application kits: www.ojp.usdoj.gov/BJA/grant/05JAGStateSol.pdf and www.ojp.usdoj.gov/BJA/grant/05JAGLocalSol.pdf.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Byrne grants overlap somewhat with other Office of Justice Programs (OJP) and Department of Homeland Security (DHS) grant and assistance programs for State and local law enforcement. Most of the other programs have a somewhat specialized focus that is not shared by the Byrne grants. However, because of the multiple purposes for which most Byrne grant funding can be used, there is a potential for significant overlap. The Bureau of Justice Assistance (BJA) attempts to mitigate duplication by monitoring State/local plans and reports. Program rules also prevent use of funds to replace or supplant State or local funds that would be used for the same purpose. BJA provides each of the state Byrne JAG recipients with a list of all the active DOJ grants in their states to help prevent resource overlap between direct and subgrant awards. To prevent duplication among other Federal programs, these same grant lists are made available to all DOJ grant-making components, and coordination meetings are held with DHS and other Federal agencies. Because of congressional direction about projects to be funded, the Byrne discretionary grant program has no specific design features to prevent duplication and redundancy from occurring.

Evidence: GAO, "Federal Assistance: Grant System Continues to Be Highly Fragmented," GAO-03-718T, April 29, 2003. (GAO has cited the Byrne grants along with other programs as examples of overlapping and duplicative "homeland security" grants.) The Byrne JAG Brochure: www.ojp.usdoj.gov/BJA/grant/JAGbrochure.pdf. The Byrne JAG 05 Application Kits: www.ojp.usdoj.gov/BJA/grant/05JAGStateSol.pdf and www.ojp.usdoj.gov/BJA/grant/05JAGLocalSol.pdf.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The Byrne JAG program functions like a block grant program. The program provides a much flexibility to grant recipients with limited restrictions. However, the Byrne JAG formula is based on population and violent (UCR Part 1) crime trends, and is not responsive to other factors that may shape State/local need for assistance, including changes taking place in trends for specific crimes (e.g., homicide, rape, etc.). Properly administered competitive grant programs can be used to more effectively target resources to the neediest of grant recipients. Another design issue in the formula design for the JAG program is that the states are direct recipients for 60 percent of funding--some of which is passed through to localities--and localities are direct recipients of 40 percent of funding. This funding split weakens the leverage that states have in attempting to coordinate use of funds to meet priority criminal justice needs, as states only have direct control over 60 percent of the funds. It also renders partially ineffective the state planning processes that attempt to determine priorities for meeting criminal justice needs within each state. There currently is no specific design for the Byrne discretionary grant program. BJA currently funds projects identified via extensive Congressional earmarking and targeting of funds.

Evidence: For information on recent congressional direction on use of funds for the Byrne discretionary grants, see the Conference Report for the FY 2005 Consolidated Appropriations Act. For information on the Byrne JAG formula, see authorizing legislation for the Byrne JAG program: www.ojp.usdoj.gov/BJA/pdf/jag.pdf. Also see Bureau of Justice Statistics Technical Report: Justice Assistance Grant (JAG) Program, 2005, July 2005, NCJ 209333: www.ojp.usdoj.gov/BJA/pdf/jag05rpt.pdf. A review of programs PARTed over the previous three years suggests that competitive grant programs are better at targeting than block grants.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Targeting of resources for the Byrne JAG program is accomplished by a six-step formula process, which funnels 60 percent of funds to states (with some pass-through of these funds to localities) and 40 percent to localities, and allocates funding based on population and generalized violent crime trends under the program's formula. While resources reach intended beneficiaries (States and localities), it is not clear that the formula provides for an effective targeting of resources. Under the program design, it is not possible to refine the targeting of resources to address mismatches in state/local requirements, to address specific crime issues, or to ensure that the neediest States/localities receive preference for funding. Allocation of resources for the Byrne discretionary grants is determined by congressional direction, preventing targeting of funds based on specific need or other criteria.

Evidence: For information on recent congressional direction on use of funds for the Byrne discretionary grants, see the Conference Report for the FY 2005 Consolidated Appropriations Act. For information on the Byrne JAG formula, see authorizing legislation for the Byrne JAG program: www.ojp.usdoj.gov/BJA/pdf/jag.pdf. The FY 2005 Byrne JAG allocations: www.ojp.usdoj.gov/BJA/grant/jagallocations.html. The Byrne JAG variable pass through percentages: www.ojp.usdoj.gov/BJA/grant/05JAGvpt.pdf. Also see Bureau of Justice Statistics Technical Report: Justice Assistance Grant (JAG) Program, 2005, July 2005, NCJ 209333: www.ojp.usdoj.gov/BJA/pdf/jag05rpt.pdf. A review of programs PARTed over the previous three years suggests that competitive grant programs are better at targeting than block grants.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: BJA is in the process of developing new measures for the Byrne grants.

Evidence: Discussions with BJA and Departmental staff.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: BJA will establish targets once measures are fully developed.

Evidence: Discussions with BJA and Departmental staff.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: BJA is in the process of developing new measures for the Byrne grants.

Evidence: Discussions with BJA and Departmental staff.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: BJA will establish baselines and targets once measures are fully developed.

Evidence: Discussions with BJA and Departmental staff.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The long-term goals for the Byrne grants are not well defined; as a result, it is difficult for the States/localities to offer specific commitments that relate to those goals. Also, the formula structure for the program--with limited ability on BJA's part to target or withhold funding--does not provide many options for holding grantees accountable to program goals. The Byrne discretionary grants are congressional earmarks, which also limit BJA's ability to hold grantees accountable to program goals.

Evidence: OJP Financial Guide: www.ojp.usdoj.gov/FinGuide/. The Byrne JAG 05 Application Kits: www.ojp.usdoj.gov/BJA/grant/05JAGStateSol.pdf and www.ojp.usdoj.gov/BJA/grant/05JAGLocalSol.pdf.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Justice Assistance Grant is too new (launched in 2005) to have been evaluated previously. Over the next six months, the Bureau of Justice Assistance will develop a comprehensive evaluation plan in consultation with OMB for the Byrne Grants. BJA has contracted for evaluations previously of its predecessors--the Byrne Formula Grant and the Local Law Enforcement Block Grant (LLEBG). These evaluations were reasonably independent; BJA contracted with external organizations for the reviews. However, the evaluations lacked sufficient quality and scope. For example, the evaluators surveyed states and localities about their uses of funds and whether they believed the funds were used effectively. Yet, the evaluators did not use rigorous evaluation methods to test for the impacts of the programs, or the major strategies/grant uses of the recipients. Also, the evaluations did not consider achievement of performance targets (although this may have been related to a lack of targets for the programs at the time). Evaluability is a key question for the two programs that comprise the Byrne grants--the Justice Assistance Grant and the Byrne Discretionary Grant. For the Byrne Discretionary Grant, which earmarks funds to a multiplicity of small criminal justice-related projects across the country, the diversity of eventual grant uses limits the kind of evaluation that can be conducted. BJA has used a number of approaches to handle the need for evaluation for the discretionary grants. BJA currently has an agreement with NIJ to conduct evaluability studies on all Byrne discretionary grant programs. BJA also has an agreement with the University of New Orleans to evaluate Byrne discretionary grants funded in 2002 and 2003. Recipients were directed to use grant funds to pay for evaluations, in 2004, using BJA/NIJ evaluation guidance. Mixed results led to a discontinuation of this practice for 2005. For the JAG Grant, the diversity of uses of what is effectively a block grant also limits the kind of evaluation that can be conducted. BJA has a $500,000 agreement with NIJ to perform an evaluation of the activities of the Byrne JAG-funded multi-jurisdictional drug task forces. The task forces were a major use for the pre-JAG Byrne Formula Grant (about 40% of funding), and probably account for 20-30% of the uses to which the JAG funds are being put in 2005. Similarly, equipment purchases accounted for about 40% of the use of pre-JAG Local Law Enforcement Block Grant funds, and probably represent 20-30% of 2005 JAG fund usage. While $500,000 is unlikely to provide for a sufficiently rigorous evaluation of a major JAG strategy, evaluating the effectiveness of the major strategies employed by States/localities with the funds may be an appropriate evaluation approach. Another evaluation strategy that could be used for either the Byrne JAG or discretionary grant programs would be to assess the extent to which States/localities accomplish the performance objectives they set for their grant use. No matter the strategy employed, however, evaluations must consider the extent to which performance targets are met, as well as ultimate program impacts, to the extent that can be measured.

Evidence: Examples of Previous Byrne Formula/LLEBG Evaluations: National Evaluation of the Local Law Enforcement Block Grant Program: Phase One Final Report. NIJ-Sponsored, 2001, NCJ 187778: www.ncjrs.org/pdffiles1/nij/grants/187778.pdf National Assessment of the Byrne Formula Grant Program, NCJ 162203; NCJ 163385; NCJ 163384; NCJ 163383; NCJ 163382: www.abtassociates.com/reports/byrne-formula.pdf Evaluation of Multi-Jurisdictional Task Forces Project: Phase I Final Report. NIJ-Sponsored, 2003, NCJ 200904: www.ncjrs.org/pdffiles1/nij/grants/200904.pdf BJA/NIJ, "Evaluating Drug Control and System Improvement Projects" available at: www.ojp.usdoj.gov/BJA/evaluation/guide/documents/nijguide.html

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: BJA is in the process of developing performance goals. Recent budget requests to OMB and Congress do not provide information about performance goals. However, with no resources requested in the President's 2006 Budget for the Byrne grants, no expectation of performance is appropriate for 2006.

Evidence: 2006 DOJ Congressional Budget Submission; 2006 DOJ OMB Budget Submission.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The key strategic planning deficiencies for the program involve the need to match program purpose to performance measures and to develop a suitable strategy for regular program evaluation. While the Administration has not requested funding for the Byrne grants in the 2006 Budget, these deficiencies should be addressed as long as the program receives funding from the Congress. Over the next six months, the Bureau of Justice Assistance will develop long-term and annual performance measures, baselines, and targets for the Byrne JAG Program.

Evidence: See answers for Questions 2.1 through 2.6.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: For the 2005 grant cycle, with the implementation of the new Byrne Justice Assistance Grant, grant awards were made without any specific performance system in place. While BJA's intention is to implement new performance measures for the program, the measures have not yet been defined. Without a performance measurement system in place for new grants, it will be difficult to demonstrate performance improvements over time. Byrne discretionary grantees are requested to submit performance reports, but due to BJA's limited discretion over such grant awards, as well as the ever-changing nature of such awards, it is difficult to use the information to improve performance.

Evidence: A variety of State grant administering agencies have posted information on the internet indicating that they are not being held to specific performance requirements for the 2005 grant cycle. See for example: www.azcjc.gov/pubs/Full_Commission051005Agenda.pdf

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: The Byrne JAG statutory authorization does not provide for withholding funds based on performance results. As a result, it is impossible to hold grantees accountable for performance results. Both Byrne JAG and discretionary grantees are subject to periodic reporting requirements, including performance reporting. Under some circumstances, funding can be withheld if certain materials or reports are not submitted. (A draft revision to the funds withholding policy is currently under development.) Additionally, on June 30, 2004, the Attorney General transmitted decisions on the SES Performance-based Pay System to OPM and OMB with the Human Capital report, including a generic work plan for all Departmental SES members, with accompanying Performance "contract" that must explicitly relate to the Department's, the President's or the AG's defined goals. On December 10, 2004, DOJ obtained approval for the SES and GS/Prevailing Rate performance orders. DOJ has completed the application package for OPM/OMB certification of the DOJ SES Performance Management and Compensation Plan. DOJ components implemented five-level performance plans for all SES. New SES and manager plans include cascading tasks/assignments that are linked to the DOJ Strategic Plan and the PMA. By Dec 30, 2004, all DOJ components certified to the Attorney General that all SES and direct report performance work plans are in place. By the end of the December 2004, the Department had performance appraisals for most of the work force that: link to agency mission, goals and outcomes in DOJ's Strategic Plan; hold employees accountable for results appropriate to their level of responsibility; effectively differentiate between various levels of performance; and provide consequences based on performance.

Evidence: P.L. 108-447, Consolidated Appropriations Act, 2005, by reference to section 201 of H.R. 3036 of the 108th Congress, as passed by the House of Representatives on March 30, 2004. BJA Draft Policy on Withholding of Funds submitted to OMB. All managers are subject to the new DOJ Performance Management System. Attorney General's Memorandum dated August 10, 2004 - "Performance Management Plans for SES Members and Managers" www.usdoj.gov/jmd/ps/memagpma-ses1.htm

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: BJA's internal award timeline and process calls for all awards to be made in 45 - 60 days after receipt of the applications. As soon as the awards are made and accepted, the full amount of funding is available for drawdown. BJA obligates most funds appropriated within the year in which they are appropriated. Grantees are required to submit quarterly financial status reports, semi-annual subgrant reports, and annual performance reports. Grantees also are subject to annual audits, monitoring and site visits to ensure that funds are spent for the intended purposes.

Evidence: 2001 thru 2004 obligations for Byrne and Byrne JAG predecessor programs (Byrne Formula, Byrne Discretionary, and Local Law Enforcement Block Grants): 2001 Budget Authority $1.099B; 2001 Obligations $1.140B; 2001 Unobligated Balance $0.98B; 2002 Budget Authority $1.247B; 2002 Obligations $1.279B; 2002 Unobligated Balance $0.080B; 2003 Budget Authority $1.028B; 2003 Obligations $1.041B; 2003 Unobligated Balance $0.060B; 2004 Budget Authority $0.875B; 2004 Obligations $0.883B; 2004 Unobligated Balance $0.050B;

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The OJP Competitive Sourcing study will enable the Department of Justice to show continued progress in advancing the Presidents Management Agenda and the commitment to government efficiency and effectiveness. The OJP study is anticipated to affect 115 full time equivalent personnel who are involved with the functions and activities of Grant Management Support Services. This 18-month study is scheduled to be completed in January 2006. The 18-month plan was approved by the Department of Justice Competitive Sourcing Official. The OJP Competitive Sourcing study is one of several studies, which were planned by the Department of Justice in coordination with the Office of Management and Budget. OJP is currently going through an A-76 competitive sourcing process, with the grants management function included in the Performance Work Statement (PWS). The PWS has been released and those responsible for currently managing the Byrne JAG Program will compete as part of the Most Efficient Organization against private sector and other governmental competitors to continue the management and administration of the program. BJA also just completed a comparison-contrast of the existing Local Law Enforcement Block Grant (LLEBG) Program system and OJP's Grants Management System (GMS) to determine which system was most cost effective and efficient for managing the Byrne JAG Program. Managers responsible for the Byrne JAG Program have performance plans directly tied to the program's implementation through the application and award process, collection of performance data, monitoring, and closeouts. Finally, BJA utilizes the OJP-approved efficiency measure for application processing times to determine the performance of the program.

Evidence: PWS solicitation for competitive sourcing, reference number 2004PR784 (www.fedbizopps.gov);

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Bureau of Justice Assistance coordinates the Byrne grants with other related programs by sharing information and data with other Office of Justice Programs and Department of Justice components, as well as the U.S. Department of Homeland Security (DHS). Examples include: ?? Monthly coordination meetings are held with the Office of Community Oriented Police Services and the National Institute of Justice. Examples of topics of coordination include BJA gang programs, cybercrime programs, police integrity/ethics programs and others, many of these projects funded under Byrne grants. At these meetings, various projects are highlighted and discussed, with efforts made to avoid duplication and utilize lessons learned. Results include jointly funded projects, jointly branded publications, and joint planning for new program areas. Proposals and draft publications are also shared at these meetings. ?? Monthly Justice Information Sharing meetings are held with the National Institute of Justice's Office of Science and Technology (OS&T). These meetings focus on developmental and implementation activities. Projects are reviewed and joint project listings are developed. ?? Quarterly Counterterrorism Training Coordination Working Group (CTTWG) Meetings are held, including various DOJ and DHS components including the FBI, the Department of Energy, COPS Office, Department of Defense and other agencies. These meetings have taken place since shortly after 9/11/01. The purpose of these meetings is to coordinate counterterrorism training activities and projects as well as related programs outside of the training arena. During 2004, this group was briefed on 15 earmarked programs funded under the Byrne Program that were closely coordinated with DHS' Office of Domestic Preparedness. DHS also was invited to participate in training delivery for the earmark recipients, discussing DHS/ODP initiatives related to their efforts/plans. BJA attempts to ensure coordination with states and local criminal justice agencies by providing each of the state Byrne JAG recipients with a list of all the active grants in their states and monitoring plans and reports. Grant lists are shared with other Justice components, as well.

Evidence: See Counterterrorism Training Coordination Working Group (CTTWG) website for examples of coordinated focus areas at: www.counterterrorismtraining.gov/mission/index.html Some examples of meeting notices and agenda submitted by BJA as evidence of coordination; also Counter-Terrorism Training Coordination Working Group 2002 Annual Report.

Does the program use strong financial management practices?

Explanation: Auditors were unable to express an opinion on the Department's 2004 financial statements because of an auditor's disclaimer on the Office of Justice Programs' 2004 financial statements. The disclaimer was the result of grant accruals, or the estimates of unspent funds. These estimates can be prepared in a variety of ways depending upon the assumptions made about grantee spending behavior--- the auditors required OJP to devise a new procedure for producing such estimates. OJP is working with auditors to resolve the issue in time for the 2005 financial statements. Discussions with DOJ and OIG staff indicate, barring an unforeseen auditor issue, that OJP is on track to resolve this issue. BJA has little input to or control over the overall OJP financial management system. In the audit report for OJP's 2004 financial statement, the OIG identified OJP grant monitoring procedures as a weakness. Because of limitations imposed by staffing and other resources, OJP directly monitors a sample of grants based on a risk assessment. For about one third of grantees for all OJP grant programs monitored (not just the Byrne grants), OJP found questionable expenditures and problems with grantee accounting procedures. In addition, financial status reports were not filed on time for 28 percent of grantees. In response to these problems, OJP has increased grantee training and monitoring in 2005, as well as boosting site visits by 12.5 percent and performing more aggressive follow-up to deal with grantee problems. In addition, Byrne grant recipients use an electronic system to request drawdown of their grant funds, which scrutinizes whether they have signed and returned their award documents, there is a current financial status report on file, and have any withholdings placed on their account before funds are electronically transferred into their account. BJA reports that it works with grantees and the Office of the Comptroller to ensure that grant awards are used in a timely fashion and for only their intended purposes. Grantees are required to submit quarterly financial status reports, semi-annual subgrant reports, and annual performance reports. Grantees are subject to annual audits, monitoring, and periodic visits.

Evidence: Discussions with BJA and Departmental staff and the FY 2005 Financial Monitoring and Technical Assistance Plan for DOJ Grants, submitted by BJA, indicate that considerable attention is being paid to resolving financial management weaknesses. OC Financial Guide: www.ojp.usdoj.gov/FinGuide/. Office of Justice Programs Financial Statement, FY 2004, U.S. Dept. of Justice, Office of the Inspector General, Audit Report 05-17, March 2005.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Bureau of Justice Assistance (BJA), and ultimately the management of the Byrne JAG Program, was reorganized in FY 2002 with a focus on State Desks. Based on this customer service model, State Policy Advisors began managing all BJA funding distributed to their state of responsibility, building stronger relationships with practitioners in the field; increasing the effectiveness of communications between BJA and states; and increasing coordination of state-local-federal responses to crime and violence. During BJA's reorganization, a Quality Assurance Manager also was added to routinely review and audit BJA's initiatives and the management of BJA's programs. One outcome as a result of instituting a Quality Assurance Manager has been the development of a withholding of funds policy, which freezes grant funds if grantees become delinquent in submitting required semi-annual reports. The Byrne Program was streamlined in FY 2003 resulting in an automated application and award process and fewer, more specific grantee reporting requirements. With the implementation of the new Byrne JAG Program, BJA hosted state and local focus groups to gain input and insight from stakeholders into what changes needed to be incorporated into the program design and administration.

Evidence: Discussions with BJA and Departmental staff. "Withholding of Funds" policy submitted in hardcopy by OJP staff to show the improved linkage between grantee reporting and BJA grant management.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: BJA uses an automated grants management system to collect and review grantee applications, process grant awards, schedule, conduct, and follow-up on monitoring activities, and collect semi-annual subgrantee information and annual performance reports. BJA uses OJP's automated SF-269 system to collect financial reporting data, and the government clearinghouse to collect annual audit reports. BJA also attempts to maintain oversight of Byrne JAG grantees and their activities through bi-weekly newsletters, monthly conference calls, semi-annual desk monitoring, annual on-site monitoring visits, regional training events, and focus group discussions. OJP has increased grantee training and monitoring in 2005, as well as boosting site visits by 12.5 percent and performing more aggressive follow-up to deal with grantee problems.

Evidence: Discussions with BJA and Departmental staff. Office of Justice Programs Financial Statement, FY 2004, U.S. Dept. of Justice, Office of the Inspector General, Audit Report 05-17, March 2005.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: BJA does collect annual performance reports from each Byrne grant recipient. However, standardized performances measures have not been implemented for 2005, making it difficult for the public and policy makers to interpret the results of the reporting for the year. State reports for the Byrne Formula Grant program, which has been combined with the Local Law Enforcement Block Grant program into the new JAG program, have been made available to the public via the internet for 2003. Reports for the other major Byrne JAG predecessor, the Local Law Enforcement Block Grant, are not available to the public.

Evidence: Byrne Formula Grant Program Annual Performance Reports for all 56 states and territories: www.ojp.usdoj.gov/BJA/grant/byrnereports.html.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Performance measures for the Byrne grants are still under development. Therefore, it is not possible to demonstrate progress in achieving long-term performance goals.

Evidence: See Discussion Under Question 2.1.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Performance measures for the Byrne grants are still under development. Therefore, it is not possible to determine whether the program is achieving its performance goals.

Evidence: See discussion under Question 2.3.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: BJA reports a number of steps taken to improve the efficiency of the Byrne grants, but needs to document cost or efficiency savings. Between 2003 and 2004, BJA reports that it reorganized its staff and projects with better efficiency as a goal. Through outreach, BJA also is sharing already existing curricula and training material with States and localities to reduce the amount of Byrne funds required for develop similar resources. Results have included lower cost training for Byrne grant recipients and their partners. Similarly, BJA is sharing the source code for IT systems previously developed with Byrne JAG funds with new grantees in order to avoid replication of costly IT development among grant awardees. Additionally, BJA promotes DOJ's Global Justice extensible Markup Language (XML) initiative to assist state and local information sharing at a fraction of the cost of developing costly new databases or designer systems.

Evidence: Discussions with BJA staff.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The lack of well-defined outcomes and outcome-related performance measures inhibits comparisons with other, related programs that assist State and local law enforcement. However, partly because of the amount of funding invested in the Byrne grants as well as the fact that Byrne grants are used for many of the same purposes for which smaller, more specialized grants are made available, sometimes the Byrne grants accomplish more (in output terms) than do the smaller, specialized grants. For example, in recent years (2003, for example), the Byrne grants have been responsible for the establishment of more drug court programs than funded under the BJA Discretionary Drug Court Program.

Evidence: Discussions with BJA and Departmental staff. FY 2003 State Byrne Formula Grant Annual Reports available at: www.ojp.usdoj.gov/BJA/grant/byrnereports.html

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The Byrne JAG program is too new to have been evaluated previously. The program is only being implemented this year. BJA should consult with OMB on a suitable evaluation strategy for the program. Previous evaluations of the Byrne JAG predecessors--the Byrne Formula Program and the Local Law Enforcement Block Grant--were not of sufficient quality to establish the effectiveness of those programs, nor of the strategies employed by the programs. Results of a review of the Byrne discretionary grants by the University of New Orleans for FY 2002 and 2003 are not yet available. Most of the evaluations conducted previously appear reasonably independent--and were conducted by outside organizations not directly affiliated with BJA or its grantees. However, the option permitted in 2004 to grantees that they reserve a portion of funds to conduct their own evaluations did not provide for sufficient evaluation independence.

Evidence: Examples of Previous Byrne Formula/LLEBG Evaluations: National Evaluation of the Local Law Enforcement Block Grant Program: Phase One Final Report. NIJ-Sponsored, 2001, NCJ 187778: www.ncjrs.org/pdffiles1/nij/grants/187778.pdf National Assessment of the Byrne Formula Grant Program, NCJ 162203; NCJ 163385; NCJ 163384; NCJ 163383; NCJ 163382: www.abtassociates.com/reports/byrne-formula.pdf Evaluation of Multi-Jurisdictional Task Forces Project: Phase I Final Report. NIJ-Sponsored, 2003, NCJ 200904: www.ncjrs.org/pdffiles1/nij/grants/200904.pdf