Detailed Information on the
Residential Substance Abuse Treatment Assessment

Improving the automation of performance data collection and handling to better track how the program is performing.

Making performance data available to the public via the internet and publications.

Developing long-term goals for reducing drug abuse relapses among participants in residential substance abuse treatment programs operated by grantees.

Instituting changes to improve the consistency and quality of grantee performance data.

Developing a model for estimating grantees' enrollment and treatment costs.

Measure: Of the offenders that complete the program, the number who have remained arrest free for 1 year following release from aftercare

Explanation:Note: Target and actual data is currently under review

Measure: Average treatment cost per inmate

Explanation:Note: Methodology for treatment cost calculation was recently determined. There has been no consistency in cost data reported by States.

Measure: Number of participants in RSAT

Explanation:Note: Measure was reworded from "Number of state and local offenders treated annually by RSAT-funded programs" to "Number of participants in RSAT" Note: Due to statutorily mandated annual report requirement, 2005 actuals are due in Mar 2006

Is the program purpose clear?

Explanation: RSAT assists state and local governments in developing, implementing, and providing residential substance abuse treatment programs within state and local correctional systems.

Evidence: RSAT assists state and local governments in developing, implementing, and providing residential substance abuse treatment programs within state and local correctional systems.

Does the program address a specific interest, problem or need?

Explanation: There is a well established link between substance abuse and criminal behavior. Though 50-60% of state prisoners have some type of substance abuse problem, only about 15% receive treatment in a given year. RSAT is designed to encourage states to address this problem during incarceration, and for a limited time after release.

Evidence: Data on drug use from National Center on Addiction and Substance Abuse. Studies by Texas Christian Univ., BOP, and others have linked prison treatment to lower recidivism. In the 1997 Survey of Inmates in State and Federal Correctional Facilities, over 570,000 of the Nation's prisoners (51%) reported the use of alcohol or drugs while committing their offense. While only 20% of state prisoners are drug offenders, 57% were using drugs in the month before their offense, and 37% were drinking at the time of their offense.

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: RSAT funding accounts for about 20% of the estimated $300+ million that states spend on adult offender substance abuse treatment, yet it 2001 it supported the treatment of roughly 10,500 prisoners, less than 10% of those estimated to be in treatment (roughly 187,000).

Evidence: Data on state treatment expenditures from National Center on Addiction and Substance Abuse report on state expenditures related to substance abuse. RSAT treatment data based project reports for 2001 submitted by grantees. Treatment estimate based upon 2001 state prison population.

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: While RSAT essentially subsidizes a state and local function, it makes a unique contribution by requiring funded programs to follow "best practices," including separate housing, drug testing, and 6-12 months of treatment prior to release. RSAT is not redundant of other Federal programs. BOP treats only Federal prisoners and HHS' Substance Abuse and Mental Health Services Administration generally does not fund offender treatment.

Evidence: Program criteria on 'best practices' are specified in the FY02 RSAT grant application kit.

Is the program optimally designed to address the interest, problem or need?

Explanation: The grant criteria have not been updated since RSAT's creation in 1994. The most significant gap in state and local resources for treating offenders is in the post-release phase, rather than pre-release, particularly as more states abolish parole. RSAT effectiveness might be improved if grantees were allowed to use a greater percentage of grant funds for post-release aftercare, though this would shift the program's current focus considerably.

Evidence: Requirements outlined in grant application criteria. Recently-enacted DOJ reauthorization legislation would provide more flexibility to support post-release programs.

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The RSAT program seeks to reduce the number of those rearrested within one year of release through the provision of substance abuse treatment. By increasing the number of offenders treated and being able to measure the effectiveness of such treatment, the RSAT hopes to demonstrate progress towards long-term outcomes of reducing recidivism and dependency on illegal drugs. However, OJP has not incorporated clear time frames or targets into these goals.

Evidence: DOJ 2003 Performance Plan provides the aggregate number treated. Recidvism targets are described in congressional justification materials.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: The primary annual performance measure is the number of inmates treated by RSAT-funded programs, which OJP is reformulating in FY04 to focus on the annual treatment level, rather than a cumulative total. A secondary goal is the number of programs funded and in compliance with "best practices" for treatment.

Evidence: Grantees must submit annual project evaluation reports with data on the number of offenders treated. OJP's FY04 request sets targets of 40,000 inmates treated in F03 and FY04.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: Grantees must agree to comply with the RSAT criteria for separate housing, drug/alcohol testing, and a focused treatment regiment. Additionally, grantees and sub grantees collect data on the number of offenders in treatment, which is sent to the state-level grantee, and then OJP.

Evidence: RSAT grant application kits outline these criteira.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: RSAT coordinates well with other OJP programs, such as the prisoner reentry initiative. However it has had only limited coordination with SAMHSA and almost none with the federal Bureau of Prisons, which has an extensive treatment program for Federal prisoners.

Evidence:  

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: CPO and the National Institute of Justice (NIJ) have developed an evaluation program that reflects the broad spectrum of the 400 active RSAT programs, that include program for adults, juveniles, males or females; State correctional facilities and local jails; programs based on different theoretical approaches; and programs in different regions of the United States. Initial results have led to increased focus on sustaining treatment after release into the community.

Evidence: In FY97 and FY99, 38 independent implementation/process RSAT evaluations were competitively funded by NIJ. A few of these evaluators were given funds to continue their studies with outcome evaluations, many of which are still in progress today. Currently, there are eleven outcome evaluations being conducted that should be completed later this year. However, few of these studies are available for public review.

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: There is a relationship between the annual BA level and the number of RSAT participants, although new funding may show a lag of a year a more in order to implement a new or expanded RSAT program. Furthermore, the expense of treatment has gone up faster than RSAT funding in many jurisdictions. OJP is encouraged develop a methodology for estimating how RSAT funds are allocated, based on each state's average treatment cost.

Evidence: The average treatment cost per inmate ($4665) can be used to estimate the impact of funding or policy changes, although it can vary +/-$1500 among states. When the RSAT program began in FY 96, it took states a year or two to plan, establish, and implement the program. Many are just now obligating past years awards because they had early delays. With additional funding, there will again be delays and start-up costs as the states bring new capacity on-line.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: RSAT is reformulating its performance goals to provide more useful information, and is beginning to assess data on treatment costs.

Evidence: FY04 performance goals will more accurately reflect annual performance. The program is working with state grantees to improve the transparency of how grant funds are used.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: While most grantees provide annual progress reports, in FY01 less than half provided requested data on drug testing and rearrests. Provision of 'within year' status reports is also highly inconsistent. However, OJP does use what data it does receive to help manage the program.

Evidence: Grant managers have used Annual Evaluation reports to identify grantees with high drop-out rates to recommend technical assistance on better screening for program candidates. OJP also monitors the length of stay data to ensure compliance with program requirements. Additionally, CPO collects quarterly Financial Status Reports (SF-269), and Semi-annual Status Reports.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: Federal grant managers are responsible for grant monitoring to ensure that grantees are compliant with the specific grant program requirements and are utilizing the funding on allowable purposes. This generally does not include grantee performance. OJP believes it has little ability to hold grantees accountable for costs or performance. Treatment costs vary widely among states, and the RSAT statute does not provide OJP with leverage to withhold funding based on poor performance.

Evidence: OJP has recently instituted controls that make the release of funds contingent upon receiving all required status reports.

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Obligations and budget authority were closely aligned, though there was some carryover from FY00 into FY01. An audit of RSAT grantees in 2000 found no evidence of improper use of funds.

Evidence: In FY2001, obligations were $61 million out of a $63 million appropriation.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: OJP has implemented an automated grant processing system through the web-based Grant Management System (GMS), however it has not been able to quantify level of efficiency savings for RSAT and most other OJP programs.

Evidence:  

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: The total adminstrative costs for the program are included in the RSAT appropriation. The portion is allocated to OJP based on a fixed percentage (2%) of the RSAT appropriation. In this way, the RSAT funding level reflects the full costs of achieving the program goals. The FY04 submission to OMB includes all indirect costs for OJP. The total funding level can be tied to the number of inmates treated, and their eventual recidivism rates.

Evidence: The M&A set aside from RSAT funds was $1.4 million in FY02, about 2% of the total. Conversely, total admin and payroll for the Correction Programs Office (which also administers holdover prison construction grants and the Offender Reentry initiative) was $1.7 million.

Does the program use strong financial management practices?

Explanation: RSAT dollars are drawn down based on OJP Comptroller standards. Recipient organizations request funds based upon immediate disbursement requirements. Funds are not paid in lump sum, but rather disbursed over time as project costs are incurred or anticipated. Recipients time their drawdown requests to ensure that Federal cash on hand is the minimum needed for disbursements to be made immediately or within a few days. RSAT funds are statutorily mandated to go to Byrne agencies, which are typically the state Criminal Justice Planning agency.

Evidence: OJP Comptroller standards specify procedures for distributing funds. Internal audit reports and a 2000 IG audit did not idenfitied any significant financial weaknesses.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: During an Office of Inspector General (OIG) review of the RSAT program, preliminary findings revealed that CPO's grant files did not contain all the grantees' reports for reporting periods that were past due and some of the grantees' files were missing. However, before the OIG completed its review, the CPO's grant files had been updated with most of the missing reports.

Evidence: OJP has recently instituted controls that make the release of grant funds contingent upon receiving all required status reports.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: OJP places a considerable importance on monitoring of grantees use of Federal funds. CPO conducts site visits at least once every 18 months, in addition to an extensive amount of desk monitoring. Additionally, the OJP's Office of the Comptroller conducts inspections of the grantees and ensures their compliance with the OJP Financial Guide.

Evidence: As the data gathered from visits is not consolidated with financial data and status reports, RSAT should make more effective use of the OJP's grant management system.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: While grantees submit annual performance data, OJP has only made the aggregate data public. RSAT program staff has expressed misgivings about releasing state or program-level data to the public because it might lead grantees to skew their self-reported data. Grantee evaluations are not readily accessible to the public.

Evidence:  

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: The program currently lacks "long term" goals other than outyear projections of its annual performance goals. OJP is trying to recast the goal to focus on the percentage of offenders rearrested within one year of release.

Evidence: While the limited post-release data appears promising, it is not comprehensive and lacks baseline data. For example, North Dakota reports a recidivism rate of 33% (after 1 year) among offenders completing the program, compared to a national rearrest rate of 39.3%. However North Dakota's overall recidivism rate is unknown.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The RSAT program has exceeded its goals of supporting incremental increases in substance abuse treatment to prisoners, however these goals were not ambitious as they were based on obsolete cumulative estimates.

Evidence: Data is presented in DOJ's FY03 performance plan, and FY04 congressonal justification.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: OJP has implemented an automated grant processing system through the web-based Grant Management System (GMS), which has improved the efficiency with which OJP processes grants. However, OJP has not quantified either the aggregate or program-specific savings.

Evidence: OJP has not provided any evidence of specific improvements within RSAT. The program appears to becoming less efficient over time, as its FY03-04 goals project a drop in the number of inmates treated.

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: While RSAT is a unique grant program, the services its funds are somewhat comparable to those provided by HHS' Center for Substance Abuse Treatment (CSAT) and DOJ's Bureau of Prisons. There has been no cross-cutting analysis of these treatment efforts, but it appears that RSAT-funded treatment is significantly more expensive that BOP treatment.

Evidence: BOP spent approximately $22 million in FY02 on residential or community transition treatment for 29,000 Federal inmates. RSAT spent more than twice as much to treat a smaller number of state inmates.

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The only comprehensive evaluation (completed in 1998) focused on states' progress towards starting or expanding their treatment programs. It gave the RSAT credit for jump-starting these efforts, but did not offer conclusive findings on the effectiveness of treatment. NIJ has funded almost 40 site or state evaluations, but most of these are either underway or are available only in draft. Other studies cited by OJP appear to simply validate the potential of in-prison substance abuse treatment, rather than effectiveness of RSAT-funded programs.

Evidence: A Texas Christian University researcher found that in-prison substance treatment is effective when it is integrated with aftercare. Additionally, the Federal Bureau of Prisons found that, "offenders who completed a drug abuse treatment program and had been released to the community for a minimum of six months were less likely to be re-arrested or to be detected for drug use than were similar inmates who did not participate in the drug abuse treatment program."