Detailed Information on the
Supported Employment State Grants Assessment

Improve the collection and reporting of supported employment related data.

Improve the use and transparency of national and State data to manage and improve the program.

Work with Congress to eliminate the Supportive Employment Grant program and integrate necessary State plan provisions into the State Plan provisions of the VR Grant program.

Develop additional measures, including a long-term measure that adequately assesses the impact of the program; collect efficiency measure data; and set targets.

Measure: Of those individuals with a supported employment goal who obtained employment, the percent that earned the minimum wage or higher.

Explanation:In the context of this measure, employment means full-time or part-time employment in an integrated setting, consistent with the individual's capabilities, interests, and informed choice. Earning the minimum wage or higher means that the individual is compensated at or above the minimum wage, but not less than the customary wage and level of benefits paid by the employer for the same or similar work performed by individuals who are not disabled. The measure is calculated by dividing the number of individuals with a supported employment goal who exited the program earning the minimum wage or higher (numerator) by the total of individuals with a supported employment goal exiting the program who obtained employment (denominator) and multiplying the result by 100.

Measure: Percentage of general and combined State Vocational Rehabilitation Agencies whose cost per supported employment outcome is within a specified range.

Explanation:The cost per supported employment outcome is calculated for each agency by dividing the amount of the Supported Employment grant by the number of supported employment outcomes reported by the agency on RSA-911 Case Service Report.

Is the program purpose clear?

Explanation: The purpose of the program is clearly stated in the authorizing language in Title VI, Part B of the Rehabilitation Act and program regulations. The Supported Employment (SE) State Grants program provides grants to State Vocational Rehabilitation (VR) agencies to assist States in developing collaborative programs with appropriate public and private nonprofit organizations to assist individuals with the most significant disabilities to achieve a supported employment outcome. By law, only individuals who have been determined by the VR agency to be an individual with a most significant disability are eligible to receive services through this program. Individuals in supported employment work in integrated work settings in the competitive labor market along with non-disabled individuals. However, they require ongoing support and other appropriate services to maintain their employment. Funds provided under this program are used to supplement the funds provided under Title I Vocational Rehabilitation State Grants Program for the cost of providing supported employment services. Both the VR State Grants and the Supported Employment State Grants programs are administered by the Department of Education's Rehabilitation Services Administration (RSA) in the Office of Special Education and Rehabilitative Services.

Evidence: Rehabilitation Act of 1973, Titile VI Part B Program Regulations at 34CFR 363.1 and 363.6

Does the program address a specific and existing problem, interest, or need?

Explanation: Only about 42 percent of working-age individuals with significant disabilities are employed as compared to 88 percent of individuals who do not have a disability. In addition, the rate of employment varies by type of disabling condition. For example, only 37% of people with a wide range of mental disorders are employed and only 22% of people with schizophrenia and related disorders have jobs. The supported employment (SE) model was developed to assist persons with mental retardation, intellectual and developmental disabilities, and later individuals with mental illness to transition into integrated work settings through the use of on-site job coaches and other ongoing support services. Supported employment services are provided by the State VR agency to individuals with the most significant disabilities for whom competitive employment has traditionally not occurred because of the nature and severity of their disabilities. These individuals need intensive time-limited ongoing job support services (not to exceed 18 months) provided by the VR agency followed by extended support services provided by another State agency, a private nonprofit organization, employer, or other source after transitioning from VR support.

Evidence: U.S. Census Bureau, 2002 The 1997 National Health Interview Survey (NHIS) Policy Brief??Supported Employment: Best Practice for People With Psychiatric disabilities,Rehabilitation, Research and Training Center for Persons with Disabilities Cornell University, September 2006 http://www.aapd.com/News/empissues/060922rrtc.htm; 34CFR 363.6 (c)(1) and 34CFR 363.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Services provided under the SE program can be provided under the VR State Grants program. However, SE funds may only be used to provide supported employment services and may only be used to supplement, and not supplant funds provided under the VR program. The SE State Grants program was designed to provide a supplemental source of dedicated funds to offset the costs of providing supported employment services and provide an incentive for State VR agencies to provide such services. However, supported employment is now an integral part of the VR program, and it is no longer necessary or efficient to continue to support the provision of supported employment services through a separate supplementary grant program. A State VR agency must consider an individual's potential for supported employment when determining eligibility for the Title I Vocational Rehabilitation State Grants program. Title I funds must be used for assessment of the individual's VR needs and for development of the Individual's Plan for Employment (IPE). A State VR agency may support an individual's supported employment services solely with Title I VR State Grant funds, or it may fund the cost of SE services in whole or in part with funds under the Title VI-B SE State Grants program. Fiscal Year (FY) 2006 data indicate that 40,082 individuals who received VR services had a SE goal in their Individualized Plan for Employment. Some Title VI-B funds were expended for 58% of these individuals, but no Title VI-B funds were expended for the remaining 42%.

Evidence: Rehabilitation Act, section 625 34CFR 363.11, 363.50, 363.53 RSA 911 FY 2005 Case Service Report

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Supported employment is an effective strategy in assisting individuals with the most significant disabilities to obtain competitive employment in integrated settings and there is no evidence that another approach would be more effective. However, two statutory provisions may limit the program's efficiency and effectiveness. First, unlike the VR program, States are not required to match the Federal funds they receive through the SE program. Second, the assumption that long-term support services will be available from other public agencies once the individual is ready to transition from the support provided by the VR agency for a period not to exceed 18 months presents a significant challenge for State VR agencies for some States and in some cases limits the effectiveness of the program. The IPE for an individual with a SE goal must include a description of the extended services needed by the individual and the source of provision of the extended services, or if the source is not known at that time, a reasonable expectation that such a source will become available. However, programs and resources for the provision of extended services may not be readily available in all areas of a State, especially in rural areas. The availability of extended support services has also decreased in recent years due to reductions in funding for public programs that provide long-term services to individuals with significant disabilities.

Evidence: 34CFR 363.11(g)(3)(i)

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Program funds may only be used to provide supported employment services authorized under the Rehabilitation Act to individuals who have been determined eligible for services under section 102(a) of the Act, have been determined to be an individual with the most significant disability, and for whom supported employment has been identified as the appropriate goal for the individual on the basis of a comprehensive assessment of rehabilitation needs. Although State VR agencies may also provide supported employments services with funds provided under the State VR program, resources are limited and the majority of States cannot serve all eligible individuals. The supplementary funds provided under the SE program are used to offset the costs of such services and increase the resources available to the VR agencies for such purpose. The authorizing language protects against supplantation and there are no unintended subsidies. A State may only use program funds to supplement, and not supplant, the funds provided under title I of the Rehabilitation Act. No more than five percent of a State's grant may be expended for administrative costs in carrying out the program. Data from the RSA FY 2006 Case Services Report and a sub-study of the "Longitudinal Study of the VR Service Program" on individuals who had a supported employment outcome indicate that SE services are reaching those individuals with most significant disabilities.

Evidence: Of the 24,205 VR consumers who had a supported employment goal and achieved an employment outcome under the VR program in FY 2006, about 87 percent were individuals with a cognitive, mental health, or other mental impairment (RSA 911 FY 2006 Case Services Report). The SE sub-study found that participants were: (1) more likely to be relying on financial assistance, especially Supplemental Security Income (SSI) and Social Security Disability Insurance (SSDI), at the time they applied for VR services, (2) performing 3-4 grade levels lower in math and reading; and (4) had a more limited work history when compared with other VR participants with significant disabilities (Longitudinal Study??Final Report on the Vocational Rehabilitation Experiences Among Individuals Who Experienced a Supported Employment Outcome, April 2000. http://www.ed.gov/policy/speced/leg/rehab/eval-studies.html#vr ) 34CFR 363.3, 34CFR 363.11(g)(4), 34CFR 363.51(b)

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Department is currently working to develop an appropriate long-term program performance measure for inclusion in the FY 2008 Program Performance Plan. Because SE is provided as part of the VR State Grants program, it is difficult to differentiate the impact of the supplemental SE funds. In the past, a long-term performance measure specific to the SE program was not developed and the SE measure was included in the VR State Grants Performance Plan. In addition, beginning in FY 2003, the Department has proposed to consolidate the program and eliminate the program's separate authorization of appropriation, consistent with the Administration's initiative to reform the Federal government's overlapping training and employment programs. Thus, the program's performance has been assessed based on it annual measure, analysis of program data, and State monitoring.

Evidence: Department of Education Congressional Budget Justifications for FY 2008 http://www.ed.gov/about/overview/budget/budget08/justifications/index.html

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Department is currently working to develop an appropriate long-term program performance measure, including ambitious targets for inclusion in the FY 2008 Revised Performance Plan.

Evidence: Department of Education Congressional Budget Justifications for FY 2008http://www.ed.gov/about/overview/budget/budget08/justifications/index.html

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program's long-term goal is to increase competitive employment (employment in an integrated setting with compensation at or above the minimum wage) for individuals with significant disabilities through the provision of on-going support services. The program's current annual performance measure assesses the extent to which individuals with who have a supported employment goal and obtain employment, achieve a competitive employment outcome. The measure includes individuals who received supported employment services funded under the VR State Grants program or under the Supported Employment State Grants program. Because the SE State Grants program is a supplementary funding source to the VR State Grants program, the SE annual measure was originally included as part of the VR State Grants Performance Plan and the SE program did not have a separate performance plan. Additional annual measures were not developed, in part, because the Administration has proposed to eliminate the program's separate authorization of appropriation in each year beginning with the FY 2003 request. The Department recently developed an efficiency measure and is currently working to develop one or more additional annual measures to provide a more comprehensive picture of the program's performance.

Evidence: Department of Education Congressional Budget Justifications for FY 2008 FY 2007 Program Performance Plan for Supported Employment, U. S. Department of Education Visual Performance System

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baseline data was collected for this measure in 1998. Performance on this measure increase significantly after 2001 in part due to the fact that, beginning in fiscal year 2002, State VR agencies could no longer consider individuals who are working in non-integrated settings (e.g. extended employment or "sheltered employment") to have achieved an employment outcome under the VR program. Due to internal procedural deadlines, 2003 and 2004 targets could not be adjusted to reflect actual performance in 2002, but they were increased for fiscal years 2005 through 2007. Targets have not been developed for FY2008 and FY 2009 because the program is proposed for termination in the FY 2008 budget request.

Evidence: FY 2007 Supported Employment Performance Plan U.S. Department of Education Visual Performance System

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: VR State agencies, extended service providers, and other program partners must work together to enable individuals with the most significant disabilities to work in the competitive labor market with supports. To be eligible to receive funds under this program, the State must submit a supplement to their VR State Plan that demonstrates evidence that arrangements have been made with program partners to carry out the goals of the program.

Evidence: The Supported Employment State Plan Supplement, Rehabilitation Act, section 625(b)(4) and (5)

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: An independent evaluation of the effectiveness of the supported employment was completed in 2000 and a second independent multi-year study is underway. The first evaluation was conducted as a sub-study of the Longitudinal Study of the VR Services Program and included a nationally representative random sample of individuals receiving supported employment services under the Title I and Title VI-B programs. The purpose of this sub-study was to examine the characteristics, services, and outcomes of consumers who achieved a supported employment outcome after exiting the VR program, including both immediate and long-term outcomes. The study was of high quality and sufficient scope. Although statutory provisions do not allow for the use of an experimental design, the study compared the characteristics of SE consumers with those of other consumers of VR services with significant disabilities who obtained employment other than SE. The study was conducted by Research Triangle Institute (RTI), who obtained the data for the study through case file abstractions, consumer interviews, and extended service provider interviews. The final report for this sub-study was issued in April 2000. The Department is currently conducting a second independent multi-year study focusing on the post-program experiences of specific subgroups of former VR consumers. Two of the subgroups in the study sample, mental illness and mental retardation, comprise the greatest percentage of individuals who receive supported employment services. The study will be able to provide information on the post-program experiences of individuals in the sample who received supported employment services. In addition, the Government Accountability Office (GAO) recently completed (January 2007) a study of 4 Federal grant programs that fund services to assist individuals with disabilities to obtain competitive employment, including the Supported Employment program (GAO-07-236). The purpose of the GAO study entitled, Stronger Federal Oversight Could Help Assure Multiple Programs' Accountability, was to determine to what extent (1) performance goals and measures have been established for these programs and (2) the agencies responsible for these programs have established adequate procedures for overseeing program implementation and assuring laws and regulations are followed.

Evidence: Final Report on the Vocational Rehabilitation Experiences Among Individuals Who Experienced a Supported Employment Outcome, April 2000. http://www.ed.gov/policy/speced/leg/rehab/vr Stronger Federal Oversight Could Help Assure Multiple Programs' Accountability (GAO-07-236) Post Vocational Rehabilitation Experiences Study Design and Survey Instrument

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Funds have not been requested for this program for the last 6 years. Beginning with the 2003 request, the Administration launched a wide-ranging multi-year reform of the Federal Government's overlapping training and employment programs. Consistent with this crosscutting reform, funds were not requested for three vocational rehabilitation programs administered by RSA, including the Supported Employment State Grants program. However, the budget requests have not been explicitly tied to accomplishment of annual and long-term performance goals. While the Department reports performance data, the Department is not able to determine the impact of particular levels of funding on the performance of the SE program.

Evidence: ED Congressional Budget Justifications for FYs 2003-2008 http://www.ed.gov/about/overview/budget/budget08/justifications/index.html

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Department is currently working to developing an appropriate long-term measure and additional annual measures, including an efficiency measure, to assess the impact of the program and State performance.

Evidence: RSA recently completed a comprehensive review of the State level SE-related performance data to assist in strategic planning and improve the program's performance measures

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: RSA regularly collects timely and credible performance information, including information from key program partners, including data from three reporting systems that provide information on the State VR agency performance with respect to the SE program. Annual data on VR consumers who had a supported employment goal, including whether funds from the SE State Grants program were spent on the individual are collected through the RSA 911 database. The RSA-911 provides information on the services received, the cost of purchased services, and the outcomes achieved by such individuals when they exited the VR program. The RSA 911 database is reviewed for quality, including numerous edit checks, before the data are released for use. Although credible, preliminary analyses of State level data have highlighted inconsistencies in SE practices among States. The data are timely and for the last two years have been available within 5 months of the end of the fiscal year. In addition, the RSA-2 Annual Cost Report collects data on the SE program expenditures and carryover funds. In the past, national aggregate data was used to manage the program and a very limited set of State level SE-related data was used to monitor performance in carrying out the SE program. This year, RSA is using State level data to manage the program and monitor State performance.

Evidence: RSA 911 Case Services Report RSA-2 Annual Cost Report SF-269 Financial Status Report

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: As a part of their performance appraisal, Federal managers are rated on their efforts to improve the performance of the program. For example, the supervisor of the unit responsible for VR and SE State Grant programs is held accountable, in collaboration with other RSA managers as needed, for the development, analysis and evaluation of new and modified program products that support State teams and improve grantee performance; identifying and addressing program policy issues, and developing processes and materials related to the review of State Plans and the monitoring of VR agencies. VR State agencies are held accountable for cost, schedule and performance results through annual State reviews and periodic onsite monitoring conducted pursuant to Section 107 of the Rehabilitation Act. RSA reviews these activities during the periodic monitoring of VR agencies conducted pursuant to Section 107 of the Act. Furthermore, VR agencies are required to monitor the contracts entered into with service providers, including community rehabilitation programs providing supported employment services. In this way, VR agencies ensure the accountability of the service providers for the cost of and performance under the contracts.

Evidence: Education Department Performance Appraisal System Section 107 of the Rehabilitation Act

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Federal grant awards are obligated in a timely manner through the Department's Grant Administration and Payment System (GAPS) and made available to the State VR agencies. In general, VR agencies obligate their SE funds within the first year of availability. However, the authorizing statute provides an additional 12 months in which funds can be carried over for obligation. VR agencies submit annual Financial Status Reports (SF-269) for the SE program that provide information on the obligation and expenditure of program funds. VR agencies also report expenditures of annual and carryover funds on the RSA-2 Cost Report. RSA monitors the submission of these reports and also notifies the agency when reports are not submitted by the due date. The accuracy of the reports is verified through an assessment of internal consistency, the comparison of prior reports, and through the on-site review of agency records supporting entries in the reports. To ensure that VR agencies are adhering to requirements related to allowable costs in accordance with the provisions of OMB Circulars and federal regulations, RSA performs on going monitoring through desk and on-site reviews that meet the oversight requirements of the program.

Evidence: GAPS Status of Funds Reports and Annual Financial Status Reports (SF-269)

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: It is difficult to measure the cost effectiveness of the supplemental SE funds because they are used in combination with VR program funds to achieve program outcomes. The Department has developed a cost efficiency measure for this program, but the baseline and targets have not been established. As part of the larger VR program, the SE allotment is subject to the same fiscal requirements, including the requirement that States follow the same procurement procedures used for non-federal procurements. State procedures to ensure that the program is being operated in a cost effective and efficient manner are evaluated as part of the larger VR program through the RSA fiscal review processes and by single State audits. VR agencies may not spend more than 5% of their SE funds on administrative costs and limited types of expenditures that can be identified as administrative costs. These requirements are intended to achieve cost efficiency and effectiveness.

Evidence: In conducting a recent study, GAO visited four States and reviewed accountability procedures for SE program funds. Three of the States visited use performance-based contracting systems requiring SE service providers to demonstrate progress toward frequently monitored milestones in order to receive payment from the VR agency. One agency's automated information management system only authorizes payments for SE services to contracted providers at the contracted rate. (Stronger Federal Oversight Could Help Assure Multiple Programs' Accountability GAO-07-236)

Does the program collaborate and coordinate effectively with related programs?

Explanation: The State VR agency must arrange for "extended services" provided by other appropriate State agencies, private nonprofit organizations or other sources for the duration of employment. Funding sources for extended services must be identified early in the rehabilitation process. VR agencies are required to identify in the IPE the Federal, State, or private programs that will provide extended services (long term supports) and the State's basis for determining that continuing support is available. The State must make maximum use of services from public agencies, private nonprofit organizations, and other appropriate resources in the community to carry out this program. A designated State unit must enter into one or more cooperative agreements or memoranda of understanding with other appropriate State agencies and private and nonprofit organizations to ensure collaboration in a plan to provide supported employment services for individuals with the most significant disabilities. Each State plan supplement must demonstrate evidence of collaboration by and funding from relevant State agencies and private and nonprofit organizations to assist in the provision of ongoing SE services following the termination of short-term Supported Employment services and traditionally time-limited post employment services.

Evidence: 34 CFR Part 363.11(e), 34 CFR Part 363.50, 34 CFR Part 363.53

Does the program use strong financial management practices?

Explanation: The program has procedures in place to ensure that payments are made properly and financial information is accurate and timely. Draw down reports are reviewed to identify grantees that have drawn down an unusually large proportion of their grant funds or who have unusually large balances and have not been drawing down funds as expected. State agencies are required to submit an annual financial report (SF-269) that provides information on the obligation and expenditure of program funds. In addition, the RSA-2 Annual Cost Report provides information on program expenditures and funds carried over for expenditure in the next fiscal year. These documents are reviewed for completeness and information on each of these reports is compared to previous years. Staff flag any unusual activity and follow-up with the agency for clarification. Issues affecting the financial operations of the program are identified during on-site monitoring visits, during the development of the annual report, and through on-going dialog and communication between RSA and its partners. Through these activities, RSA can identify weaknesses before they have a negative impact on program operations requiring the development of corrective action plans. Additionally, those areas of strength or innovative approaches to financial management are shared with other grantees to improve the management of their programs. On-site monitoring of the State agency's fiscal management allows for review of written policies, agency reporting and cost allocation practices, fiscal data, use of carryover funds, audit resolution, and procurement and contracting processes. As a test of the agency's financial system, a sample of the agency's fiscal documents are tracked through the system.

Evidence: SMPID On-site Fiscal Review Guide RSA-269 Financial Status Reports RSA- 2 Cost Report GAPS Financial Reports.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The program has taken meaningful steps to address deficiencies discovered through State monitoring visits and analyses of RSA 911 data. For example, recent monitoring visits and analyses of State data revealed inconsistencies in how State agencies operationalize the "reasonable expectation" standard in determining whether a source for the provision of extended services has been identified. In response, the Department plans to issue guidance through information memoranda and by the web-inar process on several SE program issues. Guidance is being developed to clarify the "reasonable expectation" standard in program regulations at 34 CFR 363.11(g)(3)(i). RSA staff also plans to provide guidance on the expenditure of SE program funds and encourage State agencies to expend SE program funds for supported employment services first before using VR State Grant funds for this purpose. RSA plans to issue further guidance on the reporting of SE-related RSA-911 data.

Evidence: Supported Employment related data tables

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: As noted in response to question 3.1, three reporting systems provide information on practices and outcomes of State VR agencies with respect to the SE program, including the RSA-911 Case Service Report, the RSA-2 Annual Cost Report, and the SF-269 financial status form. RSA provides fiscal oversight by conducting on-site verification activities, implementing an audit resolution process, and other methods described in items 3.3, 3.4,. and 3.6. In FY 2007, RSA has implemented a new programmatic and fiscal monitoring process for State VR agencies. Monitoring will focus on areas of interest that are unique to each State, including the SE program and its outcomes. Preliminary data reviewed as part of the monitoring process describes the grantee's SE outcomes, compares those outcomes with other comparable State agencies and with a national average. Additional data related to SE outcomes and service provision can be developed if this is an area of interest for a particular grantee.

Evidence: On-site Fiscal Review Guide RSA 911 Case Services Report RSA-269 Financial Status Reports RSA- 2 Cost ReportGAPS Financial Reports

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: RSA collects and compiles a variety of SE related data from the RSA-911 and the RSA-2 that is reported by State VR agencies on an annual basis. Beginning in FY 2007, RSA will be including key SE data in each of the State annual review report that are issued after the close of the fiscal year. In addition, detailed State agency level SE data is included as part of the tri-annual monitoring report prepared by RSA. Each monitoring report will be posted on the RSA web site and available to any interested party. The national and State level SE national data will be posted on the RSA web site at the same time as the individual agency monitoring reports before the end of the FY 2007. This simultaneous posting ensures that identical data elements for individual grantees and for the nation are reported. In addition to this detailed ED data, RSA reports annually for each of the GPRA performance indicators that are used for the SE Program. The SE GPRA indicators are posted each year on the Department's web site. The most recent GPRA Report (FY2006) is available at http://www.ed.gov/about/reports/annual/2006report/program.html

Evidence: Supported Employment related data tables

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Department is currently working to develop an appropriate long-term program performance measure for inclusion in the FY 2008 Program Performance Plan

Evidence: The program does not currently have a long-term performance goal. See question 2.1

Does the program (including program partners) achieve its annual performance goals?

Explanation: Performance on the annual measure has increased significantly since the 1998 baseline level of 69 percent. The program has consistently met, and often exceeded its annual performance targets. Performance on this measure improved significantly after 2001 in part due to the fact that, beginning in fiscal year 2002, State VR agencies could no longer consider individuals who are working in non-integrated settings (e.g. extended employment or "sheltered employment") to have achieved an employment outcome under the VR program. In FY 2006, 94 percent of individuals with a supported employment goal achieved a competitive employment outcome. In addition, 92 percent of individuals who obtained a supported employment outcome achieved competitive employment. However, additional measures are needed to adequately assess the program's annual performance.

Evidence: FY 2007 Supported Employment Performance Plan U.S. Department of Education Visual Performance System

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: Cost effectiveness is currently being measured in the context of the VR State Grants program, including cost per employment outcome, cost per participant, and consumer expenditure rate. The Department has established an efficiency measure for the supported employment program, but performance targets have not been set.

Evidence: FY 2008 Performance Plan for the Vocational Rehabilitation State Grants program U.S. Department of Education Visual Performance System

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: This State grants program is the only direct Federal program targeting funds specifically for supported employment. Supported employment programs are typically carried out at the local level through State and local funds targeted to individuals with developmental disabilities and to individuals with mental illness. The Department of Health and Human Services' Administration on Developmental Disabilities provides funds to State Developmental Disability Councils and grants awarded under the Projects of National Significance program. Some of these funds may be used at the local level to provide supported employment services. In addition, the Center for Mental Health Service (CMHS) has awarded discretionary grants to demonstrate best practices in supported employment. For example, CMHS funded a project, Employment Intervention Demonstration Program, which conducted a randomized controlled trial of the effectiveness of SE for people with psychiatric disabilities. The project found that individuals with psychiatric disabilities who participated in the SE program were more likely to be competitively employed, work more hours, and earn more income than their counterparts who did not participate in a supported employment program. However, the project is not directly comparable to the State Supported Employment State Grants program. The outcomes of individuals who receive supported employment services can be compared to the outcomes of other VR consumers with significant disabilities. The sub-study of the Longitudinal Study of the VR Services Program found that one year after exiting the program, 84 percent of those achieving SE outcomes were still working, compared with 83 percent of other VR consumers with significant disabilities.

Evidence: Policy Brief - Supported Employment: A Best Practice for People with Psychiatric Disabilities http://www.aapd.com/News/empissues/060922rrtc.html; Final Report on the Vocational Rehabilitation Experiences Among Individuals Who Experienced a Supported Employment Outcome, April 2000. http://www.ed.gov/policy/speced/leg/rehab/eval-studies.html#vr

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A sub-study of the Longitudinal Study of the VR Service Program examined supported employment outcomes and compared the post-employment experiences of VR consumers with SE outcomes to those of VR consumers with other employment outcomes. One year after exiting the VR program, 84 percent of those achieving SE outcomes were still working, compared with 83 percent of other VR consumers with significant disabilities. Over half of SE consumers (53 percent) and other significantly disabled consumers (55 percent) received an increase in pay at some point between exiting VR services and the one-year follow-up. One year after exiting VR services, between 65 and 79 percent of SE consumers who held the same job they obtained through VR services continued to receive some type of support services. Consumer satisfaction with extended services was very high; three-fourths of all former SE consumers, including 83 percent of those with a primary disability of mental retardation, reported being "very pleased" with the services they were receiving. GAO reviewed accountability processes for four federal programs, including the Supported Employment State Grants Program. In particular they looked at program performance goals and the monitoring processes. They acknowledged the difficulty in independently assessing the performance of the SE program because SE grant funds are combined with funds provided through the VR State Grants program to provide services. GAO did not recommend changes in the accountability processes for the SE program.

Evidence: Final Report on the Vocational Rehabilitation Experiences Among Individuals Who Experienced a Supported Employment Outcome, April 2000. http://www.ed.gov/policy/speced/leg/rehab/eval-studies.html#vr;Stronger Federal Oversight Could Help Assure Multiple Programs' Accountability (GAO-07-236)