Detailed Information on the
Migrant and Seasonal Farmworkers Assessment

Annually, review and analyze MSFW grantee data and RSA 911 data on migrants to ensure accuracy and to eliminate incorrect reporting by VR agencies without MSFW projects.

Calculate and analyze the efficiency measure data (cost per employment outcome) at the grantee level in order to establish targets.

Improve oversight and monitoring through teleconference reviews and on-site monitoring visits.

Use monitoring findings to improve program management and technical assistance.

Make data available to the public.

Publish a tailored annual reporting form for use by grantees in reporting uniform data.

Measure: The percentage of individuals served by MSFW projects and VR who were placed in employment outcomes.

Explanation:Annually, RSA-911 data is reported by States operating a MSFW project and serving migrant and seasonal farmworkers. In FY04 and FY05, 13 States operated a project.

Measure: Percentage of projects that report an average annual cost per employment outcome within a specified range (to be determined). (New measure, added February 2007)

Explanation:RSA will calculate project-level efficiency data by dividing the annual Federal grant amount by the number of successful employment outcomes reported by the project during the project period. RSA will use this data to establish targets for the percentage of projects whose costs per employment outcome is within a specified range.

Is the program purpose clear?

Explanation: The purpose of the Department of Education's (ED) Migrant and Seasonal Farmworkers (MSFW) program is to increase employment opportunities for migrant and seasonal farmworkers who have disabilities. Projects funded under the MSFW program focus on developing innovative methods for reaching and serving this population. These projects emphasize outreach to migrant and seasonal farmworkers, provide specialized bilingual vocational rehabilitation (VR) counseling, and provide for the coordination of VR services from other sources, including services for family members.

Evidence: Rehabilitation Act of 1973, section 304.

Does the program address a specific and existing problem, interest, or need?

Explanation: A 1987 ED study, "The Vocational Rehabilitation of Migrant and Seasonal Farmworkers," found an estimated population of 289,000 migrant and seasonal farmworkers with disabilities are in the Nation's labor force with another 60,000 dependents with disabilities. Another ED study completed in 1995, "Evaluation of Vocational Services Provided to Migratory and Seasonal Farm workers," found that although States may serve migrants and seasonal farmworkers who qualify for VR services through the VR State grants program, there does not appear to be any special effort to reach this population in States without a Migratory workers project. With few exceptions, the rehabilitation professionals interviewed in the study felt that existing VR activities failed to meet the VR needs of MSFWs with disabilities.

Evidence: Evaluation of Vocational Services Provided to Migratory and Seasonal Farm Workers, 1995; and The Vocational Rehabilitation of Migrant and Seasonal Farmworkers, 1987.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: MSFW duplicates the much-larger Vocational Rehabilitation State grant program, since both may serve the same target population. MSFW project funds are typically used to conduct extensive outreach tailored to specific needs whereas the VR State grant funds support the services provided to the migrant and seasonal farmworkers. Thus, this program is viewed as a supplement to traditional VR services provided through the VR State grants program.

Evidence: Rehabilitation Act, title I, VR State grant program authority; Budget requests from 2003 through 2007.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: MSFW has no major design flaws. Furthermore, there is no evidence that other approaches are more effective in providing services to migrant and seasonal farmworkers and improving their employment outcomes.

Evidence: Rehabilitation Act 1973, title I (VR State grants program) and Title III, section 304 (Migrant and Seasonal Farmworkers program).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: ED's Rehabilitation Services Administration (RSA) manages this program and has flexibility in directing priorities of how MSFW grant funds should be spent. Typically, however, RSA operates the program with a field-initiated approach, which allows grantees to design their specific activities to meet the needs of the MSFW within their State. MSFW resources target the right population - itinerant farmworkers with disabilities. To receive funding, grantees must describe how they will effectively serve migrant and seasonal farmworkers with disabilities. The Act also authorizes MSFW projects to serve family members when appropriate.

Evidence: Rehab Act, xxx.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: MSFW does not have long-term measures, but RSA will develop long-term measures that tie to the annual measure.

Evidence: Department of Education's PPMD; RSA-911 data; and Assisting Discretionary Grantees to Implement the Common Measures for Employment and Training Programs, RTI, December 2005.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: This program does not have long-term measures or targets.

Evidence: ED's PPMD.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The overall program goal is to increase employment opportunities for migrant and seasonal farmworkers who have disabilities. The program's sole measure compares the percentage of migrant and seasonal farmworkers with disabilities employed in States with MSFW projects with the percentage employed in States without projects. As part of the Job Training Common Measures initiative, the program has explored adopting new measures that (1) will better indicate participants' employment and earnings outcomes, as well as program efficiency, than its current measures and (2) facilitate comparisons with similar programs. The Department is assessing the results of a common measures study that measured the capacity of grantees to collect and report these data.

Evidence: Department of Education's PPMD; and RSA-911 data.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Using data from two years of grantee performance reports, ED established baselines and targets for incremental improvement for its annual measure. RSA will revise its targets as more data become available.

Evidence: Department of Education's PPMD; and RSA-911 data.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All MSFW projects work toward the annual goals of the program. Program application packages explicitly specify program goals, and RSA staff review grant application and progress reports to assess performance in relation to these goals.

Evidence: Application packages (Federal Register notices); Rehabilitation Act 1973, section 304.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Although a small program (approximately $2.5 million a year), the MSFW program used contractors to run two independent studies in the last 20 years -- in 1987 and 1995. Both studies used surveys, interviews, and reviews of administrative data of current grantees. The research questions sought to evaluate MSFW projects and State VR agencies to determine outcomes and best practices in serving this population. The results of the 1987 study helped open up the program to non-profits. And, RSA used the results of the 1995 study to modify the RSA 911 data collection instrument. Changes to the 911 data elements were made to ensure only those States with a MSFW grant reported on the farmwokers served through the grant. However, States without a MSFW grant still report on farmworkers served through a grant. Although the program is due for another evaluation, RSA should first focus on evaluating grantee data to create project profiles, best practices, and methods for addressing unmet needs.

Evidence: "The Vocational Rehabilitation of Migrant and Seasonal Farmworkers, Final Report;" E.H. White and Company, September 1987. "Evaluation of Vocational Rehabilitation Services Provided to Migratory and Seasonal Farmworkers, Final Report;" Pelavin Research Institute, June 1995.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget requests provide information on activities supported with program funds and include a discussion on the data reported by MSFW projects (and State agencies) related to the performance indicators. However, RSA has not systematically evaluated how program funding relates to program goals. In addition, in fiscal year 2003, the Administration launched an initiative to streamline job training programs and eliminate duplicative and overlapping programs. No funds were requested for Migrant and Seasonal Farmworker's program for fiscal years 2003 through 2007.

Evidence: Budget Requests for fiscal years 2003 through 2007.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: RSA needs to develop a long-term measure and impelement its efficiency measure. Further, RSA must determine whether it will adopt the job training common measures for this program and whether results from the program could be compared to those for other job training programs. If RSA elects to use the job training common measures, RSA must establish timelines to implement them.

Evidence: Department of Education's PPMD; RSA-911 data; Assisting Discretionary Grantees to Implement the Common Measures for Employment and Training Programs, RTI, December 2005.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: RSA collects timely, but not credible information from MSFW project partners. The projects report data through an annual performance report, and State VR agencies report through the RSA-911, the main VR State grants data collection. Data reported through the 911 are consistently problematic in that States report data for projects they do not have.

Evidence: Annual performance reports; RSA 911 data.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: ED's managers are subject to the EDPAS system which links employee performance to success in meeting the goals of the Department's Strategic Plan. In general, managers are provided individual performance agreements where they are given responsibility for achieving relevant action steps outlined in the Strategic Plan. These action steps and other items included in managers' performance agreements are designed to measure the degree to which a manager contributes to improving program performance. Specifically, the Chief of the unit that administers this program has EDPAS criteria holding him accountable for the program. The criteria for the Chief include:1. Develops strategies and methods to ensure the timely and accurate collection of program indicator/performance measure data. 2. Meets regularly with assigned staff to discuss strategies and methods to improve performance and discuss the feasibility of developing crosscutting initiatives. 3. Provides leadership to improve alignment of discretionary grant investments with monitoring findings. 4. Provides leadership to assigned staff to revise or establish performance measures to show improvements in employment outcomes for people with significant disabilities. 5. Provides leadership to assigned staff for the effective management of grant performance reports, grant close outs and timely liquidations of grant funds.

Evidence: Internal records; EDPAS

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Federal funds are obligated in a timely manner. This program is covered under the Single Audit Act. Recipients that receive an aggregate of $500,000 or more in federal funds are required to submit to ED an annual independent audit. The purpose of the audit is to demonstrate that the entity has a financial system in place and that federal funds are spent and accounted for properly, in accordance with OMB Circular A-133.

Evidence: Education Department General Administrative Regulations 34 CFR 80.26.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has not yet used grantee data to compute its efficiency measure of cost per employment outcome.

Evidence:

Does the program collaborate and coordinate effectively with related programs?

Explanation: At the local level, MSFW projects collaborate extensively with the larger VR State grants program. Projects reach out to eligible participants while VR provides the actual rehabilitative services. Local projects extensively collaborate with other social service providers, like schools and community clinics. At the federal level, RSA participates in the Federal Migrant Interagency Committee to share information and develop strategies to improve the coordination and delivery of services to migrant or seasonal farmworkers with disabilities. As a result of this collaboration, RSA has adopted common definitions and terms for use with Federal migratory worker programs. RSA also supports the National Alliance For Migrant and Seasonal Farmworker Vocational Rehabilitation and their annual conference that affords the grantees an opportunity to showcase their work, share information about the results and outcomes of their programs, and network for future collaboration. This support resulted in recommendations being made for RSA's use during the current reauthorization of the Rehabilitation Act. In addition, the statute requires that RSA administer this program in coordination with other programs serving migrant and seasonal farmworkers, including programs under title I of the Elementary and Secondary Education Act of 1965 (20 USC 6301 et seq.), section 330 of the Public Health Service Act (42 USC 254b), the Migrant and Seasonal Agricultural Worker Protection Act (29 USC 1801 et seq.), and the Workforce Investment Act of 1998. This coordination is working towards determining how the common measures may been implemented and uniform data collected.

Evidence: Annual conference for the National Alliance For Migrant and Seasonal Farmworker; participation in Federal Migrant Interagency Committee.

Does the program use strong financial management practices?

Explanation: The program follows Departmental financial management guidelines set forth in the Education Department General Administrative Regulations (EDGAR). Program staff monitor projects for excessive drawdown activity using the Department's Grant Administration and Payment System (GAPS) at six and nine month intervals, and prior to the obligation of a continuation award. GAPS requires grantees with excessive drawdowns to resolve the issue within two weeks after being notified. Grantees with poor audit findings and in violation of EDGAR financial management requirements must submit corrective action plans for review and approval. The program office sets a date by which the issue will be resolved. Staff will enforce any of the options available under EDGAR 74.62 (Enforcement) and EDGAR 80.12 (Special grant or sub-grant conditions for "high-risk" grantees) as appropriate. In addition, program staff monitor balances to ensure that funds do not remain unspent by the end of a budget period. Unspent balances, and any incomplete activities associated with those balances, are addressed with the grantee for appropriate resolution.

Evidence: Office of the Chief Financial Officer (OCFO)/Grants Policy notifies program staff of excessive draw downs by grantees; EDGAR.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: RSA does not have a system for identifying and correcting program management deficiencies and has not taken a systematic approach to making necessary corrections within an agreed upon timeframe. RSA needs to make the following improvements: 1) Fix the persistent reporting problems. States continue to report migrant and seasonal farmworkers being served by a project in States without a project. In order to assess the performance of the MSFW projects, steps must be taken to validate data from the States. And this data should be made available to the public. 2) Implement its efficiency measure and develop targets. 3) Provide better technical assistance to current grantees before conducting another evaluation. RSA should review historical data to create successful project profiles, best practices, and methods to address areas of unmet need. 4) Do occasional on-site monitoring visits.

Evidence: "The Vocational Rehabilitation of Migrant and Seasonal Farmworkers, Final Report;" E.H. White and Company, September 1987. "Evaluation of Vocational Rehabilitation Services Provided to Migratory and Seasonal Farmworkers, Final Report;" Pelavin Research Institute, June 1995.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: All new grants are awarded through a competitive process that includes an announcement in the Federal Register and application review by panel of external peer reviewers. However, many of the same states have received continued funding over the past 28 years.

Evidence: Federal Register application notices bi-annually.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program does not monitor project activities. A 1987 study found, "Grantees did not consider RSA monitoring or administrative oversight adequate to compel reporting; submission of reports was, essentially, voluntary." However, RSA does monitor projects' finances. For example, grantees with excessive drawdowns must resolve the issue within two weeks after being notified. Unspent balances, and any incomplete activities associated with those balances, are addressed with the grantee for appropriate resolution. Also, staff monitor projects through quarterly telemonitoring in accordance with RSA's monitoring plan.

Evidence: RSA monitoring plan; and ED's GAPS.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: RSA collects grantee performance data on an annual basis, but does not make the data readily available to the public.

Evidence:

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: RSA must establish a long term target for the current measure of "the percentage of individuals served by projects and VR who had and employment outcome."

Evidence: ED's PPMD.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program only has two years of data that address targets to assess its goals. Until RSA solves its data quality issues, it's data accuracy is questionable.

Evidence: RSA-911.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: RSA has developed, but not yet implemented, an efficiency measure. Consistent with the common measures, the Migrant and Seasonal Farmworkers program has established a "cost per employment outcome" efficiency measure. RSA will calculate project-level efficiency data by dividing the annual Federal grant amount by the number of successful employment outcomes reported by the project during the project period. RSA will use this data to establish targets for the percentage of projects whose costs per employment outcome is within a specified range.

Evidence:

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: However, the percent of individuals served by a MSFW project and VR who were placed into employment has consistently been higher than the percent of farmworkers achieving successful outcomes in States without projects. In fiscal year 2005, the 13 States with projects placed 69 percent of the individuals served into employment. VR placed 61 percent of its migrant and seasonal farmworkers into employment. This percentage does not include the 340 migrants served by VR incorrectly reported as being served by a project. A Small Extent is warranted in light of the persistent data problems which casts some doubt on the validity of these results.

Evidence: RSA-911.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Evaluation results are mixed. According to the 1995 study, "State VR agencies that received an MSFW grant were consistently more aggressive in serving MSFW clients than VR agencies that had never received an MSFW grant. (page ix)" However, the same study found there has been little success of serving migratory farmworkers who cross state lines and few State VR agencies fold MSFW project activities into their work after a grant expires. In addition, the 1987 study found "RSA has allocated the limited funding for the [MSFW] program wisely and has increased national levels of service to handicapped [sic] MSFW's dramatically. However the Section 312 [MSFW] program has not yet succeeded in establishing a national service delivery system allowing MSFW's the same level of rehabilitation services afforded to the general population, nor has it encouraged innovative approaches to service delivery. (page 198)"

Evidence: "The Vocational Rehabilitation of Migrant and Seasonal Farmworkers, Final Report;" E.H. White and Company, September 1987. "Evaluation of Vocational Rehabilitation Services Provided to Migratory and Seasonal Farmworkers, Final Report;" Pelavin Research Institute, June 1995.