Detailed Information on the
College Assistance Migrant Program Assessment

Using the newly adopted efficiency measure, the Department will analyze CAMP program costs relative to the costs of other programs that help disadvantaged students go to college.

The Department will develop data collection strategies for the program's long-term college completion goal.

The Department will develop a reporting and auditing system to verify locally reported data and to ensure that performance data are being collected consistently across grantees according to established criteria.

The program will develop a more effective method of using outcome data to hold grantees accountable.

Measure: The percentage of CAMP participants completing the first year of their academic or postsecondary program.

Explanation:

Measure: The percentage of CAMP participants who, after completing their first year of college, continue their postsecondary education.

Explanation:

Measure: The cost per participant who completes his or her first year of postsecondary education and then continues that postsecondary education.

Explanation:

Measure: The percentage of CAMP participants who graduate from a postsecondary institution.

Explanation:

Is the program purpose clear?

Explanation: The purpose of CAMP is to provide the academic and financial support necessary to help migrant and seasonal farmworkers and their children successfully complete their first year of college.

Evidence: Higher Education Act, Title IV, Part A, Subpart 5, Section 418A

Does the program address a specific and existing problem, interest or need?

Explanation: Studies have identified the lifestyle of the migrant family and the associated mobility with such lifestyle as major obstacles to migrant students in obtaining the equivalent of a high school diploma, entry into post-secondary education, entering the military, or obtaining employment; Only 15% of migrant and seasonal farmworkers complete an 8th grade level of education; migrant and seasonal farmworkers earn an average of $5,000 annually.

Evidence: Literature Review prepared by RTI Center for Research in Education, October 30, 2003; No Longer Children, p. 10, Aguirre International, November 2000; ERIC Digest, No. ED 376-997.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: CAMP focuses on a unique migrant and seasonal farmworker population that is in need of financial assistance, academic counseling, and special supporting services. This population is difficult to identify and enroll in postsecondary education. The migrant and seasonal farmworker population has access to student financial aid, but no other program focuses on enrolling these students and addressing their additional special needs to help them complete the initial year of postsecondary education and provide follow-up referral services through completion of a college degree.

Evidence: Higher Education Act of 1965, as amended, and program 34 CFR Part 206 regulations' purpose and required activities for migrant and seasonal farmworker students.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no evidence that the structure of the program is a flawed design for the program. Program eligibility requirements target students who are most impacted by the migrant lifestyle and high mobility.

Evidence: Eligibility requirements

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: There is no evidence that this program is not effectively targeted. Departmental regulations require that services are provided from grantees to the intended target population.

Evidence: 34 CFR 206. Secs. 206.10(b)(2); 206.11; 206.3

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The Department is working to create long term performance measures through 2010.

Evidence:  

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The Department is working to create targets and timeframes for long-term performance measures.

Evidence:  

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The Department has established the following performance measures for the program: (1) Eighty-five percent of CAMP participants will successfully complete the first academic year of study at a postsecondary institution; (2) A majority of CAMP students who successfully complete their first year of college will continue in postsecondary education.

Evidence: PPMD - OESE CAMP FY 2005

Does the program have baselines and ambitious targets for its annual measures?

Explanation: 2002 baseline data reported that 80 percent of CAMP students completed their first year at a postsecondary institution and 75 percent of CAMP students who successfully completed their first year contined in postsecondary education. Increasing targets have been set for 2003, 2004, and 2005 for each indicator.

Evidence: PPMD - OESE CAMP FY 2005

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Through annual performance reports, the Department confirms grantee commitment to working towards the program's goal of assisting migrant and seasonal farmworker students to successfully complete their first academic year of college and to continue their postsecondary education.

Evidence: Submissions of annual grant performance reports, OME provided technical assistance, monitoring per annual schedule and ad hoc telephone monitoring, review of budget and expenditures, conducting post-award technical assistance/guidance, and an annual grant administration meeting with grantee, OME oversight of application processing, non-competing continuation (NCC) review process, and regional technical assistance presentations all serve to establish quality control management over grant operations, including the required signed assurances from grantees' / universities' chief or key executives that university submissions of progress/data will be reliable and accurate.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Although no program funds are appropriated or authorized for conducting independent program-level evaluations, grantees are required individually to implement strong evaluation plans to shape improvement and comply with program objectives.

Evidence: Grant application requirements.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The Department has not satisfied the first part of the questions because performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable peroformance information to assess (whether directly or indirectly) the impact of the Federal investment. However, the Department has satisfied the second part of the question because the Department's budget submissions show the full cost of the program (including S&E).

Evidence:  

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The Department is working to create long-term performance measures through 2010, to be reported in the 2006 PPMD Strategic Plan.

Evidence:  

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Department collects annual grant performance reports, which are used to determine whether grantee performance meets project objectives and program and GPRA goals. Data are self-reported and not subject to verification. However, where achievement or progress is not realized, grantees develop corrective action plans that OME staff oversees as to compliance and needs for technical assistance. To receive a "yes" on this question, ED must show it has a plan for standardizing and verifying local grantee data.

Evidence: Project files

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Currently, ED cannot demonstrate how federal managers and program partners are held accountable for program goals. However, the Department has initiated several efforts to improve accountability in its programs. First, ED is in the process of ensuring that EDPAS plans -- which link employee performance to relevant Strategic Plan goals and action steps ' hold Department employees accountable for specific actions tied to improving program performance. ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance. Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results.

Evidence:  

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Funds are obligated within the timeframes set out by Department schedules and used for the purposes intended.

Evidence: Evidence suggests that grantees are drawing down funds at an acceptable rate.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has not yet instituted procedures to measure and improve cost efficiency in program execution. However, as part of the President's Management Agenda, the Department is implementing "One-ED" -- an agency-wide initiative to re-evaluate the efficiency of every significant business function, including the development of unit measures and the consideration of competitive sourcing and IT improvements. A "yes" answer is likely once the One-ED process is applied to this program's relevant business functions.

Evidence:  

Does the program collaborate and coordinate effectively with related programs?

Explanation: Program staff has collaborated with Department of Labor management/supervisory staff to establish inter-program (DOL, ED) guidance on regulatory definitions of eligible "migrant and seasonal farmworker" CAMP participants. Department of Education staff from the TRIO program have provided guidance to their grantees regarding the coordination of available services with the CAMP program. Some CAMP grantees have established working networks with State and/or local Title I Migrant Education Program (MEP) offices in order to share resources for recruiting and enrolling eligible CAMP students.

Evidence: Grantee applications; Annual OME meeting with HEP and CAMP Directors; Annual OME meeting with MEP Directors

Does the program use strong financial management practices?

Explanation: Recent agency-wide audits have not identified deficiencies in the financial management of this program.

Evidence:  

Has the program taken meaningful steps to address its management deficiencies?

Explanation: While material internal management deficiencies have not been identified for this program, the program has put in place a system to identify potential problems.

Evidence: Program staff monitor excessive drawdowns of funds to prevent high-risk situations.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The Department awards grants on a point system that is based on selection criteria published in the Federal Register. Grant information for potential applicants is published in hard copy by the Department and posted on the Department's website.

Evidence: Federal Register Notice; EDGAR selection criteria published in the Notice and Application Packages are standards for developing slates, awards and successful applicants. However, program experience has demonstrated that grantees with established operational histories and knowledgeable project faculty--along with coordination and commitment from institutional organizations--have the most successful actual performance results.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The Department maintains information on grantee activities through annual performance reports, site visits, and technical assistance activities. Program staff reviews budget expenditures in the context of mid-year and end of year reporting.

Evidence: Annual performance reports; site visit reports; revised budgets submitted by grantees

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Annual performance data is collected from grantees and compiled by program staff. Data for the indicators have been reported in PPMD for 2001 and 2002. While performance data are not published or posted on the web, results are available to the public and presented to grantees at technical assistance and grant administration meetings.

Evidence: Official project files and program compilations of performance results

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The Department has not yet established a long-term performance measure for this program. However, the first two years' of performance data (FY 1999 cohort) show that progress has been made toward achieving the long-term CAMP goals that are being developed by the Department: students completing their first year of academic study and continuing in post-secondary education.

Evidence: PPMD - CAMP FY 2004 and FY 2005; ED Grant Performance Report tabulations of FY 1999, FY 2000, and FY 2001.

Does the program (including program partners) achieve its annual performance goals?

Explanation: While the Department has established annual performance measures, only baseline data has been collected and reported. Data for the first year of performance targets (FY 2003) will be reported later in 2004.

Evidence:  

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Department has established an efficiency measure for this program: The cost per training for CAMP participants who successfully complete their first year of college and continue their postsecondary education.However, targets and baselines are still underdevelopment.

Evidence:  

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: There are no programs with readily accessible comparable data. However, ED should develop a strategy to compare CAMP and post-secondary TRIO programs.

Evidence: (a) "AIP Briefs", Fall 1998, State University at Buffalo, p. 1; (b) Cal Poly State, CA, "Preliminary Retention Report 1990-2001"; (c ) "Principal Indicators" U.S. Department of Education/IES, January 2004, p. 4; (d) FY 1999 OME CAMP 3-year Grant Performance Report tabulations

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Although no program funds are appropriated or authorized for conducting independent program-level evaluations, grantees are required individually to implement strong evaluation plans to shape improvement and comply with program objectives. Some grantees provide copies to the Department as evidence of compliance.

Evidence: Grant application requirements.