Detailed Information on the
Developing Hispanic-serving Institutions Assessment

Developing strategies to use efficiency and performance data for program improvement purposes.

Making grantee performance data available to the public in a transparent and meaningful manner.

Completing the study on the financial health of minority serving institutions of higher education and utilizing the results to validate program performance measures and improve program performance.

Measure: Graduation: percentage of first-time, full-time degree-seeking undergraduate students enrolled at two-year Hispanic-Serving Institutions who graduate within three years of enrollment.

Explanation:

Measure: Graduation: percentage of first-time, full-time degree-seeking undergraduate students enrolled at four-year Hispanic-Serving Institutions who graduate within six years of enrollment.

Explanation:

Measure: Persistence: percentage of first-time, full-time degree-seeking undergraduate students who were in their first year of postsecondary enrollment in the previous year and are enrolled in the current year at the same Hispanic-Serving Institution.

Explanation:

Measure: Enrollment: number of full-time degree-seeking undergraduate students enrolled at Hispanic-Serving Institutions.

Explanation:

Measure: Cost per successful outcome: appropriation for given year divided by the number of graduates, doctorates, first professional, and masters degrees awarded

Explanation:

Is the program purpose clear?

Explanation: The Developing Hispanic Serving Institutions (HSIs) program is designed to assist eligible Hispanic-serving institutions of higher education to improve and expand their capacity to serve Hispanic and low-income students. More specifically, to "expand educational opportunities for, and improve the academic attainment of, Hispanic students; and expand and enhance the academic offerings, program quality, and institutional stability of colleges and universities that are educating the majority of Hispanic college students and helping large numbers of Hispanic students and other low-income individuals complete postsecondary degrees."

Evidence: Section 501(b) and 501(c) of Title V of the Higher Education Act (HEA).

Does the program address a specific and existing problem, interest, or need?

Explanation: The Bureau of Census official population estimates indicate that the Hispanic community is the Nation's largest minority community representing 13.3 percent of the population. Data indicate that Hispanics have made gains in several key education areas in the past 20 years, but despite these gains, gaps in academic performance between Hispanic and non-Hispanic White students remain. Despite impressive increases in college enrollment and degree attainment, Hispanics continue to lag behind their non-Hispanic peers in overall educational attainment. In 2001, only 21.7 percent of all Hispanics in the age group 18-24 years were enrolled in degree-granting institutions, versus 39.3 percent of all non-Hispanic white students. In 2001-2002, Hispanics earned only 6.4 percent of the bachelor's degrees, 4.6 percent of the master's degrees, and 3.2 percent of the PhDs awarded in the United States compared to non-Hispanic white students earning 74.2 percent of the bachelor's degrees, 68 percent of the master's degrees, and 60.9 percent of the PhDs.

Evidence: Bureau of the Census; The Hispanic Population in the United States: March 2002; Status and Trends in the Education of Hispanics (NCES 2003-008) pgs xi, 1, 95, 107); NCES Digest of Education Statistics, 2003 (tables 264, 267, and 270)

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Title V Developing HSIs program is designed to provide discretionary grant (five-year) funding directly to eligible institutions to enhance educational opportunities for, and improve the academic attainment of, low-income Hispanic students and to expand and enhance the institutions' academic offerings, program quality, and instructional stability. The focus and funding approaches of the Title V program are unique. While there are other federal programs that target low-income Hispanic students, these programs have different purposes and do not give the institution the flexibility to design projects that will meet individual institutional needs. These programs are designed to attract graduate students specifically to the fields of science and engineering [Department of Energy (DoE), National Institutes of Health (NIH), National Science Foundation (NSF), Department of Agriculture (USDA)], promote careers in community development and post-baccalaureate education (HUD, NSF, USDA, NIH), provide funds directly to individual students or faculty (HUD), support research [National Aeronautics and Space Administration (NASA), NSF, USDA] or assist elementary and middle schools (NASA, NSF).

Evidence: An eligible institution is defined in Section 502 of the HEA. Examples of programs at the Federal level that target HSIs include DoE's Advanced Research program, NIH's Research Infrastructure in Minority Institutions program, NSF's Math and Science Partnership program, USDA's Education Grants program, HUD's HSI Assisting Communities Program, and NASA's Minority Research and Education Program.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: While no major flaws have been identified, the program may still benefit from certain statutory changes. The statute requires that HSIs provide assurances that not less than 50 percent of an institution's Hispanic students are low-income individuals. Rather than using a program-specific low-income measure, the program could use the number of Hispanic students who receive Pell Grants. Pell eligibility is based on need and may be a more accurate proxy for determining low-income eligibility in the HSI program. The Pell Grant program uses the same student financial need data that an institution must provide when meeting the "eligible institution" criteria. The resulting definition of a HSI would be an institution of higher education that is both an eligible institution and has an enrollment of undergraduate students that is at least 25 percent Hispanic. Additionally, the program would be improved if the "two-year wait out" were eliminated. Currently, a HSI is not eligible to secure a subsequent grant award until 2 years have elapsed since the expiration of the institution's most recent grant award. This has eliminated some qualified institutions that might otherwise have been awarded funding and has led to decreased service to students. Removing this provision would allow the institutions that would qualify for another grant to better serve their students.

Evidence: Sections 504(a)(2) and 502(a)(5)(c) of Title V of the HEA.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program is well targeted to reach Hispanic and low-income students attending institutions defined as HSIs. The program statute specifies that at the time of application, eligible institutions must have an undergraduate enrollment of full-time equivalent students that is at least 25 percent Hispanic students. In addition, institutions must provide assurances that not less than 50 percent of the institution's Hispanic students are low-income individuals. Grants are awarded to expand educational opportunities for, and improve the academic attainment of, Hispanic students and to expand and enhance the academic offerings, program quality, and institutional stability of colleges and universities that are educating the majority of Hispanic college students. In the 2001-2002 reporting period, HSI grantees focused nearly half of their expenditures (44 percent) on grant activities related to student services and outcomes. Grant activities that promoted academic quality accounted for 40 percent of all Title V expenditures. Substantially smaller percentages covered activities that enhanced institutional management (9 percent) and fiscal stability (7 percent).

Evidence: Section 502 of Title V of the HEA. Title V grant applicants are required to self-report data on eligibility requirements. Reported assurance data are subject to audit verification. A Profile of the Title V Developing Hispanic-Serving Institutions Program 2001-02, September 2004.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The previous HSI performance measures tracked the percentage of institutional project goals that were successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. In conducting this PART review, it was determined that these measures may not effectively measure program performance. Therefore, ED has replaced these measures with three new outcome measures reflecting enrollment, persistence, and graduation rates at HSIs. These new measures have also been established for the Title III Strengthening Institutions Program, Strengthening Historically Black Colleges and Universities, and Howard University resulting in consistent measurement across ED programs that focus on strengthening institutions programs that serve large minority populations. All of these measures, with the exception of the persistence measure, have been established for the Strengthening Historically Black Graduate Institutions program as well.

Evidence: FYs 2005 and 2006 Program Performance Plans. Data for the measures aggregated across all of the Title III and Title V programs and disaggregated by individual program are derived from the Title V electronic performance measurement system. Data for the new measures are derived from electronic annual performance reports for Title V grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitious targets and timeframes have been established for ED's new measures that focus on enrollment, persistence, and graduation rates at HSIs, using program data reported currently by institutions in IPEDS. These targets have been established in consideration of the HSI program history, the comparison of the HSI performance with the performance of all Title IV institutions and similar programs, and the demographics of the HSI recipients. HSI program goals for these measures are projected through 2011. In addition, ED had set ambitious targets and timeframes through 2011 for its former long-term measures in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes.

Evidence: Please see measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The previous HSI annual performance measures tracked the percentage of institutional project goals that were successfully completed. In conducting this PART review, it was determined that these measures may not effectively measure the performance of the overall program. Therefore, ED has replaced these measures with two new outcome measures reflecting persistence and graduation rates at HSIs. These new measures have also been established for the Title III Strengthening Institutions Program, Strengthening Historically Black Colleges and Universities and Howard University, resulting in consistent measurement across ED programs that focus on strengthening institutions programs that serve large minority populations. All of these measures, with the exception of the persistence measure, have been established for the Strengthening Historically Black Graduate Institutions program.

Evidence: FYs 2005 and 2006 Program Performance Plans; Title V Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for Title V grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ED has established baselines and targets for its new annual measures focusing on persistence and graduation rates at HSIs using program data reported by institutions in IPEDS. These targets have been established considering program history at HSIs and the comparison of the HSI performance with the performance of all Title IV institutions and similar programs. In addition, baselines and targets have been established for the program's current annual measures: improving academic quality, institutional management and fiscal stability, and student services and student outcomes of participating institutions.

Evidence: Please see measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: In response to findings and recommendations in the June 2000 Inspector General-issued audit on Title III programs and to better measure the success of Titles III and V, the Department conducted an extensive outreach effort in fiscal year 2001 to solicit recommendations for the Titles III and V program indicators and develop a new performance measurement system. The Title III and V grantee communities were informed and consulted in a number of ways, including program focus group discussions, meetings with project directors, regional grantee meetings, national conference calls, and outreach to college presidents and associations. As part of the application process all partners commit to and work toward the specific goals of the program. The goals are based on information that all postsecondary schools already measure and use for a number of purposes. Grantees must provide a five-year plan that describes their plans for accomplishing the goals and objectives of their specific projects. These projects are aligned with the purposes of the program and the program goals. Many grantee projects involve areas such as community outreach programs, tutoring, counseling, student service programs designed to improve academic success, faculty development, curriculum development, new academic programs, faculty fellowships to assist in attaining advanced degrees in the fellow's field of instruction, and improved classroom facilities and facilities for Internet and other distance learning capabilities, all of which are directly related to the new performance measures. In addition, annual performance reports are required for all grantees.

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). EDGAR 34 CFR 74.51. Annual and final reports from the Title V Performance Measurement System collect data on academic quality, institutional management and fiscal stability, and student services and student outcomes. The Performance Measurement System for Title V is designed to acknowledge the diversity of institutions that are served, use institutional data from IPEDS for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. The purpose of the analysis would be to determine whether the financial status of the institutions is improving or becoming worse and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions financial health. Mathtech, Inc. conducted the last performance measurement study of ED's Title III Programs in 1995??which at that time included HSIs. The report presents the results of a survey of all 1995-96 Part A and Part B Title III grantees as well as in-depth case studies conducted at 19 institutions. The study indicated that institutions used Title III grants primarily to strengthen academic programs rather than on institutional management. The study also found that most Title III schools were not in severe financial difficulty.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program (Mathtech, Inc., 1995) www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable performance information to assess (whether directly or indirectly) the impact of the Federal investment. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). ED's 05 integrated budget and performance plan includes the program's annual and long-term goals.

Evidence: NA

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ED has identified strategic planning deficiencies and taken meaningful steps to address these deficiencies. ED has refined its annual and long-term performance measures, and has added an efficiency measure. Further, ambitious annual and long-term targets have been established based on program and benchmarking information. ED's new performance measures are based on data already collected in IPEDS and provide for improved comparison of information across different programs. ED is also planning to review its current annual performance report and ensure that the information requested provides quality data and evidence of the success of the program, as well as of success in the individual projects.

Evidence: ED has created a new efficiency measure for the program and revised the annual and long-term performance measures to standardize the HSI measures with performance measures for other similar programs within OPE. This will enhance strategic planning, reporting, and program management. Further, the data for the performance measures will come from IPEDS data collected by NCES, which increases the credibility and reliability of the data, as well as ensuring consistent data availability.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ED requires grantees to submit regular and timely information via electronic means (i.e. annual performance reports, interim reports, and final reports). Information on grantee compliance with program requirements and objectives is collected annually and continuation awards are dependent upon a determination of substantial progress being made. The information collected is used to improve program performance and to provide technical assistance to grantees to assist them in meeting their proposed goals and objectives. In addition, ED has taken meaningful steps to address deficiencies highlighted in a recent GAO report entitled, "Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). GAO recommended that ED take steps to ensure that monitoring and technical assistance plans are carried out and targeted to at-risk grantees and that the needs of the grantees guide the technical assistance. One of those steps was to implement e-Monitoring, a new software tool that enables grant program staff to better monitor grants and assess risk. Program staff use this information to target technical assistance to grantees. Further, ED has revised its annual and long-term performance measures to more directly focus on outcomes and reflect the purpose of the program. ED uses program information to better manage the program and to make modifications, if needed, to improve program performance. In June 2005, ED met with partners to discuss regulating the definition of a program graduate. This action is in response to information learned about the wide variety of ways institutions define a "graduate."

Evidence: ED launched an electronic monitoring system (e-Monitoring) in December 2004. Program managers conduct comprehensive reviews of grantee interim reports and annual performance reports to ensure that performance measure standards are being met. Applications are reviewed in accordance with Federal statutes to ensure the planned activities are reflective of performance measures. Program monitors use interim, annual and final performance reports for targeted site assistance visits and monitoring visits as assurance that activity goals are completed as proposed or aligned to more effectively achieve intended goals. Actions such as high-risk designation leading to decreased funding, establishing special or additional reporting procedures or cancellation of grants may be imposed. Program staff places a hold on grantee's federal funds in cases of non-compliance or loss of accreditation. Further, staff approves or denies requests for budgetary changes that may affect the scope of approved projects. Based upon oversight information, institutions may have continuation funding reduced.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: While ED has initiated several efforts to improve accountability in its programs, it is not clear that Federal managers of this program are held accountable for program performance. Across the Department, ED is in the process of ensuring that "EDPAS" plans -- which link employee performance to relevant Strategic Plan goals and action steps - hold Department employees accountable for specific actions tied to improving program performance. ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance. Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results.

Evidence: The President's Management Agenda scorecard (Human Capital and Budget & Performance Integration initiatives) notes ED's efforts to improve accountability. The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Federal funds are obligated within the timeframes set out by ED schedules and used for the purposes intended. In FY 2006, ED plans to make the NCC awards by the end of April, this is two months earlier than in past years. This improvement gives institutions funding information sufficiently early to enable them to plan effectively and begin hiring processes. Further, to ensure that funds are properly managed, ED monitors grantees' financial records using GAPS and quarterly reports from the Office of the Assistant Secretary for OPE to alert staff to any grantees' excessive drawdown or inactivity of Federal funds. In addition, to ensure that funds are properly managed, ED has strengthened its financial management practices by implementing e-Monitoring. ED e-Monitoring is a new software tool that enables grant program staff to better monitor their grants and helps to prevent potential fraud, waste, and abuse.

Evidence: Title V Performance Measurement System; annual spending plans and program and partners' financial records; ED e-Monitoring was implemented December 22, 2004; and GAPS.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ED has developed an efficiency measure based on the number of students that successfully complete their education at an institution receiving funding from the HSI program. The measure is determined by dividing the program's annual appropriation by the number of students graduating in that year. A similar measure (assessing the Federal cost of the number of students graduating) is established for the HBCUs program and the HBGIs program, as well as for Howard University. This change may enable ED to assess program performance across similar types of missions and performance. In addition, ED has other procedures in place to achieve efficiencies and cost effectiveness. New competitively-sourced technical analysis support for the Developing HSIs program provides expert professional analytical skills that augment the capability of program staff. The contracting source is used to carry out administrative tasks and allows the program staff to focus more time on program review activities. The HSI program has benefited from IT improvements that support the electronic submission of information from all competition applicants, resulting in cost and time savings for both ED and the HSI.

Evidence: Data for FY 2003 show that the Federal program cost per graduating student is $1,058. (FY 2003 appropriation for Developing HSIs program: $92.4 million. Total number of graduating students: 87,326). The program's administrative contracts provide database, technical assistance, and report analysis support to help reduce the amount of time needed to accomplish administrative tasks. The technical analysis support evaluates data for grantee annual reports, and synopsizes data into an annual program report. Currently, reports are being reviewed to identify areas for improvement.

Does the program collaborate and coordinate effectively with related programs?

Explanation: ED participates in a Memorandum of Understanding with other Federal agencies providing Title V funding to educate potential grantees about pre-applications requirements. In addition, the Developing HSIs program collaborates and coordinates effectively with the HEA Title III programs??Strengthening Institutions, Strengthening Alaska Native and Native Hawaiian-serving Institutions, Strengthening Tribally Controlled Colleges and Universities, Strengthening Historically Black Colleges and Universities, and Strengthening Historically Black Graduate Institutions program. If a HSI receives funding under any of these programs, it cannot receive funding under the Developing HSI program, thus allowing resources to be managed more efficiently. Currently, these programs share common performance goals to improve academic quality, fiscal stability and institutional management, and student services and outcomes of eligible institutions and common goals and data sources (IPEDS) now that the new measures of enrollment, persistence, and graduation are in place.

Evidence: Title V is an active partner with the Department of Housing and Urban Development (HUD), the National Aeronautics and Space Administration (NASA), the Department of Agriculture (USDA), and the National Institutes of Health (NIH), conducting pre-application workshops to those interested in applying for Title V grant funding. Four pre-application workshops were presented in 2004 and 2005. Planning has begun for 2005-2006 presentations. To ensure coordination with other related ED programs, program managers have established parallel policies, administrative procedures and performance measurement and assessment tools. Additionally, National Project Directors' Workshops are held for these programs. Special rules outlined in Title III Sections 312(g), 316(d)(3), 317(e), 324(g) and Title V Section 505 of the HEA.

Does the program use strong financial management practices?

Explanation: Because of recent audit reports that indicate ED's failure to adequately monitor minority serving institutions grantees for their compliance with grant administration requirements, ED/OPE has implemented an e-Monitoring system designed to enable program staff to further mitigate against potential fraud, waste, and abuse. It allows staff to classify each grant into categories based on grantee performance, and maintains electronic records of phone, correspondence, and e-mail communications with grantees. Using ED's e-Monitoring and GAPS data along with other reports allows program officers to observe all funds applications made by an institution. Further, these systems are used to flag institutions determined not to have applied funds appropriately and to withhold funding, if necessary. ED continuously monitors grantee management and administration of Federal funds using GAPS and institutional financial reports.

Evidence: Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). All Title V funded grantee institutions are documented in GAPS and e-Monitoring systems.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: For years, ED had utilized a paper collection instrument to obtain performance data and reports. The Department has improved this process by replacing the paper collection instrument with an electronic performance reporting system to collect and report information more efficiently. In addition, each principal office must submit annual grant monitoring plans. Program offices must address the purpose, goals and objectives of the program; GPRA performance indicators and data; grantee performance; opportunities for technical assistance; and grantee or program risk factors. In response to an OIG Alert Memorandum, which found that there was confusion on the part of grantees regarding the Title V institutional eligibility requirements, ED has taken several steps to ensure grantees have clear guidance on this issue. ED has notified current grantees and applicants about the institutional eligibility requirements for this program and has modified the program application to ensure that the requirements are clear. Furthermore, recent introduction of the e-Monitoring enables program managers to have the data they need to more effectively manage and measure the progress of program participants. Results derived from individual reviews of interim and annual reports may precipitate a site visit to an institution. Program officers are required by their EDPAS performance standards to make several technical assistance visits per year.

Evidence: The new Performance Measurement System for Titles III and V is designed to acknowledge the diversity of institutions that are served, use institutional data from the Integrated Postsecondary Education Data System for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format. OIG Alert Memorandum and subsequent correspondence (February to September 2004). Section 6.4.4 of the Handbook for the Discretionary Grants Process discusses the approaches currently in use, and serves as a guide and training resource for program staff. Implementation of e-Monitoring, e-Grants, and use of GAPS. On average, approximately 14 program visits are planned each year.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Independent peer review panels are used to score and rank all applications.

Evidence: Program funds are used to pay for the peer review process. 100% of grants are subject to review.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: New procedures have been developed for improving the monitoring of expenditures based on IG concerns with the Title III Aid for Institutional Development programs and Title V institutional eligibility requirements. GAPS and ED/OPE's e-Monitoring systems are used to ensure that all related grant financial management practices are followed. Program officers provide continual oversight and coordination of activities of HSIs via on-site technical and continuous electronic communication (email, fax, and telephonic).

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). OIG Alert Memorandum and subsequent correspondence (February to September 2004). In addition to increasing efforts at on-site monitoring, program staff continues to review all performance reports that grantees are required to submit. In addition, program oversight includes documentation of grantees' use of funds, email communications, and project directors' meetings. This review process has resulted in the adjustment of grantee work plans that were not allowed within the scope of the program.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: This program collects and compiles data from performance reports and shares this information with grantees. Performance data are collected annually through the IPEDS survey by NCES and through annual performance reports submitted by HSIs to the program office. These data are available on the Department's website. Additionally, GPRA data are now reported in several formats, including on the ED website. ED has completed the first systemic profile report in September 2004. Efforts are underway to analyze the most recent performance data for future reports and to increase the timeliness of making the data available to the public.

Evidence: IPEDS data can be accessed at the College Opportunities On-line Website at http://nces.ed.gov/ipeds/cool. The Developing HSIs program profile report (Caliber Associates, September 2004) was sent to grantees and it will be available on the IDUES website shortly.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data on long-term measures are collected through the electronic Annual Performance Report for Title V programs. The antecedent measures capture the percentage of institutional project goals that are successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. The program exceeded all of its annual targets in 2002 and two of its targets in 2003. Thus, the program appears to be on target to achieve the long-term goals, which have been projected through 2011. In FY 2002, grantees completed 88% of their academic quality projects, 78% of their institutional management and fiscal stability projects, and 86% of their student services and student outcomes projects. The three targets were set at 75%. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. However, ED has replaced all of these measures with three new outcome measures reflecting enrollment, persistence, and graduation rates at HSIs compared with that of all Title IV institutions. These measures have also been established for the Strengthening HBCUs program, Strengthening Institutions program, and Howard University. These measures, except for the persistence measure, have also been established for the Strengthening HBGIs program. This change will result in consistent measurement across the ED programs that focus on strengthening institutions that serve large minority populations. HSI program goals for these measures are projected through 2011. IPEDS data show that enrollment at HSIs (defined as number of enrolled full-time undergraduate students for all HSI grantee institutions) increased by 5.6% from 2001 to 2002 compared to 5.8% for all Title IV institutions. Persistence at HSIs (defined as the median retention rate for full-time, first-time undergraduate students at HSI grantee institutions) was 67% for the fall of 2003 compared to 67% for all Title IV institutions. The graduation rate at 4-year institutions (defined as the percentage of full-time undergraduate students completing a 4-year degree within six years of entry for all 4-year HSI grantee institutions) was 33.7% for the fall of 2002 compared to 38.2% for all Title IV institutions. The graduation rate at 2-year institutions (defined as the percentage of full-time undergraduate students completing a 2-year degree or certificate (or one of less than 2 years' duration) within three years of entry for all 2-year HSI grantee institutions) was 35.1% compared to 29.5% for all Title IV institutions.

Evidence: FYs 2005 and 2006 Program Performance Plans; Title V Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for Title V grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program (including program partners) achieve its annual performance goals?

Explanation: HSI performance, as measured by the program's antecedent annual measures, has been generally positive. Grantees have exceeded their completion targets for three separate measures of activity. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. In addition, adequate information is not available to determine the program's performance on its new annual measures: persistence and graduation rates.

Evidence: Data for the new measures reflecting persistence and graduation rates (the same as the long-term measures) are derived from electronic annual performance reports for Title V grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: An efficiency measure and baseline data were recently developed, but no targets or trend data are available to demonstrate improved efficiencies. The efficiency measure is based on the number of students that successfully complete their programs. The measure is determined by dividing the program's annual appropriation by the number of students graduating in that year. The same measure has been established for the HBCUs program and the HBGIs program, as well as for Howard University. This change will enable ED to assess program performance across similar types of institutions.

Evidence: Data for FY 2003 show that the Federal program cost per graduating student is $1,058. (FY 2003 appropriation for Developing HSIs program: $92.4 million. Total number of graduating students: 87,326).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Although there are many federal, state, local, and private efforts that provide funding/services to HSIs and their students that may be comparable, outcome data are not available on these programs to provide the basis for meaningful comparison. However, ED's establishment of similar performance measures for its programs that focus on strengthening institutions may eventually make comparisons of these kinds of programs possible.

Evidence: NA

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of the Developing HSIs program have been conducted. However, the Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. A purpose of the analysis would be to determine whether the financial status of the institutions is improving or worsening and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions' financial health.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007.