Detailed Information on the
Strengthening Institutions Assessment

Completing the study on the financial health of minority-serving institutions of higher education and utilizing the results to validate program performance measures and improve program performance.

Developing strategies to use efficiency and performance data for program improvement purposes.

Establish targets for the efficiency measure.

Review the persistence and graduation rates at two-year institutions and, in collaboration with ED's Office of Vocational and Adult Education, identify other options for tracking successful outcomes.

Explore the feasibility of requiring grantees to collect data on student transfers.

Making grantee performance data available to the public in a transparent and meaningful manner.

Measure: Enrollment: number of full-time degree-seeking undergraduate students enrolled at SIP institutions.

Explanation:Increase enrollments at Strengthening Institutions Program (SIP) institutions. Fall enrollment data will be monitored annually to measure progress in meeting the long-term target, which is projected to be met in 2009. Target is derived by applying the difference between regression-based predicted values from Title IV institutions and actual grantee values for school year 2002-2003 (about 25%) -- which results in a long-term target of 253,500.

Measure: Persistence: percentage of first-time, full-time degree-seeking undergraduate students who were in their first year of postsecondary enrollment in the previous year and are enrolled in the current year at the same SIP institution.

Explanation:Increase the persistence rate for students enrolled at SIP institutions. Institutions report a persistence rate, not the numerator and denominator. As a result, the persistence rate for the SIP program is calculated as a median. The target is derived by applying the difference between regression-based predicted values from Title IV institutions and actual values for school year 2002-2003 -- which was 1.67%. Therefore, the SIP program actual persistence rate of 66% in FY 2004 was multiplied by 1.0167 to generate the long-term target (for 2009) of 68%. Annual increases are estimated to be 0.3% each year through 2009 and 0.2% beginning in 2010. (Data value for 2004 had previously been erroneously assigned to 2003).

Measure: Graduation: The percentage of first-time, full-time degree-seeking undergraduate students enrolled at four-year SIP institutions who graduate within six years of enrollment.

Explanation:Increase the graduation rate for students at four-year SIP institutions. Target for four-year graduation rate is derived by applying the difference between regression-based predicted values from Title IV institutions and actual values for school year 2002-03 -- which was 6.33%. Therefore, the SIP program actual four-year graduation rate of 45% in FY 2003 was multiplied by 1.0633 to generate the long-term target (for 2009) of 48%. Annual increases are estimated to be 1% through 2009 and 0.5% beginning in 2010.

Measure: Graduation: The percentage of first-time, full-time degree-seeking undergraduate students enrolled at two-year SIP institutions who graduate within three years of enrollment.

Explanation:Increase the graduation rate of students enrolled at two-year SIP institutions. For the two-year graduation rate projections, program experience was used to estimate targets. An increase of 0.5% was used to generate annual targets each year through 2009 and an increase of 0.3% was used beginning in 2010.

Measure: Federal cost per undergraduate and graduate degree at SIP institutions.

Explanation:Improve the efficiency of service delivery to SIP institutions. This measure is calculated as the appropriation for the Strengthening Institutions program divided by the number of undergraduate and graduate degrees awarded. The $3,975 cost/successful outcome for 2002-2003 reflects an appropriation of $81.467 million divided by 20,495 graduates. The $3,678 cost/successful outcome for 2003-2004 reflects an appropriation of $80.986 million divided by 22,021 graduates.

Is the program purpose clear?

Explanation: This Department of Education (ED) program provides grants to eligible institutions of higher education with high percentages of needy students and low per pupil expenditures to become self-sufficient by improving and strengthening their academic quality, institutional management, and fiscal stability. In establishing this program, the U.S. Congress determined that a significant number of higher education institutions serving high percentages of minority students and students from low-income backgrounds face challenges that threaten their ability to survive. Some of these institutional challenges relate to management and fiscal operations, as well as to an inability to engage in long-range planning and development activities, including endowment building. Assistance through this program is intended to enable schools serving high-risk populations to become viable, fiscally stable and independent, thriving institutions of higher education.

Evidence: Sections 301 and 311 of Title III of the Higher Education Act (HEA); Catalog of Federal Domestic Assistance, 84.031 Higher Education Institutional Aid

Does the program address a specific and existing problem, interest, or need?

Explanation: A three-year study of the so-called Title III programs (activities authorized in Title III of the HEA) completed in 1998 found that the Title III schools play an important role in serving students who have been historically underserved by postsecondary institutions. The central findings that served as the basis for creation of the Strengthening Institutions Program (SIP), are still relevant today. Although significant improvements have been made, there is still a 27 percentage-point gap between the percentages of low- and high-income high school completers immediately enrolling in college, a nine percentage-point difference between the percentages of African-American and White high school completers that immediately enroll, and a 17 percentage-point gap between the percentage of African Americans and Whites aged 25 to 29 who have completed a bachelors degree. Program funding for the SIP institutions, to be used for activities such as faculty development and improved student services, is well-justified for further closing these gaps in student achievement. Providing assistance to eligible institutions will enhance the role of such institutions in their critical role of providing access and quality education to service populations disportionately composed of low-income and minority students.

Evidence: Performance Measurement Study of the Title III Institutional Aid Program (Mathtech, 1998) - http://www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html. For college enrollment and graduation gaps, NCES, the Condition of Education, 2005. For institutional needs, Sections 301 of Title III of the HEA.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: HEA Titles III and V specify that if an institution of higher education receives funding under sections 316 or 317 of Title III Part A, Title III Part B, or Title V, it cannot receive funding under the Strengthening Institutions Program (SIP) thus allowing resources to be better targeted while also preventing duplication of activities. In addition to basic eligibility requirements, there are specific requirements in the SIP program to effectively target funds toward the intended beneficiaries. An institution must have at least 50 percent of its degree students receiving need-based assistance under Title IV of the Higher Education Act, or have a substantial number of enrolled students receiving Pell Grants. The institutions also must have low educational and general expenditures.

Evidence: An eligible institution is defined in Section 312 of the HEA.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: No major design flaws have been identified that prevent the program from meeting its defined objectives and performance goals. The institutional eligibility requirements ensure that program funding is provided to the neediest institutions.

Evidence: Application for Designation as an Eligible Institution, FY 2006

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program's design ensures that resources are well targeted to achieve the program's objectives. There are two application processes for this program -- one for identifying eligible institutions and one for grant funding. Institutions of higher education must be designated as an eligible institution before applying for funding under this program. Funding eligibility is limited to institutions of higher education serving high percentages of needy students and with low average per pupil expenditures. As a rule, many more institutions apply for benefits under the Strengthening Institutions Program (SIP) than receive them. For the 2006 competition for new awards, only 39 institutions are expected be awarded funding this year, out of approximately 300 applicants.

Evidence: Section 312 of Title III of the HEA; Application for Designation as an Eligible Institution, FY 2006 (FR Vol. 70 No. 241 dated December 16, 2005, pgs 74781-74783); Title 34, Chapter VI, Part 607 of the Code of Federal Regulations; SIP FY 2006 Grant Slate.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The previous SIP performance measures, which were developed with substantial input from the grantee community, tracked the percentage of institutional project goals that were successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. For these measures, ED now has both baseline data (2002) and trend data (2003-04), in which seven of the nine performance goals (three goals for each of the three years) have been exceeded. ED now has in place, for the second year, three long-term outcome measures that improve upon the previous measures. Whereas the previous measures captured grantees' success in completing program goals, the new measures take the assessment a step further and determine the degree to which the successful completion of project goals translate into important student outcomes. The new measures track: (1) the number of full-time degree-seeking undergraduates enrolling at SIP institutions, (2) the percentage of full-time undergraduate students who were in their first year of postsecondary enrollment in the previous year and are enrolled in the current year at the same SIP institution, (3) the percentage of students enrolled at four-year SIP institutions graduating within six years of enrollment, (4) the percentage of students enrolled at two-year SIP institutions who graduate within three years of enrollment. These measures have also been established for the Title III Aid for Institutional Development programs, the Title V Developing Hispanic-serving Institutions program, and for Howard University; providing ED with a consistent approach for measuring key student outcomes for different programs serving large minority populations. These student outcome measures use rigorously reviewed and adjudicated data and capitalize on the data still being collected on the successful completion of project goals.

Evidence: FYs 2005, 2006, and 2007 Program Performance Plans; Title III Electronic Annual Performance Report. Data for the new measures are derived from the National Center of Education Statistics' (NCES) Integrated Postsecondary Education Data System (IPEDS). IPEDS data are reported by all institutions participating in the SIP program, and are subject to NCES' established review and adjudication process, which includes multistage internal and external reviews, including a formal adjudication meeting, to ensure that statistical standards are met and that the product is of high quality.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ED established ambitious targets and timeframes in 2005 for the measures of enrollment, persistence, and graduation rates at SIP-grantee institutions using program data reported currently by institutions in IPEDS. These targets, which begin in 2006, have been established considering program history at SIP institutions, and by comparing SIP-grantee performance with the performance of all Title IV institutions and similar programs. SIP program goals for these measures are projected through 2011. In addition, ED set ambitious targets and timeframes through 2011 for its former long-term measures in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes.

Evidence: Please see measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The previous SIP performance measures, which were developed with substantial input from the grantee community, tracked the percentage of institutional project goals that were successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. For these measures, ED now has both baseline data (2002) and trend data (2003-04), in which seven of the nine performance goals (three goals for each of the three years) have been exceeded. ED now has in place, for the second year, two annual outcome measures that improve upon the previous measures. Whereas the previous measures captured grantees' success in completing program goals, the new measures take the assessment a step further and determine the degree to which the successful completion of project goals translate into important student outcomes. The new measures track: (1) the percentage of full-time undergraduate students who were in their first year of postsecondary enrollment in the previous year and are enrolled in the current year at the same SIP institution, (2) the percentage of students enrolled at four-year SIP institutions graduating within six years of enrollment, (3) the percentage of students enrolled at two-year SIP institutions who graduate within three years of enrollment. These measures have also been established for the Title III Aid for Institutional Development programs, the Title V Developing Hispanic-serving Institutions program, and for Howard University; providing ED with a consistent approach for measuring key student outcomes for different programs serving large minority populations. These student outcome measures use rigorously reviewed and adjudicated data and capitalize on the data still being collected on the successful completion of project goals.

Evidence: FY 2005, 2006, and 2007 Program Performance Plans; Title III Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for SIP grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established review and adjudication process, which includes multistage internal and external reviews, including a formal adjudication meeting, to ensure that statistical standards are met and that the product is of high quality.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ED has established baselines and targets for its new annual measures focusing on persistence and graduation rates at SIP-grantee institutions using program data reported by institutions in IPEDS. These targets, which begin in 2006, were established in 2005, considering program history at SIP-grantee institutions and the comparison of the SIP performance with the performance of all Title IV institutions and similar programs. In addition, baselines and targets have been established for the program's previous annual measures: improving academic quality, institutional management and fiscal stability, and student services and student outcomes of participating institutions.

Evidence: Please see measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The new performance measures focusing on student enrollment, retention, and graduation are well-aligned with current grantee activities. Many grantee projects involve tutoring, counseling, and other student service programs designed to improve academic success, all directly related to the performance measures. The program measures have been shared with grantees through application packages and closing date notices and have been presented at pre-application workshops, as well as Project Directors Meetings. As part of the application process, all partners commit to measuring their performance and working toward the specific goals of the program.

Evidence: ED revised FY 2006 Performance Plan; Title III and Title V Programs' 2006 and 2007 Program Performance Plans and FY 2005 Programs' Performance Report; Application for Grants Under the Strengthening Institutions Program.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: ED is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. The purpose of this independent evaluation is to analyze specific program issues, such as whether enrollment, persistence, and graduation rates are the key factors for an eligible institution's financial health. In addition, the analysis is expected to show whether the programs authorized by the HEA are having an impact on the financial health of grantees. The last performance measurement study of ED's Title III Programs was performed by Mathtech, Inc. and completed in 1998. The report presents the results of a survey of all 1995-96 Part A and Part B Title III grantees, as well as in-depth case studies conducted at 19 institutions. The study indicated that institutions used Title III grants primarily to strengthen academic programs, rather than on institutional management. The study also found that Title III schools, for the most part, were not in severe financial difficulty.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program (Mathtech, 1998) http://www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). Additionally, ED's integrated budget and performance plan includes the program's annual and long-term goals.

Evidence: N/A

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ED has refined its annual and long-term performance measures, and has added an efficiency measure. Further, ambitious annual and long-term targets have been established based on program and benchmarking information. ED's new performance measures are based on data already collected in IPEDS and provide for improved comparison of information across different programs. ED plans to review its current annual performance report and ensure that the information requested provides quality data and evidence of program impact, as well as the impact of individual projects.

Evidence: ED has created a new efficiency measure for the program and revised the annual and long-term performance measures to standardize the SIP measures with performance measures for other similar programs within OPE. This milestone will enhance strategic planning, reporting, and program management. Further, the data for the performance measures will come from IPEDS data collected by NCES, which as mentioned earlier, increases the credibility, reliability, and availability of the data.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ED requires grantees to submit regular and timely information via electronic means (i.e. annual performance reports, interim reports, and final reports). Program managers conduct comprehensive reviews of these reports and use them for targeted site assistance visits and monitoring visits to ensure activity goals are completed as proposed or aligned to more effectively achieve intended goals. ED uses program information to target technical assistance to grantees and to better manage the program and make modifications, if needed, to improve program performance. Actions such as decreasing funding for grantees designated as "high-risk", establishing special or additional reporting procedures or reduction/cancellation of grants may be imposed based on information identified during the visits. Further, ED has revised its annual and long-term performance measures to more directly focus on outcomes and reflect the purpose of the program. Current program initiatives in development include a listing of grantee exemplary practices to be posted on the program website in 2006.

Evidence: Handbook for the Discretionary Grants Process; ED e-Monitoring; the Title III Part A application for grants instructed that "Grantees are required to collect data and report on progress towards meeting the GPRA indicators and other project specific indicators in their annual performance reports."

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: ED's managers are subject to EDPAS, a system which links employee performance to relevant Strategic Plan goals, and is designed to measure the degree to which a manager contributes to improving program performance. The EDPAS agreements of SIP managers are linked with the performance of projects or the program.

Evidence: The EDPAS standards for Institutional Development and Undergraduate Education Service (IDUES) program managers require program managers to set forth strategies for implementing GPRA and Strategic Plan initiatives related to IDUES. IDUES program officers are held accountable for assessing project performance and monitoring the progress of projects in achieving program goals and objectives. SIP projects receive continuation funding on the basis of their reported progress in achieving the program goals.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Federal funds are obligated within the timeframes set out by ED schedules and used for the purposes intended. In FY 2006, ED made all non-competing continuation awards by the end of April, two months earlier than in past years. This improvement enables recipients more time to plan effectively and begin hiring processes. Further, to ensure that funds are properly managed, ED monitors grantees' financial records using Grants and Administrative Payment System (GAPS) and quarterly reports from the Office of the Assistant Secretary for OPE to alert staff of any grantees' excessive drawdown or inactivity of Federal funds. In addition, to ensure that funds are properly managed, ED has strengthened its financial management practices by implementing e-Monitoring. ED e-Monitoring is a new software tool that enables grant program staff to better monitor their grants and helps to prevent potential fraud, waste, and abuse.

Evidence: Title III Performance Measurement System; annual spending plans and program and partners' financial records; ED e-Monitoring was implemented December 22, 2004; GAPS.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While ED does not have procedures in place to fully measure and achieve efficiencies for this program, progress has been made. ED has developed an efficiency measure for SIP that is based on the number of students that successfully complete their programs of study. The measure is determined by dividing the program's annual appropriation by the number of undergraduate and graduate degrees awarded in that year. A similar measure (assessing the Federal cost of the number of students graduating) has been established for the HSI, the HBCU and the HBGI programs, as well as for Howard University. In 2006, program staff will begin to calculate efficiency measures at the grantee level as part of an OPE-wide strategy to improve efficiency data and set targets for improving efficiency. This effort will emphasize increased communication with grantees regarding performance data. ED has other procedures in place to achieve efficiencies and cost effectiveness. ED relies on competitive sourcing to provide technical and other administrative support for this program. Contractual support provides for electronic submission of applications, as well as electronic submission of performance reporting, in addition to expert administrative and technical support.

Evidence: Data for FY 2004 show that the Federal program cost per graduating student is $3,678. (FY 2004 appropriation for SIP: $81 million. Total number of graduating students: 22,021). This level reflects a 7.5% decrease from the 2003 cost of $3,975. Multiple contracts provide database, technical assistance, and reporting support. Electronic applications and annual and interim performance reports also create efficiencies in data collection, analysis, and reporting and reduce burden on grantees. Efficiency measure targets will be established in late 2006.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Strengthening Institutions Program (SIP) collaborates and coordinates effectively with other related programs within ED (Title III Part A, Tribally Controlled Colleges and Universities, Alaska Native and Native Hawaiian-Serving Institutions; Title III Part B, Historically Black Colleges and Universities; and Title V, Hispanic-serving Institutions Programs). All of these programs are designed to strengthen institutions of higher education that serve high percentages of minority students and students from low-income backgrounds. Parallel policies, administrative procedures, and performance measuring and assessment tools have been established to accomplish program goals. In regard to collaboration on the performance measures, all of the aforementioned Title III and Title V programs, as well as many other programs within OPE, now share common measures of enrollment, persistence rate, and graduation rate. SIP collaborates and coordinates with ED's Federal Student Aid offices to ensure that applicants demonstrate that they meet the legislative requirements for enrollment of needy students and maintain the required (below average) educational and general expenditure per student. Further, national Project Directors' Workshops for all IDUES programs will continue to be held jointly. SIP also collaborates and coordinates with the Federal Work-Study Program, the Federal Supplemental Educational Opportunity Grants Program, the TRIO Student Support Services Program and the Undergraduate International Studies and Foreign Language Program to allow institutions to apply for and receive waivers of non-Federal cost-share requirements under these particular programs. SIP also coordinates with the Census Bureau and the Bureau of Labor Statistics to maintain the current low-income guidelines.

Evidence: Eligibility requirements can be found in Section 312 of Title III of the HEA. Title III and Title V Programs' 2006 and 2007 Performance Plans. The FWS, FSEOG, SSS Programs are authorized under Title IV of the HEA; the UISFL Program is authorized under Title VI of the HEA.

Does the program use strong financial management practices?

Explanation: In response to an audit report that notes ED's failure to adequately monitor grantees for their compliance with grant administration requirements, ED/OPE has implemented an e-Monitoring system enabling program staff to further mitigate against potential fraud, waste, and abuse. It allows staff to classify each grant into categories based on grantee performance, and maintain electronic records of phone, correspondence, and e-mail communications with grantees. ED's e-Monitoring and GAPS data along with other reports enable program officers to track all funds applications made by an institution. Further, these systems are used to identify institutions determined not to have applied funds appropriately and to withhold funding, if necessary. ED continuously monitors grantee management and administration of Federal funds using GAPS and institutional financial reports.

Evidence: Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). All Title III funded grantee institutions are documented in GAPS and e-Monitoring systems.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: For years, IDUES had utilized a paper collection instrument to obtain performance data and reports for the SIP. IDUES has improved this process by replacing the paper collection instrument with an electronic performance reporting system to collect and report information more efficiently. Recent introduction of the e-Monitoring system has improved the program's ability to manage and monitor funds to further mitigate against potential fraud, waste, and abuse. Also, the SIP has responded to OIG concerns and findings regarding program management and has addressed these by implementing action plans that are approved by OIG.

Evidence: The new Performance Measurement System for Titles III and V is designed to acknowledge the diversity of institutions that are served, use institutional data from IPEDS for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format. Further, ED has increased the number of on-site visits to institutions and has taken steps to improve the knowledge of staff conducting site visits through training initiatives. The Handbook for the Discretionary Grants Process; implementation of e-Monitoring; e-Grants; GAPS.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: A clear competitive process is implemented based on an approved Technical Review Plan, which outlines, step-by-step, the procedures to be used in the review process. Independent panels of experts read and score all eligible applications based on how well the applicant addressed the established selection criteria. Steps are implemented to avoid a conflict of interest or appearance thereof.

Evidence: Competitions are announced in the Federal Register. The FY 2004 notice inviting applications for new awards can be found in Vol. 69 No. 12 dated January 20, 2004 on pages 2702-2706. Program funds are used to pay for the peer review process. 100% of grants are subject to review. ED used the grant slate developed for the Strengthening Institutions Program in fiscal year 2004 to make new awards in 2005 because a significant number of high quality applications remained on the fiscal year 2004 slate.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: OPE has developed new procedures for improving the monitoring of expenditures in response to IG concerns with the Title III Aid for Institutional Development programs and implementation of Title V institutional eligibility requirements. GAPS and ED/OPE's e-Monitoring systems are used to ensure that all related grant financial management practices are followed. Program officers provide continual oversight and coordination of activities of SIP grantees via on-site technical and continuous electronic communication (email, fax, and telephonic). OPE staff review annual and interim performance reports to ensure that the information requested provides quality data and evidence of the success of the program, as well as of success in the individual projects.

Evidence: In addition to increasing efforts at on-site monitoring, program staff continues to review all performance reports that grantees are required to submit. Further, program oversight includes documentation of grantees' use of funds, email communications, and project directors' meetings. This review process has resulted in the adjustment of grantee work plans that were not allowed within the scope of the program.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Performance data are collected annually through the IPEDS survey by NCES and through annual electronic performance reports submitted by SIP grantees to the program office. These data are available on the Department's website. Additionally, GPRA data are now available to the public on the ED website. Further, ED has completed the first draft of the Strengthening Institutions Program profile report. However, these data are not readily available to the public or on the Internet at this time, and do not reflect program impacts. The profile report will be posted to the ED Website in 2006.

Evidence: IPEDS data can be accessed at the College Opportunities On-line Website at http://nces.ed.gov/ipeds/cool.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The antecedent measures capture the percentage of institutional project goals that are successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. The program exceeded all of its antecedent annual targets in 2002, two of the targets in 2003, and two of its targets in 2004. Thus, the program appears to be on target to achieve the antecedent long-term goals, which were projected through 2011. Targets were set at 75% for all years. In 2005, ED developed three new outcome measures relating to enrollment, persistence, and graduation rates, which built upon the previous measures. Whereas the previous measures captured grantees' success in completing completing project goals, the new measures improve the assessment by determining the degree to which the successful completion of project goals translate into important student outcomes. Also, SIP program goals were established for 2006 through 2011. For the long-term enrollment measure, IPEDS data show, enrollment at SIP grantee institutions increased from 214,022 in 2004 to 220,764 in 2005. This is the third consecutive year of increase. The long-term persistence measure (same as the annual measure) is defined as the median retention rate for full-time, first-time undergraduate students at grantee institutions. The rate was 66% for 2004 compared to 63% in 2005. (Note the 2005 persistence rate was based on all grantees responding, whereas the 2004 persistence rate--the first year persistence was captured on IPEDS--was based on about a 50% response rate. Therefore, it is unknown to what degree the decrease in persistence rate from 2004 to 2005 reflects grantee performance.) The long-term graduation rate measure at four-year institutions, the same as the annual measure, is defined as the percentage of full-time degree-seeking undergraduate students completing a four-year degree within six years of entry for all four-year SIP grantee institutions. The rate increased from 45% in 2003 to 47% in 2004. The graduation rate at two-year institutions, the same as the annual measure, is defined as the percentage of full-time undergraduate students completing a two-year degree or within three years of entry for all two-year SIP grantee institutions. The rate increased from 25% in 2003 to 26% in 2004. Both the two- and four-year graduation rates are poised to meet or exceed their targets (established for 2006 as initial year).

Evidence: FY 2002, 2003, and 2004 Annual Performance Reports; FY 2005, 2006, and 2007 Program Performance Plans; Data for the measures are derived from NCES/IPEDS data.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Grantee performance for the years 2002-2004, as measured by the program's antecedent annual measures on the percentage of projects successfully completed, has been generally positive. The program exceeded all of its annual targets in 2002, two of its targets in 2003, and two of its targets in 2004. Thus, the program appears to be on target to achieve the long-term goals, which have been projected through 2011. Targets were set at 75% for all years. Whereas these previous measures captured grantees' success in completing program goals, the new measures improve the program assessment and determine the degree to which the successful completion of project goals translate into important student outcomes. With respect to these new measures, the two- and four-year graduation rates have both shown an increase from 2003 to 2004 and both appear poised to meet or exceed the initial (2006) annual target. While the graduation rates increased persistence rates declined. The rate was 66% for 2004 compared to 63% in 2005. (Note the 2005 persistence rate was based on all grantees responding, whereas the 2004 persistence rate--the first year persistence was captured on IPEDS--was based on about a 50% response rate. Therefore, it is unknown to what degree the decrease in persistence rate from 2004 to 2005 reflects grantee performance.) As part of the measurement development process, targets were developed beginning with 2006. Therefore, although data are as current as possible (through 2005 for persistence and 2004 for graduation) adequate information is not yet available to determine the program's performance on its new annual measures: persistence and graduation rates.

Evidence: Data for the measures on percentage of successful projects completed (the same as the long-term measures), developed with significant input from the grantee community, continue to be reported electronically via the Annual Performance Reports. Data for the new measures, reflecting persistence and graduation rates (the same as the long-term measures), are derived from NCES/IPEDS data.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: An efficiency measure -- Federal cost per undergraduate and graduate degree at SIP institutions has been developed and trend data now demonstrate improved efficiencies. Data for FY 2004 show that the Federal program cost per graduating student is $3,678, a 7.5% decrease over 2003 ($3,975). However, targets for this measure are not yet set. They are expected to be established in late 2006 and will be informed by ED's analysis of measures of individual grantee efficiency. The same measure has been established for the HSI program, the HBCU program and the HBGI program, as well as for Howard University. This approach to measuring program efficiency will enable ED to maintain a consistent approach in assessing program performance across institutions with similar types of missions.

Evidence: 2007 Annual Performance Plan for the SIP shows: The FY 2004 appropriation for SIP ($81 million) divided by the total number of graduating students (22,021) equals the cost per graduating student ($3,678). This reflects a 7.5% decrease from the 2003 cost of $3,975.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Although there are other Federal, state, local, and private efforts that provide funding/services to institutions with high percentages of needy students, these programs do not have a similar purpose. In addition to basic eligibility requirements, there are specific requirements in the Strengthening Institutions Program to effectively target the program toward the intended beneficiaries. An institution must have at least 50 percent of its degree students receiving need-based assistance under Title IV of the Higher Education Act, or have a substantial number of enrolled students receiving Pell Grants. The institutions also must have low educational and general expenditures.

Evidence: Section 312 of Title III of the HEA; Application for Designation as an Eligible Institution, FY 2006 (FR Vol. 70 No. 241 dated December 16, 2005, pgs 74781-74783); Title 34, Chapter VI, Part 607 of the Code of Federal Regulations. Other Federal programs whose funding is focused on institutions that enroll large proportions of minority and financially disadvantaged students include the Environmental Protection Agency's Greater Opportunities Fellowship Program and Greater Opportunities Research Program that strengthen the environmental research capacity of institutions with substantial minority enrollment; and the National Aeronautics and Space Administration's Minority Research and Education Program to support research and academic infrastructure.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of the Strengthening Institutions program have been conducted. However, ED is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. The purpose of this independent evaluation is to focus analysis on specific aspects of the program such as whether enrollment, persistence, and graduation are drivers of financial health of the institutions. In addition, the analysis is expected to show whether the programs authorized by the HEA are having a positive impact on the institutions' financial health.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in 2007.