Detailed Information on the
Strengthening Historically Black Colleges and Universities Assessment

Completing the study on the financial health of minority serving institutions of higher education and utilizing the results to validate program performance measures and improve program performance.

Developing strategies to use efficiency and performance data for program improvement purposes.

Making grantee performance data available to the public in a transparent and meaningful manner.

Measure: Graduation: percentage of first-time, full-time degree-seeking undergraduate students enrolled at four-year Historically Black Colleges and Universities who graduate within six years of enrollment.

Explanation:

Measure: Persistence: percentage of first-time, full-time degree-seeking undergraduate students who were in their first year of postsecondary enrollment in the previous year and are enrolled in the current year at the same Historically Black College and University.

Explanation:

Measure: Enrollment: The number of full-time degree-seeking undergraduate students enrolled at HBCU grantee institutions.

Explanation:2003 Baseline: 2105 over the prior year enrollment

Measure: Cost per successful outcome: annual appropriation divided by the number of graduates, doctorates, first professional, and masters degrees awarded in that same year.

Explanation:FY 2003 baseline: $8,631

Is the program purpose clear?

Explanation: The purpose of the Strengthening Historically Black Colleges and Universities (HBCU) program is to provide financial assistance to HBCUs to establish or strengthen their physical plants, financial management, academic resources, and endowments.

Evidence: Section 321 of Title III Part B of the Higher Education Act (HEA) states that "financial assistance to establish or strengthen the physical plants, financial managements, academic resources, and endowments of the historically Black colleges and universities are appropriate methods to enhance these institutions and facilitate a decrease in reliance on governmental financial support and to encourage reliance on endowments and private sources."

Does the program address a specific and existing problem, interest, or need?

Explanation: Congress explicitly authorized funding for HBCUs in 1965, in part, based on a finding that States and the Federal Government discriminated against Black institutions in the allocation of land and financial resources under the Morrill Act of 1862. Congress also found that the Black institutions had been discriminated against in the award of grants and contracts under the Higher Education Act and other Federal programs that benefit institutions of higher education and that the current state of Black colleges and universities is partly attributable to these discriminatory actions. While African Americans continue to lag behind their White cohorts in overall education attainment, HBCUs have contributed significantly to the effort to attain equal opportunity through postsecondary education for Black, low-income and educationally disadvantaged Americans. According to NCES statistics, in 2003, the immediate enrollment in college after graduating from high school was almost nine percentage points lower for black students (57.5%) than for white students (66.2%). Also, in 2003, full time degree-seeking black undergraduate students graduated from four-year institutions within six years at a rate that was 18.8 percentage points lower (38.5%) than the percentage for comparable white students (57.3%). In 2001-2002, African Americans earned only 9% of the bachelor's degrees, 8.4% of the master's degrees, and 5.4% of PhDs awarded in the United States, though African Americans comprise 13% of the population. Policymakers, researchers, and educators agree that Blacks are underrepresented in several academic disciplines, and are less likely to receive professional degrees in the sciences and related technology fields, and visual and performing arts.

Evidence: Section 321 of Title III of the Higher Education Act findings and purposes; NCES Digest of Education Statistics, 2003 (Tables 264, 267, and 270); Census 2000 Brief--The Black Population: 2000, Bureau of the Census.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: HEA Titles III and V specify that if an HBCU receives funding under the HEA Title III Part A programs, the Strengthening Historically Black Graduate Institutions, or the Title V Developing Hispanic Serving Institutions program, it cannot receive funding under the Strengthening HBCU program, thus allowing resources to be managed more efficiently while preventing duplication of activities. Title III Part B directs that, in any fiscal year, if Howard University or University of the District of Columbia is found eligible to receive funds under Part B they can only receive an allotment if their Part B allocation exceeds the amount received under their separate annual appropriations. While there are many other Federal, state, local, and private efforts that provide funds/services to HBCUs and their students, there are significant distinctions between these programs and the HBCU program. Similarly, within ED, even though the HBCU and HBGI programs share similar goals and objectives, their purposes differ. The major difference between the two programs is that the HBCU Program enhances the physical, financial and academic resources of undergraduate institutions, while the HBGI program is designed to enhance graduate programs, such as legal, medical, dental, veterinary, other graduate education opportunities in math, engineering, or the physical or natural sciences. Examples of programs at the Federal level include programs that are designed to promote careers in community development [Department of Housing and Urban Development (HUD)], support research and education programs [National Aeronautics and Space Administration (NASA) and Department of Commerce(DOC)], provide support in the fields of science and engineering that are important to national defense [Department of Defense (DOD)], increase the number of minority students who pursue careers in international family planning, reproductive health, and population and the environment [US Agency for International Development (USAID], and enhance the quality of science, technology, engineering and mathematics instruction and outreach [National Science Foundation (NSF)]. These efforts are due in part to Executive Order 13256 signed by President Bush on February 12, 2002. This order is the fifth in a succession of executive orders designed to highlight opportunities for these institutions to participate in and benefit from Federal programs. While ED's Strengthening HBCUs program is designed to provide financial assistance to HBCUs specific to ED's mission, HBCU programs established in other agencies address issues specific to their agency's mission.

Evidence: Title III Sections 312(g), 316(d)(3), 317(e), 324(g) and Title V Section 505. Presidential Executive Orders 12232, 12320, 12677, 12876, 13256. HUD's HBCU Program, NASA's Minority University Research and Education program, DOD's HBCUs and Minority Institutions program, USAID's Minority Serving Institutions Initiative, DOC's National Oceanic and Atmospheric Administration Educational Partnership program, and NSF's HBCU Undergraduate Program.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: No major design flaws have been identified. The authorizing legislation strictly prescribes the funding formula. Appropriations are allocated among eligible HBCUs based on the number of Pell Grant recipients enrolled, the number of graduates, and the percentage of graduates who are attending graduate or professional school in degree programs in which African Americans are underrepresented. The statute provides for a $500,000 minimum grant for each eligible institution.

Evidence: Section 324 of Title III of the HEA.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program is well targeted to reach eligible institutions designated by the Secretary of Education as any accredited, legally authorized HBCU that was established prior to 1964 and whose principal mission was, and is, the education of black Americans. Further, funds are used to directly and effectively meet the program's purpose of providing financial assistance to HBCUs to establish or strengthen their physical plants, financial management, academic resources, and endowments. The statute provides for a wide variety of uses of funds in these areas depending upon individual institutional needs. Recent 2002-2003 annual performance reports indicate that 40 percent of expenditures were spent on activities related to academic quality, 25 percent on student services and student outcomes, 26 percent on institutional management, and 9 percent on fiscal stability.

Evidence: Sections 321-323 of Title III of the HEA. Grants are distributed by formula to all eligible HBCUs. 2002-2003 Annual Performance Reports.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The previous HBCU long-term performance measures tracked the percentage of institutional project goals that were successfully completed. In conducting this PART review, it was determined that these measures may not effectively measure program performance. Therefore, ED has replaced these measures with three new outcome measures reflecting persistence and graduation rates at HBCUs. These new measures have also been established for the Title III Strengthening Institutions program, the Developing Hispanic-serving Institutions program, and Howard University. This change should enable consistent measurement across ED programs that focus on strengthening institutions programs that serve large minority populations. All of these measures, with the exception of the persistence measure, have also been established for the Strengthening Historically Black Graduate Institutions program.

Evidence: FYs 2005 and 2006 Program Performance Plans. Data for the measures aggregated across all of the Title III and Title V programs and disaggregated by individual program are derived from the Title III electronic performance measurement system. Data for the new measures are derived from electronic annual performance reports for HBCU grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: Ambitious targets and timeframes have been established for ED's new measures that focus on enrollment, persistence, and graduation rates at HBCUs using program data reported currently by institutions in IPEDS. These targets have been established considering program history at HBCUs, the comparison of the HBCU performance with the performance of all Title IV institutions and with similar programs at ED. HBCU program goals for these measures are projected through 2011. In addition, ED had set ambitious targets and timeframes through 2011 for its former long-term measures in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes.

Evidence: Please see measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The previous HBCU annual performance measures tracked the percentage of institutional project goals that were successfully completed. In conducting this PART review, it was determined that these measures may not effectively measure the performance of the overall program. Therefore, ED has replaced these measures with two new outcome measures reflecting persistence and graduation rates at HBCUs. These new measures have also been established for the Title III Strengthening Institutions Program, Title V Developing Hispanic-serving Institutions program and for Howard University, resulting in consistent measurement across ED programs that focus on strengthening institutions programs that serve large minority populations. All of these measures, with the exception of the persistence measure, have been established for the Strengthening Historically Black Graduate Institutions program.

Evidence: FYs 2005 and 2006 Program Performance Plans; Title III Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for HBCU grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: ED has established baselines and targets for its new annual measures focusing on persistence and graduation rates at HBCUs using program data reported by institutions in IPEDS. These targets have been established considering program history at HBCUs, the comparison of the HBCUs performance with the performance of all Title IV institutions and similar programs. In addition, baselines and targets have been established for the program's current annual measures relating to improving academic quality, institutional management and fiscal stability, and student services and student outcomes of participating institutions.

Evidence: Please see measures tab for specific annual targets.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: In response to findings and recommendations in the June 2000 Inspector General-issued audit on Title III programs and to better measure the success of Titles III and V, the Department conducted an extensive outreach effort in fiscal year 2001 to solicit recommendations for the Titles III and V program indicators and develop a new performance measurement system. The Title III and V grantee communities were informed and consulted in a number of ways, including program focus group discussions, meetings with project directors, regional grantee meetings, national conference calls, and outreach to college presidents and associations. The goals are based on information that all postsecondary schools already collect and use for a number of purposes. As part of the application process all partners commit to and work toward the specific goals of the program. Grantees must provide a five-year plan that describes their plans for accomplishing the goals and objectives of their specific projects. These projects are aligned with the purposes of the program and the program goals. Many grantee projects involve areas such as tutoring, counseling, student service programs designed to improve academic success, faculty development, curriculum development, and faculty fellowships to assist in attaining advanced degrees in the fellow's field of instruction, all of which are directly related to the new performance measures. In addition, annual performance reports are required for all grantees.

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). EDGAR 34 CFR 74.51. Annual and final reports from the Title III Performance Measurement System collect data on academic quality, institutional management and fiscal stability, and student services and student outcomes. The Performance Measurement System for Title III is designed to acknowledge the diversity of institutions that are served, use institutional data from IPEDS for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. A purpose of the analysis is to determine whether the financial status of the institutions is improving or becoming worse and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions financial health. Mathtech, Inc. conducted the last performance measurement study of ED's Title III Programs in 1995. The report presents the results of a survey of all 1995-96 Part A and Part B Title III grantees as well as in-depth case studies conducted at 19 institutions. The study indicated that institutions used Title III grants primarily to strengthen academic programs rather than on institutional management. The study also found that most Title III schools were not in severe financial difficulty. In addition, a recent NCES publication presents a statistical overview of the development of HBCUs. The report--Historically Black Colleges and Universities: 1976 to 2001--presents a quarter-century of Historically Black Colleges and Universities trends. The report contains summary information for HBCUs on enrollment, degrees, staff, salaries, and finances, with comparisons to other colleges and universities, in addition to detailed information on individual HBCUs, with trends in enrollment, and detailed data on degrees, staff, and finances for recent years.

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program (Mathtech, Inc., 1995) www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html. Historically Black Colleges and Universities: 1976 to 2001 (NCES 2004-062); Status and Trends in the Education of Blacks (NCES 2003-034).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: ED has not satisfied the first part of the question because program performance changes are not identified with changes in funding levels. The program, at this time, does not have sufficiently valid and reliable performance information to assess (whether directly or indirectly) the impact of the Federal investment. However, ED has satisfied the second part of this question in that ED's budget submissions show the full cost of the program (including S&E). ED's 05 integrated budget and performance plan includes the program's annual and long-term goals.

Evidence: N/A

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ED has identified strategic planning deficiencies and taken meaningful steps to address these deficiencies. ED has refined its annual and long-term performance measures, and has added an efficiency measure. Further, ambitious annual and long-term targets have been established based on program and benchmarking information. ED's new performance measures are based on data already collected in IPEDS and provide for improved comparison of information across different programs. ED is also planning to review its current annual performance report and ensure that the information requested provides quality data and evidence of the success of the program, as well as of success in the individual projects.

Evidence: ED has created a new efficiency measure for the program and revised the annual and long-term performance measures to standardize the HBCU measures with performance measures for other similar programs within OPE. This will enhance strategic planning, reporting, and program management. Further, the data for the performance measures will come from IPEDS data collected by NCES, which increases the credibility and reliability of the data, as well as ensuring consistent data availability.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ED requires grantees to submit regular and timely information via electronic means (i.e. annual performance reports, interim reports, and final reports). Information on grantee compliance with program requirements and objectives is collected annually and continuation awards are dependent upon a determination of substantial progress being made. The information collected is used to improve program performance and to provide technical assistance to grantees to assist them in meeting their proposed goals and objectives. In addition, ED has taken meaningful steps to address deficiencies highlighted in a recent GAO report entitled, "Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). GAO recommended that ED take steps to ensure that monitoring and technical assistance plans are carried out and targeted to at-risk grantees and that the needs of the grantees guide the technical assistance. One of those steps was to implement e-Monitoring, a new software tool that enables grant program staff to better monitor grants and assess risk. Program staff use this information to target technical assistance to grantees. Further, ED has revised its annual and long-term performance measures to more directly focus on outcomes and reflect the purpose of the program. ED uses program information to better manage the program and to make modifications, if needed, to improve program performance. In June 2005, ED met with partners to discuss regulating the definition of a program graduate. This action is in response to information learned about the wide variety of ways institutions define a "graduate."

Evidence: ED launched an electronic monitoring system (e-Monitoring) in December 2004. Program managers conduct comprehensive reviews of grantee interim reports and annual performance reports to ensure that performance measure standards are being met. Applications are reviewed in accordance with Federal statutes to ensure the planned activities are reflective of performance measures. Program monitors use interim, annual and final performance reports for targeted site assistance visits and monitoring visits as assurance that activity goals are completed as proposed or aligned to more effectively achieve intended goals. Actions such as high-risk designation leading to decreased funding, establishing special or additional reporting procedures or cancellation of grants may be imposed. Program staff places a hold on grantee's federal funds in cases of non-compliance or loss of accreditation. Further, staff approves or denies requests for budgetary changes that may affect the scope of approved projects. Based upon oversight information, institutions may have continuation funding reduced.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: While ED has initiated several efforts to improve accountability in its programs, it is not clear that Federal managers of this program are held accountable for program performance. Across the Department, ED is in the process of ensuring that "EDPAS" plans -- which link employee performance to relevant Strategic Plan goals and action steps - hold Department employees accountable for specific actions tied to improving program performance. ED is also revising performance agreements for its SES staff to link performance appraisals to specific actions tied to program performance. Finally, ED is reviewing its grant policies and regulations to see how grantees can be held more accountable for program results. The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Evidence: The President's Management Agenda scorecard (Human Capital and Budget & Performance Integration initiatives) notes ED's efforts to improve accountability. The Department's Discretionary Grants Improvement Team (DiGIT) recommendations indicate that ED is reviewing its grant policies and recommendations.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: Federal funds are obligated within the timeframes set out by ED schedules and used for the purposes intended. In FY 2006, ED plans to make the NCC awards by the end of April; this target date is two months earlier than in past years. This improvement gives institutions funding information sufficiently early to enable them to plan effectively and begin hiring processes. Further, to ensure that funds are properly managed, ED monitors grantees' financial records using GAPS and quarterly reports from the Office of the Assistant Secretary for OPE to alert staff to any grantees' excessive drawdown or inactivity of Federal funds. In addition, to ensure that funds are properly managed, ED has strengthened its financial management practices by implementing e-Monitoring. ED e-Monitoring is a new software tool that enables grant program staff to better monitor their grants and helps to prevent potential fraud, waste, and abuse.

Evidence: Title III Performance Measurement System; annual spending plans and program and partners' financial records; ED e-Monitoring was implemented December 22, 2004; and GAPS.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: ED has developed an efficiency measure that is based on the number of students that successfully complete their programs. The measure is determined by dividing the program's annual appropriation by the number of students graduating in that year. A similar measure (assessing the Federal cost of the number of students graduating) is established for the HSIs and the HBGIs programs, as well as for Howard University. This change may enable ED to assess program performance across institutions with similar types of missions. In addition, ED has other procedures in place to achieve efficiencies and cost effectiveness. ED relies on competitive sourcing to provide technical and other administrative support. Contractual support provides for electronic submission of applications, as well as electronic submission of performance reporting, in addition to expert administrative and technical support and increases the cost effectiveness of the programs.

Evidence: Data for FY 2003 show that the Federal program cost per graduating student is $8,631. (FY 2003 appropriation for Strengthening HBCUs program: $214.0 million. Total number of graduating students: 24,796). Multiple contracts provide database, technical assistance, and reporting support. Electronic applications and annual and interim performance reports also create efficiencies in data collection, analysis, and reporting.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The Strengthening HBCUs program collaborates and coordinates effectively with other HEA Title III programs??Strengthening Institutions, Strengthening Alaska Native and Native Hawaiian-serving Institutions, Strengthening Tribally Controlled Colleges and Universities, and Strengthening Historically Black Graduate Institutions??and the Title V Developing Hispanic Serving Institutions program. If an HBCU receives funding under any of these programs, it cannot receive funding under the Strengthening HBCU program, thus allowing resources to be managed more efficiently. In addition, program responsibility for the Strengthening HBCUs program has been placed within the Office of the Deputy Assistant Secretary for Higher Education. The Deputy Assistant Secretary also oversees the White House Initiative on Historically Black Colleges and Universities. This has led to efforts to coordinate and standardize policies and programmatic implementation for similar programs within ED. One outcome of this effort has been the common measures and data sources (IPEDS) that have been developed for key programs related to minority serving IHEs. In addition, the Strengthening HBCUs program coordinates effectively with the TRIO programs, the Federal Student Aid programs, the Howard University program, the activities of the President's Board of Advisors on HBCUs, the National Association for Equal Opportunity in Higher Education, the United Negro College Fund, and as well as other ED programs with a strong emphasis on serving minority populations.

Evidence: To ensure coordination with other related programs, program managers hold meetings and have established common performance measures and goals, as well as parallel policies, administrative procedures and assessment tools. Additionally, National Project Directors' Workshops are held for these programs. Special rules outlined in Title III Sections 312(g), 316(d)(3), 317(e), 324(g) and Title V Section 505 of the HEA.

Does the program use strong financial management practices?

Explanation: Because of recent audit reports that indicate ED's failure to adequately monitor grantees for their compliance with grant administration requirements, ED/OPE has implemented an e-Monitoring system enabling program staff to further mitigate against potential fraud, waste, and abuse. It allows staff to classify each grant into categories based on grantee performance, and maintains electronic records of phone, correspondence, and e-mail communications with grantees. Using ED's e-Monitoring and GAPS data along with other reports allows program officers to observe all funds applications made by an institution. Further, these systems are used to identify institutions determined not to have applied funds appropriately and to withhold funding, if necessary. ED continuously monitors grantee management and administration of Federal funds using GAPS and institutional financial reports.

Evidence: Low-Income and Minority Serving Institutions: Department of Education Could Improve Its Monitoring and Assistance" (GAO-04-961). All HBCU funded grantee institutions are documented in GAPS and e-Monitoring systems.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: For years, the Department of Education has utilized a paper collection instrument to obtain performance data and reports. ED has addressed this deficiency by replacing the paper collection instrument with an electronic performance reporting system to collect and report information more efficiently. In addition, each principal office must submit annual grant monitoring plans. Program offices must address the purpose, goals and objectives of the program; GPRA performance indicators and data; grantee performance; opportunities for technical assistance; and grantee or program risk factors. Furthermore, recent introduction of the e-Monitoring system has improved ED's ability to manage and monitor funds to further mitigate against potential fraud, waste, and abuse.

Evidence: The new Performance Measurement System for Titles III and V is designed to acknowledge the diversity of institutions that are served, use institutional data from the Integrated Postsecondary Education Data System for context, accommodate the diversity of institutional activities, and permit the aggregation of data in a standardized format. Further, ED has increased the number of on-site visits to institutions and has taken steps to improve the knowledge of staff conducting site visits through training initiatives. Section 6.4.4 of the Handbook for the Discretionary Grants Process; implementation of e-Monitoring; e-Grants; GAPS.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: New procedures have been developed for improving the monitoring of expenditures based on IG concerns. GAPS, e-Monitoring, annual and interim performance reports, financial status reports, and other internal reports are used to ensure that all related grant financial management practices are followed. Program officers provide continous oversight and coordination of HBCU activities via on-site technical and continuous electronic communication (email, fax, and telephonic).

Evidence: Review of Title III Programs, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators (ED-OIG/A04-90014, June 2000). Office of Higher Education Programs Needs to Improve Its Oversight of Parts A and B of the Title III Program (ED-OIG/A04-90013, December 2000). In addition to increasing efforts at on-site monitoring, program staff continues to review all performance reports that grantees are required to submit. In addition, program oversight includes documentation of grantees' use of funds, email communications, and project directors' meetings. This review process has resulted in the adjustment of grantee work plans that were not allowed within the scope of the program.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Performance data are collected annually through the IPEDS survey by NCES and through annual performance reports submitted by HBCUs to the program office. These data are available on the Department's website. Additionally, GPRA data are now reported in several formats, including on the ED website. In addition, ED has completed the first draft of the Strengthening HBCUs profile report. However, these data are not readily available to the public or on the internet at this time, and do not reflect program impacts.

Evidence: IPEDS data can be accessed at the College Opportunities On-line Website at http://nces.ed.gov/ipeds/cool.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Data on long-term goals are collected through the electronic Annual Performance Report for Title III programs. The antecedent measures capture the percentage of institutional project goals that are successfully completed in the areas of academic quality, institutional management and fiscal stability, and student services and student outcomes. The program exceeded all of its annual targets in 2002 and two of its targets in 2003. Thus, the program appears to be on target to achieve the long-term goals, which have been projected through 2011. In FY 2002, grantees completed 88% of their academic quality projects, 78% of their institutional management and fiscal stability projects, and 86% of their student services and student outcomes projects. The three targets were set at 75%. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. However, ED has replaced all of these measures with three new outcome measures reflecting enrollment, persistence, and graduation rates at HBCU grantees. These measures have also been established for the Title V Developing HSIs program, Strengthening Institutions program, and Howard University. These measures, except for the persistence measure, have been established for the Strengthening HBGIs program. This change will result in consistent measurement across the ED programs that focus on strengthening institutions that serve large minority populations. HBCU program goals for these measures are projected through 2011. According to IPEDS data, enrollment at HBCUs (defined as the number of enrolled full-time undergraduate students for all HBCU grantee institutions) increased by 9.6% from 2001 to 2002 compared to 5.8% for all Title IV institutions. Persistence at HBCUs (defined as the median retention rate for full-time, first-time undergraduate students at HBCU grantee institutions) was 64% for the fall of 2003 compared to 67% for all Title IV institutions. The graduation rate at 4-year institutions (defined as the percentage of full-time undergraduate students completing a 4-year degree within six years of entry for all 4-year HBCU grantee institutions) was 36.4% for the fall of 2002 compared to 38.2% for all Title IV institutions.

Evidence: FYs 2005 and 2006 Program Performance Plans; Title III Electronic Annual Performance Report. Data for the new measures are derived from electronic annual performance reports for HBCU grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program (including program partners) achieve its annual performance goals?

Explanation: HBCU performance, as measured by the program's antecedent annual measures, has been generally positive. Grantees have exceeded their completion targets for three separate measures of activity. In FY 2003, grantees completed 80% of their academic quality projects, 72% of their institutional management and fiscal stability projects, and 81% of their student services and student outcomes projects. The three targets were set at 75%. In addition, adequate information is not available to determine the program's performance on its new annual measures: persistence and graduation rates.

Evidence: Data for the new measures reflecting persistence and graduation rates (the same as the long-term measures) are derived from electronic annual performance reports for HBCUs program grantees and NCES/IPEDS data. IPEDS data are reported by all institutions participating in these programs, and are subject to NCES' established consistency and validity checks.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: An efficiency measure and baseline data were recently developed, but no targets or trend data are available to demonstrate improved efficiencies. The efficiency measure is based on the number of students that successfully complete their programs. The measure is determined by dividing the program's annual appropriation by the number of students graduating in that year. The same measure has been established for the HSIs program and the HBGIs program, as well as for Howard University. This change will enable ED to assess program performance across similar types of institutions.

Evidence: Data for FY 2003 show that the Federal program cost per graduating student is $8,631. (FY 2003 appropriation for Strengthening HBCUs program: $214.0 million. Total number of graduating students: 24,796).

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Although there are many federal, state, local, and private efforts that provide funding/services to HBCUs and their students that may be comparable, outcome data are not available on these programs to provide the basis for meaningful comparison. However, ED's establishment of similar performance measures for its programs that focus on strengthening institutions may eventually make comparisons of these kinds of programs possible.

Evidence: NA

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No independent evaluations of the Strengthening HBCUs program have been conducted recently. However, the Department is currently implementing a study on the financial health of institutions of higher education eligible to participate in the institutional assistance programs authorized by Titles III and V of the HEA. A purpose of the analysis would be to determine whether the financial status of the institutions is improving or becoming worse and to identify what drivers are impacting the financial health of institutions. The analysis would also examine whether enrollment, persistence, and graduation are drivers of financial health. In addition, the analysis would show whether the programs authorized by the HEA are positively impacting the institutions financial health. In October 1995, work began on a three-year performance measurement study of the U.S. Department of Education's Title III Strengthening Institutions Program. The primary purpose of this study was to document program accomplishments, and identify ways performance could continue to be measured and improved. Importantly, the study found that the Title III schools play an important role in serving students who have been historically underserved by postsecondary institutions. About half of the 21 case study respondents reported that the grants were used as seed money for developing new academic programs. Almost as many said that Title III provided funding for the introduction or upgrading of information technology. The Title III funds were primarily spent on technology, new learning labs or academic skills centers, and curriculum development. The schools describe the focus of their Title III grants as strengthening academic programs and resources, and outcomes for students, rather than on institutional management. For Part A schools pursuing academic programs and resources activities, the expected benefit cited most often as one of the top two expected benefits was "improvement in students' academic performance." For Part B schools pursuing these activities, the expected benefit cited most often was "improvement in quality or effectiveness of existing academic programs."

Evidence: Data from the Study of the Financial Health of Institutions of Higher Education Eligible to Participate in the Institutional Assistance Programs authorized by Titles III and V of the HEA will be available in FY 2007. Performance Measurement Study of the U.S. Department of Education's Title III Strengthening Institutions Program may be found on ED's website at www.ed.gov/rschstat/eval/highered/title3perfmeas/index.html (Mathtech, Inc., 1995).