ExpectMore.gov


Detailed Information on the
Health Professions Assessment

Program Code 10000276
Program Title Health Professions
Department Name Dept of Health & Human Service
Agency/Bureau Name Health Resources and Services Administration
Program Type(s) Competitive Grant Program
Assessment Year 2002
Assessment Rating Ineffective
Assessment Section Scores
Section Score
Program Purpose & Design 60%
Strategic Planning 72%
Program Management 73%
Program Results/Accountability 13%
Program Funding Level
(in millions)
FY2007 $302
FY2008 $318
FY2009 $66

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2007

Analyze grantee data submitted for 2007 and establish baselines and targets for new performance measures.

Action taken, but not completed Data collection initiated 12/30/07. Will analyze and clean grantee data and determine usefulness of the new measures. Establish baselines and targets by 12/2009. (June 08 update)
2007

Review grantee performance information and identify and disseminate exemplary best practices.

Action taken, but not completed Contract to conduct analysis of selected programs was included in the FY 2008 operating plan. Funding was not available. Program will seek approval to delete this item from the improvement plan. (June 08 update)

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2003

Proposes to continue the phase-out of most health professions grants and direct resources to activities that are more capable of placing health care providers in medically underserved communities.

Completed
2005

Undertake efforts to clarify purpose of the program.

Completed Articulated in program's strategic plan and discussed at All Grantee meeting.
2003

Establish common programmatic goals and develop new measures that hold grantees accountable for program results from program activities.

Completed New measures have been developed and approve by the agency Administrator. Grantee reporting guidance was modified to collect additional data and published for public comment. The new data elements were programed into the Comprehensive Performance Measures System and are being tested. Pending OMB approval, data collection from grantees will begin on 12/31/2007. (Dec. 07 update)
2003

Increase activities to support and promote basic nursing.

Completed
2003

Develop baseline data for Long-Term goals.

Completed
2003

Conduct independent evaluation

Completed Program has completed several evaluation studies, including: (1) Report to Congress on: The Effectiveness of Title VII Programs, (2) Advisory Committee on Training in Primary Care Medicine and Dentistry: Evaluating the Impact of Title VII, Section 747 Programs, (3) An Annotated Bibliography: Evaluations of Pipeline Development Programs Designed to Increase Diversity in Health Professions. (11/06 update)

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Proportion of persons who have a specific source of reliable, continuing healthcare


Explanation:

Year Target Actual
1999 NA 84%
2000 NA 85%
2001 NA 88%
2010 92%
2013 92%
Long-term Outcome

Measure: Proportion of health professionals completing funded programs that are serving in medically underserved communities (These communities have too few primary care physicians, higher infant mortality rates, lower family incomes and often an older population.)


Explanation:

Year Target Actual
2010 40%
2013 40%
Long-term/Annual Outcome

Measure: Proportion of health professionals completing Health Professions funded programs who are underrepresented minorities and/or from disadvantaged backgrounds


Explanation:

Year Target Actual
2001 NA 42%
2004 40% 39%
2005 43% 57%
2006 50% Data lag: Sept-08
2007 50% Dec-08
2008 50% Dec-09
2009 50% Dec-10
2010 50%
2013 50%
Annual Outcome

Measure: Increase the proportion of trainees in Titles VII- and VIII-supported programs training in medically underserved communities.


Explanation:The FY 2004 target was set in PART before baseline was known and could not be changed. Some targets may be shown as NA (not applicable) due to programmatic changes, program is developing more applicable measures.

Year Target Actual
2001 NA 52%
2004 30% 40%
2005 41% 43%
2006 43% Data lag:Sept-08
2007 43% Dec-08
2008 43% Dec-09
2009 NA NA
2010 43%
Annual Outcome

Measure: Increase the percentage of health professionals supported by the program who enter practice in underserved areas.


Explanation:The FY 2004 target was set in PART before baseline was known and could not be changed. Some targets may be shown as NA due to programmatic changes, program is developing more applicable measures.

Year Target Actual
2001 NA 19%
2004 30% 39%
2005 21% 35%
2006 35% Data lag: Sept-08
2007 35% Dec-09
2008 35% Dec-10
2009 NA NA
2010 40%
Annual Efficiency

Measure: Maintain the average cost to the program per graduate or program completer of providing pipeline and formative health professional education and training.


Explanation:

Year Target Actual
2004 Baseline $456
2005 $456 $460
2006 $456 Data lag- Sept-08
2007 $456 Dec-09
2008 $456 Dec-10
2009 $456 Dec-11
2010 $456

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The agency articulates the program purpose as addressing the failure of the market to assure an adequate distribution of health care providers to all areas of the country and all population groups. While in itself clear, this core purpose described by program managers is not cited or emphasized in the authorizing legislation, views of interested parties, or agency documents. The legislative structure and number of problems the program could conceivably address have resulted in a wide variety of purposes held by interested parties. The program primarily provides grants to academic institutions to subsidize the cost of health professions education and training. The grants include primary care, dentistry, nursing, geriatrics, pediatric dentistry, rural health, allied health, preventive medicine, public health, and health administration. The three most commonly cited purposes are to improve the supply, minority representation, and distribution of health care providers. Various efforts tie to market failure, but the variety of stated purposes presents significant challenges, including to show an impact in each area.

Evidence: The legislative history of the Health Professions program consists primarily of a layering on of authorizations, followed by limited consolidations. In 1956, the first major authorization in the Public Health Service Act for the general training of health practitioners focused on increasing the supply of nurses and mental health professionals. Today, the Health Professions constitute over 40 separate activities. Some of the Health Professions activities correspond directly with one of the frequently cited purposes, such as training for diversity. In general, the authorizing legislation itself does not specifically emphasize the most frequently cited purposes of the Health Professions program, but instead establishes a list of programs each with its own purposes and funding. The Administration has tended to focus on diversity and distribution. Congressional committees often focus on the program as a means of helping rural areas. Advocates also emphasize the financial vulnerability of funded institutions and the amount of program funding that is provided by State or discipline.

NO 0%
1.2

Does the program address a specific interest, problem or need?

Explanation: Diversity and distribution of health professionals are specific and current problems that the program is designed to address. Health Professions training grants were created nearly 40 years ago in response to an anticipated national shortage of physicians. Since that time, the program has developed to address a number of different issues. Some Health Professions grants are specifically designed to provide assistance to minority and disadvantaged individuals. In addition to the distribution and diversity of health professionals, a key specific problem that is still relevant to current conditions includes the supply of nursing professionals. Many other program purposes do not respond to currently relevant problems.

Evidence: Data are available on the problems of poor distribution and diversity of health professionals, and the supply of nursing professionals. For example, the agency projects a 13% shortage of registered nurses in 2010; under-represented minorities account for 26% of the population, but African Americans and Hispanics compose only 12% of the health professional workforce; roughly 20% of Americans live in rural areas, but only 9% of physicians practice there.

YES 20%
1.3

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: The Health Professions program is divided among various authorities with a multitude of goals and purposes and is not designed to have a significant impact on any one factor such as diversity, distribution, supply, or quality. Further, the national impact of the program in these areas is generally not known. Training of the Nation's health professionals is a large and complex problem. The program has a very broad reach. For most awards supported by the program, there are no matching requirements, but some grant activities have the effect of leveraging other funds and the program is credited with helping launch new training programs in institutions by providing seed money. In addition, disadvantaged students benefiting from scholarships and loan subsidies report the support makes a significant impact in their ability to continue their education. Also, the growth of managed care can reduce the amount of discretionary revenue available to teaching hospitals.

Evidence: The program funds 1,700 institutions nationwide, constituting a significant reach, and institutions receiving Title VII and VIII support succeed in also receiving state funds. However, each issue the program is designed to address today presents a significant challenge on its own upon which the impact of the program is not known. Health care is a labor intensive industry and requires a high level of investment in education and training. An estimated 39 million people lack health insurance. According to agency estimates, there are over 3,000 primary medical health professional shortage areas that would need over 14,000 primary care physicians to meet national standards. The US has the highest health spending as a percentage of GDP in the world. According to a UCSF report, less than 15% of medical graduates choose residencies for primary care.

NO 0%
1.4

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: Health professions institutions receive Federal support from numerous sources. The Health Professions program is different in structure and goals from Federal graduate medical education (GME) subsidy payments for Medicare and Medicaid. Federal Medicare GME statutes and Medicaid policies do not specify specific policies and purposes to drive desired outcomes. Medical schools also receive significant resources from the National Institutes of Health, but to support research and research professionals. While the Bureau of Labor Statistics tracks health careers, the program is also the only Federal entity dedicated to studying healthcare workforce supply and demand. A key focus of the program is the distribution of primary care and other health professionals. The National Health Service Corps shares that general purpose, but has an entirely different design. NHSC is focused on improving care in targeted communities and supports professionals through a different mechanism and stage in the career.

Evidence: Payment for residency training in medicine dates back to the original Medicare and Medicaid legislation of 1965. At $378 million in FY 2002, the Health Professions program is a fraction the size of Medicare and Medicaid GME payments. The FY 2002 Budget for the National Institutes of Health was $23.6 billion. The National Health Service Corps aims to improve the distribution of physicians by providing loan repayment awards and scholarships to healthcare providers in exchange for serving in an underserved community.

YES 20%
1.5

Is the program optimally designed to address the interest, problem or need?

Explanation: The program is administered through competitive grants and cooperative agreements to academic and medical institutions and contracts and awards to individual students and faculty, providing direct contact to influence changes at the institutional and student or faculty member level. Having a clearly stated purpose will aid in planning and budgeting and will also help clarify program purpose among interested parties over time.

Evidence: There is no evidence that providing support through a block grant or other mechanism would be more effective or efficient than competitive awards direct to institutions.

YES 20%
Section 1 - Program Purpose & Design Score 60%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program adopted new long-term goals during the assessment process. The long-term measures focus on the program's national impact with respect to regular access to a health care provider, the portion of program beneficiaries who go on to serve in target areas, and the portion of program beneficiaries who are underrepresented minorities and/or from disadvantaged backgrounds.

Evidence: The program has three long-term measures with targets: 1) Increase the proportion of persons who have a specific source of ongoing care to 96% by 2010; 2) Increase the proportion of health professionals trained in Titles VII and VIII Health Professions supported programs serving in medically underserved communities to 40% by 2010; 3) Increase the proportion of graduates and program completers of Title VII and VIII Health Professions supported programs who are underrepresented minorities and/or from disadvantaged backgrounds to 50% by 2010. Reliable baseline data are not yet available.

YES 14%
2.2

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: The program has adopted a limited number of annual performance goals that demonstrate progress toward desired long-term outcomes. These goals are clustered in two areas: eliminate barriers to care and eliminate health disparities.

Evidence: Health Professions annual goals include: 1. Increase the number of graduates and/or program completers who enter practice in underserved areas, 2. Increase the number of graduates and/or program completers of health professions primary care tracks and programs that support primary care, 3. Increase the number of minority/ disadvantaged graduates and program completers.

YES 14%
2.3

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: Individual service grantees provide performance data through a common reporting system to measure annual goals. Further steps to use data to reward performance could encourage additional buy-in to program goals.

Evidence: Grantees report on performance data for the annual goals through the agency's Comprehensive Performance Management System (CPMS) and Uniform Progress Report (UPR). The agency has been working to improve the timeliness and response rates for those data. Project officers review data against application targets.

YES 14%
2.4

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: The program has been looked to as a partner within the Federal government because it provides grants to such a large number of health professions institutions. The program includes a group of interdisciplinary program grants specifically designed to improve collaboration between academic institutions and states and communities, and has promoted practitioner level collaboration through its innovation awards. The program has worked with other bureaus within the Department in geriatrics, substance abuse faculty development and chiropractic demonstrations. Medicare, through its reimbursement for teaching costs related to the provision of services to Medicare beneficiaries, is the largest explicit Federal source of graduation medical education funding. However, Medicare's statutory purpose is not designed to meet physician workforce policy goals and the program is limited in its ability to collaborate with Medicare on workforce policy issues.

Evidence: The program collaborates with numerous national organizations such as the Federation of Associations of the Schools of the Health Professions, Council on Medical Education, the American Medical Student Association, and multiple professional associates. According to the National Conference of State Legislatures, the program also works with states, which are often focused on addressing health professions distribution issues. Additional collaboration with other Federal activities that share similar goals such as the National Health Service Corps in the form of meaningful actions in management and resource allocation may be beneficial.

YES 14%
2.5

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: Regularly scheduled objective, independent evaluations of the program are not supported. While the program has some outcome data in its GPRA performance report, there are insufficient data on the effectiveness of the program overall at meeting key objectives to require evaluations that merely fill gaps in performance information.

Evidence: Reports from the General Accounting Office in 1994 and 1997 pointed to a lack of comprehensive evaluations of the Health Professions program. Some targeted evaluations have been conducted. An evaluation of the Health Careers Opportunity Program was conducted in 1994 by Houston Associates, Inc. The program plans a contract with the Institute for College Research Development and Support to examine the number of HCOP program participants that enter and graduate from health professionals school. HRSA supported an evaluation of the Centers of Excellence in 2001. Some surveys have been used such as with Title VIII and Faculty Loan Repayment. Evaluations of the Area Health Education Centers and Workforce Information and Analysis are planned. Evaluations of other programs have been published in journals, such as for the Interdisciplinary Generalist Curriculum and Faculty Development Fellowships.

NO 0%
2.6

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: The program does not base a determination of the level of financial resources on what is needed to obtain annual and long-term goals. Nor does the program tie specific funding levels to each discrete output goal. The task of alignment for this program is made more difficult by the number of discrete grant activities. The program is able to estimate outputs based on past experience, but cannot estimate unit costs and cannot allocate resources by output goal. The program has struggled in advancing its strategic planning and setting budgets according to what is needed to obtain goals in part because of stark differences between annual budget request and final appropriations. Certain sub-activities such as scholarships may be more able to align budget and legislative changes with performance.

Evidence: This assessment is based on the annual budget submission to OMB and the Congress, and other information provided by the agency.

NO 0%
2.7

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: The main deficiencies highlighted in this section are in conducting comprehensive and independent evaluations, and integrating budget and performance. The agency overall is making organizational changes which will further integrate budget and performance planning. Additional work is needed to schedule comprehensive evaluations of ongoing programs.

Evidence: The assessment is based on discussions with the agency. Title VIII programs are working with George Mason University to improve their understanding of the impact of funding, policy and legislative changes on performance. Evaluations of the Area Health Education Centers and Workforce Information and Analysis are planned. The agency's electronic data system can also improve the use of performance information in budgeting and planning.

YES 14%
Section 2 - Strategic Planning Score 72%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: There is little evidence to date of the program overall using performance data to adjust program priorities, make resource reallocations, or take other management actions. Performance information is collected annually from award recipients. The agency collects data through the CPMS/UPR data management system. Other need-based programs rely on financial status reports of award recipients. These data are primarily used to monitor grantee compliance with project goals and objectives and to design technical assistance for poor performers. There are exceptions where more recently, data are being used by managers in budget and management decisions.

Evidence: Some evidence of exceptions is available, including the Health Careers Opportunity Program use of performance information to adjust future program efforts. Program managers added a funding priority for enhancing enrollment in generic baccalaureate nursing education.

NO 0%
3.2

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: The agency's senior managers are held accountable for operations of their programs, including performance results, through their annual performance contracts. HRSA reports all of its SES personnel have performance contracts with goals, standards and outcomes that are results oriented. For many Health Professions grants, continued funding requires meeting grant objectives. Accountability of award recipients could be improved and performance information could be extended to program staff performance evaluations or contracts.

Evidence: The Centers of Excellence program reports funding only those continuations that meet program goals. Scholarships for Disadvantaged Students recently increased performance levels as a condition of receiving additional funds. The Health Careers Opportunity Program rates renewal grant applications based on past performance. In the last grant cycle, of the 34 renewal applications submitted, 13 were approved. Nursing Workforce continuations are also based on past progress. The FY 2003 Primary Care Medicine and Dentistry application includes quality of objectives and outcome measures in the review criteria. The ability of the program to hold some grantees accountable through reductions in future awards could be limited by the pool of potential applicants because there are a limited number of accredited programs eligible for funding.

YES 9%
3.3

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: The program obligates funds in a timely manner. Scholarships are made in time to reallocate declined awards. Award recipients report on planned and actual expenditures. There have been very few known cases of funds being expended outside of their intended purpose. Project officers perform site reviews when possible.

Evidence: Assessment based on apportionment requests; annual budget submissions and financial reports, queries in Single Audit Database and agency grants management procedures. Many awards are made to conform to the academic calendar.

YES 9%
3.4

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: In general, the program does not have incentives and procedures in place to improve efficiency and cost effectiveness in program execution. The agency did begin collecting data from grantees electronically for the first time in FY 2002 and plans an expansion of electronic transactions.

Evidence: There is little evidence that the program has incentives and procedures in place to improve efficiency and cost effectiveness in program execution.

NO 0%
3.5

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: The program does not have a procedure for splitting overhead and other costs between outputs. The program does not have a financial management system that fully allocates program costs and associates those costs with specific performance measures, or even a consistent way to develop full cost of achieving performance goals. The program does not capture all direct and indirect costs borne by the program agency, including applicable agency overhead, retirement, and other costs budgeted elsewhere, or include informational displays in the budget that present the full cost of outputs. Given a budget total, the program can estimate indirect costs and administrative costs of awards based on ceilings established in legislation and grants policy, administrative costs and overhead, and predict the number of students trained and other outputs.

Evidence: The assessment is based on annual budget submissions to OMB and Congress. The program does not have an agency program budget estimate that identifies all spending categories in sufficient detail to demonstrate that all relevant costs had been included or a report that shows the allocation of overhead and other program costs to the program. Program managers budget for grants, grant review, travel and technical assistance. Staffing, space, and overhead are budgeted for within the agency program management budget line.

NO 0%
3.6

Does the program use strong financial management practices?

Explanation: HRSA received its first clean audit in 1999. The 2000-2001 agency financial statements showed no material weaknesses. HRSA financial statements are conducted by the Program Support Center. The OIG found in a 2002 audit of HRSA's travel, appointments, and outside activities that there was no evidence of substantive violations, but that there are technical lapses requiring improvement. The agency disagrees with the breadth of the problem and has re-issued guidance to improve oversight. The OIG FY 2001 report notes cites weaknesses in HRSA's grant accounting systems found by an independent auditor and cites for example that Health Professions expenses increased 150% despite total appropriations increasing 75%.

Evidence: In a serious of audits of universities participating in the health professions student loans program, the OIG found universities were generally in compliance, but inappropriately carrying uncollectible loans in their accounting records. The OIG has recommended that the agency better emphasize regulations on uncollectible loans in the program.

YES 9%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The agency is taking steps that could improve its efficiency, including plans to extend electronic transactions. The program is taking steps to further integrate performance in review criteria for some grants. Additional steps are needed to improve the use of performance information to make budget and management decisions.

Evidence: The agency is moving toward an electronic application process, which may improve efficiency in program execution. Federal staff office consolidations and reorganizations the agency is undergoing may improve the efficiency of Federal staff allocations.

YES 9%
3.CO1

Are grant applications independently reviewed based on clear criteria (rather than earmarked) and are awards made based on results of the peer review process?

Explanation: Most grants and cooperative agreements are awarded using a peer-review process with clear criteria. Annual appropriations bills do not earmark funds for grant recipients in the program. Overall, the agency's process is open and based on objective criteria.

Evidence: Assessment based on grant review procedures and Federal Register Notices.

YES 9%
3.CO2

Does the grant competition encourage the participation of new/first-time grantees through a fair and open application process?

Explanation: The program operates a fair and open competition within the guidelines of its authorizing legislation and provides a reasonable amount of outreach. The application process used by the program encourages the participation of new/first time grantees through preferences and priorities. Grant announcements and materials are available on the agency's web site and the agency hosts regional meetings, conference calls and one-on-one contacts to provide technical assistance to new grantees. Many Title VII program award recipients have received funds for over 30 years. The number of eligible applicants for some grants is limited to accredited programs, which increases the likelihood that the same institutions will receive grants time and again. However, increased reliance on performance data from those institutions is merited to discontinue funding to schools that do not meet standards required for the program to succeed in meeting its new performance measures.

Evidence: Assessment based on agency announcements and historical data on grant awards. The program notes that 50% of competitive applicants awarded primary care and medicine grants in FY02 had not received funding the previous year. The August 9 2002 Federal Register notice specifies a funding preference for new programs. Title VII primary care grants have provided support to 100% of the Nation's family medicine departments in medical schools. The agency cites a funding priority for Title VIII nurse managed centers that have not received funding previously.

YES 9%
3.CO3

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Award recipients provide data annually to the agency on performance and expenditures. Project officers also work directly with grantees. Site visits are made for special cases to monitor progress. Scholarship programs collect data through applications and annual financial status reports.

Evidence: Data are gathered in annual reports. Additional information is gathered from site visits and contact with project officers. The Health Careers Opportunity Program and Centers of Excellence project officers complete quarterly and annual reports on assigned grantees.

YES 9%
3.CO4

Does the program collect performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Award recipients provide data annually to the agency. Annual data are summarized in the performance report and made available on the agency web site. On a less systematic basis, performance data are also presented at conferences and other public presentations.

Evidence: Assessment based on agency GPRA reports and web site (www.hrsa.gov).

YES 9%
Section 3 - Program Management Score 73%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: The program has adopted new long-term goals for the program to measure outcomes, but needs more than one year of data to show progress for the first goal. A small or large extent would require data for the second and third measures and more definite progress on the first measure. The first measure, the proportion of persons who have a specific source of ongoing care, is a proxy measure correlated with improving access to care. Data indicate uneven progress, but some improvement. The following two measures focus on outcome of training with respect to the proportion of program beneficiaries who are serving in medically underserved communities and who are from underrepresented minorities and/or disadvantaged backgrounds. The measure on minority and disadvantaged backgrounds excludes grantees in a few states prohibited by law from collecting the data.

Evidence: The baseline year for these goals is 2001 and in most cases 2002 data are not yet available. The target year for the long-term goals is 2010. The first measure is not subject to changes in definition and area fragmentation that limit the utility of tracking impact through shortage area designations. While the measure does not capture all of the specific activities of the program, it is the most focused on final outcomes from the perspective of the problem and relates directly to the bulk of program efforts.

NO 0%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: The agency has the most direct influence over the percentage of health professionals who benefit from the program that train in these areas. Because of this influence and the correlation between training in underserved areas and eventually practicing in underserved areas, the program believes the first annual measure will provide data most useful to ongoing management with respect to improving the distribution of health professionals. The first measure is also significant for interdisciplinary grants funded by the program. Annual output data are available in the agency's annual performance plans. Performance on previously held related measures has exceeded goals in some areas including the number of students in training with organizations serving underserved areas and the number of minority/disadvantaged graduates and program completers. Actual performance has declined in some key goals, including number of graduates entering underserved areas and number going into primary care and the number of disadvantaged enrollees.

Evidence: Related to the first measure, in FY 1999, 32,629 residents/graduates, students/trainees and faculty supported by the program were training in underserved areas, up from roughly 26,300 in 1998. Related to the new second annual measure, in FY 1999 4,336 health professionals entered service in underserved areas out of roughly 89,295 total program completers (4.9%). Related to the third measure, 10,158 health professions residents/graduates and faculty are from underrepresented minority/disadvantaged backgrounds. Comparable data from FY 1998 or FY 2000 are not available.

SMALL EXTENT 7%
4.3

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The bulk of evidence shows with respect to the performance of program grantees, the program has not demonstrated improved efficiencies and cost effectiveness in achieving the program's annual goals. In addition, the OIG found in 2002 that institutions participating in the faculty loan repayment program frequently waive matching requirements, reducing the impact per Federal investment.

Evidence: The total Federal investment per placement in an underserved area has increased over the last three years. The total Federal investment per clinician trained and per minority graduate has decreased. The total Federal investment per primary care graduate, per minority enrollee, and per minority faculty has also increased. An exception involves Title VIII programs' use of conference call peer review rather than on-site review for small grants limited to $25,000.

NO 0%
4.4

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: The program is not part of the common measures exercise. However, there are some programs that support similar efforts. Federal Graduate Medical Education payments support training in the health professions. With respect to programs that share the same goals, the National Health Service Corps shares the goal of placing providers in underserved areas. Neither GME nor NHSC provides a direct comparison, but the NHSC is most closely aligned with respect to program goals. The program's performance comparison between the two programs is mixed.

Evidence: By statute, the program provides more direction than GME and its grant recipients and program completers are more likely than the national average to provide care in underserved areas and represent a minority background. GME payments are not directed to proactively encourage improvement in the diversity and distribution of the nation's healthcare workforce. With respect to Health Professions' sister entity, the NHSC, the program is less efficient in placing medical professionals in shortage areas than the NHSC. According to the most recent data available, in 2000 the average cost per placement was $77,400 for the Health Professions and $47,900 for the NHSC.

SMALL EXTENT 7%
4.5

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: The agency has not had a comprehensive evaluation on the program as a whole, or on the main components including training grants and loans and scholarships. Prior to the latest reauthorization of the Health Professions programs, GAO noted in 1997 that effectiveness has not been shown and the impact of the components will be difficult to measure without common goals, outcome measures, and reporting requirements. Academic studies of the issue indicate the underlying premise of the program, to reduce shortage areas by training professionals who may be more likely to serve there, could work. For example, researchers have found publicly owned medical schools in rural states have higher proportions of graduates entering practice in rural areas than private medical schools that are not focused on family medicine and are located in urban areas.

Evidence: No comprehensive evaluations are available, but there are some performance evaluations available with varied findings worth noting. GAO reported minority representation has improved more quickly in the health professions funded by the program than for professions requiring only a high school degree and not funded by the program. A 2001 Mathematica report found schools receiving additional Professional Nurse Traineeship funds from the program actually have fewer graduates employed in schools with medically underserved communities than schools without. The report found requiring students to sign a commitment to work in an underserved community resulted in a higher number entering service there, an important finding for program planning efforts. The OIG found in 2002 that institutions participating in the faculty loan repayment program frequently waive matching requirements, reducing the impact per Federal investment. In relative terms, a more comprehensive 2002 study of Title VII by departments of family medicine and pediatrics was published in Family Medicine. The authors of the Family Medicine article matched grant funding from 1978 to 1993 with the specialty and practice locations of graduates of departments of family medicine. The review found 1.5% of physicians trained by institutions receiving a Title VII grant between 1978 and 1993 serve in shortage areas, compared to 1.1% of those trained by institutions not funded by the program. Institutions receiving the most grants from the program had a rate of 1.3%. The only funded institutions with a rate below non-funded institutions were those receiving only faculty training grants (0.8%). Based on these data, if funded institutions placed graduates at a rate equal to non-funded institutions, 479 fewer physicians would serve in shortage areas. The authors calculate $290 million in grants to departments of family medicine over this period. Total Federal spending for the Health Professions program from 1978 to 1993 was $5.7 billion.

NO 0%
Section 4 - Program Results/Accountability Score 13%


Last updated: 09062008.2002SPR