Detailed Information on the
Homeless Assistance Grants (Competitive) Assessment

Reduce timeline for obligation of grant awarded funds through a streamlined process.

Continue to work with Federal, State, local and nonprofit partners to accomplish the goal of ending chronic homelessness.

Post performance data from Annual Progress Reports at least annually on HUD website as well as homeless population information collected from street counts of HMIS systems.

Measure: The percentage of formerly homeless persons who remain housed in HUD permanent housing projects for more than 6 months will increase annually.

Explanation:This annual measure relates to the program's purpose of helping homeless families and individuals maintain housing over time. This is a challenging measure as many chronically homeless individuals face many barriers to maintaining housing. In 2009, this measure is being revised to account for persons who stayed in permanent housing as well as those who left the permanent housing program after remaining over 6 months.

Measure: The percentage of homeless persons who have moved from HUD transitional housing into permanent housing will be 65 percent in 2009, and will increase each year.

Explanation:This measure relates to the program's purpose of helping homeless families and individuals achieve housing. Moving to permanent housing is an indicator of increased self-sufficiency; in addition, obtaining permanent housing instead of another temporary living situation improves the likelihood that homelessness will be ended for that individual or family.

Measure: The percentage of Homeless Assistance Grant funds used for housing activities, compared to the percentage used for supportive services, will be maintained to ensure an efficient use of funds.

Explanation:Since HUD began tracking this measure in 1998, the percentage of Homeless Assistance Grant funds going toward housing has increased from 43% to 60%. HUD has achieved this by targeting housing projects for funding in the Continuum of Care application process. However, there is still a great need for services that accompany the housing, in order to help individuals and families obtain self-sufficiency. For this reason, the percentage of funds going toward housing will likely not increase beyond 60%.

Measure: The employment rate of persons exiting HUD homeless assistance projects will increase one percent each year.

Explanation:This measure relates to the program's purpose of helping homeless families and individuals achieve an appropriate level of self-sufficiency. In 2006, this measure was revised to more accurately reflect the employment rates of everyone exiting projects. For example, in 2005, 10% of entrants were employed, and 17% were employed upon exiting all programs.

Is the program purpose clear?

Explanation: The overall purpose of the McKinney Act (HUD and non-HUD) programs is to use public resources in a more coordinated manner in order to help homeless families and individuals achieve housing stability and an appropriate level of self-sufficiency. The purpose of HUD's three competitive homeless assistance programs is clearly defined in three different sections of the statute. 1) The purpose of the Supportive Housing Program is to "promote the development of supportive housing and supportive services, including innovative approaches to assist homeless persons in the transition from homelessness, and to promote the provision of supportive housing to homeless persons to enable them to live as independently as possible." 2) The purpose of the Shelter Plus Care Program is to "provide rental housing assistance, in connection with supportive services funded from sources other than this part, to homeless persons with disabilities (primarily persons who are seriously mentally ill, have chronic problems with alcohol, drugs, or both, or have acquired immunodeficiency syndrome and related diseases) and the families of such persons." 3) The purpose of the Section 8 Moderate Rehabilitation Single Room Occupancy (SRO) Program is to provide rental assistance for homeless individuals in rehabilitated SRO housing.

Evidence: Evidence: Title IV of the McKinney-Vento Homeless Assistance Act, PL 100-77, section 102 (42 U.S.C. 11301, 11381, and 11401).

Does the program address a specific and existing problem, interest, or need?

Explanation: Explanation: When Congress passed the McKinney-Vento Homeless Assistance Act in 1987, it based its action on findings that a growing number of individuals and families were becoming homeless and that, "due to the record increase in homelessness, States, units of local government, and private voluntary organizations have been unable to meet the basic human needs of all the homeless??." Only an elementary methodology existed for measuring the rate of homelessness prior to 1984. HUD's Office of Policy Development and Research submitted the first systematic national survey of homelessness, which showed that a national rate of between 250,000 and 350,000 represented a higher figure than originally estimated, and consisted of more women, minorities, and families, youths, and mentally ill persons than in the past. Homelessness remains a major problem. Between 2.5 and 3.5 million people experience at least one episode of homelessness each year. Homelessness is one of the most dire consequences of America's social problems - poverty, mental illness, the drug epidemic, and domestic violence. There is no single, simple solution to the problem of homelessness because of the different subpopulations, the different causes of and reasons for homelessness, and the different needs of homeless individuals. Homeless assistance grants help meet the housing and service needs of the homeless through funding for acquisition, rehabilitation, new construction, operating costs, leasing, supportive services, and administrative expenses for local homeless assistance providers.

Evidence: "A Report to the Secretary on the Homeless and Emergency Shelters" (1984). The Urban Institute's 1996 National Survey of Homeless Providers and Clients, published in "Homelessness: Programs and the People they Serve," by the Interagency Council on the Homeless (1999) (www.huduser.org/publications/homeless/homelessness/contents.html) and The Urban Institute's "Evaluation of Continuums of Care for Homeless People: Final Report" (2002) page 4 (www.huduser.org/publications/pdf/continuums_of_care.pdf). Also see the preamble of the McKinney-Vento Homeless Assistance Act (PL 100-77, Section 102, U.S.C 11301) and Harvard University's "Innovations in American Government" award announcement (www.ashinstitute.harvard.edu/Ash/continuum.htm).

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Other Federal, state, local, and private sector programs, working through for-profit, non-profit, faith-based, and community-based organizations, address various aspects of the housing and services needs of specific subpopulations of the homeless. HUD is the lead federal agency for addressing homelessness and the only federal agency charged with serving all of the homeless populations with housing and services regardless of their special need. In 1994, HUD created a coordinated planning and application process for the three competitive grant programs, called the Continuum of Care (CoC), to minimize any duplication. The CoC process is designed to eliminate duplicate services and redundant processes at the local level, while at the same time ensuring that the resources provided by HUD's competitive programs do not supplant other efforts. In order to receive program funds, each community ensures regular and substantive participation and coordination of the following types of participants in achieving the goal: State and local government agencies, non-profit provider organizations, faith-based organizations, public housing authorities, foundations, businesses, and homeless and formerly homeless individuals.

Evidence: The competitive NOFA application requires each CoC to analyze needs and gaps in order to identify other resources and leveraging for the Continuum. The scoring rewards continuums that successfully manage the integration of these funding streams and broad participation. See Federal Register, March 21, 2005, pages 14273 - 14287. Also see Harvard University's "Innovations in American Government" award announcement (located online at www.ashinstitute.harvard.edu/Ash/continuum.htm).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The programs were consolidated administratively in 1994 into the Continuum of Care competition, and this streamlining has allowed for the capacity-building of the local Continuum of Care planning structures. The coordinated funding mechanism of the CoC includes planning and decision-making for all three programs that respond to local needs and leverage state, local, and private resources. At the same time, the competitive process requires that all programs ration funds among local partners through a priority-setting process. This process creates strong federal incentives for targeting resources to the neediest population, achieving the program purpose, and rewarding performance. Thus, the program design includes the advantages of both a block grant of federal funds to state and local governments and a tightly focused competition. The regulations are sufficiently flexible that the program's emphases can change in response to the latest research and to changing local needs. There is no other approach or program that serves all homeless populations with housing and services that would be more efficient or effective in achieving the program's purpose.

Evidence: The application process is used to competitively assess the projects proposed for funding by the CoC in order to determine the actual budgetary award to that CoC. The NOFA adjusts scoring criteria and information collected every year to reflect the latest research and other evidence on community needs and gaps in service. This adjustment can be found in the greater emphasis toward funding permanent housing units for the chronically homeless. The funding is determined based on the CoC NOFA scoring. The flexibility of the program allows CoCs to adjust individual grants as the needs of the community change. An evaluation of the CoC process by the Urban Institute found that, as a result of the process, communities "have been able to offer more support to homeless people, with more cohesion, than would otherwise have been possible." See The Urban Institute's, "Evaluation of Continuums of Care for Homeless People" (2002), page xxi, Executive Summary (located online at: www.huduser.org/publications/pdf/continuums_of_care.pdf). Also see Mission Statement (internal document) and NOFAs from 2004 and 2005 highlighting changes (in 2004 NOFA, see Federal Register May 14, 2004 pages 27506-27508 and in 2005 NOFA see Federal Register March 21, 2005 pages 14274 and 14282-14283).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The program design ensures that resources are used directly and effectively to reach the intended population (e.g. a person must be homeless in order to receive assistance). HUD generally defines a homeless person as someone who is living in a place not meant for human habitation or in an emergency shelter or in a transitional housing if the person originally came from the streets or an emergency shelter. The CoC process effectively targets funding to activities that fill gaps between needs documented at the local level and current numbers of beds available to serve the homeless. Increasingly the gaps analysis is based on actual "street counts" of the homeless, which help identify the chronic homeless population. As the HMIS is implemented across the country, HUD and communities will be better able to target funds to identified needs.

Evidence: A 1996 study by Culhane, Metraux and Hadley found that chronically homeless persons used a disproportionate amount of emergency homeless resources (as well as health, mental health and resources from other systems). As a result, the NOFA now targets permanent supportive housing resources toward the chronic population. The documentation required for these eligibility criteria is based on a body of research about the characteristics of homeless people, including the National Survey of Homeless Providers and Clients and analysis of Homeless Management Information System (HMIS) data from several major US cities. The Urban Institute's 1996 National Survey of Homeless Providers and Clients, published in "Homelessness: Programs and the People they Serve," by the Interagency Council on the Homeless (1999) pages 3.1-3.2 (located online at: www.huduser.org/publications/homeless/homelessness/contents.html); Culhane, D.P., Metraux, S. and Hadley. (2002) "Public Service Reductions Associated with Placement of Homeless Persons with Severe Mental Illness in Supportive Housing," Housing Policy Debate, Vol. 13, Issue 1; 2004 NOFA, Applying Relative Need, Permanent Housing Requirement and Chronic Homeless Requirement sections (Federal Register, May 14, 2004 pages 27508 and 27499); 2005 NOFA, Samaritan Housing Initiative, 30% Permanent Housing Requirement and 10% Chronically Homeless Requirement sections (Federal Register March 21, 2005 pages 14284-14285);

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The overall long-term goal of the Homeless Assistance Grants is to end chronic homelessness and to help homeless families and individuals move to permanent housing and to live as independently as possible. The first part of the goal is measured with the following long-term measure: "Within the next five years (2005 - 2009), HUD will create 40,000 new units of permanent housing for chronically homeless individuals. The second long-term measure is by 2009, 65% of all households that leave transitional housing will move to permanent housing.

Evidence: See the Evidence section of 1.5 for why the goal to end chronic homelessness was adopted. HUD's 2005 Annual Performance Plan, pages 110-114 (located online at: www.hud.gov/offices/cfo/reports/2005/2005app.pdf); 2005 NOFA, Performance Measurement section (Federal Register, March 21, 2005 pages 14274 and 14283); and 2005 application, Continuum of Care Exhibit 1 (Federal Register, March 21, 2005 pages 14314-14315). Also see Measures section.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: In light of current available resources, the target for the creation of new permanent housing subsidies is ambitious but realistic. In order to reach the goal of 40,000 beds within the next 5 years, HUD's targets include creating on average at least 8,000 units on a yearly basis between 2005 - 2009. Successfully moving the majority of households from transitional to permanent housing is also an ambitious goal because currently only 60% of households move directly to permanent housing. Any higher goal could lead grantees to serve easier-to-house families and individuals.

Evidence: HUD's 2005 Annual Performance Plan, pages 110-112. See HUD's 2006 Annual Performance Plan pages 114-117 (not yet published).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: HUD's annual performance measures gauge progress in moving homeless families and individuals to permanent housing and self-sufficiency. These measures, negotiated with OMB in 2003, and reflected in the 2005 Annual Performance Plan, allow Continuum of Care communities and individual grantees to evaluate their progress on each of the measures through the Annual Progress Report (APR) and annual applications for funding. The efficiency measure that assesses the percent of homeless funding for housing activities is also emphasized through the annual competitive process.

Evidence: 2005 APP Plan, pages 110-113. See HUD's 2006 Annual Performance Plan pages 114-117 (not yet published); Annual Progress Report questions 11, 12, and 14 (located online at: www.hudclips.org/sub_nonhud/html/pdfforms/40118.doc); 2005 NOFA Performance Measurement section (Federal Register, March 21, 2005 page 14283); and 2005 Continuum of Care application, CoC Project Performance - Housing and Services section (Federal Register, March 21, 2005 pages 14314-14315). Also see Measures section.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Baselines and targets are produced for the APP as well as for an annual report to Congress. The baselines for the annual goals were established in 2003 or prior years. Data on progress towards these goals is collected at the Continuum of Care level as part of the national application process and is re-tooled each year to ensure that the targets set remain ambitious. Due to the ongoing challenge of helping homeless persons become stably housed, a one percent increase in each of the measures is currently an ambitious target. Any higher goal would lead to grantees targeting easier-to-house clients.

Evidence: HUD has three annual measures: 1) The percentage of formerly homeless persons who remain housed in HUD permanent housing projects for more than 6 months will be 70 percent in 2005, and will increase one percent each year; 2) The percentage of homeless families and other homeless persons who have moved from HUD transitional housing into permanent housing will be 60 percent in 2005, and will increase one percent each year; and 3) In 2005, the employment rate of persons exiting HUD homeless assistance projects will be 10 percentage points greater than the employment rate of those entering, and will increase one percent each year. See HUD's 2005 Annual Performance Plan pages 111-113 (located online at: www.hud.gov/offices/cfo/reports/2005/2005app.pdf), and Administrative Requirements and CoC Project Performance - Housing and Services sections of 2004 and 2005 NOFAs (Federal Register, May 14, 2004 pages 27510, 27533-27534, and Federal Register, March 21, 2005 pages 14283, 14314-14315). Also see Measures section.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: HUD has incorporated its goal of moving families and individuals to permanent housing into the year-round community-based planning process through the CoC and the NOFA application. Each community must report in the APR on its progress in working toward the annual and long-term goals of the program. The three outcome measures that use APR data were changed from gross numbers to percentages so that the data can be more readily available for analysis of performance at the Continuum of Care level. Many other Federal, State, local and private agencies have embraced the goal to end chronic homelessness and have identified (both HUD and non-HUD) resources for developing local 10-year plans to end chronic homelessness.

Evidence: The CoC NOFAs for 2004 and 2005 awarded points for each CoC's progress in reducing homelessness, "measured by program participants' success in moving to and maintaining permanent housing" (i.e., through scores on two of the performance measures) (see Federal Register, May 14, 2004 Performance Measurement and CoC Project Performance - Housing and Services sections, page 27507, and 27533-27534). Also see Annual Progress Report questions 11, 12, and 14 (located online at: www.hudclips.org/sub_nonhud/html/pdfforms/40118.doc).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: HUD conducted systematic evaluations of each of the new HUD McKinney-Vento programs early in the program's implementation in order to make adjustments to program design. HUD conducts research, as needed, to ensure that major policy issues facing the the homeless programs and their implementation are informed by high-quality research. For example, evaluations have focused on the impact of the Continuum of Care planning process; on housing and self-sufficiency outcomes for mentally ill and formerly homeless clients leaving HUD-assisted Transitional Housing; and on the appropriateness of the Transitional Housing approach for homeless families. HUD will be able to use the 80-community sample created to produce nationally representative data for the Annual Homeless Assessment Report to build a research platform for future evaluations related to homelessness.

Evidence: The Department has contracted with some of the most respected social policy research firms and researchers in the country, including the Urban Institute, Westat, Abt Associates Inc., ICF Consulting, Walter R. McDonald and Associates, M. Davis and Company, Planmatics, UPENN, the Barnard-Columbia Center for Urban Policy at Columbia University, and others, to conduct these analyses. Those professionals have designed and implemented the evaluations and research to be rigorous, objective and of sufficient scope to provide implications for national program design. Additional evaluations, such as the NewYork/New York Agreement Cost Study by Culhane, Metraux, and Hadley, have been useful in providing research grounding for policies and goals.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: HUD's 2006 budget request for Homeless Assistance Grants is tied specifically to accomplishing the long-term goal of ending chronic homelessness. This goal can be accomplished by creating 150,000 permanent supportive housing units. HUD requested nearly $200 million in new funds for 2006 to create approximately 8,000 more units towards this goal. In addition, the budget request highlights the intention to focus more HAG funds towards housing. This request is in line with the efficiency measure of increasing spending on housing versus services referenced in the 2006 budget. In terms of indirect costs, the 2006 budget request identifies 64 FTE needed in HUD field offices for homeless assistance program management and monitoring, as well as 33.1 FTE for headquarters homeless assistance program management.

Evidence: 2006 Congressional Justifications for the Homeless Assistance Grants program contain the indicators developed for inclusion in HUD's Annual Performance Plan. Annual budget estimates include the following spending categories: grants, technical assistance/data analysis, working capital fund, the national homeless data analysis project, Shelter Plus Care renewals, and prisoner re-entry initiatives (requested 2005 and 2006). The effect of additional funding is reflected in out-year projections, with increased targets for specific indicators, such as the percentage of homeless that move from transitional to permanent housing. See budget document, especially Performance Measurement Table, page D-14 and Staff Requirement FTE tables on pages K-8, K-9, and K-12.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: HUD's strategic plan includes the development of outcome-based performance measures that can be assessed at the Continuum of Care level. The Homeless Assistance Grants program has taken steps to improve its performance measures and to focus on outcomes directly related to achieving the program's purpose, rather than on outputs and process. Beginning in 2003, HUD changed its performance measures to focus more directly on achieving two key outcomes: housing stability and self-sufficiency, while earlier measures focused on outputs and competition measures.

Evidence: For example, one of HUD's homeless goals in 2000 was "The share of the population living in communities with a Continuum of Care system will increase by 1 percent, to 84 percent." This measure is no longer applicable since now nearly all of the US population lives in a community covered by a Continuum of Care system, and this measure does not have an outcome related to ending chronic homelessness or to positive outcomes for clients in general. Comparison of GPRA performance indicators from the 2000 Annual Performance Plan (located online at: www.hud.gov/about/budget/hudfinal2000app.pdf) and the 2005 Annual Performance Plan (located online at: www.hud.gov/offices/cfo/reports/2005/2005app.pdf).

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Each grantee is required to submit to HUD an Annual Progress Report (APR), showing outputs and outcomes achieved for the year. Grantees who do not submit a timely APR are not allowed to continue their financial draws from HUD until the APR is submitted. The Continuums also review individual APRs as a method to evaluate grantee performance and to determine whether to renew grantee projects. Grantees and Continuum of Care communities submit aggregated key performance information that is scored in the national competition. A recent assessment of the quality of local program and grantee reporting to the APR shows that local providers are taking these reporting requirements seriously and making great efforts to report accurately. This assessment will be used to improve the instructions and guidance on completing the APR in order to increase further the accuracy of performance reporting. The APR system was used to establish the baselines for early performance measures, and it is referenced on a yearly basis to report and recalibrate our measures. Since FY 2003, Congress has appropriated funds for the purpose of establishing computerized data collection systems (HMIS) to capture client level information. Implementation of these systems will result in Continuums generating unduplicated, client-level data that can be tracked over time, resulting in improved reporting of performance and efficiency, as well as tracking trends to improve the quality and coordination of the delivery of services.

Evidence: See APR (located online at: www.hudclips.org/sub_nonhud/html/pdfforms/40118.doc); NOFA language regarding use of APR information by communities (Federal Register, March 21, 2005 Reporting section, page 14286); "APR Reporting and Outcome Measurement Among High-Performing Homeless Assistance Programs," Abt Associates Inc., 2005, page 30 (unpublished); and summary reports from HMIS section of CoC application Exhibit 1 (internal HUD data analysis). Also see HMIS Universal Data Elements (Federal Register, July 30, 2004 Data Standards for Universal Data Elements section, page 45905).

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and partners are held accountable for cost, schedule, and performance through the APR and obligation timeliness standards established in the statute. Beginning in 2002, the statute required that all HUD program funds be obligated three years following appropriation. For the competitive homeless programs, HUD initiated the same standard in 2005 for funds appropriated prior to 2002; all such funds must be obligated or will be recaptured by September 30, 2005. Each Field Office develops a Field Work Plan that is used to provide oversight and monitoring of grants. Field staff use the Line of Credit Control System (LOCCS), the monthly drawdown reports (A-67), and the Annual Progress Report to develop the field Risk Analysis that is used to identify grantees that may have cost, schedule, or performance issues. In previous years, many communities placed lower-performing renewal grantees at the bottom of the community's priority list so that they would not be awarded funding but would be credited with having a legitimate need for more funding. Beginning in 2005, Continuums are no longer required to submit all eligible renewal projects to receive full funding. This new policy gives CoCs the incentive to prioritize high performing grantees and use funds from lower priority renewals for creating new permanent housing projects.

Evidence: The schedule for timely grant execution begins with the program Operating Instructions, which provide a timeline to reach grant agreement. Field offices report on the number of grants executed and provide ongoing reviews of costs. Field staff also implement project risk analyses based on common risk factors. Finally, the program office encourages performance results by having a performance section in the CoC application. If communities score poorly in the competition they can lose a portion or all of their funding. See NOFA timeliness standards and Performance section (Federal Register, March 21, 2005 pages 14280, 14283, and 14314-14315); Operating Instructions (internal document); copy of Risk Analysis (internal document); and LOCCS screens.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: There are a variety of factors that prevent this program from obligating and spending funds in a timely manner. Grant funds are not usually obligated for 18 months after the appropriations bill. First, it takes several months for the NOFA to be approved and published in the Federal Register. Second, there is a long and cumbersome competitive process requiring HQ and Field Offices to review approximately 2,500 applications annually with limited staff. Third, it takes another few months to sign contracts and obligate the funds is also too long. After conditional awards are made, grantees are required to complete "technical submissions," which are very detailed commitments of how they intend to spend their awards, however, this is a cumbersome and duplicative process that should be modified.

Evidence: During the review process, both field office staff and HUD Headquarters reviewers assess each project budget to ensure all expenses are eligible. Expenditures are then monitored by field office staff through the LOCCS tracking system. In the Homeless Assistance Grants program, the vast majority of funds are obligated within a year of the award. The obligation rate for FY 1998, FY 1999 and FY 2000, was more than 95 percent. Beginning with the FY 2002 appropriation, funds must be obligated within three years or they will expire and be returned to the Treasury. At the end of the three-year obligation timeframe for grantees who received FY 2002 funds (FY 2004), 98.6% of the funds were obligated. Additionally, the NOFA specifies timelines grantees must meet to initiate approved projects. See NOFA guidelines (Federal Register, March 21, 2005 Timeliness Standards, page 14280) and Technical Submission booklet (internal document).

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program has in place procedures and incentives to provide more funding for housing activities, and fewer resources for supportive services. HUD's new efficiency measure identifies the percentage of homeless funds used for housing activities as increasing each year compared to the percentage used for supportive services. With limited resources available, HUD's emphasis on housing activities has achieved efficiencies by encouraging and rewarding CoCs that create housing and seek services from mainstream service providers. The annual NOFA allows for regular adjustments to the procedures for achieving cost effectiveness. Efficiencies are measured through the competitive process with aggregated accounting of the funds requested for housing and supportive services. Once HMIS is implemented, it will be used by communities to measure program outputs and outcomes and help increase efficiencies and cost effectiveness.

Evidence: See CoC competition summary reports for 1999, 2002, 2003, and 2004 (internal HUD data analyses). Also see Housing Emphasis section of the CoC application (Federal Register, March 21, 2005 Emphasis on Housing Activities section, page 14283) and sample HMIS newsletter (located online at: www.hmis.info).

Does the program collaborate and coordinate effectively with related programs?

Explanation: As noted in our response to question 1.3, the CoC planning process and NOFA application require coordination with other Federal, state, local, and private programs. This includes mainstream service and housing programs. The various agencies represented by the Interagency Council on Homelessness, the State Policy Academies, and research and interagency technical assistance further enhance collaboration and eliminate redundancy. Discussion is underway among Federal agencies on the use of Homeless Management Information Systems as databases for information and performance measurement for many different Federal and State programs. This will allow homeless providers to enter information once and to report to many different funders from the same database. An example of HUD's collaboration with other agencies includes the nation-wide effort by states to develop Policy Academies. These joint efforts involve collaboration with HHS (SAMHSA, HRSA, CMS, CSAT), DOJ, DOL, SSA, and VA.

Evidence: Three NOFAs issued jointly by HUD and other Federal agencies, including the Departments of Labor, Justice, Health and Human Services, and Veterans Affairs, demonstrate the program's efforts to collaborate and coordinate housing and service provision with related agencies. A recent example of a partnership between HUD and HHS includes development of the First Steps tool, a CD-rom for case managers to assist clients with accessing mainstream benefits programs. These programs include Food Stamps, Medicaid, and Social Security, among others. See CoC application sections on Leveraging and Coordination (Federal Register, March 21, 2005 pages 14316-14317 and 14293-14294); the Collaborative Initiative to Help End Chronic Homelessness 2003 NOFA (Federal Register, January 27, 2003); Ending Chronic Homelessness through Employment and Housing 2003 NOFA (Federal Register, July 18, 2003); and the Workforce Investment Act Prisoner Re-Entry Initiative 2005 NOFA (Federal Register, April 1, 2005). Also see the Participant List and materials from the "Developing Performance Measures for Homeless Programs" Conference in February 2005, First Steps CD-rom and website at www.hud.gov/offices/cpd/homeless/index.cfm, and report from the National Learning Meeting of the Policy Academies (located online at: www.hrsa.gov/homeless).

Does the program use strong financial management practices?

Explanation: Once funds have been awarded, grantees must adhere to timeliness standards outlined in annual NOFAs. Each grantee uses the Line of Credit Control System (LOCCS) voice response financial management system to drawdown funds and track expenditures. The LOCCS system automatically notifies field office staff of expenses requiring further clarification. The grantee is required to maintain receipts or other specified documentation to support compliance with the statutory and regulatory requirements for eligible activities. In addition, specific benchmarks must be met and verified throughout the new construction or rehabilitation process in order to allow continuing drawdown of funds. Grantees are also required to conduct an annual audit if they expend more than $500,000 in Federal funds in a year. HUD aggressively pursues recapture of unspent funds and reassigns them quickly in order to meet Congressional obligations. Grantees are informed of statutory requirements in the NOFA application, and these requirements are included in the grant agreement. For example, in 2004, over $43 million in unspent funds was used to fund additional projects in the 2004 competition. The Office of Community Planning and Development, of which the Homeless Assistance Grants office is a part, is free from material weaknesses as assessed by the Inspector General in 2004.

Evidence: Each year, field staff conduct both remote and on-site monitoring evaluations of competitive grant recipients. On-going close-out procedures are used to recapture unspent funds and these funds are then used in the current competition. Currently, the Homeless Assistance Grants program is analyzing funds awarded in previous years that have not been obligated. A review of these funds will determine if the recipient needs assistance to bring a project to completion or if the funds can be recaptured for use in a future competition. See 2005 NOFA (Federal Register, March 21, 2005, Award Information, Applicant and Sponsor Eligibility, Capacity and Performance, and Budget Instructions sections, pages 14275, 14282, and 14336). See LOCCS User Guide. See Code of Federal Regulations at 24 CFR 582-583 Subpart D- Program requirements).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The main problem regarding program management is that grantees do not always receive consistent guidance on program rules from HQ versus Field Office staff. A guidebook will be produced soon to address this concern. Technical Assistance providers are also contracted to help the Program office use data in a more systematic way. Most deficiencies have been addressed through the NOFA process and the ongoing grants management system. This year, the program office will make an ongoing concerted effort to review longstanding financial commitments that have never reached obligation. A contractor is providing technical assistance to grantees for this purpose. As needed, the contractor will provide assistance to grantees implementing a project or deciding on the amount of funds that should be withdrawn.

Evidence: See March 15, 2005 memo from the Office of Field Management entitled "Technical Assistance for Un-Obligated Grants," and FY 2005 budget reports for recaptured grants (internal document). Other relevant TA documents will be produced at a later date.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: 100% of funds are awarded competitively. Annual NOFAs published in the Federal Register specify the rules for CoC competitive submissions and describe the rating factors for assessing the quality of a CoC's: (1) planning process, (2) strategy and progress in addressing chronic and homelessness in general, (3) existing inventory of services and housing and unmet needs for emergency, transitional and permanent supportive housing, (4) housing emphasis, (5) progress in HMIS implementation, (6) discharge policy, (7) leveraging of other resources, and (8) program performance. Every CoC reviews and ranks all projects, new and renewal, as part of their annual planning and application process. Exhibit 1 requires a description of the outreach, project solicitation and project ranking process for each continuum. Local Continuums are required to assess renewal project performance before submitting funding requests to HUD. All projects--new and renewal--are reviewed by HUD staff. Projects are reviewed for merit based on need, quality, capacity, performance, and financial management. In FY2004, 613 grants, representing more than 10% of all current grants, were monitored by HUD field staff. Independent, outside experts advise HUD staff during the Continuum review process. Annual web casts, broadcast nationally, give all geographic areas the opportunity to understand precisely how applications are scored and how to score well. These web casts are also posted on HUD's website for the public to view. Technical assistance is available to provide guidance to geographic areas interested in establishing a new Continuum of Care and to existing Continuum of Care projects interested in enhancing process and capacity.

Evidence: See the table showing the number of CoC applications and the number funded (internal HUD data analysis), and the Guide to Continuum of Care Planning and Implementation (located online at: www.hud.gov/offices/cpd/homeless/library/coc/cocguide/index.cfm). Also see the NOFA process, and 2005 requirements and procedures for rating CoC Plans, especially the process reviewing project quality and rating and ranking projects nationally (Federal Register, March 21, 2005 pages 14277-14278, and 14282-14285, and Code of Federal Regulations at 24 CFR Subpart C - Application and Grant Award: §582.200). See Grants Management Process, HQ Summary Monitoring Report (internal document)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Eligible activities are identified during the CoC review process. Both field office and Headquarters staff review each grantee's proposed budget. Field staff review a more detailed budget in each grantee's technical assistance submission and then track expenses through the Line of Credit Control System (LOCCS). The LOCCS financial management system allows Field office staff to track grantee spending within each activity category. The system provides internal controls that highlight issues requiring additional analysis. HUD's APR holds grantees accountable for the numbers of homeless clients served and what assistance they received. Field office staff conduct annual risk assessments of all grantees and dedicate resources for on-site monitoring of higher-risk grantees. Resources for remote and on-site monitoring are directed at the highest risk projects each year. Federal staff and/or designated contractors provide technical assistance to grantees when deficiencies are identified in any aspect of CoC fund administration, reporting, performance, or financial operations. Grantees with "high-risk" designation are given priority-ranking for on-site monitoring.

Evidence: See example of Risk Assessment (internal document), APR (located online at: www.hudclips.org/sub_nonhud/html/pdfforms/40118.doc), Technical Submission.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: The program collects performance data annually, but at this time, this data is not made available in a timely or usable fashion for the public. There are plans to make CoC level information available on the HUD website. Each grantee is required to complete an Annual Progress Report (APR). The APR holds grantees accountable for the number of homeless clients they proposed to serve and the assistance each client received. HUD collects the APRs from grantees and reports the aggregate data to Congress The grantee-level performance data are also provided to the CoC. The CoC describes its utilization of the performance data in the application as part of the priority setting process. The CoC also presents its performance results in the application and receives a score based on performance. As a broad-based organization of stakeholders, the CoC shows the community how each renewal grantee is performing relative to HUD's performance measures. In this way HUD creates incentives for local review of grantee performance reports and local monitoring visits.

Evidence: See http://www.hud.gov/library/bookshelf16/ for Annual Performance Plan. See APR (located online at: http://www.hudclips.org/sub_nonhud/html/pdfforms/40118.doc); priority setting section of CoC application (Federal Register, March 21, 2005 CoC Project Priorities Chart and Instructions, pages 14309-14310); Web cast (located online at: http://www.hud.gov/offices/cpd/homeless/index.cfm); HMIS portal and HMIS newsletter (located online at: www.hmis.info); HUD website with Annual Performance Plans (located online at: http://www.hud.gov/library/bookshelf16/ or http://www.hud.gov/offices/cfo/index.cfm); and individual CoC debriefings (internal documents).

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Beginning in 2003, efforts to track permanent supportive housing units for the chronically homeless were initiated. Approximately 5,000 new beds were provided in 2003 and in 2004. The new long-term goal has begun to get traction throughout the Nation, so a significant increase in the number of beds developed is expected over the next five years. Within the next five years (2005 - 2009), HUD will create 40,000 new units of permanent housing for chronically homeless individuals. The long-term goal is to end chronic homelessness and to help homeless families and individuals move to permanent housing and to live as independently as possible.

Evidence: The 2005 SuperNOFA introduced a new incentive for CoCs to create new units for the chronically homeless. Bonus funding on top of their initial allocations provide new funds towards this goal. HUD's SNAPS office can provide calculations regarding the number of units established for the long-term goal.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The program has exceeded all of its annual performance goals. Each Continuum of Care is collecting information regarding its goals and performance. Each funded CoC has identified goals and is collecting a variety of performance information that is assisting the organization in planning and project assessment.

Evidence: HUD has three annual measures: 1) The percentage of formerly homeless persons who remain housed in HUD permanent housing projects for more than 6 months will be 70 percent in 2005, and will increase one percent each year; 2) The percentage of homeless families and other homeless persons who have moved from HUD transitional housing into permanent housing will be 60 percent in 2005, and will increase one percent each year; and 3) In 2005, the employment rate of persons exiting HUD homeless assistance projects will be 10 percentage points greater than the employment rate of those entering, and will increase one percent each year.. See Measures section and CoC application Exhibit 1 sections on Goals and Project Performance (Federal Register, March 21, 2005 pages 14295-14296 and 14314-14315).

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The CoC process has built-in incentives for efficiency. CoCs assess projects proposed for renewal based on past performance and effectiveness and go through a rigorous local process of prioritizing proposed projects in which incentives for funding the most cost effective projects are inherent. In addition, the match requirements, including the match requirement for services, guarantee that the community provides an investment in the project??Beginning in 2001, a particular efficiency measure emphasized by HUD is the use of HUD funds for housing rather than services. Since 2002, this has caused a shift of approximately $360 million in expenditures from supportive services to housing. HUD emphasized housing by providing more points to CoCs that use a greater share of their requests for housing than for supportive services.

Evidence: The results show that in 1998, only 42% of the homeless funds available were used for housing activities, but by 2004, 55% of the funds were being used for housing activities. HUD uses the NOFA and application scoring process each year to help achieve program goals. For example, in the 2005 NOFA, a reallocation concept was introduced to encourage local assessment of efficiencies and cost effectiveness of individual projects. HUD's emphasis on housing is an effective means of achieving housing stability, which is measured in two of HUD's outcome goals. A 1996 study by Culhane, Metraux, and Hadley found that chronically homeless persons used a disproportionate amount of homeless resources. As a result, the NOFA targets permanent supportive housing resources toward the chronic population. Housing Emphasis and Housing Bonus sections of 2002 (Federal Register March 26, 2002 Emphasis on Housing and Applying Relative Need sections, pages 13-14), 2003 (Federal Register, April 25, 2003, Emphasis on Housing and Applying Relative Need sections, page 21587), 2004 (Federal Register, May 14, 2004, Emphasis on Housing and Applying Relative Need sections, pages 27507-27508), and 2005 (Federal Register, March 21, 2005, Emphasis on Housing and Samaritan Housing Initiative, pages 14283-14284) NOFAs; CoC competition summary sheets for 2002, 2003, and 2004 (internal HUD data analyses); Cost effectiveness, match, and hold harmless components (Federal Register, March 21, 2005 pages 14273 and 14277); Culhane, D.P., Metraux, S. and Hadley. (2002) "Public Service Reductions Associated with Placement of Homeless Persons with Severe Mental Illness in Supportive Housing,"

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The funds are flexible and allow for a range of housing activities allowing communities to fit their needs versus other federal programs that are more structured to one activity. The program is more effective and efficient than other supportive housing programs that serve vulnerable populations for two reasons. First, it is targeted to the population with the greatest need for assistance and the highest risk of negative outcomes (for example, health outcomes associated with living without shelter and without health care).. The Continuum of Care approach, requiring a coordinated planning process including all stakeholders, serves all of the homeless populations with housing and services, and enables homeless persons to move to permanent housing and to live as independently as possible. Second, the program makes extensive use of the existing private market housing stock through tenant-based and sponsor-based rental assistance, rather than building new housing developments for special needs populations. This makes the program more cost-effective than programs that focus largely on production. For both of these reasons, the program compares favorably, for example, with the Section 811 program for people with disabilities, which provides housing assistance for very low-income persons, because of the flexible funding structure and ability to provide specific targeting to those most in need.

Evidence: An independent evaluation noted that, "the vast majority of projects were operational at the time of the evaluation and achieved implementation milestones in a timely manner; the costs of housing and supportive services were reasonable; projects successfully leveraged local resources; projects served the populations of homeless individuals and families intended by Congress; and PH [permanent housing] successfully retained over two-thirds of residents in stable housing for at least one year." See The Urban Institute's "Evaluation of Continuums of Care for Homeless People," (2002) (located online at: www.huduser.org/publications/pdf/continuums_of_care.pdf) and Westat's, "National Evaluation of the Supportive Housing Demonstration Program" (1995) (located online at: www.huduser.org/publications/suppsvcs/shdp.html). Also see Harvard University's "Innovations in American Government" award (located online at: www.ashinstitute.harvard.edu/Ash/continuum.htm).

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: As indicated in our response to question 2.6, the Department has supported a broad range of evaluation and research efforts relating to homelessness that speak directly to the effectiveness of the Department's competitive homeless programs. Those evaluation and research efforts, conducted by some of the most respected research firms and researchers, offer research of sufficient scope and quality to demonstrate the positive results of the Department's competitive programs and how they have been coordinated to encourage a cohesive homeless response at the local level. Independent evaluations have found that HUD's adoption of a community-wide approach has moved communities further in the direction of broad planning and program development than would have happened without the Continuum of Care approach. The ensuing networks of programs and services offer more support to homeless people, with more cohesion, than would otherwise have been possible.

Evidence: The Department has contracted with some of the most respected social policy research firms and researchers in the country, including The Urban Institute, Westat, Abt Associates Inc., ICF Consulting, Walter R. McDonald and Associates, M. Davis and Company, Planmatics, UPENN, the Barnard-Columbia Center for Urban Policy at Columbia University, and others, to conduct these analyses. Those professionals have designed and implemented the evaluations/research to be rigorous, objective and of sufficient scope to provide implications for national program design. See Evaluation Matrix (internal document). Also see The Urban Institute's "Evaluation of CoCs for Homeless People" (2002) (located online at: www.huduser.org/publications/pdf/continuums_of_care.pdf) and Harvard University's "Innovations in American Government" award (located online at: www.ashinstitute.harvard.edu/Ash/continuum.htm).