Detailed Information on the
Housing for Persons with Disabilities Assessment

HUD will propose at least two studies designed to determine appropriate and cost-effective outcome measures for the Section 811 program. Potential outcomes include improved quality of life for persons with disabilities residing in Section 811 housing, increased alternatives to "worst-case" housing needs among very-low-income persons with disabilities, and reduced overall federal expenditures as the result of Section 811 providing a lower-cost alternative to premature institutionalization. One of the proposed studies should attempt to quantify the annual Federal budgetary savings achieved through reduced Medicaid and other expenditures for persons with disabilities in supportive Section 811 housing who would otherwise need institutional care.

HUD will propose changes to the logic model and reporting requirements for the Section 811 program to provide additional information about Section 811 tenants that will improve the measurement of program results. The proposed changes will focus on the degree of disability of tenants and the number of tenants coming from or going to institutional living environments

Measure: Maintain the share of Section 811 housing properties that meet the HUD-established physical standards at no less than 95 percent.

Explanation:HUD's physical standards for Section 811 units assure that the tenats, low-income disabled persons, continue to live in a safe and decent non-institutional setting. Physical assessment data is tracked through the Real Estate Assessment Center's Physical Assessment Subsystem and the 95% compliance target will be increasingly difficult for the aging stock Section 811 projects to maintain.

Measure: Maintain the share of Section housing properties that meet HUD's financial management compliance at no less than 98 percent.

Explanation:The goal of maintaining the financial standards compliance at 98 percent ensures that safe, decent, and sanitary housing is customarily provided to clients and that the housing is maintained and available over time to the intended residents. Diligent financial management confirms that fiscal prudence is exercised when delivering quality services to the elderly clientele. This is ambitious because the portfolio is aging and limited fiscal resources must be effectively utilized to reach the target population.

Measure: Reduce the numbers of low-income persons with disabilities at risk of worst case housing needs by bringing 1,100 Section 811 units (100 projects) to initial closing during FY 2008.

Explanation:The target was established based on the number of projects in the pipeline that are targeted for closing in FY 2008. The number of projects and units is dependent on the amount of funds appropriated. As the appropriated amount remains the same (as it has recently), the number of projects/units selected declines because of the increased cost for development due to cost of living increases.

Measure: At least 70 percent of projects that are initially closed in FY 2008 will have completed the process within 24 months; and, of those, 25 percent will have completed the process within 18 months.

Explanation:Section 811 capital advance funding is linked to the expansion of quality and affordable housing for persons with disabilities. This efficiency indicator is a measure of the Department's success in achieving the intended outcome by minimizing the time needed for Section 811 projects to proceed from fund reservation to initial closing.

Measure: Maintain the number of Section 811 units at 98 percent of the prior year level, excluding new units that are added to the inventory.

Explanation:The Department is committed to preserving existing low-income rental housing for the elderly. This indicator reports on the number of Section 811 units in multifamily housing developments that serve low-income disabled persons, excluding new units added to the Department's inventory. The aggressive target established for FY 2008 uses the FY 2007 level (23,000 units) as a baseline..

Measure: To expand access to an availability of decent, affordable rental housing for low-income persons with disabilities.

Explanation:HUD is still developing metrics to demonstrate that production and maintenance of Section 811 units increases the total stock of decent, affordable rental housing for low-income disabled persons. During this process, HUD will consider whether there are substitutable alternatives to Section 811 housing.

Measure: To reduce total federal costs of supporting persons with disabilities by reducing the incidence of institutionalization among Section 811 residents compared to other very-low-income persons with disabilities.

Explanation:HUD is still developing baselines and metrics for this measure, which will determine the extent to which institutionalization can be delayed or avoided among Section 811 tenants when compared to other very-low income persons with disabilities. After establishing a baseline, HUD will incorporate specific activities in phase two of the Section 811 logic model that project owners will report on to measure their performance in reducing the rate of institutionalization, such as, the supportive services provided, the ages of the residents, the length of time the residents remain in the units, the degree of disability, the number of residents who moved into the project from institutions, and others.

Is the program purpose clear?

Explanation: The purpose of this program is to enable persons with disabilities to live with dignity and independence within their communities by expanding the supply of supportive housing that is designed to accommodate the special needs of such persons and provides supportive services that address the individual health, mental health, and other needs of such persons.

Evidence: 42 U.S.C. of Section 811 of the Cranston-Gonzalez National Affordable Housing Act (P.L. 101-625), as amended by the Housing and Community Development Act of 1992 (P.L. 102-550) authorizes the HUD Secretary to provide assistance to private non-profit organizations and consumer cooperatives to expand the supply of supportive housing for the persons with disabilities via capital advances and contracts for project rental assistance. Also, see 24 CFR 891 and HUD Strategic Plan for 2006 -2011 pages 15-27.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program targets the critical housing needs of very low-income persons with disabilities. There is consensus that supply and access to affordable housing for this vulnerable and growing section of the population is insufficient. States, units of local government, and non-profit organizations have been unable to meet the affordable housing need of very low-income persons with disabilities as evidenced by the report on "Worst Case Housing Needs." Goals for the program include providing very low-income persons with disabilities with an alternative to conditions associated with worst case housing needs, paying more than 50 percent of their income for shelter or living in severely substandard housing, or premature institutionalization or homelessness. This type of housing is more relevant and necessary today with the increased number of disabled veterans returning from the current war. Section 811 funds help meet the affordable housing needs of persons with disabilities through funding or acquisition, rehabilitation, new construction, operating cost, and leasing.

Evidence: The 2008 HUD Report to Congress, "Housing Needs of Persons with Disabilities: Supplemental Findings to the Affordable Housing Needs Report," identifies over half a million non-elderly persons with disabilities who do not receive Federal assistance, have incomes below 50 percent of the local median, and pay more than half of their income on rent or live in poor quality housing. Other estimates, using different data sources, put the number more than twice that amount. The current and former appropriation bills identify the various activities of which the funds are to be spent.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The program is specifically designed to target the very low-income persons with disabilities through the use of non-profits to provide supportive affordable housing. Although there are other programs that provide housing for persons with disabilities, they are not uniquely designed to expand the supply of housing units that accommodate the special needs of low-income persons with disabilities, through accessible design features and provision of a range of services for residents enabling them to live independently. HUD's Section 811 program for persons with disabilities is structured to provide different types of affordable supportive housing options for the target populations through acquisition rehabilitation, new construction, mainstream vouchers and operating subsidies. There are a number of ways that duplication is avoided. For the low-income housing tax credit program, duplication is generally avoided because of the different income groups that can be served without rent burdening the residents, specifically residents paying more than 30 percent of their income for rent.

Evidence: Though the Low-Income Housing Tax Credits program has been used to finance the development of housing for persons with disabilities, these units are generally not affordable to the extremely low-income population. Housing Choice Vouchers and Public Housing also serve very low-income persons with disabilities, but these programs generally do not provide supportive services or other features. According to a recent HUD study, "Implications of Project Size in Section 811 and Section 202 Assisted Projects for Persons with Disabilities", conducted for HUD by Abt Associates, Section 811 residents generally cited satisfaction with the services they are able to obtain through their provider.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Pursuant to the statute, funds are fair shared across the country for experienced, non-profit sponsors to compete for the funds in order to build projects to serve very low-income persons with disabilities. The program works well with other Federal, state and local program to deliver this unique affordable supportive housing resource. This is particular true in areas where the rental housing market is tight or the housing design does not accommodate the special physical needs of persons with disabilities. However, a 2002 GAO report estimated that the total 30 year cost of a one-bedroom Section 811 housing unit is 12% higher (total Government cost was 16% higher) on average than the 30 year cost of a similar housing voucher unit. In 2005 the Department revised the regulations so that the program could work more effectively with low-income housing tax credits. The change facilitates leveraging the Section 811 dollars and has resulted in additional units being built.

Evidence: The Notice of Funding Availability (NOFA) that is published in the Federal Register each funding cycle identifies the requirements of the program as well as the criteria that determine which applicants will be selected. Applicants selected to participate in the program must execute certain documents that state how the project is to be operated, beginning with the Agreement Letter and continuing with the Capital Advance Agreement, PRAC Agreement, Regulatory Agreement, Construction Contract, and Use Agreement. Section 811 project owners must also adhere to the statutory and regulatory requirements as stated in 24 CFR 891. A 2002 report by GAO, Federal Housing Assistance, Comparing the Characteristics and Costs of Housing Programs.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The design of the program is effectively enabling low-income persons with disabilities to live independently in affordable supportive housing. The funds are used to develop affordable housing as required by law. The appropriate use of funds is verified during the cost certification process. To ensure that the project serves the intended beneficiaries, the project owner must execute a use agreement which is recorded for the term of the 40-year mortgage. In addition, the beneficiaries occupying the units are screened to ensure that they qualify. To support the screening process, consistent with the President's Management Agenda and the provisions of the Rental Housing Integrity Improvement Project (RHIIP), the Department implemented the Enterprise Income Verification System to be used for most persons with disabilities projects. This system assists in ensuring that the right benefits go to the right persons and there is no evidence to suggest that the Section 811 program displaces the construction and maintenance of other affordable housing units targeted to the low-income elderly.

Evidence: Guidance to sponsors is provided in 24 CFR 891, the Capital Advance and PRAC Agreements, the Supportive Housing for the Persons with Disabilities (4571.2 and 4571.4), and the Occupancy Requirements of Subsidized Multifamily Housing Handbook (4350.3). Also, a Computer Matching Agreement between HUD and the Department of Health and Human Services allows HUD to confirm intended beneficiaries and eliminate making improper payments to those ineligible for the program.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: HUD has identified two long-term outcome measures for the program, but has not yet developed quantifiable timelines and targets for the new measures. The first outcome measure in development will demonstrate an expansion of access to and availability of decent, affordable rental housing for low-income persons with disabilities.

The second long-term outcome measure in development will demonstrate a reduction in total federal costs of supporting low-income persons with disabilities by decreasing the rate of institutionalization. Specific activities/outcomes will be put in place in phase two of the logic model that project owners will report on to measure their performance in reducing the rate of institutionalization, such as the ages of the residents, the length of time the residents remain in the units, the degree of disability, the number of residents who moved into the project from institutions, and others. Last year, HUD incorporated new, ambitious long-term output and efficiency performance measures into its strategic and annual performance plans. In keeping with the purpose of the program, the performance goals of the disabled housing program will ensure that low-income persons with disabilities in Section 811 housing live in an affordable decent, safe and sanitary non-institutional setting.

Evidence: Quantifiable timelines and targets for Section 811 outcome measures are still in development. The HUD Strategic Plans for FY 2003-2008 and FY 2006 - 2011 contain output and efficiency measures and the timelines for reaching the goals. In addition, the Annual Performance Plan lists specific measures to achieving these goals.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: HUD has not yet developed quantifiable timelines and targets for Section 811 outcome measures. However, HUD has made progress by incorporating output and efficiency goals into its strategic plan and annual performance reporting.

Evidence: The HUD Strategic Plan for FY 2003- 2008 and FY 2006-2011, the 2008 Annual Performance Plan, and the FY 2008 HUD Management Plan contain details of these performance measures as well as existing baseline values.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Last year, HUD incorporated long-term output and efficiency performance measures into its strategic and annual performance plans. The measures focus on timely processing and construction of new Section 811 units, and maintaining the quantity and quality of existing units. In keeping with the purpose and goals of the program, the performance measures of the disabled housing program will ensure that low-income persons with disabilities in Section 811 housing live in an affordable decent, safe, and sanitary non-institutional setting.

The following measures contribute to the goal of expanding access to and availability of decent, affordable rental housing for low-income elderly households by ensuring that new Section 811 units are constructed and that existing units are maintained:

1. The share of Section 811 properties that meet HUD-established physical standards are maintained at no less than 95 percent;
2. The share of Section 811 properties that meet HUD's financial management compliance is maintained at no less than 98 percent;
3. the number of Section 811 units serving the elderly is maintained at 98 percent of the FY 2007 level, excluding new units that are added to the inventory;
4. 1,100 Section 811 units (100 projects) will be brought to initial closing during FY 2008;
5. The FY 2008 efficiency goal is to ensure that at least 70 percent of Section 811 projects are brought to initial closing within 24 months and that, of those, 25 percent will have completed the process within 18 months, these targets will increase to 80 percent and 35 percent respectively by 2010.

Annual output and efficiency measures, as described in the FY 2008 Annual Performance Plan, are quantifiable through data available in HUD's administrative databases, thus ensuring timely tracking of short-term and long-term progress. Quality of life outcomes for individual residents are not easily measured directly and HUD has not yet incorporated true outcome measure into its performance plans.

Evidence: The annual performance measures in the FY 2008 Annual Performance Plan and the FY 2008 HUD Management Plan document the annual performance measures, including their targets and baseline values.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Building on substantial improvements to HUD's data systems, FY 2007 data is used as a baseline for the majority of the Section 811 performance measures. HUD has established recurring annual goals to maintain the quality and quantity of Section 811 units. The targets are ambitious in that they maintain current performance levels in the face of aging properties, growing demand, and steady or declining funding levels. The targets for the annual output and efficiency measures include the following: ensuring that Section 811 properties that meet HUD-established physical standards are maintained at no less than 95 percent; making certain that the share of Section 811 properties that meets HUD's financial management compliance is maintained at no less than 98 percent; ensuring that the number of units is maintained at 98 percent of those at the FY 2007, excluding new units that are added to the inventory;and maintaining the number of persons with disabilities households living in private assisted housing developments served by a service coordinator at the FY 2008 level. The construction phase output target is increasing the availability of affordable housing for persons with disabilities by bringing 1,100 units (100 projects) to initial closing in FY 2008. The FY 2008 annual efficiency target is to ensure that 25 percent of projects that initially close do so within 18 months and 70 percent close within 24 months, these goals will increase to 80 percent and 35 percent respectively by 2010.

Evidence: The Real Estate Management System (REMS) is the official source of data on Multifamily Housing's portfolio of assisted properties. The FY 2008 Annual Performance Plan documents the annual performance measures and provides more detail on the specific baselines and targets for these measures.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: Sponsors of Section 811 projects must execute a "Sponsor's Resolution for Commitment to Projects" during the application phase whereby they acknowledge their responsibilities of sponsorship, long-term support, willingness to assist the owner to develop, own, manage and provide appropriate services in connection with the proposed project, and the commitment reflects the will of the membership. They also commit to providing the required minimum capital investment (MCI) of one-half of one percent of the HUD-approved capital advance, not to exceed $10,000 and the estimated cost of any amenities or features (and operating cost related thereto) which would not be covered by the approved capital advance. The Department will waive one-half of the MCI if construction starts within the initial 18 months of the funding award. If the project is finally closed within 6 months of construction completion, HUD will approve the return of all remaining MCI funds not used to cover operating deficits during the first 3 years of operation. In addition to the sponsor's resolution of commitment to the project, the above provides additional incentives to the sponsors and owners to complete the projects within the established timeframes or earlier. In addition, if approved, each program participant signs a funding agreement letter (obligation of funds, timeframe for initial closing and use of funds); regulatory agreement (maintaining that the property in good repair, restricts use of property funds, standard for reporting and maintaining records, and annual audited financial statement); construction contract with performance standards, capital advance agreement, mortgage note, deed of trust, and Project Rental Assistance Contract (PRAC) agreement. In signing the capital advance agreement letter, the sponsors commit to closing the projects with the established timeframes as well as operating the project in accordance with other regulatory and statutory provisions of the program. HUD field offices also hold planning conferences with the sponsors to go over HUD's requirements for developing projects and the established timeframes for getting to each stage of the process. Should the owner or any of the other partners default on any of the business agreements, HUD can take to appropriate administrative action, including but not limited to flagging the HUD automated system to prevent future participation and judicial remedies, debarment, civil money penalties and criminal action.

Evidence: In addition to the documents listed above, the NOFA, program regulations (24 CFR 891), and the Supportive Housing for the Persons with Disabilities Handbooks (4571.2 and 4571.4) as modified by HUD Notice 96-102 establish project requirements and outline the performance requirements of the project sponsors and owners.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: HUD has not completed an evaluation of the Section 811 program in terms of its overall impact on poor disabled individuals.

Evidence: There are numerous studies that have shown the cost effectiveness of providing supportive services in connection with housing as an alternative to institutional care, but these studies focus primarily on housing for the disabled homeless community. A review of the literature on this subject is presented in "Public Service Reductions Associated with Placement of Homeless Persons with Severe Mental Illness in Supportive Housing", Housing Policy Debate, Volume 13, Issue 1, 2002. Research focusing on the Section 811 program is generally focused on cost rather than effectiveness. Recent studies include: "Section 202 and 811 Operating Cost Needs", March 2007; and, "Implications of Project Size in Section 811 and Section 202 Assisted Projects for Persons with Disabilities" conducted by Abt Associates.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The budget reflects the goals of maintaining and increasing the stock of decent and affordable housing. Discrete funding amounts are requested for renewal of Project Rental Assistance Contracts (PRACs) to maintain existing units and for new capital grants and PRACs to facilitate the construction of new projects. The 2009 budget request is also aligned with the long-term goals of the Section 811 program. Funding is requested for a pilot program to leverage capital grant amounts with non-federal resources. If such a pilot is successful, HUD could facilitate the construction of additional Section 811 units without increasing budgetary resources.

Evidence: Annual budget justifications. Real Estate Management System (REMS). Tenant Rental Assistance Certification System (TRACS).

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: HUD has taken a number of meaningful steps to correcting strategic planning deficiencies. In expanding the supply of affordable housing, pursuant to the Consolidated Appropriations Resolution of FY 2003, HUD conducted a summit to study recommendations for shortening the development processing time and to examine the processing of other Federal agencies. The summit was held for Section 202 projects, but the recommendations concerning the cost limits also apply to the Section 811 independent living projects. As a result of the summit, a study was conducted by the National Association of Homebuilders (NAHB) Research Center to see how the initial funding levels compared with other objective measures of development costs. Since the study, the development cost limits have been revised to the same level as the Section 221(d) (3) statutory limits. Also, performance evaluations of managers were tied to the goal of timely project development and to assess the progress the managers are making, regular pipeline conference calls are held to discuss the status of their projects and to identify solutions for those projects that are not making adequate headway. Another step taken by HUD in the expansion of the supply is through the mixed finance program.

HUD has also adopted the Logic Model concept so the sponsors and owners will evaluate their performance in enabling persons with disabilities' residents to live with dignity and independence. The Logic Model is developed and reported in two phases. During the first phase, the sponsor/owner will measure their performance in expanding the supply of affordable units and assess the timely development of the project. During the second phase, the owner will measure their performance in enabling the persons with disabilities to remain independent.

Evidence: HUD's Annual Performance Plan. The FY's Notice of Funding Availability spells out instructions concerning the Logic Model. HUD's Management Plan also spells out ways in which HUD has addressed management deficiencies. Also see, the NAHB Cost Evaluation of the Section 202 and Section 811 programs.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The agency monitors its goal achievement monthly through various automated systems. A monthly report of the pipeline is sent to the managers to help them monitor their progress in achieving the management plan goals. Quarterly conference calls are made to each of the 18 Multifamily Hub Directors throughout the country to discuss the progress of housing development, impediments to progress, and solutions for resolving issues on projects in the development pipeline. This information is used to assess HUD's performance in meeting its APP goal of increasing the availability of affordable housing for persons with disabilities. Annually, owners submit audited financial statements including balance sheets, profit and loss statements, cash flow statements, etc., to ensure that financial compliance thresholds are met and that resources are being effectively and appropriately utilized. Also, physical inspections are conducted based on established performance criteria to ensure that decent, safe, and sanitary conditions are customary. The performance information collected is matched against effectively managed comparables and is used to identify deficiencies for mitigation. In reviewing the performance information collected over the past and reviewing the strategic goals, HUD added the following two performance outcome measures to the APP last year. The number of Section 811 units serving the persons with disabilities is maintained at 98 percent of those at the FY 2007 level, excluding new units that are added to the inventory and that at least 70 percent of projects initially closed in FY 2008 will have completed the process within 24 months; and of those 25 percent will have completed the process within 18 months. Management uses these two performance measures to assess whether the units are decent, safe and sanitary so the persons with disabilities can live with dignity.

Evidence: HUD maintains several administrative databases to track the program's performance. These include the Real Estate Management System (REMS) and the Development Application Processing (DAP) system. These systems regularly provide a variety of management reports that track program performance and tenant information.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners are expected to meet time and cost objectives and the Department's Management Plan goals are critical elements in the evaluation of mangers of the Section 811 program. During the construction phase, projects are developed with a construction contract, which contains penalties for non-performance. Physical inspections of the projects are performed by HUD inspectors and private contract inspectors to ensure that the projects are on schedule. There is a scoring system for physical inspection that has the incentive of reduced HUD oversight for high performing managers who maintain their property above standard. Properties that receive and maintain an inspection score over 90 percent are inspected every 3 years instead of the normal annual inspection. HUD provides additional incentives, which includes the waiving of one-half of the MCI if construction starts within the initial 18 months of the funding award and HUD will return all remaining MCI funds that are not used to cover operating deficits during the first 3 years if the project is finally closed within 6 months of construction completion. During the term of the 40-year use agreement, owners who do not maintain properties according to contractual agreements which are directly related to the long term physical and financial goals of the Department are subject to administrative action. This action could include denial of the opportunity for future participation in HUD programs. Projects are inspected according to a prescribed schedule to verify that they are decent, safe and in good repair. Audited financial statements are submitted annually. Information on both is submitted electronically to the Department for review and monitoring.

Evidence: Regulatory Agreement, Use Agreement, Performance Appraisals, Construction Contacts, and Rental Assistance Contracts and REMS automated system.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: HUD regularly obligates funds six to eight months after Section 811 award notification. The latest unexpended balance is at $1.1 billion which is a significant decrease and indicates that funds are not only being obligated in a shorter period of time but also being expended faster. The capital advance agreement, Project Rental Assistance Contract (PRAC) agreement, and use agreement ensure that the funds are spent for the intended purpose of developing and operating housing projects for the very low-income persons with disabilities. Capital grant awards are announced annually and the amounts, project sponsor recipients, and project description are posted on HUD's website. In accordance with the program plans, funds are reserved for the projects after Headquarters complete the review of the field office's selection submission. The funds are obligated and recorded in the Department's Financial Accounting System after the sponsor/owner and the Assistant Secretary of Housing have signed the agreement letter. Funds are disbursed during the construction phase on a cost reimbursable basis based on the firm commitment application that the owner submitted and HUD approved. The disbursements are dependent upon a progress inspection conducted by an agent of HUD; HUD holds 10 percent of the disbursement until a complete cost certification is finished. The PRAC funds are disbursed monthly after occupancy for the residents and the sponsor/owner.

Evidence: The Section 811 agreement letter is the obligating document. Obligating actions are tracked in the Financial Data Mart system. Construction Cost Certification verifies appropriateness of expenditures of funds during the development phase. The annual audited financial statements verify appropriate expenditures of funds for the management and operation of the properties during the term of the 40 year use agreement. All multifamily properties receiving a new Real Estate Assessment Center (REAC) physical inspection score below 60 will be referred to the Departmental Enforcement Center for review to determine and notify the owner that they are in violation of a business agreement with HUD. The Section 811 Handbook 4571.2 provides the guidance for the Hub and Program Offices to obligate and disburse funds. The NOFA and The Processing Notice provides guidance on the use of funds, it published the cost limits based on the number of units and the disbursement requirements for the sponsor/owner prior to the capital advance application. The 2007 Capital Grant awards are accessible online (http://www.hud.gov/news/releases/pr07-157.pdf).

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: HUD does not have an efficiency measure for the Section 811 program that focuses on cost effectiveness and does not regularly conduct unit cost or other productivity analysis. However, HUD has incorporated a performance measure of time to initial closing and has established regular procedures for monitoring project timeliness including monthly pipeline reports and quarterly conference calls with each of the field managers. Through these evaluation processes, HUD identifies problems in the timely development of Section 811 projects. This identification procedure has led to increases in the development cost limits resulting in more timely development of Section 811 projects. Because sponsors/owners will not have to spend time locating other funding to complete project development, the project can reach initial closing sooner. One of the APP efficiency measures for 2008 for which the baseline was established in 2007, is to initially close 70 percent of the projects within 24 months and 25 percent within 18 months. Based on concerns raised by partners at a Section 202 Summit held in 2003, HUD contracted with the National Association of Homebuilder Research center to see how the initial funding levels compared with other objective measures of development cost. As a result of the study HUD adopted the Section 221(d)(3), which reflect the cost of developing projects in different market areas.

Evidence: The Notice of Funding Availability (NOFA). Handbook 4571.4 and Notice 96-102.

Does the program collaborate and coordinate effectively with related programs?

Explanation: HUD has worked with housing funders to more efficiently develop affordable housing. Local field staff also meets with local funders to coordinate the development of affordable housing at the local level. During the development of the Occupancy Handbook, HUD worked with the Office of Rural Housing Services at U.S. Department of Agriculture (USDA) and the IRS to ensure that determinations of rent and income were consistent among the agencies. One result of this collaboration is that the agency's Model Lease is utilized by USDA's Rural Housing office. In addition, HUD's Occupancy Handbook, 4350.3, is referenced by IRS in its "Guide for Completing Form 8823 Low-Income Housing Credit Agencies Report of Noncompliance or Building Disposition. The Guide is used by the developers of low income housing tax credit projects. There is on-going communication and coordination with USDA's Rural Housing Services Office, as well as with all State Housing Finance Agencies. HUD works with these entities to ascertain the extent of the need for housing and to ensure that there will be no adverse impacts if housing is developed. There is coordination with other HUD programs, through the Multifamily Development Office and the Office of Public and Indian Housing, to effectively market assisted housing offerings to the targeted population. HUD has also collaborated with the Federal Housing Finance Board to streamline the closing process for Section 811 projects receiving subordinate financing from Federal Home Loan Banks. This was necessary because the two agencies had different requirements.

Evidence: A memorandum of understanding was signed with the U.S. Department of Agriculture. Coordination and collaboration activities with other related programs are discussed in the NOFA. In supporting the analyses of housing needs and services, applicants must demonstrate collaboration with local housing providers. Also, throughout the life of the project, the provider must meet goals of their respective local jurisdiction. Memorandum of Agreement with the Federal Home Loan Banks.

Does the program use strong financial management practices?

Explanation: During the construction phase, funds are released on a cost reimbursable basis and dependent upon a progress inspection conducted by an agent of HUD. HUD holds back 10 percent of the disbursement until a complete cost certification is finished. The cost certification step is for HUD to make sure that the funds expended for the project were expended in accordance with the HUD-approved firm commitment application. If HUD determines that funds were not expended properly, the owner will not be reimbursed or the owner will have to pay back funds, if they were previously reimbursed. HUD reviews the operational budgets of the development to ensure costs are reasonable and appropriate for the area in which the project is located. The budgets are compared with annual audited financial statements, which are reviewed by HUD. If HUD's review indicates that funds were not spent in accordance with the approved budget, HUD will either allow the owner to resolve the issue or bring enforcement action if it cannot be resolved or depending on the severity of the offense.

Evidence: Multifamily Project Management and Asset Servicing (4350.1). Mortgage Credit Analysis for Project Mortgage Insurance (4470.1). Cost Certification Guide for Mortgagors and Contractors of HUD-Insured (4470.2). Government Auditing Standards Handbook (The Yellow Book). Inspector General Handbook, Consolidated Audit Guide for Audits of HUD Programs (2000.4)

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Aggressive steps have been taken to address identified management deficiencies. After GAO issued a finding in 2002 that there were data integrity issues in the system that was used to monitor the development process a process was instituted to routinely verify the accuracy of information in the system. After a good baseline was established regular reports are made available to the field so that errors can be corrected. When it was verified that many projects took an unnecessarily long time to reach occupancy a plan was developed to ensure that agency management and partners were held accountable for developing projects in a timely manner. Performance evaluations of managers were tied to the goal of timely development of projects. To ensure that adequate process is being made regular pipeline conference calls are held with field leadership to discuss the status of project in their jurisdictions and identify solutions for those projects that are not making adequate progress. To more effectively manage projects during the 40-year use agreement period, the agency has established objective physical and financial standards for projects in the portfolio. Sponsors annually submit an audited financial statement of project operations. In addition, physical inspections of projects are conducted on an established schedule based on the score of the last inspection.

Evidence: The HUD Annual Performance Plan offers a thorough strategy to Embrace High Standards of Ethics, Management, and Accountability. HUD's Management Plan also spells out ways in which HUD has addressed management deficiencies. HUD's under 60 policy for physical inspection.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Section 811 Capital Advances are awarded based on a clear competitive process. The Section 811 program NOFA, which is published annually in the Federal Register as part of the SuperNOFA, provides the criteria and review process for selecting the Section 811 awardees. Among the criteria on which applicants are rated are: their past experience in serving the intended population as well as experience in previously funded projects; other funds and resources committed to the project; location of the site; need for the units in the market area; and their understanding of what it takes to get the project to occupancy. After the SuperNOFA is published, a satellite broadcast is held to explain the submission and rating criteria. Local HUD offices also conduct workshops to further describe the submission, rating and review requirements for applications submitted during that fiscal year.

Evidence: The SuperNOFA. The FY Processing Notice providing instructions to HUD staff on the application processing procedures and processing schedule.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: As part of the application for funds, HUD requires applicants to provide information on their past experience in serving the proposed client population. Applicants must register with and also go through a Name Check review through Dun and Bradstreet. The officers and directors of the Sponsor and Owner boards must also undergo clearance through the 2530, Previous Participation Review Process, and a credit check is conducted of the officers of the owner board. During the construction phase of the project, regular inspections of each site are undertaken. Construction payments are made based on percentage of construction completed. A percentage of the construction budget is not released until construction is completed and the cost certification process has been completed. Once completed, properties are inspected on an annual basis. However, as an incentive, projects that score between 90 and 100 are inspected every three years; and those that score 80-89 every two years. The regulation for physical inspection protocol is outlined in 24 CFR parts 5 and 200. Annual management reviews are conducted and audited financial statements are required of each project. Finally, the business agreement signed by the project manager responsible for day to day operation, stipulates that management reviews, financial statement evaluation, and physical inspections are all evaluation tools used to oversee operating and management activities.

Evidence: There are several handbooks that outline oversight guidance and reporting requirements. Examples include: Section 811 Supportive Housing for the Persons with Disabilities (4571.2), Architectural Analysis Inspections for Project Mortgage Insurance (4460.1), Cost Certification Guide for Mortgagers and Contractors of HUD (4470.2), Occupancy Requirements of Subsidized Multifamily Housing Programs (4350.3), the Management Agent Handbook (4381.5), Multifamily Asset Management and Project Servicing Handbook (4350.1), Mortgage Credit Analysis for Mortgage Credit Insurance (4470), and Secretary-Held Servicing (4335.2). The regulation for physical inspection protocol is outlined in 24 CFR parts 5 and 200.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: HUD does not currently provide grantee performance data to the public in a clear and accessible manner. HUD is developing a central website with the name and location of project sponsors, the amount of the award, the location of the project to be developed, and other information as appropriate. Starting in 2008, HUD will post approved logic models on the Grants web site. As the project owners submit their required performance reports, they will also be posted on the web site. HUD does collect grantee performance data. Physical inspections are conducted every 1 to 3 years (for moderate to highest scoring projects, respectively), while audited financial statements are required annually of all projects. A balance sheet, profit and loss statement, applicable schedules and notes must be submitted for review. Owners must make inspection reports available for review by tenants in the property office. Information on property inspections may also be obtained at anytime through the Freedom of Information Action (FOIA). Financial performance data are proprietary and exempt from public review under Exemption 4 of FOIA.

Evidence: The Super NOFA. Executive Order 13392 published in the Federal Register on December 19, 2005 (70 FR 75373). Uniform Physical Condition Standards and Physical Inspection Requirements for Certain HUD Housing (24 CFR Parts 5 and 200, December 2, 2000). Guidance for Oversight of Multifamily Housing Physical Inspection directive issued to all multifamily Hub/Program Center Directors and Performance-Based Contract Administrators, May 24, 2001.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: HUD has not yet developed quantifiable baselines and performance goals for Section 811 outcome measures. However, to expand the supply of affordable housing for the low-income persons with disabilities, the Department continues to add new Section 811 units to the inventory and is expecting to maintain the inventory of existing units at 98 percent of the FY 2007 level. To increase the supply of housing available to persons with disabilities the agency tracks project development to ensure that projects are finished in a timely manner. Currently, most projects are available for occupancy within 36 to 48 months from the time the sponsors are advised that their project has been funded. Concurrently, a baseline for the existing portfolio of Section 811 housing was established. This allows the Department to verify that the supply of Section 811 units is in fact increasing. Performance goals and management systems have been established to monitor the financial and physical condition of the portfolio and ensure the continued quality of Section 811 housing The agency also has aggressive preservation strategies which have assisted in upgrading some of the oldest Section 811 housing. For example, many projects are refinancing to generate additional resources, which are reinvested into the project.

Evidence: Information can be found in the Department's annual Performance and Accountability Reports and year end Management Plan Reports. The REMS and DAP systems contain information on the Section 811 portfolio, existing, as well as projects currently being developed.

Does the program (including program partners) achieve its annual performance goals?

Explanation: The Section 811 program achieved its annual performance goal each year through the partnership of the Field Office and Sponsors. This is accomplished initially with the Notice of Funding Availability (NOFA) informing the potential applicants of the requirements for the program; provide program instructions, the acceptance of the awards and the planning meeting with the field office to review the sponsors timeline and plans. The Sponsors indicate the number of units they plan to develop and they commit to complete the projects in 18 months schedule with the Field Office at the Planning Conference. The sponsors are also required to report annually on their performance based on their approved Logic Model Report. In addition, the Field Offices meet with sponsor monthly for a report on the projects status during the developmental period. Housing leadership establishes a goal for each field office as a Management Program Plan and conducts monthly conference calls with the Field Office to stay abreast of the status of the goals and to ensure that the Annual Performance Plan (APP) goal of bringing 100 projects (1,100 units) to initial closing is met. Physical inspections are conducted based on established performance criteria to ensure that decent, safe, and sanitary conditions are customary. HUD accomplished the established physical standards that are maintained at no less than 95 percent and for properties that meet HUD's financial management compliance is maintained at no less that 98 percent are managed and monitored within Multifamily Housing. The owners submit annually audited financial statement to ensure that the financial management compliance is maintained at no less than 98 percent. In addition the Department has implemented automated tracking systems to monitor goal achievement throughout the year. The systems provide timely management information to ensure that all goals are on target and assist in the early identification of specific individual projects that may require more attention. Overall, the Department meets all of the annual goals in its assisted housing programs. Finally, the Department has incorporated the review of goal accomplishment into its Quality Management Review program of Regional and field offices.

Evidence: Information can be found in the Department's Annual Performance and Accountability Reports and year-end Management Plan Reports. The REMS and DAP systems contain information on the Section 811 portfolio, existing as well as projects currently being developed. The SuperNOFA provides guidance and instructions to applicants applying for Section 811 capital advances.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: HUD does not have an efficiency measure for the Section 811 program that focuses on cost effectiveness and does not regularly conduct unit cost or other productivity analysis. HUD requires that the development costs for all projects be comparable to similar projects developed in the same geographical area and do not exceed the maximum capital advance amount that HUD will allow. HUD uses the Marshall and Swift industry standards in developing local high cost standards. The development costs are certified after project completion to insure that the project was developed in accordance with the costs approved by HUD. In the application selection process, points are awarded based on the amount of additional resources the sponsor is able to leverage from public/private partners. Through the leveraging of additional resources up front, Section 811 projects can be built in a more efficient time frame. The points awarded for leveraging resource were revised in FY 2006 to give more weight to those applications with the most leverage resources.

Evidence: The HUD Strategic Plan for FY 2006-2011, the 2008 Annual Performance Plan, the FY 2008 HUD Management Plan, the NOFA, and the Development Application Processing System.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The program compares favorably with similar programs. An important source of information on the comparative costs of Federal housing assistance programs is a 2002 study by the Government Accountability Office (GAO). The GAO study compared the total per-unit costs of six active programs: Housing Choice Vouchers, Low-Income Housing Tax Credits, Hope VI, Section 202, Section 811, and Section 515. GAO determined total costs and estimated the share of total costs paid by the Federal government, tenants, and others, including State and local governments. While costing slightly more than some housing-only programs, HUD's Section 811 compares reasonably in cost to other programs when the benefits of supportive services are taken into account. HUD's Section 811 program for persons with disabilities is structured to provide different types of affordable supportive housing options for the target populations through acquisition rehabilitation, new construction, mainstream vouchers and operating subsidies. The Section 811 program also provides rental assistance vouchers to persons with disabilities through the mainstream program.

Evidence: The Centers for Medicare & Medicaid Services (CMS), SSA, in association with the Assistant Secretary of Planning and Evaluation (ASPE) at HHS, used the Section 811 program as the model for a sponsored demonstration program whereby states received assistance to provide transition options to nursing home residents who qualified to move back into the community. The GAO study, "Federal Housing Assistance Comparing the Characteristic and Cost of Housing Programs," compared the total per-unit costs of six active programs.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: No known independent and comprehensive evaluation exists that evaluates the Section 811 program's impact and effectiveness in terms of results that are linked to long-term outcomes. There have been independent evaluations that broadly describe the program's purpose and cost effectiveness and several studies that are not specific to the Section 811 program have shown the cost effectiveness of providing supportive services in connection with housing as an alternative to institutional care.

Evidence: Evaluations of a variety of supportive housing programs with similar goals have produced evidence that the Section 811 model is effective. A review of the literature on the subject can be found in "Public Service Reductions Associated with Placement of Homeless Persons with Severe Mental Illness in Supportive Housing," Housing Policy Debate, Volume 13, Issue 1, 2002. Some other examples include: "An Evaluation of the Good Shepherd Supported Independent Living Project," Center for Outcome Analysis, 2004; and "The Connecticut Supportive Housing Demonstration Program Evaluation Report," Arthur Anderson, LLP, 2000. In addition, studies conducted for HUD indicate that the Section 811 model of combining housing with certain supportive services is effective and achieves its goals. Examples include "National Evaluation of the Supportive Housing Demonstration Program," HUD, 1994 and "Implications of Project Size in Section 811 and Section 202 Assisted Projects for Persons with Disabilities," HUD, 2004.