Detailed Information on the
Housing Opportunities for Persons with AIDS Assessment

Revise HOPWA's efficiency measure to better highlight the program's achievements.

Continue targeting support to persons with the greatest needs, and assess the impact of annual increases to outcome goals on delivering assistance to those who have greater risks of poor outcomes.

Recommend a statutory update to the formula to better allocate resources based on current data on persons living with HIV/AIDS and local housing costs.

Measure: The percentage of HOPWA clients in permanent housing who maintain housing stability will be 90% by 2012 and will increase by 1% each subsequent year.

Explanation:As described in HOPWA's authorizing statute, one of the primary purposes of HOPWA is to maintain housing stability for persons living with HIV/AIDS. HOPWA measures "maintaining housing stability" by asking grantees who provide permanent housing to HOPWA clients to report annually on the housing status of clients they have served over the past year. For those clients who have left HOPWA-funded permanent housing within the year, grantees report on what type of housing situation to which they have transitioned. For example, those clients who have left HOPWA-funded permanent housing for "Emergency Shelter/Streets" are considered unstably housed, while those who have left HOPWA-funded permanent housing for "Other Subsidy" are considered stably housed. The HOPWA office aggregates this information provided by all grantees (both formula and competitive) and determines what percentage of all clients initially residing in HOPWA-funded permanent housing maintain their housing stability in any given year. HOPWA will be reviewing this measure to ensure that while grantees are improving housing stability, even those clients with the most significant challenges will continue to be served.

Measure: The percentage of HOPWA clients receiving short-term housing assistance who experience reductions in their risks of homelessness will be 70% in 2012 and will increase by 2% each subsequent year.

Explanation:As described in HOPWA's authorizing statute, one of the primary purposes of HOPWA is to reduce the risk of homelessness for persons living with HIV/AIDS. HOPWA measures "reductions in risks of homelessness" by asking grantees who provide short term housing assistance to HOPWA clients to report annually on the housing status of clients they have served over the past year. Clients who receive HOPWA-funded short-term rent, mortgage, or utility assistance (STRMU) are in immediate danger of becoming homeless because they cannot pay for their housing costs. Grantees report on if and how homelessness was averted. For example, when the STRMU assistance has ended, those clients who have ended up in "Emergency Shelter/Streets" are considered unstably housed, while those who have "Maintained private housing without subsidy" are considered among those who have successfully reduced their risk of becoming homeless. The HOPWA office aggregates this information provided by all grantees (both formula and competitive) and determines what percentage of all clients who have been assisted with HOPWA-funded STRMU have reduced their risk of becoming homelessness in any given year. HOPWA will be reviewing this measure to ensure that while grantees are reducing clients' risks of homelessness, even those clients with the most significant challenges will continue to be served.

Measure: Number of households receiving HOPWA housing assistance during a given year.

Explanation:Reporting changes issued in 2006 link use of housing components to outcomes for beneficiaries. Grantee reporting in 2007 used to identify further edits to focus assessments on results in form updates issued in 2008 along with training in evaluating annual results shown in posted grant Profiles. Outputs tracking uses a supplemental goal that funds are used in timely manner in obligation and expenditure of HOPWA resources. Use of technical support and oversight to help grants achieve long-term outcome goals with assessment of on-going housing arrangements for recipients.

Measure: HOPWA's tenant-based rental assistance costs per household will be 4% more cost-effective than the Housing Choice Voucher program's costs per household in 2009 and will increase by 1% each subsequent year.

Explanation:Both HOPWA and the HUD Housing Choice Voucher (HCV) program administer very similar types of tenant-based rental assistance. When compared to the HCV program, controlling for household size, HOPWA has demonstrated cost efficiencies. This metric challenges HOPWA to continue to find ways to be more efficient.

Is the program purpose clear?

Explanation: The statutory purpose of the Housing Opportunities for Persons with AIDS (HOPWA) program is "to provide states and localities with resources to devise and implement long-term, comprehensive strategies for meeting the housing needs" of low-income persons with HIV/AIDS and their families. The program provides housing assistance through: (1) rental assistance support; (2) residency in supportive housing facilities for clients with greater life challenges; and (3) short-term housing assistance to help prevent homelessness. HOPWA also provides resources for other non-direct housing costs, which assist clients in accessing and maintaining stable housing, including supportive services. HOPWA housing assistance and related support are intended to enable low-income persons living with HIV/AIDS, as well as their families, to avoid homelessness, meet their housing needs and enhance their access to care.

Evidence: HOPWA's statutory authority is found in the AIDS Housing Opportunity Act (42 U.S.C. 12901 et seq) available online at http://www.hud.gov/offices/cpd/aidshousing/lawsregs/laws/index.cfm. The use of HOPWA to help beneficiaries avoid homelessness is found in §12907; to help beneficiaries meet their housing needs is found in §12901; and to enhance access to care is found in §12907. The HOPWA regulations (24 CFR 574) provide further detail on the activities that can be carried out with HOPWA funds to achieve the stated purpose and are available online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html.

Does the program address a specific and existing problem, interest, or need?

Explanation: This housing assistance program addresses pressing housing needs of low-income individuals who are living with HIV/AIDS and their families. The Centers for Disease Control (CDC) reports that more than one million persons were living with HIV/AIDS in the United States in 2006. Estimates indicate that approximately 600,000 of these persons may be eligible for HOPWA assistance. During the 2006-2007 program year, grantees provided housing support to 67,000 households. Stable housing has demonstrated to help individuals access and maintain necessary medical care and needed supportive services. Program support reaches a targeted housing affordability need, as approximately 93% of competitive grant beneficiaries having extremely low-incomes (below 30 percent of area median income) or very low-incomes (between 30 and 50 percent of area median income), and approximately 80% of formula beneficiaries with monthly incomes under $1,000. This indicates that HOPWA efforts target the vulnerable population it was designed to address. As the number of low-income persons living with HIV/AIDS continues to grow, and care options improve, the need for HOPWA assistance is just as relevant today as it was when the program was first authorized to provide the base to access such care. In addition, the number of persons living with HIV is increasing: CDC estimates that among 33 states, approximately 35,000 new infections were diagnosed in 2006.

Evidence: The assessment that 60 percent of all persons living with HIV/AIDS report a lifetime experience of homelessness or housing instability is provided in the NAHC National Research Summit Fact sheet (http://www.nationalaidshousing.org/advocacy.htm#HOPWA), and the original source is as follows: Aidala, A., Cross, J.E., Stall, R., Harre, D., Sumartojo, E., "Housing Status and HIV Risk Behaviors: Implications for Prevention and Policy." AIDS and Behavior. (In Press). HOPWA performance reports from 12 formula grantees reporting on unmet needs in their area collectively report serving 6,791 households with HOPWA support, or 22 percent of their estimated 30,889 HOPWA-eligible households with housing needs in these communities. These 12 grantees are allocated 8.6% of formula funds FY07. (HUD datafiles 4/08). CDC provides current data on the number of persons living with HIV/AIDS (prevalence) as well as the number of new infections each year (incidence), which are available at http://www.cdc.gov/hiv. Specific data on yearly statistics are included in "Basic Statistics and Surveillance" (2005), found at http://www.cdc.gov/hiv/topics/surveillance/basic.htm. The most current information on diagnoses is included in CDC's fact sheet, "HIV/AIDS in the United States" (2008), found at http://www.cdc.gov/hiv/resources/factsheets/pdf/us.pdf . National AIDS Housing Coalition also released a policy paper from the Second National Housing and HIV/AIDS Research Summit, which summarizes research on the significance of housing as a mechanism to improve the health of persons living with HIV/AIDS and to prevent new infections. This paper, entitled "Transforming Fact into Strategy-Developing a Public Health Response to the Housing Needs of Persons Living With and at Risk of HIV/AIDS, is available at http://www.nationalaidshousing.org/ResearchSummitII.htm. HOPWA grantee program data is compiled from completed Annual Performance Reports (APRs) and Consolidated Annual Performance and Evaluation Reports (CAPERs). These forms are available on the HOPWA website at http://www.hud.gov/offices/cpd/aidshousing/index.cfm under HOPWA Revised APR and HOPWA Revised CAPER links. Output and beneficiary data, and unmet need planning estimates included above are from 2006-2007 APR, CAPER and IDIS data and supplemental reports, "Demographic Data for Competitive Grantees" and "HOPWA Beneficiary Summary National Report."

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The HOPWA program has the unique statutory authority to administer resources to meet the unmet housing needs of low-income persons living with HIV/AIDS and their families. HOPWA funding enables grant recipients to target various types of housing support to this population, thereby reducing their beneficiaries' risks of homelessness while fostering connections to HIV care and treatment. The support enables beneficiaries to access and consistently participate in related federal health and human service programs, such as those operating under the Ryan White Treatment Modernization Act (formerly the Ryan White CARE Act). The program's authority allows for unique targeting of affordable housing resources as part of community interventions in assisting families challenged by the combination of HIV/AIDS, housing needs and poverty. Other housing programs cannot be targeted to this purpose. Prior to HOPWA, few communities had established any directed AIDS housing plans. To ensure that HOPWA housing efforts do not overlap with other community housing efforts, HOPWA formula grantees are required to participate in their community's Consolidated Planning process. The Consolidated Plan serves as the community's tool for broad consultation, needs assessments and action plans for using HUD's Community Planning and Development formula funds. The process ensures that grantees' housing efforts are coordinated with other community housing efforts, thereby reducing duplication of effort. The Consolidated Planning process and other local housing collaborative processes also help maximize available housing resources.

Evidence: HOPWA regulations outline the responsibility of formula grantees to submit a Consolidated Plan, in 24 CFR §574.110, available online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html , and HUD general Consolidated Plan requirements are in 24 CFR §91, see: http://www.access.gpo.gov/nara/cfr/waisidx_03/24cfr91_03.html . The Ryan White HIV/AIDS Modernization Treatment Act (PL 109-415), which reauthorized the Ryan White program in 2006, is available online at http://hab.hrsa.gov/law/reauth06.htm . This statute describes the general usage of Ryan White funds and limitations on housing activities, which are restricted to emergency and transitional uses only. Additionally, the Health Resources Services Administration (HRSA) which administers the Ryan White program issued a final notice in the February 26, 2008 Federal Register. This notice, effective March 27, 2008, amends HRSA's Emergency and Transitional Housing Policy, implementing a lifetime 24 month cumulative cap on the use of Ryan White funding for housing. The notice underscores the importance of HOPWA funded housing assistance, since future Ryan White housing efforts will be limited to CARE act purposes and program requirements. See HRSA's final notice: http://edocket.access.gpo.gov/2008/pdf/E8-3607.pdf . See also CARE act performance measure assessment and data shown in the 2007 PART review, such as Q1.3 that care programs are not redundant of other federal programs (such as HOPWA), at www.whitehouse.gov/omb/expectmore/detail/10000296.2007.html. HOPWA short-term programs are for fundamentally differing purposes as guided by CPD Notice 06-07, Standards for HOPWA Short-term Rent, Mortgage, and Utility (STRMU) Payments and Connections to Permanent Housing, August 3, 2006, (replacing CPD Notice 02-09). This guidance limits STRMU use to current resident (not persons already homeless as allowed by the limited CARE Act provisions) and defines elements of housing needs assessments and related individual housing and service plans for purposes of achieving the HOPWA goals for stable housing outcomes, see: www.hud.gov/offices/cpd/aidshousing/index.cfm . See also Homeless Assistance Act performance measure assessment and data shown in the 2005 PART review, such as Q1.3 that homeless assistance programs are not redundant of other federal programs (such as HOPWA), at www.whitehouse.gov/omb/expectmore/detail/10001234.2005.html.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The HOPWA program is free of major flaws that would limit the program's effectiveness or efficiency. There is no evidence to suggest that housing for persons with HIV/AIDS should be administered through an alternative mechanism: the current design of providing funds through both formula and competitive grants is an appropriate one. The program has addressed previous PART review Program Improvement Plans, including full implementation of reporting and evaluation of client outcomes. The 2003 PART also identified a number of strengths in the program, including its flexibility to allow communities to target their HOPWA resources to meet local HIV/AIDS housing and care needs. This flexibility has allowed HOPWA to provide for a broad variety of eligible housing and services activities, which is appropriate since different solutions work in different communities (e.g., based on client characteristics, provider capacity, local housing market conditions, etc.). Further, because the range of eligible activities is broad, HOPWA activities are able to complement a variety of activities funded through state and local resources and other resources.

Evidence: See HOPWA regulations (24 CFR 574.110) for allowable uses of HOPWA funds, online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html. Also see HUD's Consolidated Plan regulations (24 CFR Part 91) for requirements governing coordination of planning efforts across HUD's housing and homeless programs, available online at http://www.access.gpo.gov/nara/cfr/waisidx_03/24cfr91_03.html. HUD's Office of Policy Development and Research "National Evaluation of the Housing Opportunities for Persons with AIDS Program " (2000) report is available online at http://www.huduser.org/publications/hsgspec/hopwa_0101.html. This report outlines the beneficial results obtained under the HOPWA program and discusses beneficiary outcomes and characteristics. HUD Chief Financial Office Risk Management Review (2000), which found that management controls are in place and sufficiently operating. HOPWA PART Review (FY 2005) and management procedures have remained consistent and reporting enhanced along with training of grantees. Performance profiles for all HOPWA grantees, which are updated quarterly, are available on HUD's website at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm. Data for leveraging is from HOPWA annual performance forms, Annual Performance Reports (APRs) and Consolidated Annual Performance and Evaluation Reports (CAPERs.) These reporting forms are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: HOPWA's funding formula needs to be updated so the program can more appropriately and fairly target its resources to the areas of the country most in need of housing assistance for persons with HIV/AIDS. Ninety percent of HOPWA funds are distributed by formula; however, the current HOPWA formula allocates funding based on cumulative AIDS data, which does not account for over one half million deaths due to AIDS, consider area housing costs differences, or utilize recent information available on persons living with HIV. As enacted in 1990, this required source of data is increasing less relevant in reflecting the state of knowledge and data on the domestic HIV epidemic. National data on persons living with HIV was recently established in 2006 as the basis for distributing federal HIV related care and treatment resources under the Ryan White Treatment Modernization Act, and almost all states now report HIV surveillance information consistent with standards issued by the Centers for Disease Control and Prevention (CDC). HUD has recommended updating HOPWA's statutory formula since 1998, and the original HOPWA PART review in 2003 addressed this fault and supported HUD's proposed update to use current data from CDC and housing costs to allocate these housing resources. GAO also reviewed HOPWA and other federal AIDS programs in using HIV and AIDS data to distribute resources and noted issues in equity. While the CARE act was modernized in part to allow for collection and adjustments in the use of HIV surveillance, similar authority was not provided to HUD for modernization of HOPWA statutory requirements. An updating of this program component is essential for HOPWA to be consistent with the state of knowledge in assisting persons living with HIV, with standards of care, allow for accurate assessments of needs and ensure equity in distribution of federal resources. Aside from the formula design flaw, HOPWA is otherwise structured to distribute its funding well. Program resources are still reaching persons with HIV/AIDS who need stable housing. Because the number of persons who qualify for HOPWA assistance far exceed the resources available, even those communities that have an inequitably large share of HOPWA resources still have an excess in demand for HOPWA assistance. Therefore, the sole barrier to HOPWA assistance effectively targeting the appropriate beneficiaries remains the outdated design of the formula.

Evidence: HOPWA's statutory authority is found in the AIDS Housing Opportunity Act (42 U.S.C. 12901 et seq) available online at http://www.hud.gov/offices/cpd/aidshousing/lawsregs/laws/index.cfm. The HOPWA formula allocation is found in §12903. The GAO report "HIV/AIDS, Changes Needed to Improve the Distribution of Ryan White CARE Act and Housing Funds," GAO-06-332, February 2006, http://www.gao.gov/cgi-bin/getrpt?GAO-06-332 recommends changing the HOPWA formula.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Following recommendations from OMB in its 2003 PART review, HOPWA created and is aggressively implementing two long-term outcome measures: (1) a housing stability outcome which measures the housing status of those clients that receive permanent housing assistance, thereby measuring whether states, localities and nonprofits have met the permanent housing needs of low-income persons with HIV/AIDS and their families; and (2) a reduced risk of homelessness outcome which measures the effect of how short-term or transitional housing interventions and support prevent the potential of homelessness and largely result in outplacement to more permanent outcomes. HOPWA clients benefit from various forms of housing assistance, depending on beneficiary needs. Tenant-based rental assistance and permanent housing facilities are often used to support clients with a greater level of independence, while short-term assistance and transitional facilities are often accessed by clients with greater support needs. By including each of these program elements in the program's two long-term outcome measures, HOPWA is able to determine whether the program uses comprehensive housing strategies to adequately assist clients with a range of needs. HOPWA grantees report on and are being assessed on their progress toward achieving these outcome measures through 2012, as identified in HUD's strategic objectives.

Evidence: As reported in PART Web updates, data on indicators used for reporting on both of HOPWA's long-term performance outcome measures are found in the APR forms (pages 10-11) for competitive grantees and the CAPER forms (pages 9-10, 13) for formula grantees. Both the APR and CAPER are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. In 2007, HUD had one outcome measure, but in 2008 divided it into two measures (with a third in development) in order to individually track sub-components of their program. To view the 2007 measure, which is very similar to the 2008 measures, please see HUD's Strategic Plan (FY 2006-2011), available online at http://www.hud.gov/offices/cfo/stratplan.cfm. This plan includes HOPWA's long-term outcome goal and further information about the program on pages 15-44. Also see the HUD Annual Performance Plans from fiscal years 2005-2009 (page 79 from FY 2009 APP) for an understanding of how the HOPWA performance measures fit into the overall framework of HUD strategic objectives, available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm . The most recently published (2007) HOPWA Notice of Funding Availability (NOFA) also includes long-term program outcomes and competitive grantees' requirements to report on these outcomes. See Federal Register, March 13, 2007, page 11664 under (2) Required Outcomes. Published NOFAs are also available online at www.federalregister.gov. Also see January 24, 2008 memo from HUD's General Deputy Assistant Secretary for Community Planning and Development. This memo introduces HOPWA's 2008 APR and CAPER forms and is available online at http://www.hud.gov/offices/cpd/aidshousing/library/memo01242008.doc.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: HOPWA has ambitious quantitative targets for both of its long-term outcome measures. 1) For its housing stability measure, HOPWA has set a long-term goal of 90%. In a study of HOPWA conducted by HUD's Office of Policy Development and Research (PDR), it was concluded that between 60 and 80 percent of clients, depending on their housing status at the time of the interview, reported that their housing was more stable or that their opportunities for obtaining stable, permanent housing were significantly increased since receiving HOPWA assistance. Based on this determination, HOPWA initially adopted a specific target of 80 percent and has since increased the goal to 90% for 2012. 2) For its reduction in risk of homelessness measure, HOPWA has set a long-term goal of 70% for 2012. Because its initial data indicate that 63% of clients in short-term and transitional housing programs reduce their risks of homelessness, achieving an outcome of 70% within the next five years is ambitious. Both HOPWA measures involve an ambitious goal for increased results over current grantee performance and the baselines established with 2007 data. In order to ensure that accurate and complete outcome data is being submitted, HOPWA has previously conducted, is currently managing, and will continue to develop comprehensive training and technical assistance plans.

Evidence: HUD's "National Evaluation of the Housing Opportunities for Persons with AIDS Program", conducted by HUD's Office of Policy Development and Research, is available online at http://www.huduser.org/Publications/pdf/hopwa_0101.pdf. On pages I-6 and IV-9, the study's findings on clients' feeling of current and future housing stability, after receiving HOPWA assistance, are included. The study states, "80 percent of clients who reported living in their own home, a group home, or an SRO said that their housing was 'more stable' or 'significantly more stable' now that they were receiving HOPWA assistance. Of those clients surveyed who reported living in a shelter, hotel/motel, or in a transitional housing program, 60 percent reported that their opportunities to obtain permanent, stable housing after they leave has significantly improved since receiving assistance." In 2007, HUD had one outcome measure, but in 2008 divided it into two measures (with a third in development) in order to individually track sub-components of their program. To view the targets for the 2007 measure, which correspond to the 2008 measures, please see HUD's Strategic Plan (2006-2011) on pages 15-44, available online at http://www.hud.gov/offices/cfo/stratplan.cfm. Also see HUD Annual Performance Plans from fiscal years 2005-2009 (page 79 for FY2009 APP), available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HUD will be updating future GPRA documents to include the revised measure targets. To help grantees improve performance and meet these targets, HOPWA conducted regional performance reporting and oversight trainings in 2006, and a national grantee training was conducted in 2007. A second round of regional training is now underway (see https://www.regonline.com/builder/site/Default.aspx?eventid=189390 for dates and locations). Finally, see also CPD Notice 06-07 "Standards for HOPWA Short-term Rent, Mortgage, and Utility (STRMU) Payments and Connections to Permanent Housing," for additional guidance that has been provided to grantees on measuring outcomes under the STRMU program (http://www.hud.gov/offices/cpd/lawsregs/notices/2006/06-07.pdf, pages 8-12).

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: HOPWA currently has reported data on four annual measures, two of which are long-term outcome measures reported annually. The third tracks grantee outputs through number of households served and the fourth measures grantees' ability to operate programs efficiently, by comparing HOPWA tenant-based rental assistance costs to those incurred under the Housing Choice Voucher program. The first two long-term/annual measures, housing stability and reduced risks of homelessness, are reported on annually so as to track progress towards the long-term outcome goals HOPWA has established. HOPWA began collecting annual grantee data on these measures in 2005, and has used this data to establish targets for the measures. HOPWA grantees will continue to report on these annual measures to track progress toward meeting the program's long-term outcome goals. HOPWA's two long-term/annual outcome measures reflect the core purpose of the HOPWA program: to develop long-term comprehensive strategies for meeting the housing needs of low-income persons with HIV/AIDS and their families. In addition to these two long-term/annual outcome measures, HOPWA has established both an output and an efficiency measure to allow the program to assess the level of assistance that it is providing.

Evidence: Indicators used for reporting on both of HOPWA's long-term performance outcome measures are found in the APR forms (pages 10-11) for competitive grantees and the CAPER forms (pages 9-10, 13) for formula grantees. Both the APR and CAPER are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. In 2007, HUD had one long-term/annual outcome measure, but in 2008 divided it into two long-term/annual measures in order to individually track sub-components of their program. To view the long-term/annual 2007 measure, which corresponds to the long-term/annual 2008 measures, please see HUD's Strategic Plan (FY 2006-2011), available online at http://www.hud.gov/offices/cfo/stratplan.cfm. This plan includes HOPWA's long-term outcome goal and further information about the program on pages 15-44. Also see the HUD Annual Performance Plans from fiscal years 2005-2009 (page 79 from FY 2009 APP) for an understanding of how the HOPWA performance measures fit into the overall framework of HUD strategic objectives, available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm . Also see January 24, 2008 memo from HUD's General Deputy Assistant Secretary for Community Planning and Development. This memo introduces HOPWA's 2008 APR and CAPER forms and is available online at http://www.hud.gov/offices/cpd/aidshousing/library/memo01242008.doc.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: All four of HOPWA's annual performance measures are set relative to established baselines, with clear timeframes and quantitative targets established, encouraging grantees to continuously improve program performance. Baselines and targets to 2012 for HOPWA's existing annual performance measures are included in the Annual Performance Plan as well as in annual updates to OMB. As was determined in Question 2.2, HOPWA's two long-term/annual measures have ambitious targets. In addition, HOPWA's annual output measure is both ambitious and realistic. In 2008, the program's performance output measure targets assisting 67,000 households. A target of assisting 70,500 households is established for 2009 and 2010 to coincide with an increased budget request and increased funding for 2008. HOPWA's efficiency measure, which compares HOPWA rental assistance costs per household to those under the Housing Choice Voucher (HCV) program, also includes ambitious targets to achieve greater efficiencies every year. These targets include an initial goal of HOPWA costs being less than 100 percent of comparable HCV costs in 2007 and 2008. The targets will decrease annually from 2009 to 2012, with the goals that HOPWA tenant-based rental assistance costs represent 96 percent of HCV costs in 2009 and one percentage point less each following year. HOPWA program data on progress toward these goals are based on annual grantee performance reports submitted by all formula and competitive grantees. Extensive and ongoing technical assistance and training on the implementation of performance outcomes has been provided for grantees to clarify project objectives and ensure consistent and accurate reporting by recipients.

Evidence: In 2007, HUD had one long-term/annual outcome measure, but in 2008 divided it into two long-term/annual measures in order to individually track sub-components of their program. To view the long-term/annual 2007 measure targets, which correspond to the long-term/annual 2008 targets, please see HUD's Strategic Plan (2006-2011) on pages 15-44, available online at http://www.hud.gov/offices/cfo/stratplan.cfm. Also see HUD Annual Performance Plans from fiscal years 2005-2009 (page 79 for FY2009 APP), available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HUD will be updating future GPRA documents to include the revised measure targets. In order to familiarize grantees on HOPWA's outcome measures and reporting, HOPWA conducted regional performance reporting and oversight trainings in 2006, and a national grantee training was conducted in 2007. A second round of regional training is now underway (see https://www.regonline.com/builder/site/Default.aspx?eventid=189390 for dates and locations). See also CPD Notice 06-07 "Standards for HOPWA Short-term Rent, Mortgage, and Utility (STRMU) Payments and Connections to Permanent Housing," for additional guidance that has been provided to grantees on measuring outcomes under the STRMU program (http://www.hud.gov/offices/cpd/lawsregs/notices/2006/06-07.pdf, pages 8-12). Current output, outcome, and financial data reported by grantees is available online in HOPWA Grantee Performance Profiles, at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm .

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: HOPWA has implemented performance reporting protocols which ensure that all program participants are committed to achieving HOPWA's performance goals and reporting program accomplishments accurately. Grantees were actively involved in the development of HOPWA's long-term outcome measures, and their comments were also used to revise annual performance reporting forms. HOPWA is working actively with technical assistance providers to conduct a series of 14 regional trainings across the US during the spring and summer of 2008 on the revised APR and CAPER forms, to ensure that grantees understand HOPWA's performance measures and how to use the reporting forms to evaluate their programs. Grantees' reported data will be updated to grantees' performance profiles quarterly and posted on the internet. This effort will improve grantees' understanding of their own data and the importance of accurate reporting. In addition, HOPWA grantee and project sponsors sign grant agreements that outline compliance with program management requirements including performance reporting. Community Planning and Development (CPD) Field Office staff are engaged with review of results under standards for showing achievement of annual and long-term goals through the field management tools, annual grantee risk assessments, report tracking and through the conducting of on-site monitoring. Program requirements on achieving outcomes were updated in regulations, grant agreement provisions, monitoring handbooks, the HOPWA Grantee Oversight Resource Guide, and related training materials.

Evidence: HOPWA regulations, grant agreements, and related operating instructions for both formula and competitive grants establish broad agreement in program tools for meeting reporting requirements and responsibilities. For example, the formula grant agreement specifies that grantees must ensure that each project sponsor will "establish housing output measures in the approved Consolidated Plan for the grant with specific one year goals for the number of households to be provided housing through the use of HOPWA." These operating instructions are available online at (http://www.hud.gov/offices/cpd/aidshousing/programs/formula/operatinginstructions/grantagmt.doc, page 2). The CPD Grantee Monitoring Handbook outlines guidance for HUD Field Office staff with regard to monitoring the activities and performance of HOPWA grantees (http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm, Chapter 10). Additionally, the HOPWA Oversight Resource Guide outlines grantee responsibilities with regard to monitoring and oversight of project sponsors (pending release, 2008). The Office of HIV/AIDS Housing, along with its team of national technical assistance providers, conducted a series of regional training sessions for HOPWA grantees and project sponsors on performance reporting and oversight in 2006. A second round of regional training is now underway (see https://www.regonline.com/builder/site/Default.aspx?eventid=189390 for dates and locations). Finally, HOPWA Grantee Performance Profiles allow grantees to access and interpret their own reported data, and they also allow grantees to use other grantees' data to better understand their own. These profiles are available online at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Research on HIV and housing has been a key focus of HUD and program partners. HUD has been collaborating with CDC on completing a joint "Housing and Health Study" to examine the impact of providing rental assistance to persons living with HIV/AIDS who are homeless or at severe risk of homelessness. CDC utilized a randomized controlled trial and worked with HOPWA grantees at three study sites: Baltimore, Chicago, and Los Angeles. In 2007, initial client intake data and the study objectives were published, and CDC will publish a final study later this year. The study represents a collaborative partnership between two federal agencies, which will produce a high quality study with unbiased results and a sufficient scope. The Chicago Housing and Health Partnership (CHHP) represents another significant randomized controlled trial, conducted from 2002 to 2007, to determine the effect of HOPWA funded supportive housing, including case management services, on homeless adults with HIV/AIDS. The Collaborative Research Unit of the Cook County Bureau of Health evaluated the results and presented them at the 2008 NAHC Research Summit, and will issue a report on study outcomes later this year. Both the Housing and Health and CHHP studies examine the impacts of housing assistance and related supportive services, which will provide insight on the two most critical components of the HOPWA program. Further studies on the importance of housing as an intervention for persons living with HIV/AIDS and other chronic illnesses have been conducted by faculty at Columbia University, Emory University, Johns Hopkins University, and the University of Pennsylvania. All of these evaluations were conducted by third parties not subject to direct oversight of HUD. In addition, HUD conducted several program management evaluations of the HOPWA program early in the program's implementation to ensure that the program was designed and operating effectively, including evaluations by HUD's Chief Financial Office and Office of Policy Development and Research (conducted through ICF Consulting). These studies found that management controls were in place for the HOPWA program, and that the HOPWA program was reaching its intended results??developing long-term housing strategies to benefit clients with housing assistance and support. Also, in partnership with HHS, Columbia University conducted a study on the specialized efforts of HHS and HOPWA to reach and serve clients with multiple diagnoses (1996-2002).

Evidence: See the November 2007 supplement to AIDS and Behavior, which contains 18 articles on the connection between housing and HIV/AIDS, including articles on HUD/CDC Housing and Health study. (To purchase the journal, go to http://www.nationalaidshousing.org/PurchaseAIDSandBehavior.htm) For additional information on the Housing and Health and CHHP studies, see NAHC's Research Summit site at http://www.nationalaidshousing.org/HousingandHIV-AIDSResearchSummit.htm. GAO's reports on HUD include number 06-332, "Changes Needed to Improve the Distribution of Ryan White CARE Act and Housing Funds," which includes an evaluation of specific program aspects (http://www.gao.gov/new.items/d06332.pdf). For earlier research on HOPWA program design, results, and management, see the 2000 HUD PD&R study National Evaluation of the Housing Opportunities for Persons with AIDS Program (http://www.huduser.org/publications/hsgspec/hopwa_0101.html) and HUD Chief Financial Office Risk Management Review (2000).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: HUD's FY 2009 Congressional Justification for HOPWA details annual accomplishments under the Department's strategic goals. Goals for both annual and long-term measures are clearly presented in the context of this budget request. The FY 2009 budget projects an increase in grant performance based on use of 2008 resources, tied to a needs analysis of the size of the HIV epidemic, qualification of new formula grantees, and solid program budgetary outlays. The budget request estimates that an increase in appropriations will support a greater number of affordable housing units for persons with HIV/AIDS and their families. HOPWA uses annual reporting data for these estimations and links the budget justifications to the established performance goals. HUD received $300.1 million for the HOPWA program in FY 2008, representing an increase of $13.99 million over the FY 2007 appropriation. However, to account for the fact that budgetary changes generally affect the program's performance the year after they are implemented, HOPWA's performance targets for 2008 are consistent with those from 2007. HOPWA has increased its performance target for FY 2009 to account for the FY 2008 budgetary increase: the additional $13.99 million is expected to help serve an additional 3,500 households, so the FY 2009 target will increase to 70,500.

Evidence: HUD's Congressional Justifications for HOPWA describe the program and include information about past financial and performance information, as well as future targets. Please note that in 2007, HUD had one long-term/annual outcome measure, but in 2008 divided it into two long-term/annual measures in order to individually track sub-components of their program. The efficiency measure was revised slightly, as well. Though the FY 2009 budget documents discuss the 2007 measures, all future budget documents will reflect the revised measures. HOPWA's portion of the FY 2009 HUD Congressional Justification is on pages Q-1 through Q-20 and available online at http://www.hud.gov/offices/cfo/reports/2009/cjs/cpd1.pdf . In addition to reporting direct costs that are needed to meet performance targets, the Congressional Justification also reports the indirect resource and workload costs needed to meet HOPWA performance targets. HUD's FY 2009 Congressional Justification for the HOPWA program includes estimates of FTEs and costs for salaries and expenses including personal services, travel, printing, other services, and supplies. See CPD's section on Salaries and Expenses (http://www.hud.gov/offices/cfo/reports/2009/cjs/cpd2.pdf ) for these estimates on page C-8. Additional resources which outline indirect costs needed to support the HOPWA program include pages 107, 109, and 113 of HUD's FY 2007 Performance Accountability Report (PAR) and pages 24, 26 and 30 of HUD's FY 2008 Annual Performance Plan (APP). Both the APP and PAR are available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HOPWA's performance measures and targets are included in HUD's FY 2006-2011 Strategic Plan (http://www.hud.gov/offices/cfo/reports/hud_strat_plan_2006-2011.pdf) on pages 15-44.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: HOPWA has addressed each of the Performance Improvement Plans that were created as a result of its 2003 PART Review. The program has worked with its grantees to develop two long-term outcome measures: (1) By 2012, the percentage of HOPWA clients in permanent housing programs who maintain housing stability will be 90 percent; and (2) By 2012, the percentage of HOPWA clients in short-term or transitional housing programs who have reduced risks of homelessness will be 70 percent. HOPWA revised its performance measurement forms in 2006 to allow the program to collect data for these outcome measures. Since 2006, HOPWA has implemented a fully comprehensive technical assistance approach, including a current series of regional trainings, to ensure grantees' success in collecting and reporting performance data. In response to grantee comments, HOPWA issued an updated APR and CAPER form in 2008 to collect grantee performance data. HOPWA has initial data on client housing stability and reduced risks of homelessness, and the program is anticipating that data from the 2008 reports will continue to inform HOPWA of grantees' results. For grantees that are reporting below the outcome targets, HOPWA is providing targeted technical assistance to enhance grantees' ability to operate and report on programs with successful outcomes.

Evidence: In 2007, HUD had one long-term/annual outcome measure, but in 2008 divided it into two long-term measures in order to individually track sub-components of their program. To view the long-term 2007 measure targets, which correspond to the long-term 2008 targets, please see HUD's Strategic Plan (FY 2006-2011), available online at http://www.hud.gov/offices/cfo/stratplan.cfm. This plan includes HOPWA's long-term outcome goal and further information about the program on pages 15-44. Additional evidence includes HUD's Also see the HUD Annual Performance Plans from fiscal years 2005-2009 (page 79 from FY 2009 APP) for an understanding of how the HOPWA performance measures fit into the overall framework of HUD strategic objectives, available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HUD will be updating future GPRA documents to include the revised measure targets. Indicators used for reporting on both of HOPWA's long-term performance outcome measures are found in the APR forms (pages 10-11) for competitive grantees and the CAPER forms (pages 9-10, 13) for formula grantees. Both the APR and CAPER are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: HUD offices review performance documents and make use of tracking tools and data verification steps to manage HOPWA performance reporting. HOPWA formula grantees are required to submit accomplishment data in Consolidated Annual Performance and Evaluation Reports (CAPERs) that quantify outcomes and outputs over the previous year and provide for comparing actual accomplishments against planned accomplishments. Similarly, Annual Progress Reports (APRs) are used for reporting by competitive grants. HOPWA used grantees' data, as reported in these performance forms, to establish baselines for output and outcome measures, as well as to set meaningful, ambitious performance targets. HOPWA also uses grantees' data to analyze key factors associated with successful performance of HOPWA programs, and to assist grantees in their efforts to perform successfully and achieve the best possible results. Assessments of program data reported in APRs and CAPERs are also used to improve the instructions and guidance on completing the forms, as well as to target technical assistance resources to further the accuracy of performance reporting. HOPWA grantee performance profiles??which are distributed to grantees and HUD Field Offices, and posted online??include output and outcome information, by housing type for each grantee. An additional tool is also being issued at HOPWA's 2008 trainings: the "Grantee Oversight Resource Guide," which provides guidance to grantees on programmatic oversight of project sponsors and their activities. The issuance of this guide, in addition to the 2008 performance measurement trainings, will assist grantees in conducting oversight over their project sponsors, making use of actual performance data, identifying corrections as needed to enhance the quality of this data, and thereby improving project planning and client results.

Evidence: HOPWA Consolidated Annual Performance and Evaluation Reports (CAPERs) and HOPWA collects annual grantee data through Consolidated Annual Performance and Evaluation Reports (CAPERs) for formula grantees and Annual Progress Reports (APRs) for competitive grantees. Both the CAPER and APR are available online at http://www.hud.gov/offices/cpd/aidshouding/index.cfm. Current output, outcome, and financial data reported by grantees is available online in HOPWA Grantee Performance Profiles, at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm. The HOPWA Oversight Resource Guide outlines grantee responsibilities with regard to monitoring and oversight of project sponsors (pending release, 2008). This guide builds upon CPD's Monitoring Handbook that is used by Field Offices to evaluate performance of grantees and is available online at http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm. For information on current technical assistance plans for training on performance reporting and introducing grantee performance profiles and the Grantee Oversight Resource Guide, see https://www.regonline.com/builder/site/Defalt.aspx?eventid=189390.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and partners are held accountable for grantees' achieving performance results and for their management of finances. HUD managers' job performance is linked to the Department's GPRA goals, under the Performance Accountability and Communication System (PACS) and the Leadership Development and Recognition System (LDRS). Forty-four CPD Field Offices develop Field Office work plans that identify oversight activities of grantees to identify monitoring, technical assistance, or corrective actions. For example, within the last four years $2.5 million from 29 competitive, 7 formula, and one technical assistance grant has been recaptured and reallocated into the competitive grants competition. Field staff use the Line of Credit Control System (LOCCS), CAPERs and APRs to develop annual risk analyses, which are used to identify grantees that may have financial or performance issues. Formula grantees are required to develop annual action plans through the Consolidated Planning process to identify how resources will be used, and performance is tracked and reviewed consistent with program requirements and grant agreements. Competitive grants are reviewed consistent with NOFA standards and renewal notice requirements. HOPWA formula and competitive grantees must also report on how their program accomplishments contribute to HOPWA's performance goals, including annual housing outputs and annual achievement toward long-term client outcomes for housing stability, reduced risks of homelessness, and improved access to care.

Evidence: Under HUD employee performance standards in the PACS and LDRS systems, ratings, promotions and monetary awards are appropriate to managers' accomplishments in meeting the Department's GPRA goals. Program partners also are assessed in meeting performance objectives. CPD risk assessment procedures and Monitoring Handbook, used to evaluate the performance of grantees, are available online at http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm. HOPWA-specific information is found in Chapter 10. Additional CPD standards for risk assessment procedures are included in CPD Notice 07-07, available at http://www.hud.gov/offices/adm/hudclips/notices/cpd/07-7CPDN.doc . See HOPWA operating instructions for standard program grants management tools, such as grant agreements, award conditions, financial practices and performance reporting elements and forms (http://www.hud.gov/offices/cpd/aidshousing/library/index.cfm ). HOPWA's most recent renewal notice, CPD Notice 08-03, outlines requirements for competitive renewal grantees and is available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/08-03CPDN.doc. Field staff report on the number of grants executed and provide ongoing reviews of costs through the LOCCS system. The HUD Performance Accounting System (PAS) includes program records on financial recaptures.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: HUD financial system controls and HOPWA data collection tracking tools are used to ensure timely obligation of unobligated resources. Grant funds are consistently obligated within the required timelines, abiding by the statutory limits established in FY 2002 for obligation within 2 years (formula), and 3 years (competitive), and expenditures within 5 years of commitment. The program demonstrates a consistently high level of overall outlays, which generally equal or exceed levels of new funds made available. HUD's CFO compiles a report of all grantee obligations and inputs this information into the FAADS. Additionally, as HUD offices award contracts through the Office of the Chief Procurement Officer, they are entered into the FPDS-NG system. According to the HUD Reform Act, all awards must also be published in the Federal Register. Once awards are obligated, field staff monitor the expenditure of funds through the Line Of Credit Control System (LOCCS) and Integrated Disbursement Information System (IDIS). The Office of HIV/AIDS Housing receives monthly status reports on grant program accounts, showing obligation, expenditure and recaptured funds, and evaluates need for technical assistance and field management actions. Information is also shown in HOPWA grantee performance profiles, which have been distributed to grantees and Field Offices, and are used to support grant management needs in reviewing timeliness of reporting and outlays. The profiles use a financial performance ratio similar to one developed and applied to Community Development Block Grant programs to compare timeliness based on formula grantees' balance of funds on hand to annual funds allocated. This tool enables the Field Office to target grantees that need additional oversight or corrective actions taken. This tool enables all program participants to easily understand and address financial irregularities. In addition to HOPWA grantee performance profiles, HUD CPD Field Office staff regularly monitor and address audits issues as filed through the Federal Audit Clearinghouse, and use a process to identify those that pertain to HOPWA. HUD staff work with grantees to determine that corrective actions address the audit concerns.

Evidence: HUD financial reports for HOPWA grants from 2003-2008 include information on obligations. Annual Congressional Justifications for the HOPWA program also include information on obligations and outlays, with the most recent Congressional Justification (FY 2009) available online at http://www.hud.gov/offices/cfo/reports/2009/cjs/cpd1.pdf (see page Q-1). The Department of Housing and Urban Development Reform Act of 1989 includes requirements for the Department to publish a notice of available funding, as well as information about the awardees subsequent to their selection. See Section 102(a)(4) of the act, available online at http://thomas.loc.gov/cgi-bin/query/F?c101:7:./temp/~mdbsj28ZGa:e4108. Grantee performance profiles can be found at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm. Field Offices track audits that are filed through the Federal Audit Clearinghouse, available online at http://harvester.census.gov/sac/. Field Offices request a copy of the audit from the audited grantee, submit correspondence to the grantee, and determine a plan for corrective actions. Grantees spending levels on various program components of HOPWA (housing costs and supportive services) were noted in GAO's February 2006 report on HOPWA, "Changes Needed to Improve the Distribution of Ryan White CARE Act and Housing Funds." This report (06-332) is available online at http://www.gao.gov/new.items/d06332.pdf. See page 20 for breakdown of grantees' usage of HOPWA funds.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The HOPWA program has an overall efficiency measure in place with a baseline and targets, which compares HOPWA's costs of rental assistance per household (adjusted to average HOPWA household size) to the Housing Choice Voucher program's costs of rental assistance per household. HOPWA is also working with researchers on the HUD-CDC Housing and Health Study and the Chicago CHHP Study to interpret and utilize cost analysis and related efficiency results of these studies. In 2008, HOPWA competitive grant application reviews will serve as a pilot for HUD to test community approaches to a return on investment model to help evaluate cost effectiveness of affordable supportive housing. Results of this and related program assessments may lead to replacing the efficiency measure in future years, and the developmental use of related performance data is under review. Further, in 2008 HUD is conducting training with HOPWA grantees on measuring cost efficiency and how this information can be used to plan for and adjust projects. HUD uses standards on efficiency that are issued in Notice of Funding Availability (NOFA) and Renewal Grant Notices, along with standard federal grant requirements, when selecting competitive grantees. New evaluation tools make use of outcomes, costs per unit, and leveraged funds, and they assist grantees evaluate their program results. Other efforts to measure and achieve efficiencies in program execution include technical assistance, which provides support and training to improve project management and accountability. Technical assistance resources are targeted to address under-performers and grantees confronting impediments with implementing their grant activities. Finally, HUD continues its efforts to enhance IT changes to systems such as IDIS and E-Grants.

Evidence: Annual efficiency measure actuals are included in HOPWA's PART Web updates and are available online via HOPWA's performance measures section of the Expect More website: http://www.whitehouse.gov/omb/expectmore/detail/10001163.2003.html#performanceMeasures . One initial article that has been released on the cost analysis results of the Housing and Health Study is Holtgrave, D.R., Briddell, K., Little, E., Bendixen, A.V., Hooper, M., Kidder, D.P., Wolintski, R.J., Harre, D., Royal, S., Aidala, A., "Cost and Threshold Analysis of Housing as an HIV Prevention Intervention" (2007) AIDS And Behavior. HOPWA anticipates using this information and other forthcoming articles to support the program's ongoing evaluation of its cost efficiencies. Annual HOPWA competitive grant application standards are included in the Department's Notice Of Funding Availability and are published in the Federal Register, posted online at http://www.federalregister.gov. HOPWA technical assistance plans, maintained by program staff, outline technical assistance training and objectives. Throughout spring of 2008, HOPWA is conducting a series of regional technical assistance trainings, for which the schedule and details are available online at https://www.regonline.com/builder/site/Default.aspx?eventid=189390. Guidance for the IDIS program, used by HOPWA formula grantees, is available online at http://www.hud.gov/offices/cpd/systems/idis/reporting/index.cfm.

Does the program collaborate and coordinate effectively with related programs?

Explanation: As part of HUD's special needs programs, HOPWA is integrated into HUD's Consolidated Planning requirements, as well as within HUD's Strategic Plan and mission. In addition to the review of annual action plans that identify proposed activities, other Consolidated Planning processes include an assessment of coordination of related efforts and connections with other federal programs and state and local resources. HOPWA grantees and project sponsors are further required to coordinate with their areas' local governments, State governments, private organizations, and other public organizations that are responsible for providing services relevant to the HOPWA population. The relationship between the HOPWA and Ryan White programs is reflected in area AIDS service planning efforts and local coordination of support for clients, including case management efforts to improve access to this related care for beneficiaries of stable housing support as well as in agency cross-training of grantees at national training events hosted by HUD and HRSA. HOPWA also reviews competitive grant applications and has selected projects as models of service delivery to populations in collaboration with other federal efforts, including assisting persons who are chronically homeless, persons living in rural areas, persons experiencing substance abuse issues, and persons experiencing mental illness. When redesigning its reporting forms, HOPWA worked with both the Ryan White program at HHS and the Special Needs Assistance Programs office at HUD, and these programs share reported grantee data with HOPWA. HOPWA also collaborated with several federal agencies to create the AIDS.gov website in 2006 to serve as the portal for information dissemination to the public and service provider network. Collaborative efforts to maintain and advance information sharing are on-going. Also, as detailed in answer 2.6, HOPWA has been collaborating with CDC in a research effort to clarify the relationship between stable housing, health outcomes, and the progression of HIV.

Evidence: Consolidated Plan requirements are in 24 CFR §91, available online at http://www.access.gpo.gov/nara/cfr/waisidx_03/24cfr91_03.html. HOPWA regulations outline the responsibility of formula grantees to submit a Consolidated Plan are in 24 CFR §574.110. HOPWA regulations requiring grantees and sponsors to collaborate with local and State governments and agencies are in 24 CFR §574.420. HOPWA regulations limiting grantees from paying for health services that are otherwise expected to be covered by Federal or State insurance programs, compensation programs, or prepaid health service programs are in 24 CFR §574.310(2). HOPWA regulations are available online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html. HUD's collaboration with CDC for the Housing and Health Study has produced several articles to date, available in the AIDS and Behavior November 2007 Supplement, "Housing and HIV/AIDS." The HUD and HHS Multiple Diagnosis Initiative "Lessons Learned" publication was produced in 2001. HOPWA's collaborations with the National AIDS Housing Coalition include presentations at three NAHC research summits and communication on numerous program matters. HOPWA's collaboration with the National Association of People With AIDS includes a presentation at HOPWA's 2007 National Grantee Training on individual needs, from client perspectives. HOPWA also collaborates with several organizations on CPD outcome performance measurement reporting and IDIS enhancements, including discussions with the Council of State Community Development Agencies, National Community Development Association, National Association for County Community Economic Development, and the National Association for Housing and Redevelopment Officials. See CPD Notice of Draft Outcome Performance Measurement System for Community Planning and Development Formula published in FR June 10, 2005 noting this collaboration with national organizations to enhance outcome reporting and assessment of formula programs, available online at http://a257.g.akamaitech.net/7/257/2422/01jan20051800/edocket.access.gpo.gov/2005/pdf/05-11619.pdf and related Q&A in implementation of resulting program changes http://www.hud.gov/offices/cpd/about/performance/QandA_111805.pdf.

Does the program use strong financial management practices?

Explanation: HUD management systems and program tracking tools support the program's ability to monitor the use of HOPWA resources. HOPWA and other CPD program's audits, which are reviewed annually, have been found to be free from material internal control weaknesses by HUD's Office of Inspector General (2004). A CFO review of HOPWA, which was conducted in 2000, determined that sufficient management controls were in place for financial transactions to reduce risks of waste, fraud, abuse and mismanagement. HOPWA competitive grantees use the Line of Credit Control System (LOCCS) to draw down funding and track expenditures, in accordance with agency and CPD timeliness standards. Formula grantees utilize the IDIS system. Grantees must maintain documentation for all expenditures, and they must also meet certain benchmarks to continue spending on construction and rehabilitation projects. Field Office staff use the LOCCS tool to monitor grantees' expenditures, including monthly reports that allow for project oversight. A HOPWA tracking tool and grantee performance profiles are also used to support field management needs in reviewing timeliness of reporting and outlays. CPD conducts annual risk assessments along with on-site monitoring to ensure grantee compliance with HUD financial management standards. Further, where appropriate based on risks, HUD grants oversight has occurred and resulted in corrections, recaptures, and other grants management actions.

Evidence: Program performance and accountability, including financial controls, audit reviews, and management actions addressing identified deficiency or problem, are shown in HUD's annual Performance and Accountability Report. See FY2007 PAR at http://www.hud.gov/offices/cfo/reports/2007par.cfm. HUD's Chief Financial Office Risk Management Review (2000) found that management controls are in place and sufficiently operating. CPD Field Offices consider timeliness standards when conducting risk assessments of grantees. The procedures for risk assessments are found in CPD Notice 07-07 and available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/07-7CPDN.doc. Also, the CPD Monitoring Handbook includes information on timeliness of expenditures (http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm). HOPWA Operating Instructions include standard program grants management tools, such as grant agreements, award conditions, financial practices, management plans for on-site monitoring, and performance reporting elements and forms (http://www.hud.gov/offices/cpd/aidshousing/index.cfm). LOCCS and IDIS screens show grantees' requirements for tracking spending and represent HUD's mechanism to also track grantee drawdowns. HOPWA's technical assistance efforts are outlined, including schedules and locations, at https://www.regonline.com/builder/site/Defalt.aspx?eventid=189390. Performance profiles can be found at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: HOPWA regularly uses contact with grantees, technical assistance providers, contractors' ongoing reviews of grantee data and grantees' reported barriers and recommendations to identify possible program management deficiencies. Additionally, CPD Field Offices conduct risk assessments and monitoring of grantees to further identify and address possible management issues and develop related guidance. HUD has taken management actions to update program regulations establishing output/outcome requirements, issue and update reporting tools, clarify HOPWA objectives in HUD strategic plans, guide grantees in training efforts and ensure compliance with these program standards in monitoring performance consistent with the CPD Monitoring Handbook. This aggressive set of management actions is demonstrating results in grantees' annual reporting data, and their data is further used to guide technical support and inform evaluations and planning. HOPWA reports actual financial and performance data, and maintains regular contact with grantees to engage in effective service delivery, which is shown in overall program performance. Related data cleanup efforts have also been undertaken and reporting materials have been updated, including pending IT enhancements to support reporting accuracy. Consultation and training with grantees is a regular part of program management efforts. While no serious management deficiencies are currently identified, 2008 training efforts and use of the Grantee Oversight Resource Guide will increase accountability throughout the program.

Evidence: HUD's Chief Financial Office Risk Management Review (2000) found that management controls are in place and sufficiently operating. Departmental actions to ensure accountability are report annually see Program performance and accountability, including financial controls, audit reviews, and management actions addressing identified deficiency or problem, are shown in HUD's annual Performance and Accountability Report. See FY2007 PAR at http://www.hud.gov/offices/cfo/reports/2007par.cfm. HOPWA's Grantee Oversight Resource Guide is currently in draft form and is being presented at regional TA trainings, allowing grantees to comment on the Guide's usability. The Guide is pending publication. The procedures for risk assessments are found in CPD Notice 07-07 and available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/07-7CPDN.doc. Also, the CPD Monitoring Handbook includes information on timeliness of expenditures (http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm). HOPWA Operating Instructions include standard program grants management tools, such as grant agreements, award conditions, financial practices, management plans for on-site monitoring, and performance reporting elements and forms (http://www.hud.gov/offices/cpd/aidshousing/index.cfm).

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: In addition to data collection efforts, HOPWA takes steps to validate its data and uses information for evaluation and oversight. HOPWA formula grantees use the Integrated Disbursement and Information System (IDIS) to draw down funding and track expenditures, in accordance with agency and new HOPWA timeliness standards. Grantees must maintain documentation for all expenditures, and they must also meet certain benchmarks to continue spending on construction and rehabilitation projects. Field Office staff monitor grantees' expenditures through IDIS??a real-time disbursement system that collects detailed funding activity on every draw. This system includes internal controls that automatically alerts Field Office staff of expenses requiring further clarification. A HOPWA tracking tool is also used to support field management needs in reviewing timeliness of reporting and outlays. CPD conducts annual risk assessments along with on-site monitoring to ensure grantee compliance with HUD financial management standards. In 2007, field staff reported conducting reviews of 27 percent of HOPWA formula grants. HOPWA elements of the CPD Monitoring Guide and other related tools are updated in the monitoring handbook, annual performance reports, and IT manuals. Field office and headquarters staff review grantee performance for eligibility of activities and results consistent with approved plans. Technical assistance planned during 2008 includes a focus on how grantees should conduct oversight of their project sponsors, using the forthcoming "Grantee Oversight Resource Guide".

Evidence: The IDIS system is described in the user guide found at http://www.hud.gov/offices/cpd/systems/idis/reporting/. HOPWA's NOFA's also outline financial reporting requirements for grantees, and the most recently published program NOFA is online at http://www.federalregister.gov. The procedures for risk assessments are found in CPD Notice 07-07 and available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/07-7CPDN.doc. Also, the CPD Monitoring Handbook includes information on timeliness of expenditures (http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm). HOPWA Operating Instructions include standard program grants management tools, such as grant agreements, award conditions, financial practices, management plans for on-site monitoring, and performance reporting elements and forms (http://www.hud.gov/offices/cpd/aidshousing/index.cfm). Grantee information is also provided through required public consultations for strategic plans and annual performance reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Grantee performance profiles update the level of HOPWA's transparency to the public by providing information of client outcomes, housing outputs, and timeliness of expenditures. Profiles are based on data collected in the national HOPWA database, which is shared with all program participants and the public at large. Formula projects report their results annually in standard HUD reports. These performance data are featured in grantee performance profiles, which are posted online at the HOPWA website, and are used for HOPWA evaluation training. The profiles are updated quarterly to reflect grantee performance and financial data. HOPWA's website also includes grantee contact data. In addition to this grantee-level data, HUD posts aggregated program data to the HOPWA website as well. HUD reports annual grantee outputs and outcome data to the Department for the Performance and Accountability Report (PAR), as well as the Annual Performance Plan (APP), and these reports are also posted on the HUD website. HOPWA's program information is also linked to the AIDS.gov portal for public access to information on federal HIV/AIDS resources. Finally, HOPWA program results have been included in presentations and reports at three National AIDS Housing Coalition research summits and reported in national journals.

Evidence: HOPWA's grantee performance profiles are available online at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm. Formula grantees report performance results annually in CAPER forms, available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. HOPWA's performance measures and actuals are reported in the PAR (pages 167, 169, 207 and 237 of the FY 2007 report) and APP (page 79 of the FY 2009 report), both available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. Information from NAHC research summits is available at http://www.nationalaidshousing.org.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Ten percent of the annual HOPWA appropriation is designated for a competitive process, and among this 10%, all of the awards are consistently competed fairly and openly. Competitive grants consist of two categories: the three-year renewal of expiring permanent supportive housing projects that were originally competitively awarded; and new grant awards for: (1) long-term projects in geographic areas that are not eligible for formula allocation or in the balance of the state areas outside of an eligible metropolitan statistical area; and (2) Special Projects of National Significance (SPNS) that will undertake innovative housing service models. Funding availability for new project awards are announced through the Department's annual SuperNOFA (Notice of Funding Availability) that is published in the Federal Register. Renewal applications are assessed against a pass/fail threshold review for eligible activities and budget line funding requests. The review includes an assessment of current grant performance and a determination that project is continuing to fulfill its original intention of providing permanent supportive housing. With respect to new projects, the SuperNOFA specifies the application procedures and describes the rating factors and criteria used to score and rate each application: (1) capacity of the applicant and project sponsors and relevant organizational experience; (2) need/extent of the problem; (3) soundness of approach: responsiveness, coordination and public policy priorities, and model qualities; (4) leverage and sustainability; and (5) achieving results and program evaluation. HUD reviews each application to ensure that it meets the threshold requirements outlined in the NOFA: HUD will not select an application that is rated below 75 points, receives a Rating Factor 1 (Capacity) score lower than 14 points, or receives a Rating Factor 4 (Leveraging) score lower than one point. Both review procedures consist of application review conducted by HUD staff. Annual web casts are broadcasted nationally to provide all potential applicants with the opportunity to understand precisely how applications are to be submitted and scored. These web casts are posted on HUD's website for public view.

Evidence: Information about HOPWA competitive grants, including amounts available, eligible applicants, and application requirements, are included in HOPWA's regulations in 24 CFR §574.200 through §574.260, available online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html. HOPWA's most recent renewal notice, CPD Notice 08-03, outlines requirements for competitive renewal grantees and is available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/08-03CPDN.doc. The most recently published program NOFA, which includes components of grantees' applications and corresponding scoring information, is online at http://www.federalregister.gov. Archived webcasts are available at HOPWA's website at http://www.hud.gov/offices/cpd/aidshousing/index.cfm.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: During the competitive grants review process, eligible grant activities are identified and reviewed against published standards used to determine eligibility of grant activities for both the renewal of expiring permanent supportive housing projects and for new project activities. Upon grant award, a grant agreement is signed by the grantee and HUD that establishes grants management procedures to ensure compliance with programmatic and financial management requirements. Both HUD field staff and HQ staff monitor and review expenditures through the Department's Line of Credit Control System (LOCCS) that enables grantees to draw down grant funding. Grantees are required to maintain documentation for all grant expenditures and for those undertaking construction and rehabilitation activities must also meet established benchmarks to continue drawing down grant funding. LOCCS provides HUD staff with the ability to track grantee expenditures within each budget line activity category. The system provides internal financial management controls that highlight issues requiring additional analysis. HUD field staff routinely monitor grantee expenditures and a HOPWA tracking tool is used to support field staff needs in reviewing timeliness of reporting and outlays. Field staff conduct annual risk assessments to determine those grant activities that are identified for on-site monitoring (a priority ranking is identified for the on-site monitoring of grantees designated as "high risk"). The CPD Monitoring Handbook establishes the guidelines for evaluating grantee compliance with both programmatic and financial grants management activities. In addition, updated NOFA requirements and standards are used to guide reviews of proposed activities and approved budget line item activities. The submission of the grantee's Annual Progress Report (APR) provides HUD with specific accomplishment and client outcome data that is used to assess grantee compliance. Technical assistance and support is provided to grantees when performance reporting deficiencies are identified through annual risk assessment, reporting, performance, or financial reviews.

Evidence: HOPWA's NOFA's outlines financial reporting requirements for grantees, and the most recently published program NOFA is online at http://www.federalregister.gov. The procedures for risk assessments are found in CPD Notice 07-07 and available online at http://www.hud.gov/offices/adm/hudclips/notices/cpd/07-7CPDN.doc. Also, the CPD Monitoring Handbook includes information on timeliness of expenditures (http://www.hud.gov/offices/cpd/library/monitoring/handbook.cfm). HOPWA Operating Instructions include standard program grants management tools, such as grant agreements, award conditions, financial practices, management plans for on-site monitoring, and performance reporting elements and forms (http://www.hud.gov/offices/cpd/aidshousing/index.cfm ). Grantee information is also provided through required public consultations for strategic plans and annual performance reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Annual performance data is collected for each competitive grant through the submission of an Annual Progress Report (APR). HOPWA Grantee Performance Profiles that are posted online at the HOPWA website provides public transparency through the identification of client outcomes, housing outputs, and the timeliness of expenditures. Profiles are based on data collected in the national HOPWA database which is shared with all program participants, technical assistance providers, and the public at large. These profiles are updated quarterly to reflect grantee performance and financial data. The HOPWA website also includes grantee contact data and information. In addition to this grantee level data, HUD posts aggregate program data to the HOPWA website. HUD reports annual grantee outputs and outcome data to the Department for the Performance Accountability Report (PAR), as well as the Annual Performance Plan (APP), and these reports are also posted on the HUD website. In addition, HOPWA program information is linked to the AIDS.gov portal for public access to information on federal HIV/AIDS resources.

Evidence: HOPWA's competitive grantees report performance results annually in APR forms, available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. HOPWA's performance measures and actuals are reported in the PAR (pages 167, 169, 207 and 237 of the FY 2007 report) and APP (page 79 of the FY 2009 report), both available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. Grantee performance profiles are available online at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: HOPWA is on track to exceed its targets for both of its long-term performance outcome measures. For the first measure, "marinating stable housing," for which data and targets have been available since 2005, HOPWA has consistently exceeded its targets. The second measure, "reducing risks of homelessness," which was established as a separate target in 2008, has had available data since 2005. Though the 2008 target is ambitious, HOPWA expects to exceed the new target as well. In order to ensure that accurate and complete outcome data is being submitted, HOPWA has developed a comprehensive training and technical assistance plan. HOPWA conducted a webcast on the grantee reporting forms, and, is conducting 14 regional trainings across the country to support grantees' data reporting. These efforts and the issuance of grantee performance profiles will help grantees to report accurately and will contribute toward focused program evaluations using a verified baseline on HOPWA outcomes.

Evidence: Data indicators used for reporting on both of HOPWA's long-term performance outcome measures are found in the APR forms (pages 10-11) for competitive grantees and the CAPER forms (pages 9-10, 13) for formula grantees. The APR and CAPER are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. In 2007, HUD had one outcome measure, but in 2008 divided it into two measures (with a third in development) in order to individually track sub-components of the HOPWA program. To view the 2007 measure, which is very similar to the 2008 measures, please see pages 15-44 of HUD's Strategic Plan (FY 2006-2011), available online at http://www.hud.gov/offices/cfo/stratplan.cfm. Also see HUD's Annual Performance Plans from fiscal years 2005-2009 (page 79 from the FY 2009 APP) for the long-term outcome measures and their actuals, available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HOPWA Grantee Performance Profiles are available online at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm.

Does the program (including program partners) achieve its annual performance goals?

Explanation: HOPWA has met or exceeded its annual performance goals that were established prior to 2008. In addition, the focus on client outcomes is fostering transformation in project planning: grantees conducting short-term assistance are assessing their results and adjusting efforts to achieve long-term results. HOPWA's first and second annual measures are long-term goals measured each year, and the program's success in meeting these measures are discussed in Q 4.1. HOPWA's third performance measure indicates the number of rental households receiving assistance. The program successfully achieved its housing output goal of providing assistance to 67,000 households in 2007. HOPWA's fourth annual measure is its efficiency measure, which compares HOPWA cost per household assisted for Tenant-Based Rental Assistance to cost per households assisted in the Housing Choice Voucher program. The target for this efficiency outcome measure is that HOPWA will be able to show that HOPWA is cost effective on a per unit basis, and HOPWA has demonstrated this consistently. Data from 2007 show that HOPWA cost per household was 94.8% of the Housing Choice Voucher cost per household.

Evidence: Data indicators used for reporting on both of HOPWA's long-term performance outcome measures are found in the APR forms (pages 10-11) for competitive grantees and the CAPER forms (pages 9-10, 13) for formula grantees. The APR and CAPER are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. In 2007, HUD had one annual outcome measure, but in 2008 divided it into two annual measures (with a third in development) in order to individually track sub-components of the HOPWA program. To view the 2007 measure, which is very similar to the 2008 measures, please see pages 15-44 of HUD's Strategic Plan (FY 2006-2011), available online at http://www.hud.gov/offices/cfo/stratplan.cfm. Also see HUD's Annual Performance Plans from fiscal years 2005-2009 (page 79 from the FY 2009 APP) for the long-term outcome measures, reported on annually, and their actuals, available online at http://www.hud.gov/offices/cfo/reports/cforept.cfm. HOPWA Grantee Performance Profiles are available online at http://www.hud.gov/offices/cpd/aidshousing/reports/perfpro.cfm.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The HOPWA program has an efficiency measure in place which compares HOPWA's costs per household receiving Tenant-Based Rental Assistance against the Housing Choice Voucher program's cost per household. Performance reports and studies document that HOPWA's rental assistance program provides stable housing for clients that is cost effective in comparison to HUD's Housing Choice Voucher program. When comparing HOPWA's performance on this efficiency measure, HOPWA achieved greater cost efficiency in 2007 (94.8% of the Housing Choice Voucher household costs) than 2006 (97.5% of Housing Choice Voucher household costs). HOPWA requests cost information from grantees in their annual reports, and this data is used to compare efficiencies. In 2008 HUD is conducting training with HOPWA grantees on measuring cost efficiency and how this information can be used to plan for and adjust projects. HUD uses standards on efficiency that are issued in Notice of Funding Availability (NOFA) and Renewal Grant Notices, along with standard federal grant requirements, when selecting competitive grantees. New evaluation tools make use of outcomes, costs per unit, and leveraged funds, and they assist grantees evaluate their program results. Other efforts to measure and achieve efficiencies in program execution include technical assistance, which provides support and training to improve project management and accountability.

Evidence: Annual efficiency measure actuals are included in HOPWA's PART Web updates and are available online via HOPWA's performance measures section of the Expect More website: http://www.whitehouse.gov/omb/expectmore/detail/10001163.2003.html#performanceMeasures. HOPWA's performance reporting forms, the APR for competitive grantees and the CAPER for formula grantees, are available online at http://www.hud.gov/offices/cpd/aidshousing/index.cfm. Expenditure and output information from these forms are used to track program's cost efficiencies. HOPWA anticipates receiving additional information on cost efficiencies within the program through the HUD-CDC Housing and Health Study. One initial article that has been released on the cost analysis results of this study is Holtgrave, D.R., Briddell, K., Little, E., Bendixen, A.V., Hooper, M., Kidder, D.P., Wolintski, R.J., Harre, D., Royal, S., Aidala, A., "Cost and Threshold Analysis of Housing as and HIV Prevention Intervention" (2007) AIDS And Behavior.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The HOPWA program's performance compares favorably to other programs with similar purposes and goals. The program is both effective and efficient in serving its special needs population by: (1) targeting housing resources to a population with the greatest need for financial assistance and high risks of negative health outcomes; (2) providing supportive housing in an effective, comparatively lower-cost intervention than Medicaid for persons with housing and medical needs, including mental health and substance abuse; and (3) requiring formula grantees to participate in communities' Consolidated Planning processes, ensuring that HOPWA is used as part of a continuum to serve the needs of persons living with HIV/AIDS. HOPWA also provides a range of housing options to its beneficiaries, enabling clients to receive assistance that is most appropriate for their housing needs, including short-term, transitional and permanent support. The program makes extensive use of the existing private market housing stock (TBRA and STRMU, which comprise 86 percent of the program), enabling beneficiaries to receive housing support within a more immediate timeline. Also, HOPWA's use of TBRA and STRMU makes the HOPWA program cost-effective, as compared to facility-based housing programs such as Section 811 for persons with disabilities and Section 202 for elderly persons. HOPWA has also been shown to be cost-effective compared to the Housing Choice Voucher program, when considering the two programs' costs for tenant-based rental assistance. HOPWA's purpose and its population are comparable to those served under the Ryan White HIV/AIDS program at HHS and the Homeless Assistance Grants at HUD. HOPWA achieves all of its performance goals in serving this population, and the Ryan White HIV/AIDS program and Homeless Assistance Grants do as well.

Evidence: Homeless persons and those with housing instability incur high health care costs, often billed to Medicaid. Overall cost-efficiency can be achieved by providing supportive housing, such as HOPWA, according to "Chronic Homelessness and High Users of Health Services: Report from a Meeting to Explore a Strategy for Reducing Medicaid Spending While Improving Care," by the National Academy for State Health Policy, available online at http://www.nashp.org/Files/chronic_homelessness.pdf. HOPWA Consolidated Plan requirements are included in regulations 24 CFR §574.110, available online at http://www.access.gpo.gov/nara/cfr/waisidx_05/24cfr574_05.html. Section 811 and 202 both rely on the development of facilities for housing and service delivery purposes. Development programs, such as Section 811 and Section 202, were rated to be more costly than voucher programs (such as HOPWA's TBRA and STRMU forms of assistance) in GAO's 2002 report, "Federal Housing Assistance: Comparing Characteristics and Costs of Housing Programs." See page 16, available online at http://www.gao.gov/new.items/d0276.pdf. HOPWA's cost effectiveness measure and actual performance on this measure are detailed in questions 3.4 and 4.3. HOPWA, Ryan White and Homeless Assistance Grants' performance actuals are available online at OMB's Expect More website, under the performance measures section of each of the program's pages.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Though the body of knowledge on the relationship between housing and HIV care has been significantly advanced in recent years, two key studies that will demonstrate the impact of HOPWA-funded interventions have not been published as of May 2008. Initial publications for these studies have described study objectives and methodologies, and both utilize randomized control trials. Complete study findings are expected to be published in late 2008 and early 2009. However, other research indicates that the HOPWA program is operating well. In 2000, HUD's Chief Financial Office conducted a risk management assessment and found that HOPWA management controls were significantly in place to mitigate against waste, fraud, abuse, and mismanagement. Also in 2000, a HUD Policy Development and Research evaluation, conducted by ICF Consulting, resulted in a series of findings that the HOPWA program is reaching its intended results, benefiting clients with housing assistance, and leveraging other resources in coordination with AIDS care. HOPWA grantees also conducted studies of their programs and the effectiveness of housing support, often through third party researchers. One study, conducted in 2007 by Rainbow Research, Inc., supported the conclusion that housing stability improves health and increases quality of life.

Evidence: See the November 2007 supplement to AIDS and Behavior, which contains 18 articles on the connection between housing and HIV/AIDS, including articles on the HUD-CDC Housing and Health Study. To purchase a journal, go to http://www.nationalaidshousing.org/PurchaaseAIDSandBehavior.htm. For additional information on the Housing and Health and CHHP studies, see NAHC's Research Summit site at http://www.nationalaidshousing.org/Housingand HIV-AIDSResearchSummit.htm. For earlier research on HOPWA program design, results and management, see the 2000 HUD PD&R study "National Evaluation of the Housing Opportunities for Persons With AIDS Program," online at http://www.huduser.org/publications/hsgspec/hopwa_0101.html, and the HUD Chief Financial Office Risk Management Review (2000).