Detailed Information on the
Developmental Disabilities Grant Programs Assessment

Strengthening performance measures and monitoring results and progress made toward the program goals.

Implementing an efficiency measure to demonstrate how well the program is performing.

Completing research design for an independent evaluation of the impact of program activities and conducting the research.

Completing research design for an independent evaluation of the impact of program activities and conducting the research. Milestone: Develop draft evaluation indicators.

Strengthening performance measures and monitoring results and progress made toward the program goals. Milestone: Revise guidelines for reporting on program measures to ensure national uniformity to reporting standards.

Implementing an efficiency measure to demonstrate how well the program is performing. Milestone: Report FY 2006 result.

Measure: By the end of FY 2007, the percentage of individuals with developmental disabilities who are independent, self-sufficient and integrated into the community, as a result of State Council efforts, will increase to 14 percent. (SCDD)

Explanation:

Measure: By the end of FY 2007, the percentage of trained individuals who are actively working to improve access of individuals with developmental disabilities to services and supports will increase to 94 percent.

Explanation:

Measure: By the end of FY 2007, percentage of individuals who have their complaint of abuse, neglect, discrimination or other human or civil rights corrected will increase from 87% to 93%. (P&As)

Explanation:

Measure: Percentage of individuals who have their complaint of abuse, neglect, discrimination or other human or civil rights corrected compared to total assisted. (P&A)

Explanation:2002 baseline of 87%

Measure: Increase the number of individuals with developmental disabilities reached by the Councils who are independent, self-sufficient and integrated into the community, per $1,000 of federal funding to the Councils. (Measure added, February 2007)

Explanation:The State Councils on Developmental Disabilities program is a force within state governments for systems change and capacity building, as well as providing training to individuals with developmental disabilities and their family members to prepare them to participate in the process of policy making, since they often have a deeper appreciation of their own needs than do even professionals in the field. This efficiency measure reflects performance data reported to ADD on existing annual reports from the States. Increased data quality efforts have made the FY 2005 actual result lower but also more accurate than the FY 2004 actual result. The ongoing data quality improvements are expected to be fully in effect for the FY 2009 actual result.

Is the program purpose clear?

Explanation: The Administration of Developmental Disabilities' (ADD) purpose is ''to assure that individuals with developmental disabilities and their families participate in the design of and have access to needed community services, individualized supports, and other forms of assistance that promote self-determination, independence, productivity, and integration and inclusion in all facets of community life, through culturally competent programs.'' Included in this review are three grant pograms: (1) State Councils on Developmental Disabilities (SCDDs) to help communities create systems of supports and services for individuals with developmental disabilities; (2) Protection and Advocacy (P&A) systems to protect individuals with developmental disabilities from abuse, neglect, and violation of rights; and (3) University Centers for Excellence in Developmental Disabilities Education, Research, and Service (UCEDDs) to provide education, training, technical assistance, public information, and research.

Evidence: The Developmental Disabilities Assistance and Bill of Rights Act of 2000 (the Act), Sec. 101(b).

Does the program address a specific and existing problem, interest, or need?

Explanation: ADD's grantees address the problems of approximately four million individuals with developmental disabilities in the United States, many of whom need long-term if not lifetime services and supports to successfully and safely live in their communities. Grantees assist states and local communities in examining service systems, pursuing individual and systemic advocacy efforts, and coordinating the resources of universities to enhance community living for individuals with developmental disabilities in such areas as education, employment, housing, and health care.

Evidence: Almost every State has lists of eligible individuals waiting for supports to remain in or return to their communities. ADD tracks measures directly or indirectly related to assisting individuals with developmental disabilities access services and opportunities in community settings. Through its grantees and national data surveys ADD has learned: in 2002, there were 254,762 individuals with developmental disabilities on various waiting lists for housing or other community-based services (SCDD Program Performance Report (PPR)); in 2000, 672,994 adults with developmental disabilities had parents 60 years or older as their primary caregivers (Braddock, David, editor Disability at the Dawn of the 21st Century and the State of the States, 2002); 88 percent of individuals with developmental disabilities live with their parents or in their own households.

Is the program designed so that it is not redundant or duplicative of any Federal, state, local or private effort?

Explanation: Although there are other entities that serve individuals with developmental disabilities, ADD is the only Federal effort that systematically assesses the state of services to individuals with developmental disabilities. All grantee applications require strategic plans. For example, SCDDs are required to submit State Plans that include a comprehensive review and analysis of availability of services, identifying unmet needs and opportunities for collaboration with State, local and private entities. Grantees provide technical assistance (TA) and direct support, and collaborate to expand, create and improve services. ADD and its grantees strive to provide services that are not redundant or duplicative, but rather fill service gaps. Federal monitoring and legal oversight provided by ADD ensures program accountability and implementation of the Act.

Evidence: The Act specifies planning and reporting requirements for SCDDs (Sec 124 (c)(3)); P&As (Sec. 143); and UCEDDs (Sec. 153).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: ADD's grantees have flexibility in the programs they administer to ensure that programs remain responsive to needs of consumers in a particular State. SCDDs, P&As, and UCEDDs all have advisory or governing boards comprised mostly of consumers who identify, on an on-going basis, needs and problems and resolve them in an efficient and effective manner. This process includes verification through collection of consumer satisfaction surveys and goal assessments. ADD monitors grantees on an on-going basis, and when problems are identified they are resolved through corrective action plans and TA. The Act also includes a provision to protect funds from supplantation or substitution.

Evidence: In 2002 approximately 3,235 individuals served on governing and advisory bodies to components of the DD program. Of these 1,736 were individuals who had disabilities. The Act includes requirements for governing bodies, the assessment of goals and objectives, and that funds be used to supplement, not supplant non-Federal funds (Sections 124, 125, 144, 154).

Is the program effectively targeted, so program resources reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The resources and activities of ADD's programs are targeted toward individuals with substantial life-long disabilities that originated before they reached the age of 22. ADD's grantees identify service gaps and address systemic issues that impact individuals with substantial disabilities. Although not measured, it is possible that sponsored activities could benefit individuals with substantial disabilities who are not considered to be individuals with developmental disabilities. Grantees are required to report annually on the characteristics of the individuals they serve and the activities provided.

Evidence: 'Developmental disability' is defined in Sec. 102(8) of the Act. References for composition of SCDDs (Sec. 125(b)), P&A boards and advisory councils (Sec. 144), and UCEDDs advisory councils (Sec. 154(a)(3)(E)). ADD grantee Program Performance Reports.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: ADD has meaningful long-term, outcome-based goals for FY 2003 through FY 2007. Since 1998 ADD has tracked performance measures under GPRA reporting requirements; however, some measures were based on outputs instead of outcomes. The new measures directly reflect the program's purpose that individuals with developmental disabilities and family members: (1) have access to community services, individualized supports, and other forms of assistance that promote self-determination, independence, productivity, independence, and inclusion in all facets of community life, and (2) participate in the design of services.

Evidence: Administration on Children and Families FY 2004 Performance Plan. Revised measures will be reflected in the FY 2005 Performance Plan.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: ADD and its grantees developed ambitious and achievable targets that impact an increased number of individuals with developmental disabilities relative to the national population (ambitious), while being based on data-driven strategic planning (achievable). The timeframes coincide with the next scheduled reauthorization of the legislation.

Evidence: Administration on Children and Families FY 2004 Performance Plan. Targets will be reflected in the FY 2005 Performance Plan.

Does the program have a limited number of specific annual performance measures that demonstrate progress toward achieving the program's long-term measures?

Explanation: ADD developed seven annual performance measures to support its three long-term goals. ADD is working to develop a meaningful efficiency measure.

Evidence: Administration on Children and Families FY 2004 Performance Plan. Revised measures will be reflected in the FY 2005 Performance Plan.

Does the program have baselines and ambitious targets and timeframes for its annual measures?

Explanation: Baselines are generated from the grantees annual performance reports. Annual measures are consistent with targets for the long-term measures and are developed through the same strategic planning process.

Evidence: Administration on Children and Families FY 2004 Performance Plan. Revised measures will be reflected in the FY 2005 Performance Plan.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, etc.) commit to and work toward the annual and/or long-term goals of the program?

Explanation: ADD developed long-term and annual performance measures. ADD's grantees commit to the performance goals and measures through required planning documents and annual reporting of progress on the performance measures. The planning documents are monitored to ensure the commitment of grantees and subgrantees to ADD's goals. Grantees have the flexibility to select state-specific goals in any area of emphasis (e.g., employment, housing) to support ADD's long-term goals.

Evidence: The Act outlines specific measures as indicators of progress (Sec. 104(3)(D)(ii)(I-III)), which are required to be addressed in grantee plans (SCDD - Sec. 124(c)(4)(B)(I); P&As - Sec. 143(a)(2)(C); and UCEDDs - Sec. 153(a)(1)). Grantee Program Performance Reports.

Are independent and quality evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Although ADD regularly reviews its grantees through audits and its Monitoring and Technical Assistance Review System, it does not undertake a regular independent evaluation of program effectiveness. ADD intends to conduct a design study of an independent evaluation in FY 2004 and to begin the independent evaluation of the three grant programs in FY 2005.

Evidence: Monitoring and Technical Assistance Review System Guidelines

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: SCDDs and P&As are formula grant programs determined on the basis of State population, per capita income, and estimates of individuals in the State with developmental disabilities. While ADD reviews data provided by grantees regularly, allocation of resources to particular priorities, based on statute, is influenced by individuals with developmental disabilities, families of and advocates for individuals with developmental disabilities. ADD sponsors forums and meeting to emphasize key areas of need and to recognize best practice, that often reinforce State-based priorities. Grantees participated in the drafting of ADD performance measures and are committed to long-term tracking of them.

Evidence: Administration on Children and Families' Annual Performance Plan and congressional justification.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: ADD developed long-term and annual performance measures and targets. The Roadmap to the Future, ADD's strategic plan, is a document that is revised periodically in response to recommendations by programs and self-advocates, and authorizing legislation. Focus groups reviewed the strategic plan and reporting documents in response to the Act of 2000 and made recommendations to correct deficiencies. These recommendations have been implemented.

Evidence: FY 2005 Performance Plan Roadmap to the Future Roadmap to the Future Update

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: SCDDs and P&As report data and performance electronically on an annual basis, while the UCEDDs submit their annual reports in paper copies. ADD also has an agreement to access annual data from the Association of University Centers on Disability's (AUCD) National Information Reporting System (NIRS), which includes data sets on trainees, projects, activities/impact and products. The data collected from these sources was being used by ADD to develop their performance measures baselines. Data are reviewed/approved and used for on-site monitoring by ADD staff. ADD's goal is to conduct on-site monitoring of 25-30 percent of States' grantees annually. ADD staff report their finding back to the State along with recommendations for management improvements. Electronic Data Systems (EDS) data reviews contribute to the determination of ADD's annual goals achievement. Program improvements are made through corrective action plans.

Evidence: ADD information gathered through monitoring is used to assist in the determination of non-compliance with the Act and in the provision of technical assistance. Grantee Annual Performance Reports.

Are Federal managers and program partners (grantees, subgrantees, contractors, cost-sharing partners, etc.) held accountable for cost, schedule and performance results?

Explanation: The Secretary and ADD require fiscal and program accountability to ensure adherence to legislative intent. Program staff are responsible for monitoring the programs and assisting in the development and application of technical assistance. Federal accountability is also reflected in the Senior Manager's Performance Contract with the Assistant Secretary, and all staff performance plans, which are linked to the senior manager's performance. ADD will not release funds until the grantee submits an acceptable plan. Noncompliant grantees may be subjected to a designation of high risk status. Generally, when deficiencies are identified, corrective action plans are required and monitored. There are no monetary incentives built into the Act for superior performance.

Evidence: ADD manager performance contracts. Grantee Program Performance Reports (PPR).

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: ADD obligates funds to grantees in a timely manner, and limited funds remain unobligated at the end of the year. SCDDs, P&As and UCEDDs submit annual financial reports (SF-269s) and ADD conducts periodic on-site monitoring to ensure the funds are spent on their intended purposes. Program audits are performed by independent auditors in accordance with OMB Circular A-133 and other relevant OMB Circulars, and through PPRs by ADD staff.

Evidence: Financial management requirements. SF-269. Single State Audits.

Does the program have procedures (e.g., competitive sourcing/cost comparisons, IT improvements, approporaite incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: While the program has implemented procedures to improve efficiency, there are no procedures in place by which to measure such efficiencies. For example, ADD is implementing an EDS to enable staff to collect, analyze and report data more efficiently. Over the past year, ADD experienced a reduction in staff and managed a large increase in grant activity. ADD is developing a meaningful efficiency measure.

Evidence: ACF Extranet Outsourcing Contract

Does the program collaborate and coordinate effectively with related programs?

Explanation: ADD collaborates with other Federal agencies to ensure that (1) legal and advocacy services are coordinated and available to individuals with developmental disabilities; (2) related programs are jointly monitored; (3) technical assistance activities of related programs are coordinated to prevent overlap. ADD contributes to the coordination of programs and services to people with developmental disabilities through Federal councils and committees. SCDDs, P&As and UCEDDs are required to participate on the boards of their sister organizations, as well as collaborate with numerous state agencies, councils, and committees.

Evidence: Interagency Agreements with the Rehabilitation Services Administration and the Substance Abuse and Mental Health Services Administration for the provision of legal advocacy services. The Federal partners meet monthly with the Federal contractor to plan, develop and monitor the training and technical assistance activities provided, make joint decisions and evaluate the progress and outcomes of grantees.

Does the program use strong financial management practices?

Explanation: The SF-269s are used to determine financial compliance with law and regulations. Staff review SF-269s to ensure expenditures and obligations are for authorized purposes.

Evidence: SF-269. Single State Audits. FIMA report

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Annually ADD reviews of the Monitoring and Technical Assistance Review System (MTARS) Manual and monitoring procedures and makes revisions needed to improve the monitoring process and the provision of TA. ADD reviews all monitoring corrective action plans for concurrence with regulations. In 2001-2002, ADD conducted 21 monitoring visits resulting in corrective actions being implemented by grantees and two grantees being designated as 'high risk'. A 'high risk' designation confirms that substantial non-compliance issues have been cited with the potential for monetary restrictions until the grantee has corrected deficiencies. In cases involving corrective actions, ADD staff monitor compliance and engage in extensive technical assistance, and track action in corrective action plans.

Evidence: MTARS Manual and Monitoring Guide. On-site MTARS Reports.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Grantees are required to submit program and financial reports annually, which describe goals and objectives undertaken and their outcomes. Monitoring of programs is conducted by ADD staff with the assistance of regional, consumer and peer reviewers. The monitoring process includes review of program documents, on-site reviews which include consultations with grantee staff and sub-grantees and contractors of grantees, and live town meetings to gather input from individuals with developmental disabilities and family members. This information is compiled in a final report of compliance, non-compliance and recommendations for grantee program and improvements. Grantees are monitored on a rotating basis, with 25-30% of grantees monitored each year. The quality of programs' data are not currently assessed; however, in FY2004, ADD intends to design an independent evaluation of the three programs.

Evidence: SF-269. Annual Program Performance Reports. Review of independent audits. On-site MTARS Reports.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Program Performance Reports, State Plans (SCDDs and P&As) and Statements of Goals and Priorities are received via the EDS and made available to the developmental disabilities network. Information is also made available to the public through the Bi-Annual Report to President, Congress, and National Council on Disability; presentations at national meetings; progress reports on the President's New Freedom Initiative; and ADD's website. UCEDD data is collected by the TA contractor and put into NIRS. Grantee specific information is made public via the State agencies.

Evidence: Information is received and provided via EDS data sheets. ADD Web site (www.acf.dhhs.gov/programs/add/index.htm). Numerous publications produced by program components funded by ADD such as National Association of Councils on Developmental Disabilities, National Association of Protection and Advocacy Systems, the Association of University Centers on Disabilities.

Has the program demonstrated adequate progress in achieving its long-term outcome performance goals?

Explanation: ADD is in the process of implementing new performance measures. While data indicates that many of the prior goals were met some of the targets were not ambitious.

Evidence: ADD's Annual Reports to Congress and Reports from the Councils and P&As on achieving long-term and annual goals.

Does the program (including program partners) achieve its annual performance goals?

Explanation: ADD is in the process of implementing new performance measures. In the past, ADD indicated that some if its annual performance goals were met (e.g., exceeded the prior employment goal) while others were not (e.g., did not meet housing goal). Some of the targets were not ambitious.

Evidence: ADD's Annual Report to Congress; Performance Reports from the Council and P&A grantees; and?? UCEDD National Information and Reporting System.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program performance goals each year?

Explanation: While ADD has taken steps to improve efficiency, it cannot measure these efficiencies against established targets and baselines. Efforts such as implementing the EDS paperless reporting system have enabled ADD to manage increasing grant workloads while experiencing reductions in staff. ADD is working to develop a meaningful efficiency measure.

Evidence: ACF Extranet Outsourcing Contract

Does the performance of this program compare favorably to other programs, including government, private, etc., that have similar purpose and goals?

Explanation: Although there are other programs that serve individuals with disabilities, including individuals with developmental disabilities, none are similar to ADD's role. ADD and its grantees provide technical assistance and collaborate with other Federal, State, and private entities with direct services responsibilities or interests. The work of ADD and its programs helps to ensure the effectiveness and responsiveness of other Federal, State, and local programs affecting the lives of individuals with developmental disabilities and their families.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: A comprehensive, independent evaluation of ADD programs and grantees has not been conducted to date. In Fiscal Year 2004, ADD will issue a request for proposals to explore the feasibility and design of a national level program evaluation for ADD's grant programs. ADD will also ensure that the findings and recommendations from the FY2004 feasibility study and the resulting national evaluation of the three programs in future years are available to the public.

Evidence: