Detailed Information on the
Head Start Assessment

Working to utilize child outcomes information to help local grantees improve children's school readiness.

Continuing to work with states to promote a coordinated early childhood education delivery system. Milestone: Convene a 2-day meeting of all Head Start Collaboration Office Directors, State Child Care Administrators, State Head Start Association representatives and State Pre-K Administrators to develop strategies for Head Start/child care/pre-k collaboration at the state and local levels.

Developing performance measures to: assess social/emotional outcomes of children in Head Start; assess positive child development outcomes of children in Early Head Start; and capture reductions in repeated financial management and service provision deficiencies among Head Start grantees.

Continuing to work with States to promote a coordinated early childhood education delivery system. Milestone: Convene 3 day meeting of all Head Start State Collaboration Offices and their state-level partners to provide information and guidance that promotes coordination of Head Start with statewide early childhood education delivery systems.

Developing performance measures to: assess social/emotional outcomes of children in Head Start; assess positive child development outcomes of children in Early Head Start; and capture reductions in repeated financial management and service provision deficiencies among Head Start grantees. Milestone: Complete first year of data collection on the National reporting System (NRS) Social Emotional measure.

Implementing and operating a National Reporting System designed to assess every 4-year-old in Head Start in the fall and spring of their preschool year. Milestone: Complete and submit Spring 2007 National Reporting System (NRS) data for analysis.

Measure: Increase the percentage of programs that achieve average fall-spring gains of at least 12 months in word knowledge (PPVT) in the National Reporting System.

Explanation:Children's word knowledge (i.e., vocabulary) upon entry to kindergarten is correlated with improved academic performance in kindergarten (FACES 2000 Fourth Progress Report). This skill is a precursor to learning to read and write; thus, this indicator provides a direct and useful measure of how well Head Start is succeeding in attaining its mission of promoting school readiness among low-income children. Furthermore, this measure provides information on how well individual Head Start programs are doing in meeting this goal. An analysis of NRS data from the Fall 2003 data collection suggests that Head Start children begin the program nearly 12 months behind the average child (HS children were 55 months of age on average when entering the program, and had PPVT scores consistent with the average 43.5-month-old). The proposed threshold gain of 12 months in a 6-month period (the time differential between the fall and spring assessments) would reduce the gap between HS children and non-HS children by 50% (6 months) over the course of the Head Start year. The 12 month threshold gain requires children to advance at twice the rate that would result from maturation. Preliminary analysis of NRS and FACES data indicates that there is a significant positive, moderate-to-strong relationship (p<0.0001) between the NRS PPVT task and Kindergarten Vocabulary and General Academic Knowledge. The predictive validity of scores on the PPVT task indicates that a higher score would correspond to later vocabulary and academic knowledge. Currently (as of Spring 2005), 51.8% of HS programs are achieving an average fall-spring gain of 12 months, and 80.8% of programs are achieving an average fall-spring gain of 9 months. Progress on this measure over time will also help to show how well Head Start is targeting training and technical assistance to improve program performance in this area. (This measure does not penalize already high performing grantees as long as they continue to achieve gains over time.) Data for this measure will come from the National Reporting System (NRS), Head Start's annual assessment of all kindergarten eligible four-year olds on literacy, math, and language skills (see Report to Congress and Question 2.6 for more information on the NRS). (It should be noted that the NRS was not operational as of the last PART evaluation.)

Measure: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring by the National Reporting System.

Explanation:Children's letter identification skills upon entry to kindergarten have been shown to be correlated with improved academic performance in kindergarten (FACES 2000 Fourth Progress Report). This skill is a precursor to learning to read and write; thus, this indicator provides a direct and useful measure of how well Head Start is succeeding in attaining its mission of promoting school readiness among low-income children. Furthermore, this measure provides information on how well individual Head Start programs are doing in meeting this goal. Congress established a requirement in the HS Act that all Head Start children know ten or more letters at the time they complete Head Start. Preliminary analysis of NRS data indicates that there is a significant positive, moderate-to-strong relationship (p<0.0001) between the NRS Letter Naming task and Kindergarten Basic Reading Skills and General Academic Knowledge. Children come into HS knowing an average of 6 letters. Progress in this measure over time will also help to show how well Head Start is targeting training and technical assistance to improve program performance in this area. (This measure does not penalize already high performing grantees as long as they continue to achieve gains over time.) Data for this measure will come from the National Reporting System (NRS), Head Start's annual assessment of all kindergarten eligible four-year olds on literacy, math, and language skills (see Report to Congress and Question 2.6 for more information on the NRS). (It should be noted that the NRS was not operational as of the last PART evaluation.)

Measure: Increase the percentage of programs that achieve average fall-spring gains of at least 4 counting items in the National Reporting System.

Explanation:Children's early numeracy skills (i.e., counting) upon entry to kindergarten has been shown to be correlated with improved academic performance in kindergarten (FACES 2000 Fourth Progress Report). This skill is a precursor to learning to do mathematics; thus, this indicator provides a direct and useful measure of how well Head Start is succeeding in attaining its mission of promoting school readiness among low-income children. Furthermore, this measure provides information on how well individual Head Start programs are doing in meeting this goal. The threshold gain is set at 4 counting items. Children enter HS, on average, being able to count to 10. A target of 4 counting items means they would be able to count to 14 upon leaving HS. Preliminary analysis of NRS data indicates that there is a significant positive relationship between the NRS Math Reasoning task and Kindergarten Math Reasoning and General Academic Knowledge. There is no way to quantify the exact significance of scoring an additional 4 counting items, but the predictive validity of scores on the Early Math Skills task indicates that a higher score would correspond to later math reasoning and academic knowledge. Progress in this measure over time will also help to show how well Head Start is targeting training and technical assistance to improve program performance in this area. (This measure does not penalize already high performing grantees as long as they continue to achieve gains over time.) Data for this measure will come from the National Reporting System (NRS), Head Start's annual assessment of all kindergarten eligible four-year olds on literacy, math, and language skills (see Report to Congress and Question 2.6 for more information on the NRS). (It should be noted that the NRS was not operational as of the last PART evaluation.)

Measure: Increase percentage of parents of children in their pre-kindergarten Head Start year who report reading to child three times per week or more, as measured in the Spring of their pre-kindergarten Head Start year

Explanation:Head Start FACES Study has demonstrated a link between frequency of parental reading and children's level and gain in early literacy activities, and corresponding school readiness. This long-term measure requires increases in the percentage of parents reading to their children over time, thus providing ongoing incentives to programs to improve their performance in this area. Setting a program goal of supporting parent reading helps to take literacy activities from the classroom into the home learning environment as well as emphasizes the primary role of parents in children's learning. The baseline for this measure (74.0% of parents) was determined based upon data from the FACES cohort in Spring 2004. Nationally, 85% of parents from all income levels reporting reading to their child (age 1 to 5) three or more times a week (Urban Institute, Survey of America's Families). The targets propose to completely close the gap between how frequently Head Start parents read to their children compared to non-Head Start parents by 2013. In order to meet this target, Head Start is working with the parents to improve their reading skills at the same time that they are educating parents about the importance of reading to their children. Data from the FACES will be used to determine the extent to which this long-term performance measure is being met.

Measure: Increase the percentage of program in which children make prescribed gains on a measure of social skills between the Fall and Spring of their pre-kindergarten Head Start year, as measured in the National Reporting System.

Explanation:This indicator measures the gain in social skills. A number of social skills are essential for school success, including, among others, paying attention, communication skills, and cooperation skills such as following directions. The NRS behavior ratings are indicators of school adjustment and social competence and have been shown to be predictive of kindergarten behaviors that promote learning and those that impede learning. When available, data from the NRS will be used to determine whether this long-term performance measure is being met.

Measure: Increase the percentage of Early Head Start children completing all medical screenings. [OMB proposes to keep this measure for now and include discussion of a developmental measure focused on child outcomes in the improvement plans section.]

Explanation:Early childhood screenings are vitally important for ensuring that children develop and are able to learn to their maximum potential. Medical screening during a child's first three years of life enables the diagnosis and treatment of such conditions as failure to thrive, anemia, lead poisoning, and hearing and vision impairment as well as developmental delays. Early identification increases the likelihood of successful treatment. For example, children with undiagnosed hearing impairments may have lifelong speech and language problems that could have been eliminated or significantly reduced with early treatment. Low income children are at greater risk because they are less likely than middle and upper class children to receive medical screenings and related referrals when necessary. Access to medical providers, lack of health insurance, transportation barriers and lack of knowledge about the importance of medical screenings are among the reasons for this disparity. Head Start requires grantees to obtain from a health care professional that a child is up-to-date for age appropriate preventive and primary health care screenings within 90 days of a child's entry to the program. The requirements are specified through the Early and Periodic Screening, Diagnosis, and Treatment Program of the Medicaid agency of the State in which they operate. In 2005, 82% of Early Head Start children received the required screenings within the allowable 90 days. Data for this measure is collected through the annual Program Information Report that all grantees complete each year. In 2005 (baseline), 82% of Early Head Start children received all required screenings within the 90 days required by HS regulations. The 2007 target has been set at 85%; the 2010 target is 91%--these long-term targets represent a 2% improvement each year in the percent of children completing all medical screenings??more than twice the rate of improvement between 2004 and 2005. These targets are ambitious given trend data, cultural or family preferences, and lack of family health insurance, transportation, and health care providers in some areas.

Measure: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring by the National Reporting System.

Explanation:Children's letter identification skills upon entry to kindergarten have been shown to be correlated with improved academic performance in kindergarten (FACES 2000 Fourth Progress Report). This skill is a precursor to learning to read and write; thus, this indicator provides a direct and useful measure of how well Head Start is succeeding in attaining its mission of promoting school readiness among low-income children. Furthermore, this measure provides information on how well individual Head Start programs are doing in meeting this goal. Congress established a requirement in the HS Act that all Head Start children know ten or more letters at the time they complete Head Start. Preliminary analysis of NRS data indicates that there is a significant positive, moderate-to-strong relationship (p<0.0001) between the NRS Letter Naming task and Kindergarten Basic Reading Skills and General Academic Knowledge. Children come into HS knowing an average of 6 letters. Progress in this measure over time will also help to show how well Head Start is targeting training and technical assistance to improve program performance in this area. (This measure does not penalize already high performing grantees as long as they continue to achieve gains over time.) Data for this measure will come from the National Reporting System (NRS), Head Start's annual assessment of all kindergarten eligible four-year olds on literacy, math, and language skills (see Report to Congress and Question 2.6 for more information on the NRS). (It should be noted that the NRS was not operational as of the last PART evaluation.)

Measure: Increase the percentage of teachers with AA, BA, Advanced Degree, or a degree in a field related to early childhood education

Explanation:Teacher credentials are tied to classroom quality. FACES findings indicate that teachers with higher levels of education tend to be in classrooms rated higher on a number of quality indicators, including the ECERS-R Language subscale, the Arnett Caregiver Interaction Scale, and the ECERS-R total score. Teachers with higher levels of education also tend to have more positive attitudes and knowledge of early childhood education practices, which in turn are associated with higher levels of classroom quality. In 1998, Congress mandated that by September 30, 2003, at least 50% of Head Start teachers in center-based programs must have an Associate, Bachelor, or Advanced Degree in early childhood education, or a degree in a field related to early childhood education. (See Head Start Act, Section 648A(a)(2)). Head Start has shown a steady increase in the number of teachers with degrees in early childhood education and met the 2003 goal required by the Head Start Act. (The current Head Start reauthorization, however, may further raise the bar on this requirement.) For FY 2004, 65 percent of Head Start's teachers had an AA degree or higher. Of the 56,208 teachers, 31 percent had an AA degree, 30 percent had a BA degree, and 4 percent had a graduate degree. The total FY 2004 figure represents an increase of 6,071 degreed teachers over the previous year (7.3 percentage points). The target 3% growth set for 2007 and 2008 recognizes the difficulty recruiting and retaining credentialed teachers. HS salaries are approximately 50% lower than salaries for elementary school teachers. Turnover for HS teachers is high (15% in FY 2004) and requires ongoing intensive professional development, including assisting staff without degrees to pursue academic training, and recruiting efforts for HS. The targets also account for the methods traditionally used by Head Start programs in their professional development. In particular, Head Start programs often recruit parent volunteers and develop them into teachers - in fact, many effective Head Start teachers began as Head Start parents. Since many Head Start parents have low education levels, it often takes time for them to move through the higher education system and attain the credentials. Note: the large target increase from 2006 to 2007 is due to the fact that the 2006 target was exceeded in 2005, with 69% of teachers possessing the specified credentials. Because the 2006 target has already been set (per the FY 2007 Congressional Justification), the 2007 target was revised to account for the 2005 actual. The Office of Head Start tracks progress in meeting this Congressional mandate through its Program Information Report (PIR), which all Head Start grantees complete annually. For detail on the data collection methods and status the PIR, please see 3.1.

Measure: (Developmental) Reduce the percent of grantees with repeat financial deficiencies through the provision of targeted technical assistance.

Explanation:This indicator measures the extent to which targeted technical assistance helps grantees make the systemic changes they need to prevent being cited for repeat financial deficiencies during their next onsite monitoring review. The GAO report, "Head Start: Comprehensive Approach to Identifying and Preventing Grantee Financial Management Weaknesses," found that 53% of grantees with financial management findings were again cited in the grantee's next review. TA providers work with grantees to correct deficiencies and to ensure that all management, financial, reporting, and programming systems comply with all applicable Federal regulations.

Measure: (Developmental) Decrease the number of grantees with deficiencies in early childhood development.

Explanation:The Head Start education and early childhood development performance standards require that grantees provide for the development of each child's cognitive and language skills including supporting emerging literacy and numeracy development (Section 1304.21(a)(4(IV)). Additionally the Standards require that the child development and education approach provide for the development of cognitive skills by encouraging each child to organize his or her experiences, to understand concepts, and to develop age appropriate literacy, numeracy, reasoning, problem solving and decision-making skills which for a foundation for school readiness and later school success (1304. 21(c)(ii). Grantees are also required to conduct ongoing assessment of each enrolled child. Compliance with these requirements is examined during triennial onsite reviews. Identifying and correcting deficiencies in early childhood development, including grantee curriculum should result in improved child outcomes.

Measure: Decrease under enrollment in Head Start programs, thereby increasing the number of children served per dollar

Explanation:Head Start must assure that grantees are serving the full number of children that they are approved to serve. By assisting grantees in moving to full enrollment or deobligating funds previously awarded which have not been used because of under-enrollment the program will decrease the national total of under enrolled children, and ensure the most efficient use of Federal funds by lowering the average cost per child. The national under enrollment rate is an indicator of how efficient the program is in maximizing the use of its capacity to deliver services to children. An un-enrolled space or "vacancy" in Head Start is defined as a funded space that is vacant for over 30 days. Using this definition, a vacancy of 31 days is counted the same as a vacancy of 250 days. (This is important to understand in order not to overstate underenrollment .) Head Start programs are required to maintain a waiting list to ensure that vacant positions can be rapidly filled. However, there are a number of reasons that this may not occur, or may not be implemented effectively. Regardless of the possible reasons, the Office of Head Start has consistently maintained that Head Start programs are required and expected fill each funded space. Technical assistance is available to programs to help develop strategies for reducing chronic under enrollment problems. More serious consequences, such as deobligation of funds previously awarded which have not been used because of underenrollment and the reduction of prospective awards based upon a grantees' inability to achieve full enrollment or termination, for chronic under enrollment problems. This data is collected through the annual Program Information Report that all grantees complete annually. The baseline was established at 4.4% in 2004. Reduction in under enrollment ensures that funding is being used for intended purposes and maximizes the number of children (and days of service for those children) and thus continues to be a priority for the OHS. Targets were chosen based upon a 10% improvement rate each year (4%, 3.6%, 2.8%, and 2.7% for 2005-2008). A reduction of 10% corresponds to approximately 4000 vacancies a year in 2005. The targets set are ambitious and require continuous improvement, but they recognize that HS uses a stringent definition of under enrollment. Programs are unlikely to be able to dramatically reduce many short-term vacancies due to the work required to contact families and secure all needed documentation, as well as for the family to make needed arrangement for beginning a new child care situation. Note: The FY 2006 target was set in 2004, prior to the availability of FY 2005 actual data, and cannot be changed. It does not reflect an expectation that under-enrollment will increase in FY 2006.

Is the program purpose clear?

Explanation: Head Start's purpose, although multifaceted, is clear and is reinforced through various program mechanisms. As stated in Head Start's authorizing legislation, the Head Start program is designed to?? "promote school readiness by enhancing the social and cognitive development of low-income children through the provision, to low income children and their families, of health, educational, nutritional, social and other services that are determined, based on family needs assessments, to be necessary." (Sec. 636, 42 USC 9831 et seq.) To help ensure that children are ready and able to learn, Head Start aims to provide children, from birth to age five, with individualized, comprehensive child development services, including activities to promote early literacy and numeracy skills as well as healthy socio-emotional development. Health services, including medical, dental, and mental health, disability services, nutritious meals, transportation, and other support services are also provided. Head Start works with parents to ensure that the family has access to the support services it needs. While Head Start provides services to preschool-aged children and their families, Early Head Start serves pregnant women, and infants and toddlers and their families. Ninety percent of Head Start children must be from families who live at or below the Federal Poverty Line ($20,000 for a family of four in 2006) or receive public assistance through the Temporary Assistance to Needy Families program (TANF). At least 10% of available spaces must be reserved for children requiring disability services. Head Start services come in various forms. Some children come to a Head Start center daily; others participate in home-based services, and others participate in both. To ensure that all programs are providing the highest quality services to children and their families in line with the purpose of Head Start as defined by Congress all Head Start and Early Head Start programs are held to the Head Start Program Performance Standards (HSPPS), a set of research-based standards for the provision of quality comprehensive child development services to children. Additionally, as required by Congress in the 1998 Reauthorization of Head Start, programs must ensure that children meet specific early literacy and numeracy goals. Head Start also works to develop children's socio-emotional skills.

Evidence: -Head Start Act, program purpose
-Office of Head Start website - Budget/Policy section, Good Start, Grow Smart
-Head Start Overview - general program information
-Federal Poverty Guidelines
-Head Start History
-Original Report Recommending Head Start
-Early Head Start History
-Head Start Program Performance Standards

Does the program address a specific and existing problem, interest, or need?

Explanation: Head Start's mission is to help children from low-income families start school ready to learn. Young children living in poverty are much less likely than non-poor children to be ready to learn by the time they reach school-age. Immigrant children are at an even greater risk. Achievement gaps follow these children into elementary and high school. Head Start aims to close the gap early. The achievement gap is stark: Children entering Head Start score significantly below national norms--entering at the 16th percentile for vocabulary and early writing, the 31st percentile in letter recognition, and the 21st percentile in early math. (FACES) The U.S. Department of Education's Early Childhood Longitudinal Study found that nearly half of children entering kindergarten coming from families with multiple risk factors score in the bottom quartile in early reading and math skills. Head Start regulations require that 90% of Head Start families be at or below the Federal Poverty Line (FPL) ($20,000 for a family of four in 2006). In 2004, there were 5 million children under age 6 who lived in families whose income was below the FPL, a 14% increase since 2000. U.S. Census Bureau estimates show that 1.64 million 3- and 4-year old children are eligible for Head Start services by virtue of their families' income level. Head Start serves approximately half of these children. Approximately 21,000 Head Start children come from homeless families. National evaluation studies of Head Start programs have demonstrated that Head Start can be effective in helping to close the achievement gap. These evaluations have found that Head Start centers have positive effects on cognitive skills, and promote school readiness skills and indicators (e.g., physically healthy, well-nourished, adequate communication skills, and socio-emotional health). Head Start programs cannot shield children from all that comes along with growing up in poverty. However, research is clear that high quality early childhood programs, including Head Start, can and do make a difference in closing the achievement gap and preparing children to learn in school.

Evidence: -Original Report Recommending Head Start: Cooke Report
-Head Start Act
-US Census Data
-Numbers of children in poverty in US
-Abecedarian Project Evidence - school readiness indicators & long-term outcomes
-ECLS-K study - school readiness & achievement gaps
-Head Start Program Information Report Data
-Research/Evidence of continuing gap in school readiness for low-income kids
-Research on need for comprehensive services
-FACES data
-Head Start Impact Study findings

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: Several characteristics distinguish Head Start from other child care and early childhood program models: 1. Head Start's flexible design allows it to target needs of families and local communities. 2. Head Start is a comprehensive program, grounded in early childhood education research, and includes health, nutrition, transportation, and social services, as well as services for children with disabilities. 3. Active parent participation, both in program governance and in each child's education, is a core operating principle and is required by the Head Start Program Performance Standards. 4. Inclusion is another core principle??10% of the spaces in the program must be reserved for children with disabilities. 5. Head Start was designed, in part, to be a national laboratory for early childhood knowledge and has long been a national model for quality early childhood programming. 6. Collaboration with other community agencies has always been central to Head Start's service delivery design. Since Head Start is not able to provide all mandated services, collaboration with agencies that can is crucial. Each Head Start program is required by the Head Start Program Performance Standards to coordinate with local early childhood programs and school districts. Furthermore, each state (and the District of Columbia) has a State Collaboration Director charged with developing collaborations. Head Start children and families??those at the lowest end of the income scale??are most likely to benefit from the type of comprehensive services offered by Head Start. Providing comprehensive services to the most needy, however, does cost more, and few states are able to provide this level of service. Federal, State, local, and private efforts, such as Even Start, the federally funded Child Care and Development Fund, State preschool and pre-K, and local preschool programs, do not offer the same comprehensive services to children and families as Head Start, nor are they able to reach all eligible children. All together, the 40 or so states that provide some kind of preschool program serve approximately 800,000 children (compared to the 905,000 served by Head Start). Head Start provides an important complement to state-funded early childhood programs and a safety net of quality early childhood services for the lowest income children and families. Head Start's comprehensive service-delivery model, capacity to serve, and role in facilitating collaboration in the early childhood arena, distinguish it from all other efforts of its kind.

Evidence: -Even Start information
-Child Care Bureau
-Child Care & Development Block Grant information
-Head Start Act - population eligibility requirement
-Head Start Act - Early Head Start authorization
-Head Start Act - scope of services to be provided by Head Start
-Program statistics for 2005 - # enrolled in HS
-States with universal pre-K or other pre-K programs
-Data on available of slots for low-income children in local and private cc

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Current flaws in Head Start's program design inhibit the ability of the program to fully realize its mission. Limited competition: Head Start funds are currently distributed in a way that significantly limits competition. Currently, priority for funding is given to existing HS programs, unless they have blatantly failed to meet program or financial management requirements, limiting the ability of the OHS to recompete grant funds at the end of each cycle. Congress and the Administration recognize this problem, and the proposed reauthorization of Head Start contains provisions to enhance competitiveness, so that most grantees must recompete for funds every five years. Improving coordination/collaboration: Although Head Start has begun to develop mechanisms for enhanced collaboration, there is considerable room for improvement in the way that Head Start programs collaborate and coordinate with other agencies and organizations in early childhood educations and low-income service delivery. Improved coordination with the States is an Administration priority. The Administration supports House Bill HR 2123 which specifies that States require early leaning collaboration grants to promote a coordinated early childhood education delivery system. Program performance and technical assistance: Head Start has undertaken significant improvements to the monitoring system to address underenrollment, erroneous payments, child outcomes, and grantee compliance. In addition, the OHS has implemented the National Reporting System (NRS) to collect data on children's progress towards a set of school readiness indicators and compile individual program performance data for these measures. The OHS of Head Start has begun developing mechanisms to use NRS results to inform program improvement and technical assistance efforts. The OHS responded to recent GAO concerns about Head Start's Training and Technical Assistance (T/TA) System by reorganizing it to provide individualized, targeted and intensive onsite support to grantees. This is a step in the right direction to solidify the link between program monitoring and proactive technical assistance. The Administration has proposed additional changes in Technical Assistance in order to focus efforts more on low performing and deficient grantees. The Office of Head Start is studying the TA system and plans to make changes based on results.

Evidence: -Head Start Program Performance Standards (45 CFR 1301 et seq.)
-Head Start-State Collaboration Annual Report 2005
-The Head Start Management Initiative (DHHS, May 2004)
-Head Start Act, requirements for tri-annual monitoring
-PRISM instrument
-Head Start Act, requirement for 20% nonfederal match
-Head Start Act, requirements for priority designation
-Pros & cons for federally managed grants vs. block grants to states
-Government Accounting Office, "Head Start: Comprehensive Approach to Identifying and Preventing Grantee Financial Management Weaknesses", GAO-05-176
- Head Start Act reauthorization-bills before Congress

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Head Start's program design ensures that the program purpose is met and resources reach intended beneficiaries: Federal to Local Design As a federal to local program, Head Start awards grant funds directly to over 1600 local public, nonprofit, and for-profit organizations, which helps to ensure that money reaches children and families in local communities. A Federal to state to local structure would require an added layer of government to disseminate, administer, and monitor standards and funding. Head Start also leverages additional funding and services in each community by requiring a 20% local match. National performance standards help to ensure that comprehensive child development services are provided consistently throughout the nation to promote school readiness for low-income children. Financial Management and Program Accountability Head Start has rigorous financial management requirements and protocols to ensure that funds are directed to services for children and families (see 3.3, 3.6 and 3.CO2), including a strict limitation on administrative costs--no more than 15% of grants. Every grantee is also monitored onsite at least once every three years (see 3.1), during which compliance with all program standards is closely examined. Grantees that are found deficient are given a limited period of time to correct the deficiency, and failure to do so may result in termination of grants. Reduction in Under enrollment In response to the higher-than-desired under enrollment seen in the past, the Office of Head Start issued guidance (Nov 04) reinforcing that grantees have contractual responsibilities to fully enroll programs and outlining possible consequences of chronic under-enrollment. Federal staff and contracted technical assistance staff follow up with under-enrolled grantees to assist them in addressing problems. When problems are not resolved, termination proceedings are initiated. Under-enrollment rates dropped from 4.4% in 2004 to 2.8% in 2005 (annual PIR data). (See 3.4) Reduction in Erroneous Payments The Head Start Act and regulations specify that all children served by Head Start programs must be age eligible and that 90% must be at or below the Federal Poverty Line or receiving public assistance (tribal grantees excepted). The Office of Head Start made significant progress in FY 2005 in assuring that Head Start funds served intended beneficiaries. In FY 2004, ACF designed a study to assess compliance and conducted onsite reviews of enrollment files randomly selected to be a nationally representative sample. Approximately 3.9% of all enrolled children were not eligible (in 50% of these cases, adequate documentation did not exist in the files to verify income eligibility). In response, the Office of Head Start issued guidance to grantees reinforcing Head Start eligibility requirements and the expectation that these requirements be verified. In FY 2005, Head Start reviewed another 50 grantees, using the same rigorously designed protocols. The number of ineligible children had been reduced by 100%--to 1.5%. This effort will continue in FY 2006 and further reductions are expected.

Evidence: -FY2006 Head Start Funding
-Head Start Act, eligibility requirements
-Head Start Act, Administrative Cost cap of 15%
-2005 PIR income eligibility data
-Federal Poverty Guidelines
-Underenrollment data from PIR
-ACYF-HS-PI-04-03 - Achieving and Maintaining Full Enrollment
-Erroneous payment data
-Guidance on erroneous payments
-Data Collection Form Used by Erroneous Payment Reviewers
-ACF Regional Office responsibilities
-Requirement for community assessment from HS Program Performance Standards

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: Long-term outcome measures focus on Head Start's (HS) progress in providing high-quality comprehensive services to young children and their low-income families to improve children's overall development and prospects for school. PROGRAM GOAL: Child development: Enhance children's growth and development through improved literacy, numeracy, and language skills. Measure #1: Increase the percentage of programs in which children average fall-spring gains of at least 12 months in word knowledge (PPVT) in the National Reporting System (NRS). Measure #2: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring by the NRS. Measure #3: Increase the percentage of programs in which children average fall-spring gains of at least 4 counting items in the National Reporting System. The measures for this program goal: receptive vocabulary (PPVT), Letter Naming, and Early Math (Woodcock-Johnson) are predictive of later success in literacy, numeracy, language skills, and general academic knowledge. Preliminary analysis of NRS data indicates that there is a significant positive, moderate-to-strong relationship (p<0.0001) between the NRS PPVT task and Kindergarten Vocabulary and General Academic Knowledge. A similar predictive relationship appears to exist between the NRS Letter Naming task and Kindergarten Basic Reading Skills and General Academic Knowledge, and between the NRS Math Reasoning task and Kindergarten Math Reasoning and General Academic Knowledge. The predictive validity of scores on the NRS tasks indicates that a higher score would correspond to later academic success. (Note: The analysis of predictive validity was done by assessing the correlation between the tasks and kindergarten success for children in both the NRS sample and FACES 2003 cohort). NRS results are based on assessments of all Head Start four-year-olds administered initially in the Fall and then in the Spring of the academic year. Programs receive information on how children are doing "on average" in their program; they use this information to plan improvements to their curriculum and staffing. The National measures establish targets for an increasing percentage of programs to meet the specified thresholds. Because the national measures build upon data that programs receive and use for their internal management purposes, this information is also used to support program improvements through targeted training and technical assistance. PROGRAM GOAL: Parent Role: Strengthen role of families in enhancing children's growth and development Measure #4: Increase percentage of parents of children in their pre-kindergarten Head Start year who report reading to child three times per week or more, as measured in the Spring of their pre-kindergarten Head Start year. PROGRAM GOAL: Social Skills: Enhance children's social skills to improve their ability to interact with peers and teachers to improve school readiness. LT Measure #5: Increase the percentage of programs in which children make prescribed gains on a measure of social skills between the Fall and Spring of their pre-kindergarten Head Start year, as measured in the National Reporting System (Developmental) PROGRAM GOAL: Health; Children demonstrate improved physical health. LT Measure #6: Increase the percentage of Early Head Start children completing all medical screenings.

Evidence: -Head Start Act - primary purpose of HS
-GPRA Strategic Plan
-FACES - data on quality of interaction, parents reading to children, and gain in cognitive scores
-PIR-teacher qualifications
-NRS Report to Congress
-Head Start Impact Study - data on parents reading to children
-EHS Research and Evaluation Project - data on parents reading to children

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The targets establish ambitious timelines for improving the performance of Head Start grantees and narrowing the gaps in child achievement. The Targets account for performance trends, research in early childhood education, and a realistic assessment of program challenges and environmental constraints. Baselines and targets are shown in the Measures section. LT Measure 1: Increase the percentage of programs that achieve average fall-spring gains of at least 12 months in word knowledge in the National Reporting System. These targets are ambitious because: 1) the proposed threshold gain of 12 months in a 6-month period (between the fall and spring assessments) requires children to advance at twice the rate that would result from maturation; and 2) they would require a 27% increase in the proportion of programs achieving this accelerated gain over seven years. The gain would reduce the gap between HS children and non-HS children by 50% (6 months) in the face of numerous obstacles. LT Measure 2: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring by the National Reporting System. The threshold of 10 letters directly responds to the legislative mandate for children to have learned 10 letters by the time they exit Head Start. Currently over 90% of Head Start programs meet the threshold - the goal is ambitious because it requires that ALL Head Start programs meet the threshold for letter recognition, a precursor to early literact, within five years. LT Measure 3: Increase the percentage of programs that achieve average fall-spring gains of at least 4 counting items in the National Reporting System. The baseline for this measure is set at Spring 2004 at 731%. The target is set for 82% by 2007 and 84% by 2010. Performance improved rapidly from 2004 to 2005 (7%), coinciding with implemention of the NRS and having intentional instruction focused on the development of numeracy skills. By 2010 84 percent of programs will achieve the average 4 item gain. On the other hand, according to NRS data, HS children enter their pre-kindergarten year able to count to ten on average. The 4 point gain means that, on average, HS children would exit the program being able to count to 14. Though it is difficult to compare the progress of HS children to average children on the task because it is not normed, the gain of 4 counting items seems significant. LT Measure 4: Increase percentage of parents of children in their pre-kindergarten Head Start year who report reading to child three times per week or more, measured in the Spring of their pre-K HS year. The targets are ambitious because they propose to completely close the gap between how frequently HS parents read to their children compared to non-HS parents by 2013, despite the fact that parents often experience literacy problems of their own. HS is working with the parents to improve their reading skills at the same time that they are educating parents about the importance of reading to their children. LT Measure #5 (developmental): Increase the percentage of programs in which children make prescribed gains on a measure of social skills between the Fall and Spring of their pre-kindergarten Head Start year, as measured in the National Reporting System. LT Measure #6: Increase the percentage of Early Head Start children completing all medical screenings. In 2005 (baseline), 82% of Early Head Start children received all required screenings within the 90 days required by HS regulations. The 2007 target is 85%; the 2010 target is 91%--these long-term targets represent a 2% improvement each year in the percent of children completing all medical screenings??more than twice the rate of improvement between 2004 and 2005. These targets are ambitious given trend data, cultural preferences, and lack of health insurance, transportation, and health care providers in some areas.

Evidence: -Data on parents reading to children
-US DoE's Early Childhood Longitudinal Study-Kindergarten (ECLS-K) - information on performance of low-income children upon entry to kindergarten
-Abecedarian Project
-GPRA and PART timeframes
-Head Start Program Performance Standards- standards requiring that programs serve those in greatest need

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Head Start's (HS) five annual performance and one efficiency measures support HS's mission and long-term performance goals. Data that support these measures is available annually and provides local programs and the Office of Head Start (OHS) with ongoing information on individual program performance and progress toward national goals. Note: Details on each of the proposed annual measures are provided in the "measures" section of the PART document, and information about the proposed targets is provided in Question 2.4 and in the measures section. Annual Performance Measure 1: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring of the pre-kindergarten Head Start year by the National Reporting System. (Because of the critical importance of this measure for the learning potential of children, it is used as both a long-term and annual measure.) Annual Performance Measure 2: Increase the percentage of teachers with AA, BA, Advanced Degree, or a degree in a field related to early childhood education. Annual Performance Measure 3: Increase the percentage of Early Head Start children completing all medical screenings. (Because of the critical importance of this measure for the healthy development and learning potential of children, it is used as both a long-term and annual measure.) Annual Performance Measure 4: (developmental) Reduce the percent of grantees with repeat deficiencies through the provision of targeted technical assistance. Annual Performance Measure 5: (developmental) Decrease the number of grantees with deficiencies in early childhood development. Annual Efficiency Measure: Decrease under enrollment in Head Start programs, thereby increasing the number of children served per dollar.

Evidence: -GPRA measures
-FACES methodology & specific places in FACES with the data on quality of teacher-child interaction, & parents reading to children, and gain in word knowledge, letter identification, social skills would be found
-PIR methodology & specific places in PIR that the data shows teacher qualifications and underenrollment would be found
-NRS methodology & specific places in the NRS where the data on gains in word knowledge, mathematical skills, and letter identification would be found
-Impact Study methodology & specific places in the Impact Study where the data on parents reading to children would be found
-EHS Research and Evaluation Project methodology & specific places in the EHS R &E project where the data on parents reading to children would be found
-GAO-05-176 "Head Start: Comprehensive Approach to Identifying and Preventing Grantee Financial Management Weaknesses"

Does the program have baselines and ambitious targets for its annual measures?

Explanation: Annual Performance Measure 1: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring of the pre-kindergarten Head Start year by the National Reporting System. As of FY 2005, 92 percent of programs met this threshold, and under the measure, all programs are expected to meet it by FY 2010. The targets for the next five years are: FY 2005 92%; FY 2006 94%; FY 2007 96%; FY 2008 97%; FY 2009 99%; FY 2010 100%. These targets are ambitious since reaching this goal within such a short timeframe will establish Head Start's success in promoting letter identification, a predictor of early literacy. Annual Performance Measure 2: Increase the percentage of teachers with AA, BA, Advanced Degree, or a degree in a field related to early childhood education. The baseline (45%) was set in 2001. Ambitious targets (65% in 2006; 72% in 2007*; 75% in 2008) require improved program performance each year, yet recognize the reality that HS must compete with local school districts for skilled teaching staff while offering lower pay. See the Measures section for discussion of why these targets are ambitious. **Note: the large target increase from 2006 to 2007 is due to the fact that the 2006 target was exceeded in 2005, with 69% of teachers possessing the specified credentials. Because the 2006 target has already been set (per the FY 2007 Congressional Justification), the 2007 target was revised to account for the 2005 actual. Annual Performance Measure 3: Increase the percentage of Early Head Start children completing all medical screenings. Currently 82% of Early Head Start children receive all required screenings within the 90 days required by HS regulations (baseline set at 2004). Future targets are set at 83%, 85% and 87% for 2006-2008. See the Measures section for discussion of why these targets are ambitious. Annual Performance Measure 4: (developmental) Reduce the percent of grantees with repeat deficiencies through the provision of targeted technical assistance. Annual Performance Measure 5: (developmental) Decrease the number of grantees with deficiencies in early childhood development Annual Efficiency Measure: Decrease under enrollment in Head Start programs, thereby increasing the number of children served per dollar. The baseline was established at 4.4% in 2004. Targets were chosen based upon a 10% improvement rate each year (4%, 3.6%, 2.8%, and 2.7% for 2005-2008). A reduction of 10% corresponds to approximately 4000 vacancies a year in 2005. See the Measures section for discussion of why these targets are ambitious.

Evidence: Targets for GPRA annual measures, Evidence Binder

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: All Head Start (HS) partners commit to and work toward HS's annual and/or long-term goals. HS's partners include the HS grantees, Federal partners, contracted HS training and technical assistance providers, and HS State-Collaboration Offices. Each HS program must comply with the Head Start Program Performance Standards. The Performance Standards require implementation of comprehensive child development and parent involvement services designed to improve children's prospects for school success and help to ensure that performance targets related to early learning and literacy, staff credentials, parent involvement, and full enrollment are met. Several PART performance measures rely on the Head Start National Reporting System, which provides annual data on individual program progress with respect to children's achievement gains in word knowledge, mathematical skills, and letter identification. The National measures establish targets for an increasing percentage of programs to meet the specified thresholds. Because the national measures build upon data that programs receive and use for their internal management purposes, national performance goals are fully integrated into performance goals at the grantee level. This information is also used to support program improvements through targeted training and technical assistance. The ACF Regional Offices are responsible for grants management, procurement and general program support, participating in the program monitoring process and working with programs to assist them in meeting all program requirements and goals. ACF's program monitoring systems support ongoing assessment of grantee performance in achieving HS's long-term goals. The ACF Regional Technical Assistance (TA) Contracts improve outcomes for children and families, including improving early literacy and mathematic skills for young children, implementing family literacy initiatives, strengthening staff qualifications and teaching strategies, developing program strategies to ensure full enrollment and sound grants management and fiscal administration as well as full compliance with the Head Start Program Performance Standards.

Evidence: -Head Start Program Performance Standards (full-version)
-ACF Regional TA Provider contract (excerpts)
-Roles & responsibilities of ACF Regional Offices
-SCO Annual Report 2005
-Head Start Act excerpts on SCOs
-Head Start Reauthorization 2006
-PRISM Instrument and Guidance
-NRS Infornation for programs (NRS tool; PI/IM)
-Head Start Management Initiative strategy document (May 2004)
-Family Literacy Cooperative Agreement request for proposal (September 2005)
-Mentor Coaching information: IM
-Parenting Mentoring information: IM
-Higher Education Grantee Compendium of 2005 grantees and new RFPs

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Three major independent national evaluations of the Head Start program are being conducted to describe and evaluate the effectiveness of the program and to support program improvement. The major studies include the Head Start National Impact Study, the Head Start Family and Child Experiences Survey, and the Early Head Start Research and Evaluation Project. Additionally, a number of other research projects and data collection measures, such as the Head Start University Partnerships grants, support ongoing program improvement and program effectiveness. The Head Start National Impact Study uses a nationally representative, random assignment study design in order to assess the impact of Head Start. It is only implemented in communities where there are a larger number of eligible children than can be served by the program, given funded enrollment levels in which a randomly assigned control group can be established. The comparison group may use any other non-Head Start services in their community. Data collection began in 2002, and only the first year's results have been published as of this writing. Begun in 1997, the Head Start Family and Child Experiences Survey (FACES) is a nationally representative longitudinal study that provides information on the characteristics, experiences, and outcomes of children and families served by Head Start. Data is collected on children while in Head Start as well as when they graduate from Kindergarten and at the end of first grade. The Early Head Start Research and Evaluation Project is a rigorous, random assignment evaluation. This two phase study includes an implementation study, an impact evaluation to assess program impacts on children through their third birthday, and local research projects. The pre-Kindergarten follow-up phase follows children and their families until they enter Kindergarten. In addition to the major research studies, Head Start has a number of other means for collecting data on program performance, including the National Reporting System, the Program Information Report, and the PRISM program monitoring system. (see 3.1 for more detail.) Head Start also has an Advisory Committee on Head Start Accountability and Educational Performance Measures, which is charged with assessing the progress in developing and implementing educational measures in Head Start, including the National Reporting System (NRS). The Committee will be providing recommendations to HHS for integrating NRS with other ongoing assessments of the effectiveness of Head Start including recommendations as to how NRS and other assessment data can be included in the broader Head Start measurement efforts found in the Family and Child Experiences Survey (FACES), the National Head Start Impact Study, Head Start's Performance Based Outcome System, and the ongoing evaluation of the Early Head Start program.

Evidence: -National Head Start Impact Study methodology
-National Head Start Impact Study research findings reports
-Head Start Family and Child Experiences Survey (FACES) research methodology
-Head Start Family and Child Experiences Survey (FACES) research findings reports
-Early Head Start Research and Evaluation Project methodology
-Early Head Start Research and Evaluation Project research findings
-Office of Head Start, Research website, general
-PIR Instrument and process
-PRISM Instruments (PRISM tool, Checklists, Income Eligibility Data Collection form) and process
-National Reporting System Assessment Instrument and Process

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: For each ACF program, including Head Start (HS), the ACF FY 2007 Congressional Justification provides summaries of the budget requests and performance analysis and explicitly ties them together in the rationale for the budget request. The rationale provided for HS programs is based on the legislative authority provided by the Head Start Act The Congressional Justification also includes Resource and Program Data sheets that present resource needs in a complete and transparent manner. Resource data show funding designated for grants, research and evaluation, demonstration and development, training and technical assistance, and program support. Program data shows the number of grants, contracts and intra-agency agreements. (See Section 5.2 HS Program Goals: Enhance Children's Growth and Development; Strengthen Families; Children receive educational services; and Children in HS receive health and nutritional services) Additionally, funding decisions have been made based on accomplishing the long-term and annual performance measures of the program. Starting in FY 2005, HS initiated a more rigorous approach to dealing with grantees not serving the number of children funded. These grantees are given a short- period of time to reach full enrollment, and if not successful their funding is reduced and the "freed up" funds are used to fund a different grantee??chosen competitively??that can provide services to all funded children. Moreover, as HS grantees lose their grants, the competition to select a replacement grantee includes cost effectiveness as one of the central factors used in determining the successful applicant. The Assistant Secretary and other ACF senior staff regularly participate in Departmental and ACF meetings to review financial information and performance information. o The Assistant Secretary reviews the latest performance information for all ACF programs, including HS, and discusses this information in meetings with program heads. For measures that have fallen short of targets, he tasks program heads to identify specific activities during the remainder of the year to improve performance, using a logic model framework to link financial, staff, and other resources with activities, outputs, and performance outcomes. Following these discussions, ACF senior staff work together to develop and refine these plans for review by the Assistant Secretary. o The Assistant Sec meets with senior staff to review financial and performance information to make decisions regarding succession planning and staffing for ACF components. o The Assistant Sec and senior ACF staff present the findings to Departmental officials and the approved policy decisions are reflected in the FY 2007 Congressional Justification budget request. The regionally based HS Technical Assistance contracts are targeted to improve program performance and compliance, including performance related to performance measures. Contracts are awarded based upon organizations' ability to meet specific tasks outlined in the Program Announcement, including a number of criteria related to improving program performance in the long-term and annual performance measures, such as: - providing technical assistance to grantees to remedy problems associated with under-enrollment, - providing technical assistance where grantees are unable to recruit and retain qualified staff, - analyzing PIR data to determine areas for improvement and recommending technical assistance strategies to address these needs, - assisting grantees in working with local school systems to facilitate the transition from HS to kindergarten and to assure HS children are entering school with appropriate skills in early literacy. Supplemental TA funds are also provided to grantees that have been found to be deficient or have specific identified needs, and grantees were awarded additional funding to enhance teacher qualifications in order to meet the Congressional mandate.

Evidence: -ACF FY 2007 Congressional Justification, February 2006
-Grantee Budget Request (GABI)
-Head Start Bureau Guidance on Underenrollment
-Reauthorization Legislation - excerpts on 5-year recompetition, etc.
-Head Start Budget - nationwide program budget

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Head Start corrected several strategic planning deficiencies since its last PART evaluation, including?? ?? Incorporating a long-term and annual performance measure which assesses individual program progress toward performance goals (Long Term Measures #1, 2, and 3 and Annual Performance Measures #1, 3 and 4), thus providing Head Start with means for evaluating individual program performance toward Head Start goals ?? Implementing the National Reporting System, which provides annual data on individual program progress with respect to children's achievement gains in word knowledge, mathematical skills, and letter identification, and incorporating this data into the PART performance measures (Long Term Measure #1, 2 and 3 and Annual Performance Measure #1) ?? Developing a new annual measure to track individual program compliance in meeting performance standards related to early learning and literacy (Annual Measure #5). ?? Developing a new annual measure to track the effectiveness of the TA system in preventing repeat program deficiencies (Annual Measure #4). ?? Making continued progress through the Head Start-State Collaboration Offices in improving coordination in state systems for child care, early childhood education, and other services that support child and families ?? Working with Congress to write legislation that would further improve the integration of Head Start State Collaboration Offices with child care and state-operated preschool programs ?? Improving and standardizing PRISM monitoring reviews to improve program performance, as noted in detail below ?? Developing a risk assessment strategy and an integrated database of all program information systems, including those related to funding, monitoring, program services, and program performance, thus improving resource allocation, program planning, and decision-making. ?? Within the constraints of the Head Start authorizing legislation, working to tie performance to funding, including for demonstration projects, one-time funding, expansion funding, and grantee re-competitions Head Start is also in the process of developing a logic model to guide the work of the national training and technical assistance system. This model reviews the major activities of the T/TA system, the output, indicators, outcomes, and responsible parties. This model will help to ensure that Head Start's strategic objectives are integrated into and implemented throughout the Head Start T/TA system and lead to desired program performance outcomes at the grantee level. Additionally, in FY 2006, Head Start is implementing several significant changes that are designed to improve its oversight capacity and thus its ability to monitor strategic objectives and, ultimately, improve the performance of Head Start programs. These include refining the PRISM Monitoring System and developing an integrated management information system which will house Head Start databases grants funding and budget information, performance and compliance information, and information about the children and families served by each Head Start program.

Evidence: Overview of Recent Changes to the Head Start Program Monitoring System (PRISM)

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The Office of Head Start (OHS) systematically collects information related to program goals from a variety of sources, and regularly uses it to identify and correct problems and improve performance. OHS recently implemented a number of major oversight initiatives to improve program performance and reporting and to address GAO's findings (in GAO-05-176) that: "identified flaws that limit the quality and reliability of information that ACF collects." The following are recent changes: - OHS nationalized the monitoring system and improved reviewer selection, training, and reviewer performance tracking to ensure high quality monitoring and consistency in interpretation and application of all ACF guidelines and HS regulations (see 3.7). To ensure objectivity, Federal Team Leaders no longer review grantees within their own Region. - The risk-based Fiscal Checklist has been implemented to assist reviewers in identifying fiscal management problem areas. - OHS now requires data verification of selected PIR indicators during onsite reviews, and is conducting a validity study of PIR data using a random sample of grantees. - OHS is developing an integrated data management system called the Enterprise System (operational fall 2006), and a Risk Assessment System (operational FY 2007) (see 3.4) to allow better identify potential grantee problems and direct technical assistance to them. - OHS is hiring five fiscal specialists for the new HS Financial Integrity Unit to provide fiscal oversight to grantees, training to federal managers and technical assistance providers, and resources to support sound financial management practices. The following information collection methods provide continuous feedback in support of ongoing program improvement. -PRISM: Head Start grantees are monitored on site at least once during every 3 year period. Qualified teams conduct onsite reviews and document grantee compliance with all applicable regulations using the Program Review Instrument for Systems Monitoring (PRISM). Grantees are required to correct each area of noncompliance within a prescribed period. Failure to correct deficiencies results in OHS initiation of termination procedures. -Training and Technical Assistance Monitoring results drive much of the work of HS's technical assistance (TA) services. Each grantee must develop a TA Plan to address weaknesses identified through self-assessment and Federal monitoring. TA providers and Federal staff ensure that TA resources are allocated to address these needs. An evaluation of the TA System is in process to assess performance. -Program Information Report: OHS collects extensive program performance information through the annual Program Information Report (PIR). All grantees are required to submit an annual PIR, which provides descriptive information about HS grantees, children and families, and services. PIR data is aggregated and available to grantees, Federal officials, and the public and is used for setting program priorities and allocating resources. -National Reporting System: The National Reporting System (NRS) collects information related to the school readiness, such as letter recognition, vocabulary, and early math skills, and uses this to measure children's progress while in HS. NRS data helps to determine the allocation of TA resources at the grantee, regional, state, and national levels. Once longitudinal data is available and instruments and methodologies are fully tested and validated, NRS data will be used, along with other information, to provide performance data for individual HS grantees. -Research: OHS funds several significant research projects. Each of these provides information which is used to improve program services and outcomes as well as to track performance. (see 2.6)

Evidence: -Head Start Act, requirement to monitor every three years
-Most recent PRISM instrument
-Head Start Act, requirements for terminating grantees
-Overview of Recent Changes to the Head Start Program Monitoring System (PRISM)
-Report to Congress: Head Start Monitoring for FY 2004
-February 2005 GAO Report #05-176: Head Start: Comprehensive Approach to Identifying and Addressing Risks To Help Prevent Grantee Financial Management Weaknesses
-TA Provider contract
-Head Start Training and Technical Assistance Quality Assurance Study: A Design Report
-2005 PIR Summary Data
-Head Start Act, requirements for teacher degrees
-IM/PI on Quality Improvement Funding for Improving Teacher Qualifications
-2003 Summary PIR data on Teacher Qualifications having met legislated goal
-IM on launch of NRS, as part of President Bush's Good Start, Grow Smart initiative
-Good Start, Grow Smart Initiative
-NRS Methodology
-NRS Findings
-FACES research
-Child Outcomes Framework (ACYF-IM-HS-00-18)
-IM on Training for Early Literacy Mentor Coaches (Summer 2002 and Fall 2002)
-Information on STEP Network and the Instructional Design for Mentor Coaching (2003-2005)
-IM on The Head Start Conference for STEP Early Literacy Mentor-Coaches (November 2002)
-The Head Start Leaders Guide to Positive Child Outcomes and The Head Start -Path to Positive Child Outcomes (2nd edition)
-The Early Learning and Literacy Work Zone (collaborative online work environment for Head Start Early Learning and Literacy TA Specialists)
-Early Head Start Research and Evaluation Project
-Early Head Start Tip Sheets, including one on program design based on needs of families
-RFP for Child Welfare Services Project
-RFP for Enhanced Home Visiting Project
-RFP for new social emotional national center
-Examples of Early Head Start "research to practice" materials
-Head Start Impact Study

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal managers and program partners, including program managers from grantees and delegate agencies, are held accountable for cost, schedule, and performance results in a number of ways. -Federal managersFederal managers have clearly defined performance standards. The Assistant Secretary for Children and Families has a performance plan that includes critical elements that hold him accountable for cost and schedule and outcomes. The performance contract includes achieving specific gains in letter identification, mathematical skills, word knowledge, and social skills as performance objectives. Increasing the percentage of teachers with degrees in early childhood education or a related field is also identified as a key performance objective. These objectives are reflected in the Office of Head Start's position descriptions and employee performance plans. For example, all staff have clearly identified responsibilities related to grant and contract administration and must demonstrate expertise in the areas of early childhood education and development, including among others early literacy and numeracy, program management, and grant and contract administration processes. -Head Start grantees Head Start grantee managers are held accountable for meeting Head Start Program Performance Standards and other applicable Federal regulations. Grantee managers are held accountable for adhering to these performance standards and are monitored with an intensive onsite review once every three years to ensure that these and other regulations are met or exceeded. Grantees who are not meeting Performance Standards or other required Federal regulations are given a specified period of time to correct problems. If problems are not corrected, termination proceedings are initiated. In response to Congressional and Office of Inspector General concerns about executive compensation of Head Start grantee executives, OHS implemented a number of decisive measures. OHS launched a Head Start Salary and Other Compensation Survey in December 2003 which required every Head Start grantee to submit information on executive salaries and benefits. In 2005, OHS issued a ACYF Program Instruction (PI) to all grantee and delegate agencies outlining the requirements of Section 205 of the HHS Appropriations bills, limiting the use of Head Start funds for employee compensation to an Executive Level II salary ($162,100 currently). This condition is also stated on each Financial Assistance Award. This PI informed grantees that ACF would conduct uniform systematic reviews as part of the annual grant refunding process to determine if the grantee was adhering to all statutory requirements, including those governing staff compensation. Salary compensation is now regularly monitored during onsite PRISM reviews to ensure compliance. -Program managers Head Start Program managers have additional incentives for ensuring that their programs meet or exceed Head Start Program Performance Standards: funding for new initiatives, one-time quality funding, pilot or demonstration projects, and program expansion grants are allocated only to those programs in good standing. Efficiency measures, such as cost per child, are also taken into consideration in making these awards.

Evidence: -Head Start Program Performance Standards (45CFR Parts 1304 and 1308) -List of other applicable Federal regulations -Info on development of the Head Start Program Performance Standards -RFP requirement for programs being in good standing ACYF-PI-HS-03-02, Head Start Survey of Salaries and Other Compensation ACYF-PI-HS-05-01, Employee Compensation Cap

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: Head Start's FY 2005 appropriation level was $6,843,000,000. Virtually 100% of these funds were obligated during this fiscal year by 1,600 HS grantees to serve approximately 905,000 children. Grantees must obligate these funds in a timely manner to ensure the continued provision of services to children and families. In response to concerns cited by GAO in GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks", the Office of Head Start (OHS) enacted several measures to improve program fiscal accountability and compliance, including: - strengthening the HS monitoring system, (see 3.7) including implementing a risk-based fiscal checklist to help reviewers assess potential problem areas, improving selection of and training for fiscal reviewers, and conducting special onsite fiscal reviews; - redesigning the technical assistance system to create a stronger link between monitoring and technical assistance (see 1.4); - establishing the Early Alert System to review specific performance and accountability indicators prior to each grantee's annual refunding (see 3.CO2); - creating a centralized HS Fiscal Integrity Unit to improve oversight, training, and resources for fiscal management (see 3.6); - developing the Enterprise System, an integrated data management system, to facilitate the analysis of program performance and potential areas of concern (see 3.CO2); - designing a Risk Assessment Strategy for early identification of potential problems; - conducting a nationally representative randomized study of erroneous payments to ensure only eligible children are served (see 3.6) With these systemic improvements, grantee compliance is expected to significantly improve, technical assistance will be more efficiently and effectively targeted, and grantees not complying with applicable regulations will suffer more severe and immediate penalties. In addition to the targeted efforts undertaken by the OHS to ensure that funds are being spent for their intended purpose, as noted above, HS has a number of strict financial management requirements and protocols, including the timely obligation of funds. -Obligation of Funds and Sound Financial Management Grantees are required to meet financial management standards in accord with 45 CFR 74.20 - 28. ACF Regional Offices conduct onsite reviews for grantee compliance with the Cost Principles of OMB A-122, and Federal financial management standards. Grantees are required to submit financial reports on the SF-269 to report the status of the grant funds on a semi-annual and final basis of expenditures in accord with 45 CFR 74.52. Information reported on the SF-269 must be reconcilable to the grantee's accounting records. The Final SF-269 must be reconciled by the auditor to the final audit report for the year, as well as reconciled by the grantee to the figures on the PSC 272, Federal Cash Transactions Report, for the final quarter of the budget period. ACF Regional Offices review 269s for timeliness and accuracy. Unobligated funds, as reported in the 269s, are de-obligated from the grant award. -Annual Audit Review Process Each grantee that expends in excess of $500,000 of federal funds is required to have an audit conducted in accordance with OMB A-133. The grantee is required to complete and submit the completed audit to the Federal Audit Clearinghouse, not later than 9 months from the end of the grantee fiscal year. Based on the results, ACF assigns an analyst and takes appropriate actions with the grantee to fully resolve the audit finding in compliance with the applicable program regulations. Upon resolution of audit findings, ACF is responsible for completing an "Audit Determination Letter" to inform the grantee of ACF resolution actions, any disallowances, and to give the grantee their appeal rights and close the audit. ACF is responsible for any follow up necessary to ensure that disallowed costs are reimbursed by the grantee.

Evidence: Data sheet on FY 2005 Head Start Expenditures, Children Served, etc. -Head Start Act, requirements on eligibility -Guidance on Erroneous Payments -Results of study on Erroneous payments -45 CFR 74.20 - 28 -Cost Principles of OMB A-122 -45 CFR 74.21 and other Part 74 administrative requirements -SF-269 Financial Status Report form, and description of SF-269 process from the Head Start Fiscal Assistant -PSC--272 Federal Cash Transaction Report -45 CFR 74.52 -424 Budget Applications -OMB A-133 Audits of States, Local Governments and Non Profit Organizations, revised June 27, 2003 -Head Start Compliance Supplement -General Terms and Conditions -ACF External Audit Resolution Manual -Real Time, On Line GATES Audit Data

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: Head Start has in place standard procedures to monitor and improve cost efficiency and effectiveness. While it could do more in the area of competitive sourcing, Head Start regularly takes advantage of IT improvements appropriate to the program structure. -Efficiency Measure As noted in Section II, the Office of Head Start uses the efficiency measure of decreasing under enrollment in Head Start program. As under enrollment decreases, the average cost per child also decreases. Baseline data for this efficiency measure was established in 2004 at 4.4%. Ambitious targets have been set at decreasing rates of 4.0%, 3.6%, 2.8%, and 2.7% for 2005, 2006, 2007, and 2008 respectively. In 2005, the under enrollment rate dropped to 2.8%, significantly exceeding expected targets. -Demonstrated Efficiencies through Budget Reallocation Currently, unless a grantee has unresolved deficiencies, they can automatically renew their grant at the end of each five-year award cycle. The proposed Head Start reauthorization bill would require grantees not meeting program or performance standards to recompete their grant every cycle. However, the Office of Head Start and grantees continue to demonstrate improved efficiencies and cost effectiveness despite the limitation of the current authorizing legislation. For example, in FY 2005, Head Start served the same number of children it did in FY 2004, despite receiving an increase well below the rate of inflation. Head Start will maintain these enrollment levels again in FY 2006, despite a 1% funding reduction of all discretionary programs. -Competitive Funding/Sourcing In awarding all Head Start pilot projects and demonstration grants, one-time funding, recompetitions, and expansion grants, cost is an important consideration. Additionally, all Head Start grantees and contractors must abide by Federal requirements and seek competitive bids for procurement transactions. - Efficiencies through IT Improvements The Office of Head Start has invested in technologies and strategies that will provide significant cost savings and benefits to Head Start. First, the Office of Head Start recently developed the Head Start Enterprise System, a database which will centrally house all of the information collected on individual Head Start programs. In addition, Head Start is working on a Risk Assessment Strategy, to be operational in FY 2007, that will allow the Bureau to identify grantees at risk of management problems, so that staff or TA providers can intervene early. The Early Childhood Learning and Knowledge Center, an online hub for Head Start providers, allows staff ongoing access to research-based best practices. It is a powerful tool for training and development and has the potential to provide substantial cost savings. Lastly, the Fiscal Assistant, an online resource for information on all fiscal management and grants administration issues, regulations, and guidance, provides this information to all grantees in a cost-efficient way.

Evidence: -Head Start Bureau guidance on underenrollment
-Reauthorization bill (still pending)
-Competitive costing regulations (45 CFR 74.43)
-Early Childhood Learning and Knowledge Center website
-Head Start Fiscal Assistant

Does the program collaborate and coordinate effectively with related programs?

Explanation: Head Start (HS) promotes collaboration at all levels to achieve its program mission and objectives. In July 2003, an HHS report "Strengthening Head Start" noted that "Head Start is not eliminating the gap in educational skills and knowledge needed for school," citing the fragmented pre-K system. Improved coordination with the States is an Administration priority, and the Administration supports House Bill HR 2123, which would give States early learning collaboration grants to coordinate early childhood education. While changes to the authorizing legislation are appropriate and there is more to be done, Head Start has taken substantial steps towards coordination. In addition to the requirements discussed in 1.4, HS Program Performance Standards require that all HS and Early Head Start (EHS) programs coordinate with a broad array of community organizations, including local education agencies, providers of health care, disability, and family support services, and child protective services, among others. Because HS funding levels are not adequate to allow programs to provide all required services to children and families, there is a strong financial incentive for programs to coordinate with other area service providers. Head Start State Collaboration Office (HSSCO) grants require that HSSCOs help to build early childhood systems, and facilitate the involvement of HS in State policies, plans, processes, and decisions affecting its target population. No nationwide assessment of of HSSCOs exists since they function differently in each state; however, preliminary assessments indicate that HSSCOs are improving coordination of early childhood systems (see HSSCO Annual Report). HSSCOs are active in State Early Learning Councils, SEA school readiness committees, and advisory and planning committees in states with universal pre-K, and all HSSCO directors and State Education Agencies (SEAs) have negotiated Memoranda of Understanding (MOUs) regarding services to children with disabilities. HSSCOs work on aligning standards, developing assessments, facilitating MOUs between agencies, and developing state-wide early childhood professional development opportunities. The Office of Head Start formally collaborates with entities on the federal, state, and local levels on various projects: Maternal and Child Health Bureau: Oral Health Intra-agency Agreement; U.S. Dept of Education and Child Care Bureau: Good Start, Grow Smart; Children's Bureau: Child Welfare Services Project; U.S. Department of Education's Office of English Language Acquisition; White House Initiative on Educational Excellence for Hispanic Americans; Memorandum of Understanding on Homelessness: HS and ACF OCS; National Governors Association: Integrated Policies and Services for Children; and Interagency Agreements with State Departments of Education. Additionally, nearly 450 local school districts and 150 city and county governments are HS grantees.

Evidence: -Head Start Act, provisions requiring collaboration
-HSB/MCH-HRSA Interagency Agreement
-Head Start-State Collaboration Offices Annual Report 2005
-OCS/HS-State Collaboration Office supplement funding agreement
-Interagency Workgroup for Good Start, Good Smart
-EHS CWS Project - RFP and research/evaluation contract
-2005 Birth to Three Conference - program
-CSEFEL website
-U.S. Department of Education's Inclusion Institute in Chapel Hill, NC announcement (August 2005)
-Smoke Free Homes Agreement with EPA
-Hispanic Institute Program Book
-Homelessness MOU
-National Governors' Association grant award announcement
-Agreement for contribution of $60,000 to the Pan American Health Organization (PAHO) to promote three International Vaccination Weeks
-2005 Head Start-State Collaboration Offices Annual Report
-Website for state interagency agreements-Disabilities Services - website on ECLKC
-State Interagency Agreements with State Departments of Education - website on ECLKC
-Head Start Program Performance Standards - excerpts on collaboration
-Contract/Agreement - Special Quest: Hilton partnership website
-Executive Training Program: Johnson & Johnson partnership website

Does the program use strong financial management practices?

Explanation: Head Start (HS) strictly adheres to ACF's financial management systems, practices, and policies. ACF's financial management systems meet all statutory and regulatory requirements, and financial information has been deemed accurate and timely. ACF is developing the Unified Financial Management System, scheduled for implementation of UFMS in October 2006, to produce automated financial statements and to achieve economies of scale and several other financial management goals of the Department. ACF has a strong Federal Managers Financial Integrity Act (FMFIA) program. Individual program managers are required to provide an annual assessment and certification attesting to whether or not internal controls are in place and functioning as intended. If not, managers are required to prepare an action plan and track progress until the issue has been resolved. The implementation of the revisions to OMB A-123/FMFIA, effective for FY 2006, will strengthen the requirements for conducting management's assessment of internal controls and provide the opportunity to improve internal controls and overall operational efficiency and effectiveness. ACF, as the second largest operating division in the DHHS, prepares annual financial statements and is audited by an independent firm in accordance with the CFO Act. ACF has received six consecutive unqualified or "clean" audit opinions. To correct findings cited in GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks", OHS enacted several measures to improve program fiscal accountability and compliance, including: - strengthening the HS monitoring system, (see 3.7) implementing a risk-based fiscal checklist to help reviewers assess potential problem areas, improving selection of and training for fiscal reviewers, and conducting special onsite fiscal reviews; - redesigning the TA system, including for fiscal TAl, to strengthen the link between monitoring and technical assistance (see 1.4); - establishing the Early Alert System to review specific performance and accountability indicators prior to each grantee's annual refunding (see 3.CO2); - creating a centralized Fiscal Integrity Unit to improve oversight to grantees, training to Federal managers and TA providers, and resources for fiscal management; - tightening oversight on erroneous payments (see below); - developing the Enterprise System, an integrated data management system to facilitate the analysis of program performance and potential areas of concern (see 3.CO2); and - designing a Risk Assessment Strategy for early identification of potential problems. - Erroneous Payment In FY 2004, HS began an erroneous payments initiative to determine to what extent HS funds were being used inappropriately. HS worked with statisticians to design a study using a national representative random sample. HS reviewed another random sample of grantees in 2005 using the same protocols, and the number of ineligible children was 1.5%, less than the FY 2004 level of 3.9% OHS expects to see even lower rates in 2006, based upon reports from TA providers -PRISM & T/TA The fiscal systems and documentation of HS programs are carefully monitored during onsite PRISM reviews, and providing training in financial management at national, regional and state training events is also a high priority for the OHS, ACF Regional Offices, and TA providers.

Evidence: - GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks"
-GAO Standards for Internal Control in the Federal Government (GAO/AIMD-00-21.3.1)
-OMB A-133 Audits of States, Local Governments and Non Profit Organizations, revised June 27, 2003 - requirement for annual audit
-Head Start Compliance Supplement
-GATES system (is used to issue grants within statutory and OMB limitations)
-FY 2004 ACF Federal Financial Managers Integrity Act Report
-FY 2004 HHS CFO Audit Opinion
-Unified Financial Management System Business Case
-Results from Erroneous Payment Review in FY 2004 and 2005
-PRISM - Fiscal Checklist
-2006 Directors Institute program book
-Fiscal Assistant website

Has the program taken meaningful steps to address its management deficiencies?

Explanation: Head Start has taken meaningful steps to address management deficiencies, including significant improvements to the grantee monitoring system, decisive action on under enrollment and erroneous payments, and an ACF reorganization to improve management efficiencies for HS and its other programs. -PRISM: the OHS has made major improvements to program monitoring (PRISM) and follow-up activities to address grantee management deficiencies, including a risk assessment-based approach, utilization of standardized methodologies and protocols, more rigorous selection and training for monitoring reviewers, assessment of reviewer performance, centralized analysis of monitoring results, and application of uniform standards for deficiency determinations. A number of checklists, protocols, and instruments have been developed to provide a consistent framework and an "early warning" system for grantees at risk. For example, the Fiscal Checklist prioritizes indicators designed to identify "red flags" (i.e., areas most likely to have the greatest impact on grantee fiscal health). Grantees with "red flags" are subject to more extensive reviews of fiscal records. Findings are provided to ACF Regional Offices (ROs) so they can strengthen risk management strategies and target TA resources. PRISM data is also used in planning to enhance performance/compliance. For example, when PRISM revealed that program governance was a major source of program deficiency, OHS allocated funding for governing board training regarding board legal and fiduciary responsibilities, and is now studying results to determine its effectiveness. OHS also ensures that annual training for HS and EHS directors covers governance, internal controls, and other relevant issues. The new annual performance measure to decrease the number of grantees with deficiencies in early childhood development will help target OHS oversight of and immediate provision of TA to grantees with performance/compliance problems related to the development of children's cognitive skills. -Underenrollment and Erroneous Payments: As previously noted, the OHS also acted to improve under enrollment and erroneous payments in HS programs. (See 3.3 and 3.4). -ACF Reorganization The recent reorganization of HS will improve ACF's oversight of HS grantees. Formerly, the OHS was housed within the Administration for Children, Youth, and Families, while the ten ACF ROs were located within the Office of Regional Operations. The disconnect between the policy making office and those working directly with grantees led to communication inefficiencies and inconsistencies in policy implementation. A recent GAO report identified several areas of weakness in ACF's oversight of HS programs, including, for example "??inconsistencies in the process used by the regional offices to determine the severity of [grantee] problems (p.3)." What was labeled as a deficiency in one region was viewed as a less serious noncompliance in another; and while some regions routinely took action to terminate grantees that failed to correct deficiencies, other regions rarely used this authority. OHS will now report directly to the Assistant Secretary of ACF, and HS managers and program specialists in the Regional Offices will report to OHS (renamed from "Office of Head Start"). The realignment of the ROs will link HS regional staff directly to program administrators in the central office. The new structure will enhance the ability of HHS's operating division to work with states and local governments and HS grantees, reduce administrative layers, expedite decision making, and improve communication, resulting in improved Federal oversight of HS grantees.

Evidence: -Overview of Recent Changes to the Head Start Program Monitoring System (PRISM) GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks"
-2006 Directors Institute Program Book
Federal Register: May 23, 2006, Vol 71, No 99, pp 29649-29650

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: Although new Head Start and Early Head Start grants are awarded based upon a competitive process at initial award, recompetition only occurs when grantees demonstrate blatant deficiencies. Statute dictates that Head Start grantees are refunded each year, on a non-competitive basis, as long as they meet Head Start Program Performance Standards and other applicable Federal regulations. Within the constraints for the Head Start authorizing legislation, Head Start does use peer-reviewed open competitions for new service areas, one-time funding opportunities, demonstration or pilot projects, and replacement grantees. Head Start grants are refunded annually by the ACF Regional Offices and AIAN and Migrant Program Branches after submission of budgets and program data and plans. As stipulated in the Head Start Act, priority for funding must be given to existing Head Start and Early Head Start programs, unless it is determined that they have failed to meet standards. This limits the ability of the OHS to recompete grant funds at the end of each grant cycle. The Administration believes that Head Start grantees with any unresolved deficiencies undergo recompetition as suggested in the proposed Head Start reauthorization. All interested parties are provided with additional information about the Head Start and/or Early Head Start program as well as a Request for Proposal which includes a clear set of criteria by which applications are evaluated. All applications received are reviewed by independent three-person panels composed of individuals knowledgeable about Head Start and early childhood and trained in the ACF grant review process, including researchers, program staff, PRISM reviewers, and early childhood independent consultants. Applicants are scored according to evaluation criteria including need for services, expected benefits, staff qualifications, proposed approach, cost per child, and budget. A "Decision Memo", or ranking of applicants based upon their scores, is submitted to the ACYF Commissioner for review and final determination. The applicants receiving approval (i.e., typically those with most competitive and/or geographically desirable applications) are visited.

Evidence: -Head Start Act, renewal of grant award
-Head Start Act, criteria for selection
-Head Start Program Performance Standards
-2006 Head Start Reauthorization
-Sample newspaper announcement of grant recompetition
-Examples of recompetition RFP
-Information on grant review process (ACYF Grant Review Manual; Selection Criteria; Score Sheets)

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: Head Start (HS) has a number of oversight practices that provide sufficient knowledge of grantee activities, both fiscal and programmatic, including a number of recent changes designed to address concerns cited in GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks": - strengthening the HS monitoring system, (see 3.7) including developing a risk-based checklists and protocols to help reviewers assess potential problem areas, and conducting special onsite fiscal reviews (see below); - redesigning the TA system to create a stronger link between monitoring and TA (see 1.4); - establishing the Early Alert System to review specific performance and accountability indicators prior to annual refunding (see 3.CO2); - creating a centralized Fiscal Integrity Unit to improve oversight, training, and resources for fiscal management; - developing the Enterprise System, an integrated data management system to facilitate oversight and the analysis of program performance and potential areas of concern (see 3.CO2); and - designing a Risk Assessment Strategy for early identification of potential problems. In addition to PRISM monitoring reviews (see 3.1 and 3.7), Regional Offices (ROs) conduct special program/fiscal onsite reviews to determine compliance on specific Head Start issues, including grantee compliance with HS salary cap limitations, under enrollment, facility issues, safety issues, and others. ACF RO staff and regional TA providers have ongoing contact with grantees to ensure adequate oversight. Federal and TA staff regularly review, for example, annual budgets, grantee TA Plans, community assessments, PIR data, PRISM monitoring reports. OHS is developing a Risk Assessment system to allow for early detection of potential concerns so that proactive TA can be provided. The TA system was recently redesigned to strengthen the link between monitoring and TA. Grantee program data is carefully reviewed 60 days before refunding by the Early Alert System, a meeting of Federal managers to discuss key indicators of program performance and compliance. This provides Federal managers at OHS and the ROs with indicators of program success and problems. Correction or a plan for correction must be in place prior to refunding. The OHS is investing in an integrated database system, the Enterprise System, to bring together all HS databases, including those related to funding, program services, program performance, monitoring, and TA. Federal staff and TA providers will have immediate access to all information related to a grantee or set of grantees, allowing early identification of problems as well as possible correlations in service provision, outcomes, and funding; and ultimately, facilitating more effective planning and efficient resource allocation. Independent auditors audit grantees annually (OMB A-133). The HS Compliance Supplement details the financial management requirements of the program. Auditors examine the day-to-day systems, financial reporting, eligibility determination systems, payments to providers, and systems to ensure overpayments are properly recorded, for compliance with financial and programmatic requirements. ACF ROs review grantee A-133 single audit reports for unqualified opinions, questioned costs, program findings, financial ratios and overall financial viability. The resolution process may include site visits to verify correction. Audit findings may result in return of Federal funds, revision of expenditure reports, and/or confirmation of corrective action by the grantee for management or non-monetary finding. As of September 30, 2005, 279 of the ACF audits in the resolution process were Head Start audits. (also see 3.3)

Evidence: -GAO-05-176, "Comprehensive Approach to Identifying and Addressing Risks"
-OMB A-133
-Head Start Compliance Supplement
-PRISM monitoring tool
-TA Provider contract
-Sample of annual budget
-Sample of TA Plan
-Sample of community assessment
-Sample PIR data form

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Head Start collects a wide range of grantee performance data on an annual basis, which is available to the public online and by request. Head Start collects annual performance data from all its programs through the Program Information Report (PIR). Data collected include: Child enrollment statistics, program services provided, staff numbers, qualifications, and salaries, health and medical information, child care services, services for disabled children, and Head Start family information. These data are collected at the program level and are made available to anyone requesting them, including members of the grantee's community, other Federal agencies, and Congressional members and staff. These data are aggregated and included in the statutorily mandated Biennial Report to Congress. In addition, any performance data on Head Start acquired through ACF research studies, including FACES, the Head Start Impact Study, Early Head Research and Evaluation Project, and others, are available at ACF's Office of Planning, Research, and Evaluation website. Data on Head Start grantees collected through the on-site monitoring system (PRISM) is also made available, upon request, to anyone requesting such data. Requests may be made in writing to the Office of Head Start. Aggregate PRISM data is also reported in the Biennial Report to Congress, which is available to the public online. While the National Reporting System (NRS) is still being refined (as noted in question 3.1), a Report to Congress on the status of the project was issued in February 2005. This lengthy report reviews the methodology and results of the NRS, including program performance on the Congressionally mandated early learning indicators of vocabulary, letters identification, and early math skills. Each program also receives information on its performance relative to the performance of all Head Start program nationwide.

Evidence: -Program Information Report (PIR) - sample of individual program report
-2003 Biennial Report to Congress
-ACF sponsored Head Start research studies, including FACES, Head Start Impact Study, Early Head Research and Evaluation Project, and others
-PRISM report sample
-Report to Congress on NRS Findings

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: Head Start (HS) is on track to meet the majority of its long-term performance goals, according to data from the NRS, FACES and the EHS Research and Evaluation Project. Results are shown Program Performance Measures section of the PART submission. HS has and will continue to commit substantial resources and design systems to ensure that ongoing progress toward meeting these long term performance measures. Measure #1: Increase the percentage of programs in which children average fall-spring gains of at least 12 months in word knowledge (PPVT) in the National Reporting System (NRS). Measure #2: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring by the NRS. Measure #3: Increase the percentage of programs in which children average fall-spring gains of at least 4 counting items in the National Reporting System. As shown in the Performance Measures Section, HS has demonstrated significant progress is on track to meet each of the long-term cognitive measures, although there is no comparable data yet available for Long-term Measure #1. These gains are likely a reflection of the intentional and intensive efforts made by HS programs to ensure that HS children develop early literacy and math skills (see below). Continued gains are expected as HS continues to focus its efforts on these skills. Measure #4: Increase percentage of parents of children in their pre-kindergarten Head Start year who report reading to child three times per week or more, as measured in the Spring of their pre-kindergarten Head Start year. As shown in the Performance Measures Section, HS has increased the percentage of parents reading to their children three or more times a week from 68% to 74% between Spring 2001 and Spring 2004. This is likely the result of the intensive efforts to improve early literacy activities. HS anticipates continued gains as it continues to focus T/TA resources in this area. Measure #6: Increase the percentage of Early Head Start children completing all medical screenings. This is a new measure for EHS and performance data is only available for 2004 and 2005. In 2005 (baseline), 82% of EHS children received all required screenings within the 90 days required by HS regulations. The 2007 target has been set at 85%; the 2010 target is 91%--which is a higher rate of improvement than was evident between 2004 and 2005, where a .9% improvement was seen. OHS has invested substantial resources to improve program performance in achieving program goals, including by targeting resources and TA; providing oversight; working with state partners; improving the PRISM monitoring system; implementing the National Reporting System; enforcing the HS Management Initiative; improving teacher qualifications; and conducting a number of early learning and literacy initiatives including the Family Literacy Cooperative Agreement, Project STEP Mentor Coaching, Parent Mentoring, and the Head Start Leaders Guide to Positive Child Outcomes to ensure that teaching staff are using best-practices teach early learning, literacy and numeracy skills. (see 2.5.)

Evidence: -The Center on the Social and Emotional Foundations for Early Learning
-GPRA performance report
-Other evidence of achieving LT goals (sources that document achievement toward goals even if the data supports measures not used in annual goals)?? FACES
-Other evidence of achieving LT goals (sources that document achievement toward goals even if the data supports measures not used in annual goals)??Impact Study
-Other evidence of achieving LT goals (sources that document achievement toward goals even if the data supports measures not used in annual goals)??EHS Research & Evaluation

Does the program (including program partners) achieve its annual performance goals?

Explanation: Head Start has achieved or exceeded its annual performance measures for each measure in 2006. As discussed in depth in question 2.5, the Office of Head Start intentionally targeted resources, training, and technical at early learning skill development (i.e. letter identification), teacher qualifications, and under enrollment to ensure that annual performance measures would be met and that there will continue to be improvement in the performance measures over time. The performance measures section shows progress on each measure. Annual Performance Measure 1: Increase the percentage of programs in which children on average can identify 10 or more letters of the alphabet as measured in the Spring of the pre-kindergarten Head Start year by the National Reporting System. According to FACES data, Head Start children are "closing in" on the gap between their performance and national norms, coming closer to national norms for letter recognition. The NRS also shows significant progress in achieving this annual measure. While data for Spring 2006 is not yet available, the trend toward improved performance is clear. Annual Performance Measure 2: Increase the percentage of teachers with AA, BA, Advanced Degree, or a degree in a field related to early childhood education. Head Start has achieved and significantly surpassed its annual goals for this measure??a reflection of the Office of Head Start's effort to enhance teacher credentials nationwide. Annual Performance Measure 3: Increase the percentage of Early Head Start children completing all medical screenings. Early Head Start has made progress in this measure, increasing the percentage of children receiving all medical screenings from 81% in 2004 to 82.10% in 2005. Continued progress is expected (targets are set at 83% in 2006, 85% in 2007 and 87% in 2008); however, there may be a realistic ceiling to the percentage of children that receive all screenings based on several reasons, including lack of health insurance, lack or transportation, lack of access to medical providers, and the fact that even with information about the benefits of screenings some parents do not want their children to receive screenings. Annual Efficiency Measure: Decrease under enrollment in Head Start programs, thereby increasing the number of children served per dollar. Head Start has met is goal for 2005 for under enrollment, and greatly exceeded it. 2005 under enrollment was 2.8%. The Office of Head Start believes this is a result of clarifying the procedures for counting vacancies in the program, reiterating the contractual obligations of grantees, and undertaking firm and decisive action with grantees with chronic under enrollment problems.

Evidence: -GPRA performance report - word knowledge
-GPRA performance report - mathematical skills
-GPRA performance report - letter identification
-GPRA performance report - % of grantees meeting or exceeding NRS targets
-GPRA performance report - parents who report reading to child three times per week or more
-GPRA performance report - percentage of teachers with AA, BA, Advanced Degree, or a degree
-GPRA performance report - average lead teacher score on an observational measure of teacher-child interaction
-Additional historical performance data which shows progress towards annual measures from FACES
-Additional historical performance data which shows progress towards annual measures from Impact Study
-Additional historical performance data which shows progress towards annual measures from EHS Res & Eval
-Center for Family Literacy - goals and participation info (HSB's and programs' commitment to working toward annual performance measures)

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The Office of Head Start has demonstrated improved efficiencies and cost effectiveness in several areas. In FY 2005, Head Start continues to serve the same number of children it did in FY 2004, despite receiving an increase well below the rate of inflation. Moreover, in FY 2006, Head Start, with a 1% funding decrease from FY 2005, will continue to maintain FY 2005 enrollment levels. Other measures taken to improve program efficiency and cost effectiveness include decreasing under enrollment (Head Start's annual efficiency measure), reducing erroneous payments to grantees, and developing cost efficient means for information distribution to grantees and Head Start partners. -Decreasing the Rate of Underenrollment: Head Start's annual efficiency measure examines under enrollment in Head Start programs. Because grantees range in size from super-grantees with multiple delegate agencies to individual centers, a national under enrollment rate is a better illustration of under enrollment than the proportion of grantees meeting enrollment targets. The Office of Head Start has implemented management practices designed to achieve efficiency gains by reducing under enrollment of children in Head Start programs. Underenrollment is the gap between the number of funded children compared to the actual number of children served. In FY 2004, Program Information Report (PIR) data, as well as non-compliance findings from on-site program reviews, indicated an increasing number of grantees with problems maintaining their full funded enrollment. The most recent data available indicate that, during the 2005 program year, Head Start grantees had, on average, not enrolled 2.8% of the children for which they had been funded to serve. This is a significant improvement in efficiency in just one year, dropping from 4.4% in 2004, or a decrease of approximately 15,000 vacancies. (It is important to note that the definition of a vacancy is a funded space for a child that has not been filled for 30 days; thus, the total number of vacancies includes vacancies that may have been open for several months or even the full year as well as vacancies of, for example, 31 days.) -Reducing Erroneous Payments to Grantees In FY 2004, the Office of Head Start undertook a new initiative to determine whether and to what extent Head Start programs were serving ineligible children??that is, children from families with incomes above the poverty line and in excess of the 10% "over-income" children that all programs are allowed to serve. In FY 2005, the erroneous payment error rate decreased by more than 60 percent to 1.5%. This reduction means that the number of ineligible children dropped from approximately 35,000 such children in FY 2004 to 14,000 in FY 2005. In other words, in FY 2005 21,000 more income eligible children were served in Head Start than was the case in FY 2004, a significant increase in the program's efficiency. -Other Projects Leading to Greater Efficiency The Office of Head Start pursues cost efficiency in a number of other ways. Three examples that assist grantees in meeting Head Start requirements and improving the quality of services to children and families include the development of an online knowledge management resource that will be rolled out to the full Head Start community in 2006; the Fiscal Assistant, an online resource for fiscal management and grants administration resources as well as the Head Start Program Performance Standards and other required Federal regulations, which provides Head Start grantees with an efficient source of information on assistance for resolving, designing, or implementing fiscal and grants management policies, procedures, and requirements; and the development of a Risk Assessment Strategy for the monitoring system that, when operational in FY 2007, will allow OHS to identify grantees that seem to be potentially at risk.

Evidence: -General program statistics on # served & budget for FY2003, 2004, and 2005
-HSB response to reducing erroneous payments
-Report on erroneous payment study results
-ACF Program Instruction ACYF-HS-PI-04-03 - Achieving and Maintaining Full Enrollment (November 30, 2004)
-PIR data on underenrollment for 2004 and 2005
-Website for Early Childhood Learning and Knowledge Center
-Website for Fiscal Assistant

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The Head Start Impact Study (HSIS) provides some information about the effect of HS versus other early childhood (EC) programs. Based on HSIS data, the HS program is comparable, and in many cases superior, to related programs. (Note: the PART guidance does not require that programs show they are better than similar programs, but rather "comparable".) After 1 year, HS was found to have positive impacts, relative to the control group, on: vocabulary (for 3-yr olds); pre-reading and pre-writing skills and parents' report of children's literacy skills (for 3- & 4-yr olds); problem behaviors (3-yr-olds & English-speaking 4-yr-olds); parents' reports of access to dental care; and parent reports of children's health status (3-yr olds). Compared to children in other center-based care, HSIS found that HS centers had more positive ratings of teacher-child interaction, more often used an instructional curriculum and activities to enhance children's skills, and had higher scores on the ECERS-R--a measure of quality in the child care environment. This important finding demonstrates that, at the grantee level, HS programs are of higher quality than similar programs. Several differences in HS's design and the design of HSIS are important to note: -Few EC programs have goals or comprehensive services comparable to HS -HS children come from more disadvantaged backgrounds where known risks to educational achievement are prevalent (see 1.2). The typical HS child enters HS at about the 16th percentile in vocabulary and early writing skills, 31st percentile in letter recognition, and 21st percentile in early math (FACES). -HSIS results reflect only 1 year of HS--no information exists about the impacts on 3-yr-olds who completed the whole program, or on 1st grade impacts since HSIS is still in progress -HSIS includes a great deal of "crossovers"--nearly 1/5 of the children in the control group received HS services from another provider -HS has been a leader in EC by promoting comprehensive services and high quality instruction, which would lead HSIS to understate the true impact of HS (the 60% of the control group children who participate in other EC programs and settings may have benefited from HS's leadership in EC) HSIS is ground-breaking in the field of EC education in its use of a nationally representative sample within a randomized control design. While admirable, this methodological rigor requires consideration when comparing HSIS findings to other EC program evaluations. For example, state studies of pre-K programs have shown short-term cognitive benefits of universal pre-K, sometimes with larger effect sizes than HSIS, yet as Thomas D. Cook, an expert in research methodology and EC, noted, these studies "creamed" pre-K programs, selecting only those that were of the highest quality for evaluation. Unlike the nationally representative HSIS, these state pre-K studies likely overstate effects. These studies also examine only a limited range of outcomes and do not evaluate effects on children's socio-emotional development or other key outcomes that affect school success. Finally, these studies are not directly comparable to HSIS: they include children from all socioeconomic backgrounds, and fewer children in control groups participate in group child care settings. However, other EC programs, such Even Start and Comprehensive Child Development Program, evaluated with rigorous designs and populations more comparable to HS's, have found many fewer effects than HSIS. The EHS Research & Evaluation Project found a broad pattern of positive impacts for children and parents. Children benefited in social emotional, language and cognitive development. EHS parents were more supportive of children's emotional, cognitive and language development and were more likely to be in education or job training. Two years after completion of EHS, some impacts remained, particularly in children's social emotional development and parenting.

Evidence: -Impact Study - comparative data
-FACES
-EHS Research and Evaluation Project findings
-Even Start information
-Thomas D. Cook, "Pre-K Programs: Which Ones Make a Difference?" Institute for Policy Research Briefing: Children's Achievement: What Does the Evidence Say About Teachers, Pre-K Programs, and Economic Policies. http://www.northwestern.edu/ipr/events/briefingmay06-cook/slide1.html

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: A number of independent evaluations indicate that Head Start is effective and is achieving results. The three major studies include the National Impact Study, FACES, and the Early Head Start Research and Evaluation Project. Their findings are presented below. -National Head Start Impact Study, First Year: Preliminary Results from the First Year of the National Impact Study report positive child and family outcomes in the Cognitive Domain, the Social-Emotional Domain, the Health Domain, and the Parenting Practices Domain. -Family and Child Experiences Survey (FACES) According to the FACES study, Head Start classrooms rate higher than other preschool programs using the Early Childhood Environment Rating Scale (ECERS), which measures a variety of classroom characteristics related to effectiveness. FACES also showed that a large number of programs collaborate with pre-kindergarten programs in their communities. FACES results from the 1997 and 2000 cohorts also indicated that child outcome gains were evident in vocabulary knowledge and early writing skills, social skills and reduced hyperactive behavior, and significant gains in English vocabulary among Spanish-speaking children without declines in their Spanish vocabulary skills. The gains that children made while in Head Start were of a level predictive of their achievement levels by the end of kindergarten. -Early Head Start Research and Evaluation Findings Early Head Start programs produced statistically significant, positive impacts on standardized measures of children's cognitive and language development; favorable impacts on more aspects of social-emotional development at age 3 than at age 2; significant favorable impacts on a wide range of parenting outcomes; significant positive impacts on parents' participation in education and job training activities; and favorable impacts of factors related to positive relationships between children and fathers.

Evidence: -Head Start Impact Study
-FACES
-Early Head Start Research and Evaluation Study and Survey