Detailed Information on the
Senior Community Service Employment Program Assessment

Continue to strengthen program accountability through common performance measures, including developing a new measure to gauge cost-effectiveness.

Publishing a final rule to implement the 2006 OAA amendments.

Adopting efficiency measures that are linked to performance outcomes, account for all costs, and facilitate comparisons across Department of Labor training and employment programs.

Continue to award national grants competitively to strengthen program design and service delivery.

Ensure that the annual and long-term performance goals established for CSEOA under the new common measures are sufficiently challenging.

Publish a proposed rule to implement the 2000 OAA amendments.

Providing software to facilitate reporting by grantees through their existing systems and developing an internet-based reporting system to facilitate grantee reporting and improve accuracy.

Publishing a proposed rule to implement the 2006 Older Americans Act amendments.

Measure: Entered Employment: Percentage of program participants employed in 1st quarter after program exit (Note: New measure; Targets to be determined. 2002 shows performance against a similar previous measure.)

Explanation:Performance Target: New Measure: result of common measures initiatives; targets to be determined in 2004. Actual Performance:Progress will be reassessed based on DOL's targets and data for new common measures goals.

Measure: Retention in Employment: Percentage of program participants employed in 1st quarter after program exit who remained employed in the 2nd and 3rd quarters after exit (Note: New Measure; Targets to be determined.)

Explanation:Performance Target: New Measure: result of common measures initiatives; targets to be determined in 2004. Actual Performance:Progress will be reassessed based on DOL's targets and data for new common measures goals.

Measure: Earnings: Percentage change in earnings for program participants: (1) pre-enrollment to program exit; and (2) 1st quarter after exit to 3rd quarter after exit. (Note: New measure; Targets to be determined.)

Explanation:Performance Target: New Measure: result of common measures initiatives; targets to be determined in 2004. Actual Performance:Progress will be reassessed based on DOL's targets and data for new common measures goals.

Measure: Cost per participant.

Explanation:

Is the program purpose clear?

Explanation: The program seeks to provide useful community services through temporary, part-time subsidized jobs and foster individual economic self-sufficiency through work experience and job placement in unsubsidized jobs.

Evidence: Authorizing statute (42 U.S.C. 3056d(a), as amended by P.L. 106-501).

Does the program address a specific interest, problem or need?

Explanation: CSEOA is intended to provide a source of short-term income and unsubsidized employment for low-income individuals who are 55 years old or over, the number of which is growing. The program grew out of a pilot project first funded in 1965 to employ poor, chronically unemployed adults, primarily in rural areas. In 1973 the program was refocused on older individuals.

Evidence: Between 1998 and 2008, the number of over-55 individuals is projected to grow by 46%, to 43.7 million. During the same period, the target CSEOA population is expected to grow by 27% (from 8 million to 10 million). [Fredrica Kramer and Demetra Smith Nightingale/Urban Institute, "Aging Baby Boomers in a New Workforce Development System," (Washington, DC: DOL, January 2001)].

Is the program designed to have a significant impact in addressing the interest, problem or need?

Explanation: Program impact data are limited to the rate of placement into unsubsidized employment; there are no data to show the effect of the program on a significant portion of the eligible population, or at a broader level (and relative to other factors and programs).

Evidence:  

Is the program designed to make a unique contribution in addressing the interest, problem or need (i.e., not needlessly redundant of any other Federal, state, local or private efforts)?

Explanation: Similar programs exist (such as the National Senior Service Corps programs, which provide stipended, part-time service opportunities for low-income seniors). The Workforce Investment Act can also serve many of the same individuals. WIA programs provide services, but historically fewer than 5% of participants are over 55. A 1995 GAO report noted duplication within the program, despite regulatory requirements for coordination with other relevant agencies. In most states, both State agencies and nonprofit national sponsors receive CSEOA funding, and there are often multiple national sponsors. There has been no mechanism to ensure coordination of effort, which has resulted in service overlap in some areas, and gaps in others. (The 2000 OAA amendments require the development of State Senior Employment Services Coordination Plans, which could fix these problems.)

Evidence: GAO, Senior Community Service Employment Program Delivery Could be Improved through Legislative and Administrative Actions (November 1995, HEHS-96-4), Program regulations at 29 CFR 89.12. Workforce Investment Act of 1998 (WIA; P.L. 105-220) Information on the National Senior Service Corps programs is accessible at www.cns.gov. When the results of the recent National SCSEP Grantee competition are made final by July 1, 2003, the number of program operators in many counties and in a few states will be reduced. Moreover, one of the amendments to the OAA called for the development of a State Senior Employment Coordination Plan that solicits involvement from all organizations interested in senior employment and could include those in the National Senior Service Corps programs. In addition this State plan addresses gaps and over -service among all SCSEP operators within a State.

Is the program optimally designed to address the interest, problem or need?

Explanation: National program awards (about 78% of program funding) are non-competitively awarded; statute requires contractors to be held at their 2000 level of activity, which in the past has been treated as a hold harmless. The 2000 Older Americans Act (OAA) amendments anticipate competition of funding in cases where contractors repeatedly fail to perform. Broader competition could strengthen program design and service delivery. In December 2002, a broad competitive SGA was issued to compete the entire national grantee portion of the program ($342 million and the positions of 47, 000 low income seniors). New grantees will begin operations on July 1, 2003. Draft regulations were published for comment on April 28, 2003, with the comment period ending on June 12, 2003. Regulations to implement the amendments have not yet been finalized, but are targeted to be in place in Fall 2003. If competition is introduced, and regulations implementing the 2000 OAA amendments are finalized, this answer could change to a "yes."

Evidence: Authorizing statute (42 U.S.C. 3056, as amended by P.L. 106-501). Information on the SGA is accessible at http://wdsc.doleta.gov/seniors/html_docs/OAA_SGA.htm. Draft regulations are accessible at http://wdsc.doleta.gov/seniors/.

Does the program have a limited number of specific, ambitious long-term performance goals that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: CSEOA is part of the Job Training Common Measures initiative Accordingly, the program has adopted four goals that will better measure the impacts of the program and allow comparisons across similar programs. DOL is implementing the common measures and will establish numerical targets over the next year. The draft regulations published on April 28, 2003, include the common measure for earnings increase. In consultation with OMB, CSEOA plans to include the common measures for entered employment and retention as reporting items until the OAA is amended to accommodate these commonmneasures. For PY 2003 - 2004, CSEOA has also set a target for 37% of participants to be placed in unsubsidized jobs, which is higher than any previous program achievement level. DOL will negotiate grantee performance levels for PY03 individually to encourage grantee organizations to focus on performance. The FY04 performance budget also establishes the CSEOA goal: increase the employment, retention, and earnings of individuals participating in the Senior Community Service Employment Program.

Evidence: Information on performance indicators is available in Volume 1 of FY 2004 Budget Justifications for Appropriation Estimates for Committee on Appropriations. Draft regulations and information on the common measures are accessible at http://wdsc.doleta.gov/seniors/.

Does the program have a limited number of annual performance goals that demonstrate progress toward achieving the long-term goals?

Explanation: CSEOA is part of the Job Training Common Measures initiative. Accordingly, the program has adopted four specific long-term goals that will better measure the impacts of the program and allow comparisons across similar programs. DOL is implementing the common measures and will establish numerical targets over the next year. The draft regulations published on April 28, 2003, include the common measure for earnings increase. In consultation with OMB, CSEOA plans to include the common measures for entered employment and retention as reporting items until the OAA is amended to accommodate these common measures. For PY 2003 - 2004, CSEOA has also set a target for 37% of participants to be placed in unsubsidized jobs, which is higher than any previous program achievement level, and which ties to the strategic goal. DOL will negotiate grantee performance levels individually for PY03.

Evidence: The OAA amendments mandate the establishment of certain performance goals, including placement and retention in unsubsidized employment, number of persons served, community services provided, customer satisfaction, and any other appropriate measures. Information on performance indicators is available in Volume 1 of FY 2004 Budget Justifications for Appropriation Estimates for Committee on Appropriations. Draft regulations and information on the common measures are accessible at http://wdsc.doleta.gov/seniors/. A preliminary paper work reduction package has been shared with OMB and substantial refinements to it are being developed.

Do all partners (grantees, sub-grantees, contractors, etc.) support program planning efforts by committing to the annual and/or long-term goals of the program?

Explanation: Grantees must commit to program goals as a condition of their grant award, and the majority of grantees exceed the national goals by substantial amounts. To address under-performance, DOL negotiates corrective action plans with grantees, develops and releases lists of grantees ranked by their unsubsidized placement rate and their participation rate, and holds training to help grantees improve their performance. Historically, there have been no funding consequences for poor performance. The OAA amendments strengthen accountability by mandating numerous performance indicators and requiring recipients to be assessed against, and held accountable for, their performance in these areas. Draft regulations were published on April 28, 2003, and grantees may make comments on them and the proposed performance measures. Additional guidance on the performance measures and the indicators will be published separately in the Federal Register for comment. DOL also expects to provide grantees with software and training that will assist them in reporting.

Evidence: Older Americans Act of 2000 (P.L. 106-501). Draft regulations and information on the common measures are accessible at http://wdsc.doleta.gov/seniors/.

Does the program collaborate and coordinate effectively with related programs that share similar goals and objectives?

Explanation: WIA requires CSEOA coordination with Workforce Investment Act programs (it is a mandatory One-Stop partner under the law), although there is no evidence of effective collaboration. Program has an interagency agreement with HHS' Administration on Aging. However, the program has not collaborated with the Corporation for National and Community Services' Senior Service Corps programs.

Evidence: WIA Section 121(b) (1) (B) (vi). GAO reports that the CSEOA offers special services to older workers, and is accessible to these individuals through the One Stop system. Seventy-eight percent of the local area respondents to a GAO survey said that CSEOA staff were located in One-Stop Centers. (GAO-03-350 page 22).

Are independent and quality evaluations of sufficient scope conducted on a regular basis or as needed to fill gaps in performance information to support program improvements and evaluate effectiveness?

Explanation: A comprehensive study of CSEOA has not been done since 1986. There have been more focused independent evaluations done since then--the last such assessments were in 1995 (GAO) and 1996 (Urban Institute). No impact or outcome evaluations of this program are financed under the long-term research plan. However, DOL has developed a revised evaluation plan that will establish a regular cycle of evaluation for all major job training and employment programs, including CSEOA.

Evidence: Urban Institute (Nancy Pindus and Pamela Holcomb), Analysis of the Impacts of Proposed Legislative Changes in the Senior Community Service Employment Program (March 1996); GAO, Senior Community Service Employment Program Delivery Could be Improved through Legislative and Administrative Actions (November 1995, HEHS-96-4).

Is the program budget aligned with the program goals in such a way that the impact of funding, policy, and legislative changes on performance is readily known?

Explanation: Program funding is linked to outputs, rather than outcomes. Cost per "slot" (unit cost), which is the estimated annualized cost of administrative expenses plus an enrollee's wages, benefits, training, and incidental expenses is divided by the appropriation to arrive at the targeted number of enrollees. Federal administrative costs are not shown alongside programmatic costs; they are carried in ETA's Program Administration budget. In preparing its FY 2004 submission, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels on performance outcomes.

Evidence: In the FY 2004 budget, DOL intends to provide full program costing and integrate performance goals with budget information. This program performance and cost integration needs to permit measurement of the full costs of each program and an assessment of the impact of budget levels on performance outcomes.

Has the program taken meaningful steps to address its strategic planning deficiencies?

Explanation: DOL published draft regulations on April 28, 2003. Among other things, the regulations will establish new performance indicators and sanctions for the program. DOL is also preparing a new reporting system for CSEOA to capture performance information.

Evidence: Draft regulations are accessible at http://wdsc.doleta.gov/seniors/.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: ETA gathers from funding recipients various data, including enrollment, placements and placement rate, types of services, and enrollee characteristics. ETA has taken steps to improve data validation, working with a contractor to set up new reporting systems. Based on the information it collects, ETA determines the need for corrective actions or technical assistance (although historically, low performers have continued to receive awards every year without funding consequences). ETA is strengthening its efforts to improve performance by conducting technical assistance to help grantees--particularly those with low placement rates--boost their unsubsidized placement rate (the goal for which has been raised for PY 2002 - 2004).

Evidence: DOL's Annual Report on Performance and Accountability for FY 2002, which is accessible at: www.dol.gov/_sec/media/reports/annual2002/. Volume 1 of FY 2004 Budget Justifications for Appropriation Estimates for Committee on Appropriations.

Are Federal managers and program partners (grantees, subgrantees, contractors, etc.) held accountable for cost, schedule and performance results?

Explanation: The OAA amendments mandate certain performance indicators (see above) and requires the assessment of funding recipients against those indicators. At the Federal level--the Department of Labor has recently tied its performance goals to performance ratings for managers; this new appraisal system has been cascaded through the Department.

Evidence: Older Americans Act 2000 (P.L. 106-501). Revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001).

Are all funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: For the past 5 years, appropriations have been obligated in a timely manner and expended within a 14-month period. The program reviews obligations and outlays monthly. However, past GAO and DOL OIG findings have revealed inappropriate use of funding. A 1995 GAO report found that most recipients spent above the permissible amount (up to 15%) for administrative activities. In addition, a 2000 OIG report questioned $6 million in costs claimed by one of CSEOA's largest grantees, raising concerns about the purposes for which funds were being spent. ETA has taken some action to address these issues (although it questioned GAO's findings regarding administrative expenditures). The OAA amendments sought to address these concerns by clarifying appropriate administrative expenditures and establishing "responsibility tests" for funding. Once completed, the implementing regulations should reinforce these controls.

Evidence: GAO, Senior Community Service Employment Program Delivery Could be Improved through Legislative and Administrative Actions (November 1995, HEHS-96-4), DOL OIG (March 2000, OA Report No. 18-00-006-03-360. Since the release of the GAO report program grantees have been provided with additional information on cost categorization. The draft regulations published on April 28, 2003, are designed to track the cost categorization approach used under WIA. The legislation and the regulations establish limits on administrative costs, and a minimum that must be spent on the wages of program participants.

Does the program have incentives and procedures (e.g., competitive sourcing/cost comparisons, IT improvements) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program does not have cost-effectiveness or efficiency measures. Once DOL submits completed common measures data that include an efficiency measure, OMB and DOL will work together to craft a methodology for using these data to assess cost effectiveness and this answer could be changed to a "Yes."

Evidence: The program is currently collecting information on under-expenditures from prior appropriations which can be used for incentive grants, technical assistance and training, and other purposes pursuant to OAA section 515(c). A notice was published in the Federal Register on June 5, 2002. When regulations are final additional effort will be made to access cost effectiveness.

Does the agency estimate and budget for the full annual costs of operating the program (including all administrative costs and allocated overhead) so that program performance changes are identified with changes in funding levels?

Explanation: The amount budgeted for the program does not include the cost of Federal staff who administer the program. These staff are funded (and displayed) in the Employment and Training Administration's Program Administration account. Apart from changes in participation levels, there is no sense of how changes in funding will affect performance. Like the rest of DOL, ETA does not have an integrated accounting and performance management system to identify the full cost of achieving this program's performance goals and support day to day operations. In preparing its FY 2004 budget submission, DOL moved closer to providing full program costing and began to integrate performance goals with budget information. However, this program performance and cost integration does not yet permit measurement of the full costs of each program or an assessment of the impact of budget levels on performance outcomes.

Evidence: FY 2003 and FY 2004 Budgets for CSEOA. Budget requests are available at: www.dol.gov/_sec/media/reports/.

Does the program use strong financial management practices?

Explanation: No material weaknesses were identified last year.

Evidence: DOL's FY 2002 Annual Report on Performance and Accountability, which is accessible at www.dol.gov/_sec/media/reports/annual2002/

Has the program taken meaningful steps to address its management deficiencies?

Explanation: DOL has completed work on a proposed rule, which will implement the 2000 OAA amendments. The draft regulations published on April 28, 2003. Among other things, the rule will establish more rigorous and comprehensive performance standards for the program. DOL also hopes to strengthen financial and non-financial performance incentives for grantees. In addition, DOL has recently tied its performance goals to performance ratings for managers; this new appraisal system is to be cascaded through the Department.

Evidence: Revised performance management plans for senior executives (Form DL 1-2059, Rev. 10/2001) and for supervisors and managers (Form DL 1-382, Rev. 10/2001).

Has the program demonstrated adequate progress in achieving its long-term outcome goal(s)?

Explanation: Although the program exceeded its placement goal, its rigor is questionable. CSEOA has adopted four new long-term goals as part of the Job Training Common Measures initiative. With the exception of job placement and unit cost, performance in the Common Measures cannot be assessed with current data. DOL is implementing the common measures and will establish numerical targets over the next year. DOL is developing an updated CSEOA reporting system that will collect base line data for the common measures in PY 2003. DOL will use this information to negotiate and establish new performance levels with individual grantees. OMB will reconsider the answer to this question once this work is complete.

Evidence: FY 2003 Performance Plan; Volume 1 of FY 2004 Budget Justifications for Appropriation Estimates for Committee on Appropriations. The draft regulations published on April 28, 2003 include the common measure for earnings increase. In consultation with OMB, CSEOA plans to include two other common measures - entered employment and retention - as reporting items until the OAA can be amended to accommodate these measures. Draft regulations are accessible at http://wdsc.doleta.gov/seniors/.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Although the program exceeded its placement goal, its rigor is questionable. CSEOA has adopted four new long-term goals as part of the Job Training Common Measures initiative. With the exception of job placement and unit cost, performance in the Common Measures cannot be assessed with current data. However, CSEOA is implementing the common measures and will establish numerical targets over the next year. Along with the development of a system to collect information during PY 03, base line information will be obtained on the new performance measures. DOL will use this information to negotiate and establish new performance levels with individual grantees. OMB will reconsider the answer to this question once this work is complete.

Evidence: FY 2003 Performance Plan; Volume 1 of FY 2004 Budget Justifications for Appropriation Estimates for Committee on Appropriations. The draft regulations published on April 28, 2003 include the common measure for earnings increase. In consultation with OMB, CSEOA plans to include two other common measures - entered employment and retention - as reporting items until the OAA can be amended to accommodate these measures. Draft regulations and information on the common measures are accessible at http://wdsc.doleta.gov/seniors/.

Does the program demonstrate improved efficiencies and cost effectiveness in achieving program goals each year?

Explanation: The program has no cost-effectiveness or efficiency measures, so there is no basis on which to assess this. However, DOL is participating in the development of one common measure of efficiency and cost-effectiveness for the 2004 Budget.

Evidence:  

Does the performance of this program compare favorably to other programs with similar purpose and goals?

Explanation: CSEOA is part of the Job Training Common Measures initiative. Leaving aside differences among the various job training programs being measured (e.g., different populations), the program does not compare favorably. Its placement rate is among the lowest, and it does not measure employment retention or earnings growth.

Evidence:  

Do independent and quality evaluations of this program indicate that the program is effective and achieving results?

Explanation: There have been few evaluations of the program (and no comprehensive evaluations since 1986), but those that have been conducted show some evidence of effectiveness. A 1996 Urban Institute review of past evaluations and surveys (evaluations from 1981, 1985, and a phone survey from 1995) concluded that the program is well regarded and successful, and exceeds its statutory 20% placement goal. A 1995 study by GAO raised concerns about the program (e.g., how funding was used, duplication of effort). More recent evidence of program effectiveness is limited. However, DOL has developed a revised evaluation plan that will establish a regular cycle of evaluation for all major job training and employment programs, including CSEOA.

Evidence: Urban Institute (Nancy Pindus and Pamela Holcomb), Analysis of the Impacts of Proposed Legislative Changes in the Senior Community Service Employment Program (March 1996); GAO, Senior Community Service Employment Program Delivery Could be Improved through Legislative and Administrative Actions (November 1995, HEHS-96-4), DOL OIG (March 2000, OA Report No. 18-00-006-03-360.