Detailed Information on the
AmeriCorps National Civilian Community Corps Assessment

Working with Congress to reform the program to be more cost effective, reduce the number of campuses, and focus on disaster response and preparedness.

Completing a rigorous program evaluation.

Incorporating cost-sharing into the program.

Recruiting and engaging a diverse membership from a wide variety of socio-economic and ethnic backgrounds.

Working with Congress to reform the program to be more cost effective, reduce the number of campuses, and focus on disaster response and preparedness.

Measure: Number of AmeriCorps members who use their Education Award

Explanation:This measure relates to the usage of the education award and how it helps enhance member skills and facilitiate the transition into higher education.

Measure: Cost per Member Service Year

Explanation:Controlling cost per member is important to NCCC and will be a proxy for the efficiency in generating civic leaders.

Measure: Percentage of NCCC members who complete their service.

Explanation:Completion of service is an indicator of the programs success. NCCC anticipates a decline in members completing their service given the challenges of accomodating a more diverse corps and meeting its statutory target of 50% enrollment of economically disadvantaged youth.

Measure: Percentage of NCCC members who accepted public service employment (including government and non-profit work) one year after completing AmeriCorps service.

Explanation:This measure addresses NCCC's purpose of developing leaders who demonstrate their ongoing civic engagement.

Measure: Number of volunteers generated/coordinated per NCCC member.

Explanation:This measure relates to NCCC's statutory charge "to significantly increase the support for national service and community service by the people of the United States."

Is the program purpose clear?

Explanation: The AmeriCorps NCCC mission is to strengthen communities and develop leaders through team-based national and community service. The NCCC program was created in 1993 as a demonstration program charged with determining (1) whether federally funded residential service programs can significantly increase the support for national and community service, (2) whether such programs can expand the opportunities for young men and women to perform meaningful, direct, and consequential acts of community service in a manner that will enhance their own skills while contributing to their understanding of civic responsibility in the United States, (3) whether retired members of the armed forces can provide guidance and training under such programs that contribute meaningfully to the encouragement of national service and, (4) whether domestic national service programs can serve as a substitute for the traditional option of military service.

Evidence: The National and Community Service Act of 1990, as amended (42 U.S.C 12611; Subtitle E, Sec. 151). AmeriCorps National Civilian Community Corps Strategic Plan 2002-2007 (November 2001).

Does the program address a specific and existing problem, interest, or need?

Explanation: A study funded by the Pew Charitable Trusts reveals that nearly six in ten youth between the ages of 15 and 25 are "completely disengaged from civic life." NCCC was created to help address this problem by providing a civilian complement to military service. NCCC is a 10-month residential national service program for young women and men between the ages of 18 - 24. Members serve on environmental projects, education, public safety, disaster relief, and other community needs across the country. Members live at one of five regional campuses located in Denver, CO, Charleston, SC, Sacramento, CA, Perry Point, MD, and Washington, DC. Retired military currently comprise 21 percent of the NCCC staff, including three out of five NCCC Region Directors.

Evidence: "The Civic and Political Health of the Nation: A Generational Portrait," September 2002, The Center for Information & Research on Civic Learning & Engagement.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: While NCCC is the only residential service program operated directly by the Corporation, it's goals of encouraging volunteerism, promoting civic participation and meeting human needs through community volunteers, are shared by many state, local, private and non-profit organizations. The residential design was created to provide NCCC with the flexiblity to respond to disaster relief activities. However, in FY 2004 only 7 percent of a members' service was dedicated to disaster relief. Similar to the AmeriCorps State and National program, the majority (93 percent) of an NCCC members' time is focused on non-disaster relief activities such as tutoring and mentoring youth, building homes for low-income families and building trails in national parks. Federal residential programs such as Job Corps and the National Youth ChalleNGe program have similar program characteristics including a community service focus. In addition, there are several efforts underway at the Federal, State, and local level that provide community service, civic engagement and first responder opportunities. One example is the Citizen Corps, created following the attacks of September 11th to help coordinate volunteer activities that will help make communities stronger, safer and better prepared to respond to emergency situations. Currently, there are 55 State/Territority Citizen Corps Councils and 1,662 county, local and tribal Citizen Corps Councils serving 67 percent of the U.S. population. NCCC members work with Citizen Corps through its Memorandum of Understanding with FEMA and Red Cross training in disaster relief services.

Evidence: "AmeriCorps*NCCC Community Outcomes," WESTAT March 2002; CNCS Performance and Accountability Report 2004. Citizen Corps "Uniting Communities Preparing the Nation," on www.citizencorps.gov.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: Established on the military structure of the 1940's Civilian Conservation Corps, NCCC was charged with determining if federally funded residential service programs can significantly increase the support for national and community service. Recent studies have focused on member satisfaction and future volunteering behaviors but have not assessed the effectiveness or impact of the residential design. Currently, over 74,000 AmeriCorps members are enrolled annually in non-residential community service and more than 1 million are engaged in service learning, suggesting that there are viable alternatives to NCCC's high-cost residential model. The program is extremely expensive on a per- participant basis, with an average federal cost per member of $27,859 and annual enrollment levels of 1,000 participants. NCCC is by far one of the most expensive Corporation-run service programs that supports the least number of participants compared to the two other AmeriCorps programs (AmeriCorps State/National and AmeriCorps VISTA). The cost differences among the AmeriCorps programs is related to program design. NCCC is funded entirely by the Corporation and does not have a cost-sharing component that is encouraged in the other AmeriCorps programs but they do encourage NCCC sponsors to contribute in-kind support, such as housing, food, and other resources. Until a comprehensive cost benefit evaluation is conducted, the full effectiveness of the NCCC program cannot be assessed.

Evidence: """Serving County and Community: A Longitudinal Study of Service in AmeriCorps,"" Abt Associates, Inc., December 2004. AmeriCorps Funding and Benefits (GAO Report B-282553; February 2000) "

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The authorizing statute charges NCCC with ensuring that "participants are from economically, geographically, and ethnically diverse backgrounds "??and "to the extent practicable, at least 50 percent of the participants in the national service program shall be economically disadvantaged youths." While the NCCC includes wide geographic representation. Only 21 percent of participants are economically disadvantaged youth and 15 percent are ethnic minorities. In late FY 2004, NCCC developed a plan to recuit a more diverse corps of members.

Evidence: The National and Community Service Act of 1990, as amended; NCCC Demographic Profile, ASPEN contractors; 2004 National Benchmarking Surveys, Urban Institute; CNCS e-Grants database.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The program does not meet the criteria for a "yes" answer. NCCC included ten long-term measures in the 2002-2007 Strategic Plan but additional work is needed to identify a limited number of long-term outcome measures that gauge progress toward achieiving the program's purpose of "strengthening communities" and "increasing civic responsibility". NCCC is currently working on implementing long-term measures that directly relate to the program's purpose and mission, such as the "Percent of NCCC members who accepted public service employment (including government and non-profit work) one-year after completing AmeriCorps service".

Evidence: NCCC 2002-2007 Strategic Plan; Report from May 2005 NCCC Directors Planning Retreat.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: NCCC is in the process of identifying a limited number of long-term outcome measures that gauge progress toward achieiving the program's purpose of "strengthening communities" and "increasing civic responsibility". Targets and baselines will be developed accordingly.

Evidence: NCCC 2002-2007 Strategic Plan; Report from May 2005 NCCC Directors Planning Retreat.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: NCCC does not meet the criteria for a "yes" answer. The program has established one adequate annual outcome measure (see Measures Tab) and has two output measures. NCCC needs to develop additional annual outcome measures that directly link to the programs purpose of "strengthening communities" and "increasing civic responsibility".

Evidence: "NCCC 2005 Operating Plan; FY 2006 Congressional Justification Example of Annual Outcome Measure: (1) Percent of current members successfully obtaining post secondary credit through service in NCCC. Example of Annual Output Measure: (2) Number of NCCC members who use their education award.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: As noted in question 2.3, the NCCC program does not have adequate annual outcome measures. As NCCC develops new annual outcome-based measures, they will be able to set ambitious annual targets for these measures.

Evidence: "NCCC 2005 Operating Plan; FY 2006 Congressional Justificaton

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The NCCC project application specifies the priority areas for service. Community sponsors must commit to specific training and must apply service learning methodology to contribute to members' development of civic responsibility and skill building. Their applications are reviewed by NCCC project developers to determine whether or not they include essential elements such as a clear set of goals and output measures. CNCS conducts site visits to new community partners before the first team of NCCC members arrives for service and are evaluated for preparedness. Random site visits are conducted by NCCC staff during project implementation to ensure program goals are being met. After project completion, members report on the benefits they received from the project and on their assessment of community benefit and project outputs. The community sponsor must review and verify the members' report. The report is then entered into the Corporation's e-Grants database.

Evidence: NCCC Project Application; NCCC Project Completion Reports; CNCS Egrants database

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: NCCC was established as a demonstration project to determine (among other factors) whether retired military members can provide guidance and training that encourage national service. Over its eleven-year history, NCCC has been the subject of a number of independent evaluations that have primarily assessed member satisfaction and volunteer behaviors. A comprehensive cost benefit evaluation or audit of the program analyzing the four factors outlined in the authorizing statute has never been conducted. The NCCC program has relied on outcome analysis provided through program data, member surveys including longitudinal member surveys and project sponsors. In addition, the OIG has never conducted an evaluation or audit of the program.

Evidence: "Progress Report on the Implementation of the AmeriCorps*NCCC," 1995, DynCorp; "AmeriCorps*NCCC Accomplisments Report for Class IV," May 1999, WESTAT; "Two AmeriCorps Programs' Funding and Benefits," February 2000, GAO; "AmeriCorps*NCCC Community Outcomes," March 2002, WESTAT; "A Longitudinal Study of Service in America," December 2004, Abt Associates, Inc.; National Performance Benchmarking Surveys, March 2005, Urban Institute.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: NCCC's budget request includes output and outcome measures, but does not present resource needs for its campus or program needs in a transparent manner. NCCC needs to develop a formal capital asset plan to identify operations, rehabilitation and acquisition activities associated with its five residential campuses. NCCC plans to develop and submit a formal capital asset plan for the 2007 budget.

Evidence: NCCC 2005 Operating Plan; FY 2006 Congressional Justifcation

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: NCCC has begun to take steps to address its deficiencies through its strategic planning process, but has not initiated work on a formal capital asset plan. NCCC plans to develop a formal capital asset plan for the 2007 budget.

Evidence: Report from the 2005 NCCC Directors Planning Meeting; 2005 NCCC operating plan; FY 2006 Congressional Justification

Has the agency/program conducted a recent, meaningful, credible analysis of alternatives that includes trade-offs between cost, schedule, risk, and performance goals, and used the results to guide the resulting activity?

Explanation: The primary asset in which NCCC invests is its five residential campuses. However, NCCC does not have a formal capital asset plan. Although the program has existed for 10 years, it only recently established a working group to examine the campus needs and create a schedule for repairs. NCCC has not yet identified an outside party to review and validate the analysis. NCCC plans to develop a formal capital asset plan that documents the operations, rehabilitation and acquisition activities associated with its five campuses for the 2007 budget.

Evidence: FY 2006 Congressional Justification; FY 2006 Questions for the Record submitted to Congress in response to inquiries on the NCCC program structure and campus needs.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Performance information is collected from sources internal and external to NCCC. Internally, NCCC headquarters conducts regular reviews of field operations and program performance. Annual member end-of-year surveys and training evaluations have been administered as standard operating procedure. Members report project outputs through their project completion reports, which the community partnering organization must verify. NCCC participates in the annual external performance benchmarking surveys conducted by the Urban Institute and the imminent implementation of a customer satisfaction survey using the American Customer Service Index as a standard. The data are regularly monitored and used to help guide project, campus and program management, as well as strategic planning and budgeting.

Evidence: NCCC member training evaluations; Member training needs assessments; In-process reviews conducted of field operations by NCCC HQ; NCCC team weekly progress reports; NCCC project completion reports, Egrants database; CM end-of-year surveys; CNCS bench mark surveys; and Customer Satisfaction Surveys.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: In July 2004, CNCS implemented a new management appraisal system geared towards performance outcomes and accountability measures. All NCCC managers are subject to the new review, and their 2005 work plans were developed in light of the new system. As partners, NCCC project sponsors must detail performance expectations in their applications, submit work and training schedules and sign an agreement to fulfill the responsibilities detailed in their proposal. For example, nearly all community partnering organizations provide housing for members who are serving with them and are not close enough to stay at the regional NCCC campus. Many community partners provide member meals. NCCC staff members conduct periodic site visits to gauge community partners' fulfillment of commitments. NCCC teams give formal debriefs of projects to the NCCC staff and address project management issues through corrective action.

Evidence: CNCS management appraisal system; NCCC management workplans; NCCC project application and sponsor agreement form, Sponsor Evaluation form.

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: NCCC prepares an annual operating plan that details all program obligations. The operating plan is compared to actual expenditures to determine the quality of planning and to ensure that funds were properly obligated and expended. As a residential program, a high percentage of NCCC's costs are fixed (rent, healthcare, etc.) and predictable. Unanticipated surplus of funds infrequently arises predominately through unexpected levels of member attrition. Because NCCC has two-year obligational authority, unobligated balances of annual appropriations are carried over to the following program year. The amount of carryover funds is typically minor, for example in FY 2004 only $55,000 was carried over into FY 2005.

Evidence: NCCC 2005 Annual Operating Plan

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: NCCC has not demonstrated effiencies or cost effectiveness in controling costs per participant. In FY 2000, GAO issued a report on the funding and benefits of AmeriCorps and compared NCCC to Job Corps. The report found that cost per participant were higher for Job Corps ($28,933) than those of NCCC ($23,426) due to the array of postservice assistance Job Corps offers. Today, the cost per NCCC participant has increased to $27,859. The NCCC program recently established the following efficiency measure: "Cost per member", to help evaluate the program's success of improving cost effectiveness.

Evidence: "Report on AmeriCorps*NCCC: Meeting the Nation's Disaster and Public Safety Needs," March 2005, Prepared for Congress in response to requirement in Senate Report, 108-353 for FY 2005 Appropriations.

Does the program collaborate and coordinate effectively with related programs?

Explanation: The NCCC program is part of the CNCS integrated budget and program planning process. NCCC priorities are coordinated with other CNCS programs through an agency-wide logic model. All NCCC projects are carried out in collaboration with community or government partners. NCCC has developed large-scale service partnerships with Habitat for Humanity and the Boys and Girls Club. In addition, NCCC has a formal agreement with the American Red Cross and partners with FEMA through a Corporation Memorandum of Understanding. These parternerships ensure that NCCC is integrated into the national disaster response network and thus can fulfill its responsibilities effectively. Since its inception, NCCC members have responded to every major natural disaster and coordinated disaster reponse with State and local officials. An initiative is under development with Job Corps to transition appropriate Job Corps graduates into the NCCC program as a leadership development opportunity.

Evidence: CNCS and NCCC '05 operating plan & '06 budget submission; CNCS/FEMA MOU; NCCC/Red Cross SOU

Does the program use strong financial management practices?

Explanation: NCCC does not have a record of any significant negative audit findings. The regional and national resource managers have bi-weekly conference calls to discuss financial accountability, procedures, and possible cost-saving measures. The internal process review (IPR) is used to review program field operations, including financial management, on a quarterly basis.

Evidence: Internal Process Review reports; 2004 Performance Accountability Report (pages C-225-229 and D-257-264).

Has the program taken meaningful steps to address its management deficiencies?

Explanation: NCCC has a formal review process called the In-process review (IPR). The IPR is used to review program management practices, to identify deficiencies, and to specify required steps to address the deficiencies. The implementation of the Corporation-wide Management Appraisal System in 200X has resulted in management work plans more closely tied to program performance measures in the PAR, annual operating plan, and budget submission. Employee work plans have also been adjusted to reflect program priorities and incorporate targeted outcomes. A process is currently underway to identify "like" positions across NCCC campuses and to standardize major elements of the positions' workplans, including performance tagets. Senior NCCC staff and senior management convene bi-weekly conference calls to review management issues. Bi-weekly conference calls of headquarters and field staff are held to review progarm performance.

Evidence: IPR form and sample reports; management appraisals and employee work plans.

Is the program managed by maintaining clearly defined deliverables, capability/performance characteristics, and appropriate, credible cost and schedule goals?

Explanation: NCCC does not have a formal capital asset plan and does not manage its capital according to established federal "best practices" such as the Capital Programming Guide in OMB Circular A-11. Instead, NCCC uses an informal process through a Faciilities Infrastructure Form to identify facility needs across years and according to priority and type of activity. The form includes a description of how the regional campuses arrived at an estimation for the costs of the facility work. To improve its facilities planning, the Corporation will develop a formal capital asset plan that documents the operations, rehabilitation and acquisition activities associated with the five campuses for the 2007 budget.

Evidence: FY 2006 Congressional Justification; FY 2006 Questions for the Record (House Appropriations Committee)

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: As noted in questions 2.1 and 2.2, additional work is need to identify a limited number of long-term outcome measures. Currently, there is minimal data to adequately measure NCCC' progress toward its mission and strategic goal of "strengthening communities" and "increasing civic responsibility".

Evidence: NCCC 2002-2007 Strategic Plan; Report from May 2005 NCCC Directors' Planning Retreat.

Does the program (including program partners) achieve its annual performance goals?

Explanation: As noted in question 2.3 and 2.4, the program only has one adequate annual outcome measure. As the program develops new outcome measures, they will be able to better demonstrate progress toward achieving their annual performance goals.

Evidence: 2004 and 2005 National Performance Benchmarking Effort, Urban Institute.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: NCCC has not demonstrated effiencies in controlling costs per participant, they recently developed a cost efficiency measure as a proxy for efficiency. Compared to other federally funded residential programs, NCCC is far more expensive. For example, the average per day program cost for an NCCC participant with an average 10-month length of stay is estimated at $72.00, compared to $27.00 for the National Youth ChalleNGe program, a 17-month program that has a 5 1/2 month quasi-military residential phase followed by a 12-month post-residential phase. In FY 2000, GAO compared the costs and benefits between NCCC and the Job Corps program. The report found the costs per participant for Job Corps ($28,933) was significanlty higher than NCCC ($23,426) due to the array of postservice assistance Job Corps offers. Today, the cost per NCCC participant has increased to $27,859.

Evidence: Department of Defense---National Guard Youth Challenge Program; 2004 Performance and Accountability Highlights, www.ngycp.org. Job Corps FY 2006 Congressional Justification. CNCS Questions for Record (Data on Cost Per Participant)

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: A comprehensive evaluation comparing NCCC's effectiveness to other residential programs has neven been conducted. Evaluations have focused mainly on member satisfaction. In FY 2000, GAO conducted a report on the funding and benefits of NCCC compared to those of the Job Corps program and new military recruits. The report found that the benefits and post-service assistance in Job Corps and the military far exceeded those provided by NCCC, but did not focus on the program's effectiveness in creating lifelong civic engagement or the impact of the program's residential structure. Other program evaluations and data demonstrate some progress in NCCC's success. For example, eighty-three percent of a random sampling of NCCC community sponsors who have used other corps teams rated NCCC teams as better, considering such factors as being disciplined, being productive, motivated and work ethic.

Evidence: 2004 National Performance Benchmarking Effort, Urban Institute. "Two AmeriCorps Programs' Funding and Benefits," February 2000, GAO Report B-282553.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Independent evaluations over the last eleven years have focused on member satisfaction and volunteer behaviors. A recent report from the Longitudinal Study of Service in AmeriCorps noted statistically significant positive impacts on NCCC members' connection to community, local civic efficacy, participation in community-based activities, and future volunteering behaviors. However, the final effects of participation may not be known for several years. An OIG audit or evaluation of the program has never been conducted.

Evidence: "Progress Report on the Implementation of the AmeriCorps*NCCC," 1995, DynCorp; "AmeriCorps*NCCC Accomplisments Report for Class IV," May 1999, WESTAT; "Two AmeriCorps Programs' Funding and Benefits," February 2000, GAO Report B-282553; "AmeriCorps*NCCC Community Outcomes," March 2002, WESTAT; "A Longitudinal Study of Service in America," December 2004, Abt Associates, Inc.; Annual Benchmarking Pilot, March 2005, Urban Institute.

Were program goals achieved within budgeted costs and established schedules?

Explanation: "NCCC does not have a formal capital asset plan and does not manage its capital according to established federal "best practices" such as the Capital Programming Guide in OMB Circular A-11. To improve its facilities planning, the Corporation will develop a formal capital asset plan that documents the operations, rehabilitation and acquisition activities associated with the five campuses for the 2007 budget.

Evidence: