Detailed Information on the
Compassion Capital Fund Assessment

Monitor program performance, analyze data, assess performance, and institute appropriate program improvements.

Monitoring and evaluating technical assistance and capacity building grants to faith-based and community-based organizations focused on gang prevention activities.

Conducting a rigorous, experimental program evaluation to assess program's overall impact.

Instituting new data reporting guidelines and mechanisms to establish baseline data and to support performance measures. Milestone: Provide technical assistance on suggested reporting format with intermediaries and mini grantees.

Conducting a rigorous, experimental program evaluation to assess program's overall impact. Milestone: Provide guidance to grantees regarding requirements of impact evaluation.

Monitoring and evaluating technical assistance and capacity building grants to faith-based and community-based organizations focused on gang prevention activities. Milestone: Convene conference to provide technical assistance to grantees.

Monitoring and evaluating technical assistance and capacity building grants to faith-based and community-based organizations focused on gang prevention activities. Milestone: Propose evaluation design for CEY program.

Conducting a rigorous, experimental program evaluation to assess program's overall impact. Milestone: Collect baseline surveys for impact study.

Measure: Increase the proportion of CCF-funded Faith-Based and Community Organizations (FBCOs) awarded funding from Federal, State, local, or private sources. (CCF Demonstration Program)

Explanation:CCF has participated in, and contributed to, efforts resulting in increased Federal participation and funding of FBCOs each year since the program's inception in 2002. Projected increases in the number of FBCOs awarded funding as a result of CCF assistance are ambitious considering the dramatic improvements achieved during the early years of the Faith-Based and Community Initiative and the CCF as a result of concerted outreach and direct involvement of the President in program promotion. Projected increases are based on continued funding of the CCF at current levels.

Measure: Increase the number of capacity building activities implemented by FBCOs per $50,000 of federal assistance through the CCF Demonstration Program and Targeted Capacity Building Program funding.

Explanation:Capacity building outcomes include administrative knowledge and skills, management or infrastructure improvements, enhanced services, and organizational development. Progress on these individual elements would be quantified in order to measure progress on the measure. Currently, the Survey instruments from the CCF retrospective and prospective evaluations ask FBCOs (both receiving subawards and/or intensive TA) to complete a checklist listing specific "capacity building" outcomes in 7 areas; examples include "Established Board of Directors and Bylaws" and "Obtained computers and related hardware and software needed to manage the organization." By summing the total number of outcomes for all FBCOs, CCF would be able to determine progress on this measure. CCF considers this an ambitious performance measure given the extensive effort of the CCF program to reach and assist a large number of FBCOs during the early years of the President's Initiative. Most FBCOs are expected to achieve capacity improvements within the period they work with CCF intermediaries. Over the period of measurement and reporting, it is likely that much of CCF funding will be in the form of new grants to organizations not currently receiving funds.

Measure: Increase the proportion of FBCOs served through the CCF Demonstration Program which report expanding or enhancing services.

Explanation:This measure builds on the efficiencies inherent in the CCF grant structure. It allows the program to determine efficiency by focusing on ACF's ability to use Compassion Capital Fund dollars to build capacity among hundreds of FBCOs throughout the country by means of grants awarded to a small number of intermediaries. An increased ratio of FBCOs assisted to the number of total intermediaries indicates intermediaries' growing ability to promote organizational capacity throughout their communities.

Measure: Increase the proportion of CCF-assisted FBCOs awarded funding from federal, state, local, or private sources. (CCF Targeted Capacity Building Program)

Explanation:CCF has participated in, and contributed to, efforts resulting in increased Federal participation and funding of FBCOs each year since the program's inception in 2002. Projected increases in the number of FBCOs awarded funding as a result of CCF assistance are ambitious considering the dramatic improvements achieved during the early years of the Faith-Based and Community Initiative and the CCF as a result of concerted outreach and direct involvement of the President in program promotion. Projected increases are based on continued funding of the CCF at current levels.

Is the program purpose clear?

Explanation: The Compassion Capital Fund (CCF) helps faith-based and community organizations (FBCOs) increase their effectiveness, enhance their ability to provide social services, expand their organizations, diversify their funding sources, and create collaborations to better serve those most in need. This is accomplished by making grants in support of public-private partnerships that help small FBCOs replicate or expand model social service programs. Specifically, CCF awards the following two types of competitive grants: 1) The Demonstration Program: Grants to intermediary organizations who in turn, provide subawards and direct capacity building training and technical assistance to FBCOs in defined geographic areas; and 2) The Targeted Capacity Building Program: One-time $50,000 mini-grants to smaller FBCOs for specific capacity building initiatives.

Evidence: 1) Catalog of Federal Domestic Assistance 93.009 Compassion Capital Fund. 2005. Available at http://12.46.245.173/pls/portal30/CATALOG.SEARCH_PGM_TXT_DYN.SHOW?p_arg_names=prog_nbr_in&p_arg_values=93.009&p_arg_names=keyword_in&p_arg_values=compassion%20capital%20fund&p_arg_names=_title&p_arg_values=; and 2)"Conference Report on H.R. 3061." 2001. Congressional Record, Dec. 18, H10239. Senate Report 108-345, Departments of Labor, Health and Human Services, and Education, and Related Agencies Appropriation Bill. 2005.

Does the program address a specific and existing problem, interest, or need?

Explanation: The program addresses the problem of faith-based and smaller community organizations not having equal access to Federal resources for social service delivery. The bulk of public funding goes to a relatively select group of large social service and health nonprofits. There are many FBCOs who serve needy families in communities across the U.S., but are unable to secure public funding due to serious obstacles. In 2002, the Government Accountability Office (GAO) conducted a 50-state survey on this issue. GAO found that faith-based organizations were precluded from applying or being awarded Federal grants, because they lacked access to information about government funding opportunities, were inexperienced with dispensing public dollars, and had organizational barriers??financial and administrative??not unlike other small, community-based organizations. In addition, a study conducted by the Urban Institute identified concerns about the ability of small FBOs to understand the contract process, write high quality applications, meet data collection and reporting requirements, and meet licensing and credentialing requirements for some services such as substance abuse treatment. All of these challenges serve as barriers to FBCOs' ability to obtain Federal funding. The White House Office of Faith-Based and Community Initiatives (WHOFBCI) also reviewed 130 federally administered programs at six agencies (the Departments of Health and Human Services, Housing and Urban Development, Education, Labor, Justice, and Agriculture) and 28 program areas at the U.S. Agency for International Development. They found that as of 2005, secular nonprofits still received nearly six times as much funding as faith-based organizations (or FBOs).

Evidence: 1) Bane, Mary Jo, Brent Coffin, and Ronald F Thiemann. 2000. Who Will Provide? The Changing Role of Religion in American Social Welfare. Boulder, Colo.: Westview Press; 2) Government Accountability Office. 2002. "Charitable Choice: Federal Guidance on Statutory Provisions Could Improve Consistency of Implementation." Available at http://www.gao.gov/new.items/d02887.pdf/; 3) Kramer, Finegold, De Vita, and Wherry. 2005. "Federal Policy on the Ground: Faith-Based Organizations Delivering Local Services." Washington, D.C.: The Urban Institute; 4) White House Office of Faith Based and Community Initiatives (WHOFBCI). 2005. "Grants to Faith-Based Organizations: Fiscal Year 2005." Available at http://www.whitehouse.gov/government/fbci/data-collection.html/; and 5) White House Office of Faith Based and Community Initiatives (WHOFBCI). 2001. Unlevel Playing Field: Barriers to Participation by Faith-Based and Community Organizations in Federal Social Service Programs. Available at http://www.whitehouse.gov/news/releases/2001/08/unlevelfield.html/.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: CCF is the only nationwide capacity building program that helps FBCOs provide social services to a targeted needy population. It is also the only Federal program, which funds a national network of intermediary organizations that are in geographic proximity to the grassroots FBCOs they serve and are familiar with the issues in the communities in which they work. While some States and cities have FBCO initiatives, assistance is limited to providing information and resources about partnering with the state government or other entities. Few are established to help grassroots organizations build greater organizational capacity to compete for and secure funding, and to better manage their programs. In fact, a study conducted by the Roundtable on Religion and Social Welfare Policy at the Rockefeller Institute found that, as of 2003, only two states provided capacity building or startup grants (Ragan, Montiel, and Wright 2003). Also, the state and local programs that are similar to CCF are complementary, as opposed to duplicative and CCF seeks these programs out to serve as intermediary organizations. CCF understand the unique position of States and local governments to create a more pervasive infrastructure to support FBCOs by leveraging resources, creating permanent liaison and support offices on the ground, and spurring new philanthropic investments for FBCOs. Local public officials serve as a key entry point of connectivity to marginalized FBCOs, and local faith-based offices play a key role in disseminating information about funding opportunities of potential interest to FBCOs. Their efforts complement and broaden that of CCF intermediaries, reaching some communities not currently covered by the CCF program.

Evidence: A key word search on the Catalog of Federal Domestic Assistance (CFDA) found that there were 56 programs that support capacity building. However, these programs focused on different target populations and topics. For example, there were programs providing capacity building for community organizations to address health and clean water concerns; housing deficiencies; and economic development. CCF was the only program, which provided capacity building to better serve a targeted needy population including the homeless, prisoners reentering the community, children of prisoners, at-risk youths, and addicts. Other evidence include the following: 1) GAO. 2002. "Charitable Choice: Federal Guidance on Statutory Provisions Could Improve Consistency of Implementation." Available at http://www.gao.gov/new.items/d02887.pdf/; 2) "HHS Program Announcement HHS-2005-ACF-OCS-EJ-0035." 2005. Federal Register, April 29, 22,331-41; 3) Ohio of the Governor News Release. 2006. "Governor Announces Grants to Faith-Based and Community Organizations. Commends grantees for providing social services for at-risk Ohioans." Available at http://www.governor.ohio.gov/releases/050906Grants.htm/, May 9; 4) Ragan, Montiel and Wright. 2003. Scanning the Policy Environment for Faith-Based Social Services in the United States: Results of a 50-State Study. The Roundtable on Religion and Social Welfare Policy. Available at http://www.religionandsocialpolicy.org/publications/publication.cfm?id=41/. WHOFBCI. 2006. "New Agency at Homeland Security." White House press release, May 12; and 5) White House Office of Faith Based and Community Initiatives (WHOFBCI). 2005. "Grants to Faith-Based Organizations: Fiscal Year 2005." http://www.whitehouse.gov/government/fbci/data-collection.html/.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: CCF's program design is free from major design flaws and builds on existing literature and research that identify best practices for enhancing capacity in nonprofit organizations. Model practices include the following: 1) Strive to be holistic in nature; 2) Combine grantmaking with training and specialized technical assistance; 3) Be contextual and field based, with culturally relevant coaches; 4) Work through localized intermediaries who have longstanding relationships with the targeted participating organizations and communities; 5) Use multiyear programming; and 6) Build in organizational assessments. A key component of CCF's program design is the use of intermediary organizations with well-developed connections and working relationships within the nonprofit community. Intermediaries play an instrumental role in helping CCF have a wide reach through a diverse group of FBCOs throughout the country. Also, the public-private partnerships established between the Compassion Capital Fund and intermediary organizations provide an efficient method for delivering high-quality capacity building services to smaller FBCOs. This is supported by research, which states that intermediary organizations "make enormous contributions to the scope, scale, and effectiveness of grassroots, faith-based social service agencies, and often do so at low cost. [I]ntermediaries work best at a city scale. This is a target large enough to engage the broad vision of intermediaries (a community building paradigm rather than exclusive focus on service-building paradigms) yet small enough to facilitate the deep relationships genuine intermediaries build with their constituent ministries." (Sherman, 2002) Specifically, intermediaries "help grassroots groups overcome their problems of limited reach and immature organizational and administrative prowess. They bolster street saints' internal operations; connect them to new financial and human resources; teach them critical skills in management, fundraising, and outcomes evaluation; and perform the often unheralded service of simply helping overwhelmed community healers maintain their own sanity." (Anglin and Montezemolo, 2004) The program also makes one-time mini-grants to grassroots FBCOs whose annual operational budget is less than $500,000, or whose tribal membership is less than 5,000. Mini-grants are not a substitute for the intermediary grantees whose knowledge and experience are invaluable to smaller FBCOs seeking to improve their organizational capacity. However, it greatly expands CCF's reach to those FBCOs who are more sophisticated and have a demonstrated capacity to manage Federal funds without the oversight and assistance of an intermediary. Such organizations benefit from receiving one-time assistance, which help them compete for grants at the Federal level. Also, directly funding FBCOs through the mini-grants avoids the overhead costs associated with working through another organization. While 44 intermediaries assisted 1275 FBCOs in the 2005 operational year - with an average of 29 FBCOs per intermediary -- the mini-grant program reached an additional 300 FBCOs with capacity building support.

Evidence: 1) Anglin and Montezemolo. 2004. "Supporting the Community Development Movement: The Achievements and Challenges of Intermediary Organizations." In Building the Organizations That Build Communities: Strengthening the Capacity of Faith- and Community-Based Development Organizations. U.S. Department of Housing and Urban Development, Office of Policy Development and Research. Available at http://www.huduser.org/publications/commdevl/buildOrgComms.html/; 2) Backer, Bleeg, and Groves. 2004 The Expanding Universe: New Directions in Nonprofit Capacity Building. Alliance for Nonprofit Management. Available at http://www.allianceonline.org/publications/universetc.page/; 3) Branch Associates, Inc., with Abt Associates, Inc. 2005. "Promising Practices for Improving the Capacity of Faith- and Community-Based Organizations."; 4) Connolly and York. 2003. Building the Capacity of Capacity Builders: A Study of Management Support and Field Building Organizations in the Nonprofit Sector. Available at http://www.tccgrp.com/pdfs/buildingthecapacityofcapacitybuilders.pdf#search='building%20the%20capacity%20of%20capacity%20builders'/; 5) Fine, Alison. 2002. Echoes from the Field: Proven Capacity Building Principles for Nonprofits. Innovation Network, Inc. Available at http://www.envsc.org/bestpractices.pdf/; 6) "HHS Program Announcement HHS-2005-ACF-OCS-EJ-0035." 2005. Federal Register, 22,331-41; and 7) Sherman, Amy L. 2002. "Empowering Compassion: The Strategic Role of Intermediary Organizations in Building Capacity Among and Enhancing the Impact of Community Transformers: A Report on the Hudson Institute Faith in Communities' Intermediaries Study." Available at http://www.hudsonfaithincommunities.org/intermediaries.html/.

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Compassion Capital Fund grants provide capacity building activities, which are designed to increase an organization's sustainability and effectiveness, enhance its ability to provide social services, and create collaborations to better serve those most in need. It targets capacity building activities by awarding about 70% of its resources to intermediary organization and the remaining 30% directly to smaller, grassroots FBCOs. Experienced intermediary organizations provide training, technical assistance, and sub-awards to FBCOs. Specifically, intermediary organizations assist with capacity building activities in five critical areas: 1) leadership development, 2) organizational development, 3) program development, 4) revenue development strategies, and 5) community engagement. Successful applicants submit a plan in which they show a demonstrated ability to provide training and technical assistance in these five areas. They also must describe how they will assist FBCOs in identifying organizational needs, and how they will measure the impact of training, technical assistance, and sub-award activities. Intermediaries are also required to use at least 40 percent of the CCF grant to make sub-awards to smaller, grassroots FBCOs. The program works closely with intermediaries to ensure that funds are used appropriately to meet the program's purpose. For example, intermediaries are required to develop a detailed plan for making sub-awards. This includes a strategy for the types and number of organizations expected to receive funding and the purposes to which sub-awards may be put. It also describes the procedures the grantee will employ to identify and select organizations to receive sub-awards. For example, all intermediary sub-award plans must include evidence of outreach to both faith-based and community organizations, and sub-grants awarded using a fair and open competition. The plan must also specify if sub-awards will be used to assist organizations at different stages of development and how the sub-awards will be distributed in amounts that are manageable for a small organization. Mini-grants also reach intended beneficiaries since they are awarded directly to grassroots FBCOs. Recipients of the mini-grants use the funding to expand or enhance their organizational capacity in a variety of areas such as leadership effectiveness, organization structure, program policies and procedure, financial management and funding and community engagement . For example, an organization may use the funds to strengthen their board composition and formalize its functions, identify and provide appropriate development to staff, establish first time or better approaches for monitoring program performance, establish a financial sustainability plan, or enhance their information technology system. This is consistent with the program's purpose because the goals of the min-grant program are the same as the intermediary grant program, which requires grantees to use the awards to increase efficiency and organizational capacity.

Evidence: 1) Fiscal Year 2002, 2003, and 2004 Grantee Subaward Plans, Assessment Plans, and Semiannual Progress Reports; and 2) "HHS Program Announcement HHS-2005-ACF-OCS-EJ-0035." 2005. Federal Register, 22,331-41.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: CCF has performance measures, which tie to the program's purpose. The program aims to help small, grassroots FBCOs diversity their funding by securing new Federal, state, and local funds. It also aims to help FBCOs build capacity so that they may expand and enhance service delivery. The program's long-term measures are as follows: 1) Increase the proportion of CCF-assisted FBCOs awarded funding from federal, state, local, or private sources by 15 percent for the Demonstration Program and 10 percent for the Targeted Capacity Building Program by fiscal year 2011; and 2) Increase the ratio of capacity building outcomes achieved by CCF-assisted FBCOs to CCF Demonstration Program funding by 20 percent by fiscal year 2011.

Evidence: 1) Increase the proportion of CCF-assisted FBCOs awarded funding from federal, state, local, or private sources by 15 percent and 10 percent by fiscal year 2011, for the Demonstration Program and Targeted Capacity Building (mini-grant) Program, respectively. The numerator reflects the # of FBCOs awarded Federal, state, local, or private grants and the denominator reflects total # of FBCOs assisted. 2) Increase the ratio of capacity building activities achieved by CCF-assisted FBCOs to CCF Demonstration Program and Targeted Capacity Building Program funding by 20 percent by fiscal year 2011. The numerator reflects # of capacity building activities and the denominator reflects CCF funding to intermediaries and mini-grants.

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The program does not have baselines for the long-term performance measures. However, it is in the process of instituting new reporting requirements to gather baseline data. Without the baseline data, it is not possible to evaluate whether proposed targets are ambitious and meaningful.

Evidence: See measures tab.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The program's long-term measures will serve as the annual measures. Grantees will report annual progress made, which will show whether the program is on track in meeting the long-term goals.

Evidence: See measures tab.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The program does not have baselines for the annual measures. However, it is in the process of instituting new reporting requirements to gather baseline data. Without the baseline data, it is not possible to evaluate whether proposed targets are ambitious and meaningful.

Evidence: See measures tab.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: While partners support the overall goal of the program, they do not currently measure and report on their performance as it relates to accomplishing the program's long-term and annual goals. The program will require new reporting data to assess partner's contribution to the annual and long-term performance goals.

Evidence: New data requirements for the diversifying resources measure are as follows. Number of applications submitted, approved, and total funding received from: 1) Grants/ contracts from Federal government agencies; 2) Grants/ contracts from state/local government agencies; 3) Grants/ contracts from foundations; 4) Grants from other federated giving groups (e.g. United Way); 5) Direct mail fundraising; 6) Special fundraising events; 7) Fundraising in church or community; and 8) Door to door fundraising Instructions to gather new data for the capacity building measure are as follows: 1) Summary of services provided- Detail your organizations major activities and accomplishments during the project period. Detail CCF activities as originally outlined in your application's proposed strategy, and discuss major revisions to the originally proposed strategy during the course of your project. Use of the project task chart from approved grant application and/or project work plan is recommended. Concisely describe how your project's actual activities reflect your proposed and/or amended approach. Include final counts of services provided where possible (i.e. number of hours of technical assistance, number of organizational assessments, number and amount of sub-award). 2) FBCO capacity-building outcomes - Describe the outcomes that organizations have achieved related to your CCF activities. Provide information from any internal or external evaluation of program outcomes or processes, including but not limited to: 2a) Improved knowledge and skills gained from training sessions; 2b) Improved organizational development based on pre- and post-organizational assessments; 2c) Management, leadership & infrastructure improvements/ key organizational best practices. List each accomplishment in table format provided below; 2d) Expansion and diversification of funding. Please complete the table below with any supporting data for each organization; and 2e) Expansion or enhancement of social services. List in table format provided below and provide supporting data for each organization.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The program has an independent evaluator to conduct a multi-pronged evaluation of the program. The evaluation is of sufficient scope and uses an experimental random assignment design to assess potential improvements in multiple domains of organizational capacity among faith-based and community organizations. The following are the three components of the evaluation: 1) Retrospective Study: To provide interim information on program performance, 173 FBCOs were surveyed to assess how sub-awards had helped smaller, grassroots FBCOs. Findings showed that FBCOs were able to diversity funding as well as garner skills to build capacity. For example, there was an 11% increase in the receipt of federal funds, a 4% increase in state/local government funds, and a 6% increase in foundation funding. The survey also found that FBCOs reported the intermediaries had a positive influence on long-term planning, improved financial management, ability to serve more clients and to start new programs. 2) Prospective Outcome Study: The study includes all 44 intermediary grantees operating in FY 2006, and the smaller FBCOs they helped with the grant funding. The survey addresses the following outcome areas for about 1,000 grantees in the study: program services (number served, benefits measured); financial management (audits, written financial procedures, formal budgeting); funding diversity (applications, sources of new funds; grant writing training); human resources/staff development (number of paid/volunteer staff, written job descriptions, training); governance (established Board, formal procedures, functions); technology (availability of computers, internet access, web presence); programmatic record keeping (types of records kept); and community engagement (partnerships, needs assessments). Baseline data collection will continue through mid-2006, follow-up survey conducted about 15 months following receipt of sub-award/TA services, and findings are expected to be available in late 2007. 3) Prospective Impact Study: The Impact Study component will include all new intermediaries (about 10) selected under the FY 2006 grant announcement. The evaluation contractor will work with each of the new intermediaries to implement a random assignment process with a treatment and control group for about 500 FBCOs. The Impact Study data collection/survey forms are expected to be the same as those used in the Outcome Study. By obtaining the same data from both the treatment group and the control group, the Impact Study component will be able to assess the net impact of the CCF Program - that is, the difference between what is achieved with CCF support and the counterfactual (what would have happened without CCF services). The baseline data collection is expected to begin in early 2007, the follow-up data collection in mid-2008, and findings available in late 2008.

Evidence: 1) CCF Evaluation Contract Scope of Work (Abt and Branch Associates); 2) Baseline and Follow-up Survey - National CCF Program Evaluation; and 3) Retrospective Study Survey - National CCF Program Evaluation.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: The program's budget request is not tied to the accomplishment of the annual and long-term performance goals. Performance goals were established recently in April of this year. To date, budget requests have been based on existing research which point to the need for a "level playing field" for FBCOS to compete for Federal grants and other funding sources. Funding requests have been tied to number of grants awarded to building capacity among FBCOs, and increases in program funding went to support additional grantees. The White House Faith-Based Office (WHFBO) tracks Federal dollars awarded to FBCOs from select Federal agencies. The number of FBCOs receiving funding from HHS has more than doubled since FY 2002. However, this is not an evaluation and it is not possible to assess the Compassion Capital Fund program's impact on increased HHS dollars awarded to FBCOs. In other words, it is not possible to project new Federal dollars that FBCOs secure with each dollar spent on the Compassion Capital Fund. Although budget requests are not tied to performance targets, the program presents resource needs in a complete and transparent manner. This is evidenced by the "Resource and Program Data Sheet" in the Congressional Justification. The data sheet shows funding designated for grants, research and evaluation, demonstration and development, training and technical assistance, and program support. Included in program support are indirect costs for information technology support, grant/ paneling review, conference, and contract fees.

Evidence: 1) White House website - Faith-Based and Community Initiative http://www.whitehouse.gov/government/fbci/factsheet.html; 2) ACF Compassion Capital Fund program information website http://www.acf.hhs.gov/programs/ccf/; 3) CCF Grant Announcements - Fiscal Year 2005; and 4) Congressional Justification - FY 2003 and FY 2005.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: Since the program was first created three years ago, CCF has acted to correct strategic planning deficiencies. For example, CCF determined after two years of program operation that the grant cycle for intermediary funding needed to be shortened from three years to 17 months to enhance the competitiveness of the grants and for improved oversight and monitoring. Specifically, the shortened funding cycle helps the program better assess intermediary performance and make necessary adjustments in funding. CCF used the early years to implement the program and obtain initial program information. This year, the program deemed itself ready to develop performance measures. In FY 2006, CCF will accomplish the following: 1) Establish long-term performance measures, associated targets, and baselines; 2) Establish annual performance targets to achieve long-term outcomes, including baselines; 3) Initiate an on-going process for performance measurement training and coordination with program grantees and stakeholders; and 4) Initiate the impact evaluation portion of the independent evaluation of the CCF program by an independent evaluator.

Evidence: 1) CCF Report Format for Fiscal Year 2005 grantees; 2) CCF Evaluation Contract Scope of Work (Abt and Branch Associates); and 3) National Evaluation Baseline and Follow-Up Survey.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The program monitors grantees' activities through semi-annual and final reports, but does not collect high quality performance data. The program has taken steps to remedy this by recently adopting new performance measures and establishing new reporting requirements to collect this data. As such, it has not collected baseline data for the long-term and annual performance measures to set meaningful performance targets.

Evidence: See newly adopted performance measures in the measures tab. The program is taking steps to collect baseline data.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Federal program staff are held accountable for cost, schedule and performance results, as evidenced by the Employee Performance Management Plan (EPMP). Employees who achieve Outstanding or Excellent performance ratings traditionally receive monetary awards, subject to the availability of funds. Staff with poor performance ratings must develop a plan to support improved performance. Beyond the program staff, CCF grantees are held accountable for costs, schedule and performance results in accordance with the grants management policies and requirements. CCF-funded intermediaries are expected to adhere to a schedule of activities and associated outcomes, which were included in the application for funding. Intermediaries, in turn, monitor the progress of their sub-grantees and collect semiannual and annual information on cost and performance. Both recipients of mini-grants and intermediaries must account for the progress of expenditures, schedule of activities, and performance targets in the semiannual and final performance reports to CCF. For grantees who fail to make adequate progress, CCF and the program's National Resource Center (NRC) staff work closely with the grantees to establish corrective actions. At the extreme, nonperforming grantees may have awarded funds suspended or recaptured. For example, a CCF intermediary in Montana made a subaward to a parish nurse center, which in turn used the funds to support religious activities. This was a direct violation of Federal regulations that specify "organizations that receive CCF funds may not engage in inherently religious activities, such as worship, religious instruction, or proselytization, as part of the programs or services funded with CCF funds." A Federal court judgment ruled against the CCF grantee in October 2006 and the intermediary reassigned the grant to another sub-awardee. Finally, the CCF National Resource Center contractor submits reports on its activities, performance, and costs on a monthly basis. These reports are reviewed to ensure activities and expenditures conform with the approved contract.

Evidence: 1) ACF Grants Management manual and other instructions; 2) CCF Semiannual and Final Program Guidance to Intermediary and minigrantees; 3) Contractor reports on National Resource Center usage and requests for information; 4) EPMS for CCF Program Staff; 5) Program announcements for CCF Intermediary and minigrant funding; 6) Subaward Assessment Plan for Intermediary grantee; and 7) Employee Performance Plan.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: CCF obligates program funds in a timely manner in accordance with the intended program purpose. It also monitors grantees' financial and progress reports to ensure timely and appropriate obligation of funds. Intermediaries ensure that accountability exists for sub-awards by establishing reporting requirements and making frequent contacts with the sub-grantees. Intermediaries also ensure that funds are spent in a timely manner by disbursing the funds in increments throughout the year instead of providing a lump-sum. For example, some intermediaries disburse half of the sub-awards by 1/1; a quarter of the sub-awards upon receipt of the second program and financial status report on 7/15; and the final quarter of the funds upon receipt of the third program and financial status report on 10/15. Also, there are established procedures that reflect a sound check and balance system for approval and obligation of funding. Funding is approved by the ACF Assistant Secretary and the OCS Budget Officer obligates funds through the automated grants support system, GATES. The funds are certified and awarded through GATES by the Office of Grants Management prior to the end of each fiscal year. Once funds are obligated and released to grantees, grantees are subject to basic audit requirements. Specifically, intermediaries with grants exceeding $500,000 are subject to a single audit whereas FBCOs with less than $300,000 conduct self-audits. Self-audits require grantees to maintain all appropriate records to allow a private auditor to assess their financial statements. Although no formal protocol is used, grantees must be able to make all accounting information available to an independent auditor upon request. Examples of materials grantees need to make available to independent auditors include but are not limited to accurate current and complete disclosure of the financial results of each HHS sponsored project; records that identify the source and application of funds for HHS sponsored activities; and accounting records, including cost accounting records, that are supported by source documentation. To date, no negative findings from grantee audits have been reported.

Evidence: 1) CCF Program Announcements for Intermediary and mini grants; 2) CCF Semiannual and Final Program Report Guidance; 3) Sub-award Assessment Plan for Intermediary grantees http://www.acf.hhs.gov/programs/ccf/resources/gbk_gm/gm_gbk_audit.html; and 4) Wichita sub-award plan.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The program design ensures that it achieves efficiencies in program execution. CCF reaches hundreds of FBCOs to increase their sustainability and organizational capacity by working through a limited number of intermediaries. For example, 44 grants were awarded competitively to intermediaries for new or continuation grants in FY 2004 and operated through 2005. These intermediaries assisted 1,275 FBCOs through subawards, technical assistance, and training. In addition, the de-layered CCF management structure increases the effectiveness of CCF program. The CCF staff work as a team to implement the program. The team is a relatively flat structure with one program manager to whom three program specialists report, along with an administrative professional. The use of a National Resource Center to assist in delivering information, training and technical assistance is another example of an important management efficiency strategy for the program. The NRC model enables the small CCF staff to extend their reach and increase program/grantee communication and oversight. Technology is a major part of this effort. The NRC uses conference calls, ListServs and Web-based conferencing to augment national conferences and face-to-face meetings in an effort to stay in close contact with grantees, to support and reinforce technical assistance and training content, and to achieve economies of scale in program delivery.

Evidence: The program's efficiency measure is as follows: The ratio of the total number of FBCOs assisted with CCF funds annually to the number of direct CCF grants to intermediaries. This measure will not create perverse incentives for intermediaries to meet the targetse by hosting large workshops. This is because the program requires that intermediaries allocate 40% of their Federal CCF grant to sub-awards to FBCOs and that a minimum of 50% of the training and technical assistance provided be provided in the form of direct and individualized assistance.

Does the program collaborate and coordinate effectively with related programs?

Explanation: CCF collaborates with Federal partners, which have resulted in meaningful management actions. For example, ACF discretionary grant announcements now reflect the CCF mission of "creating a level playing field" by identifying FBCOs as eligible entities for the grant funding. This includes HHS programs such as the Community Services Block Grant program, the Assets for Independence Living program, and the Job Opportunities for Low-Income program. CCF also works collaboratively with other initiatives within HHS that have the same target population as CCF. For example, CCF grantees are encouraged to apply for grants for building strong families, strengthening fatherhood, and mentoring of children of prisoners. Similarly, the program helps to identify grants in other cabinet agencies with the same target CCF population by working closely with the Office of Faith-Based and Community Initiatives. Special emphasis is placed on funding opportunities in the areas of community development (HUD and the Department of Commerce), job training and placement (Department of Labor), and services to individuals in the criminal justice system (Department of Justice). The faith-based offices identify discretionary grant funding opportunities for FBCOs, including CCF grantees, by posting this information on the CCF National Resource Center. This information is also shared through a CCF e-newsletter, which has a distribution list of more than 3,000 FBCOs.

Evidence: 1) CCF Program Announcements; 2) Grant Announcements for other OCS programs - CSBG, JOLI, CED; 3) Grant Announcements for ACF Discretionary Grant Programs (CSBG, AFI, JOLI); and 4) White House Office of Faith-Based and Community Initiatives Web site.

Does the program use strong financial management practices?

Explanation: There is no evidence of the program integrating financial and performance systems to support day-to-day program operations. Also, there are clean audit opinions and no material internal control weaknesses at the ACF level. While there are clean audits of some individual CCF grantees, no such opinion exists at the program level.

Evidence: OCS guidance for program and financial reports - semiannual and final

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Faith-Based and Community Initiative is a part of the President's Management Agenda. CCF, along with Access to Recovery and Mentoring Children of Prisoners, are a part of the faith-based and community initiative. Each quarter, the initiative is rated on its progress and status. CCF has contributed to the green progress rating for the initiative by meeting the data collection requirements, as well as initiating an outcome based evaluation. The program continues to make management improvements in these areas by expanding the set of data collected by adopting new performance measures and has planned additional outcome-based evaluations. Outside of the President's Management Agenda, the program has taken other meaningful steps in addressing management deficiencies. For example, the duration of grant award for intermediaries was shortened from 3 years to 18 months to ensure that grants remain competitive. The program also has monitoring and oversight procedures in place to correct any deficiencies at the grantees and sub-grantees levels

Evidence: President's Management Agenda: Faith-Based and Community Initiative Scorecard

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: CCF grants are awarded in accordance with the discretionary grant competition policies and procedures of the Administration for Children and Families. CCF funds are awarded through fair and open competitions in which all CCF grants are awarded on a competitive basis. To illustrate the competitive nature of the grants, there were 446 applicants in FY 2005 competing for 20 awards. As for the mini-grants, there were 1,596 applicants in FY 2005 for 310 awards. The intermediary grants are awarded for 17 months and remain competitive, because continued funding throughout the 17 month grant period is dependent on grantees meeting clearly set performance standards. Upon completion of the 17 month grant period, grantees can re-compete for funds, and are subject to the same criteria outlined within the program announcement. The program is also committed to fair and open competition and reaches out to potential applicants regularly. For example, information about the availability of CCF grants are often included in the CCF e-newsletter which goes to approx 4000 subscribers; CCF also reaches out to potential applicants through cooperation with the WHOFBCI, which conducts regional outreach conferences, and publishes information regarding grant opportunities for FBCOs on its website. In addition, the CCF NRC goes to a variety of conferences to speak to FBCOs about CCF funding opportunities. Finally, Intermediary conduct outreach as part of their sub-award plans (SAP). Specifically, intermediaries develop and later execute a marketing/outreach plan which details how intermediaries publicize the availability of CCF sub-awards. . Further, all grant announcements are published through grants.gov, and non-federal listserves which are widely accessible. The criteria for evaluation and selection of applications are clearly defined in the program announcement for the general public and potential applicants. The program announcement includes specific categories of assessment and points are assigned to each of these categories. Grant applications are reviewed and evaluated by an independent panel of reviewers with appropriate expertise in the programmatic area. OCS recruits individuals from across the country who have relevant knowledge and expertise to serve as grant reviewers for the CCF program. Reviewers receive extensive training on the review process, including the evaluation criteria and scoring of applications. All applications are rank-ordered based on the evaluation scores assigned by reviewers.

Evidence: 1) CFF Grant Announcements; 2) OCS Peer Review Process - Contract; 3) OCS Peer Review Manuals and Forms; and 4) Compassion Capital Fund Fact Sheet.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The program collects program and financial reports to ensure that grantees adhere to the terms of the grant. All reports are submitted electronically to CCF staff. Upon receipt, the staff logs the report into the "CCF Intermediary Reports Log." This log lists all grantees, the number of reports submitted, and when they were submitted. CCF staff crosscheck semiannual and final reports with grantee proposals to ensure that grantees met the deliverables. If consistent, no action is taken and the report is logged as sufficient. However, if discrepancies or issues are identified, CCF staff contact the grantee directly, discuss the issue and make suggestions/recommendation to rectify the issue. The conversation is confirmed by email and a revised report is submitted or documentation of corrective actions is included in the next report. Program information collected includes work plans detailing intermediaries' activities on outreach, selection, award, reporting and monitoring of subawardees. The intermediary semiannual progress report collects information on project milestones, units of service provided, a list of training and technical assistance engagements and a list of organizations receiving subawards. With the development of new performance measures, grantees will now be required to report their progress and contribution made in helping the program meet the performance targets. The final report for intermediaries solicits capacity building outcomes from evaluations conducted by the intermediary. Finally, the program collects a financial status report, Form 269, which includes total outlays, refunds, rebates, and net outlays, including unobligated balance. Grantees submit this report with the semiannual report and the final report to the Office of Grants Management. A more detailed cash transactions report, Form 272, is submitted electronically each quarter to the Division of Payment Management. This form requires organizations to report payment vouchers credited to an account, treasury checks received, and cash on hand at the beginning of the reporting period. The 272 form requires the grantees to report any advances to subgrantees or subcontractors. By regularly reviewing the program and financial reports, the program has oversight of grantees' activities and also ensures that funds are used for the designated purpose. In the first three years of the program, the program employed NRC to conduct 66 site visits for intermediary grantees from FY 2002 through FY 2004. Currently, site visits are conducted when OCS and the NRC recognizes that a grantee is behind in their projected outcomes or timeline.

Evidence: 1) CCF Intermediary Development Series guidebook, "Outcomes Measurement," teleconference announcement for subaward assessment plan guidance; 2) CCF Semiannual and Final Program Report Guidance; 3) Final Report Guidance for Intermediary Organizations??FY 2002 or updated for FY 2003 and 2005 TBD; 4) Guidance for Subaward assessment Plan; 5) Guidance for Subaward Plan/ teleconference training; 6) Subaward plan Review Template; 7) National Resource Center Web site overview??http://www.acf.dhhs.gov/programs/ccf/the_nrc/index.html; 8) Online subaward assessment plan wizard; 9) Regional Office monitoring reports; 10) CCF Intermediary Development Series guidebook, "Outcomes Measurement"; 11) Site Visit Form and example of a completed Site Visit Report.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Currently, the program does not make grantees' performance data available to the public. To remedy this, CCF is instituting a system for gathering performance information from FBCOs receiving CCF intermediary organization support, training and technical assistance. In addition, CCF will collect and report information on new funding sources that intermediary FBCO subgrantees have secured as a result of CCF assistance. Prior to the new data collection system, CCF tracked the number of new CCF grant awards made to FBCOs since the program's inception. This information was submitted to various offices within HHS and the White House Office of Faith-Based and Community Initiatives. These offices, in turn, made the information available to the public through reports and news releases.

Evidence: 1) HHS Press Releases on CCF Grant Awards and rates of awards of other Federal funds to FCBOs; 2) White House Office of Faith-Based and Community Initiatives Website and reports; and 3) CCF survey form for intermediary FBCO subgrantees. http://www.tacs.org/training/files/sos/rfp/BaselineSurvey.pdf/.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: CCF is in the process of instituting new reporting requirements to gather data for the newly proposed performance measures. While the program has yet to collect actual data, there are some findings that show a positive trend. For example, a retrospective study of the CCF program found that a select sample of the FBCOs diversified funding and self-reported positive capacity building outcomes. However, CCF needs to complete a more rigorous impact evaluation before applying and attributing the findings as program-wide outcomes. Also, the White House Office of Faith-Based and Community Initiatives tracked funding to FBOs awarded by key cabinet agencies in FY 2005. The report noted grants to FBOs increased by 37% between FY 2003 and 2005. However, this was not an evaluation study which confirmed a link between CCF program efforts and increased ability of FBOs to obtain new Federal grants.

Evidence: 1) CCF Retrospective Study findings; and 2) White House Office of Faith-Based and Community Initiatives. 2006. Grants to Faith-Based Organizations??Fiscal Year 2005. Available at http://www.whitehouse.gov/government/fbci/final_report_2005.pdf/, March 9.

Does the program (including program partners) achieve its annual performance goals?

Explanation: CCF is in the process of instituting new reporting requirements to gather data for the newly proposed performance measures. While the program has yet to collect actual data, there are some findings that show a positive trend. For example, a retrospective study of the CCF program found that a select sample of the FBCOs diversified funding and self-reported positive capacity building outcomes. However, CCF needs to complete a more rigorous impact evaluation before applying and attributing the findings as program-wide outcomes. Also, the White House Office of Faith-Based and Community Initiatives tracked funding to FBOs awarded by key cabinet agencies in FY 2005. The report noted grants to FBOs increased by 37% between FY 2003 and 2005. However, this was not an evaluation study which confirmed a link between CCF program efforts and increased ability of FBOs to obtain new Federal grants.

Evidence: 1) Branch Associates and Abt Associates, Inc. 2006. "An Initial Assessment of the Compassion Capital Fund." April 10.; and 2) White House Office of Faith-Based and Community Initiatives. 2006. Grants to Faith-Based Organizations??Fiscal Year 2005. Available at http://www.whitehouse.gov/government/fbci/final_report_2005.pdf/, March 9.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The program has established baseline data for its efficiency measure. The baseline data shows that 44 CCF intermediary grantees provided subawards and associated technical assistance and training to 1,275 FBCOs in FY 2005. Because the program recently adopted the efficiency measure, it has yet to show improved efficiency over the prior year.

Evidence: See measures tab.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: Compassion Capital Fund is the only Federal program, which has a national scope in reaching FBCOs to build organizational capacity. It targets grassroots FBCOs with services to a special target population, which includes the homeless, prisoners reentering the community, children of prisoners, at-risk youth, and addicts. While some states (such as Indiana, Oklahoma, Texas) and cities (such as Indianapolis) have FBCO initiatives, according to the Roundtable on Religion and Social Welfare Policy, most do not (Ragan, Montiel, and Wright 2003). The Roundtable on Religion and Social Welfare Policy also found that, as of 2003, only two states provided capacity building or startup grants (Ragan, Montiel, and Wright 2003), however, no studies currently exist that evaluate the effectiveness of the two states providing the capacity building grants. Rather than providing funding or training and technical assistance, existing state and local FBCO initiatives most frequently serve to provide information and resources to FBCOs about partnering with the state or local government or others to provide services. CCF takes this further, providing intensive training, technical assistance and targeted investments. There are other Federal programs, which build capacity among community organizations in affordable housing or international development. Though CCF is modeled after some of these successful programs, the outcomes cannot reasonably be compared due to the following factors: 1) no common measures or national comparison data exist across cities, states, or regions for these programs, 2) the outcomes are programmatically different (e.g., outcomes of capacity building training and investments in the housing program are based on the number of affordable housing units constructed, improved or occupied), or 3) the goals of the funding or intermediary structures were geographically or neighborhood explicit and thus do not correlate with the scope of CCF.

Evidence: 1) Backer, Bleeg, and Groves. 2004. The Expanding Universe: New Directions in Nonprofit Capacity Building. Alliance for Nonprofit Management. Available at http://www.allianceonline.org/publications/universetc.page/; 2) Fine, Alison. 2002. Echoes From the Field: Proven Capacity Building Principles for Nonprofits. Innovation Network, Inc. Available at http://www.envsc.org/bestpractices.pdf/; and 3) Ragan, Montiel and Wright. 2003. Scanning the Policy Environment for Faith-Based Social Services in the United States: Results of a 50-State Study. The Roundtable on Religion and Social Welfare Policy. Available at http://www.religionandsocialpolicy.org/publications/publication.cfm?id=41/.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: CCF is in the process of conducting a comprehensive program evaluation through Abt Associates. The evaluation consists of the following three components: 1) A retrospective study of program outcomes through a survey of 173 FBCOs completed in 2006; 2) A prospective study of program outcomes through a baseline and follow-up survey of more than a thousand FBCOs receiving services from all operating CCF intermediaries in 2006, to be completed in 2007; and 3) A prospective impact study based on an experimental design comparing outcomes among FBCOs in a treatment group with those in a control group and involving all intermediaries awarded new grants in FY 2006, to be completed 2008. To date, only the retrospective study has been completed, but this is not of sufficient scope since the sample size is 15% of 2003 sub-awardees. Also, the study does not meet the criteria for a quality evaluation since it is not of an experimental or quasi-experimental design. The retrospective study informs the prospective outcome and impact study, which are of sufficient scope and rigorous evaluation design.

Evidence: 1) Branch Associates and Abt Associates, Inc. 2006. "An Initial Assessment of the Compassion Capital Fund." April 10; 2) CCF Evaluation contract; and 3) Retrospective Study findings.