Detailed Information on the
Black Lung Clinics Assessment

Establishing baselines and ambitious targets for the long-term performance measures.

Exploring the parameters and feasibility of conducting an independent evaluation of the program.

Collecting data on the location of miners to better target resources and further enhance outreach.

Making grantee performance data available to the public and further communicating with grantees on best practices.

Measure: The percent of miners that show functional improvement following completion of a pulmonary rehabilitation program.

Explanation:The Black Lung Clinics Program provides medical care to persons with Black Lung disease. This is a health outcome measure relevant to the disease. Given that people with Black Lung disease cannot be cured, the next best option is improvements in quality-of-life and ability to function within the constraints of the disease. This measure relies on information from the application of a pre- and post- pulmonary rehabilitation program 6 minute walk test, applying uniform standards of measurement established by the American Thoracic Society (ATS). According to ATS the 6 minute walk test provides information that may be a better index of the patient's ability to perform daily activities than is peak oxygen uptake; the test correlates better with other measures of quality-of-life. This measure is in the developmental stage.

Measure: The number of miners served each year.

Explanation:The Black Lung Clinics program supports clinics that diagnose, treat, and rehabilitate active and retired coal miners who have been exposed to coal dust. For a number of reasons, it is difficult to encourage miners to seek care for occupation-related ailments. Tracking the number of miners served helps assess the reach of the program and the impact of outreach efforts.

Measure: The number of medical encounters for Black Lung each year.

Explanation:Once miners have been diagnosed as having black lung disease, they should return to clinics for follow-up, treatment and rehabilitation appointments, generating encounters. Tracking encounters helps gauge the impact of the program with regard to ongoing care of miners.

Measure: The number of medical encounters per $1 million in federal funding.

Explanation:Encounters can be considered a basic unit of service of Black Lung Clinics. Given the rising costs of health care and level Federal funding of the program, improvements on this measure indicate efficient and cost effective operations.

Is the program purpose clear?

Explanation: The purpose of the Black Lung Clinics Program (BLCP) is to establish and operate clinics that identify, diagnose, treat, and rehabilitate active and retired coal miners with occupational exposure to airborne particles resulting in respiratory and pulmonary impairments. Through competitive grants to States, organizations, individuals, and private or public entities, the BLCP coordinates services and benefit programs for the coal miner population. The BLCP grantees focus on the following five elements: 1) outreach, 2) primary and specialty care (particularly screening, diagnosis and treatment for black lung disease), 3) patient education, 4) pulmonary rehabilitation, and 5) patient care coordination.

Evidence: The Black Lung Benefits Reform Act of 1977 (Public Law 95-239), as amended, February 27, 1985, authorized support of the BLCP to evaluate and treat coal miners with respiratory impairments. The programs purpose along with grantee responsibilities are clearly stated in the Federal Register (42 CFR Part 55a) and in HRSA's BLCP guidance (PIN 2002-08 BLCP Expectations and Principles of Practice).

Does the program address a specific and existing problem, interest, or need?

Explanation: Congress created the BLCP to provide unique pulmonary and respiratory care to coal miners, who otherwise could not access specialized health care. Black Lung and Coal miner's Pneumoconiosis (CWP) remains a prevalent illness among current and retired coal miners. For example, in 2005, the BLCP provided services to 10,790 miners. The Bureau of Labor Statistics reported that in November 2004 there were more than 70,000 individuals currently working in the coal mining industry. The National Mining Association estimates that the U.S. will need approximately 50,000 new coal miners over the next ten years to meet future demands and replace retiring miners.

Evidence: The CDC report, Changing Patterns of Pneumoconiosis Mortality - United States, 1968-2000 states CWP continues to occur among working coal miners, even among those first employed after the current federal exposure limit became effective. The CDC examined pneumoconiosis prevalence among working coal miners in federal chest radiograph surveillance programs for 1996-2002 and found that the prevalence of CWP increased with age. According to the CDC Surveillance Report 2002, there are over 1,000 annual deaths directly attributed to CWP.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Department of Labor (DOL) and the Center for Disease Control and Prevention's National Institute for Occupational Safety and Health (CDC-NIOSH) have programs specific to coal mining and Black Lung Disease; however, neither program provides comprehensive clinical services and/or health education to the entire population effected by coal miner's pneumoconiosis (CWP). The DOL and BLCP work in tandem to ensure both current and retired coal miners, those with 100 percent disability and those with early symptoms, have access to appropriate medical care. The DOL Coal Mine Workers Compensation program only serves coals miners who are totally disabled. However, the BLCP provides pulmonary screening, diagnosis, and treatment specific to black lung disease in areas where these specialized services are not readily available or to miners whom are not covered through the DOL program. Moreover, the BLCP provides health education and screening not covered by public or private insurance and to patients that do not have the ability to pay. The CDC-NIOSH educates companies and workers about prevention and safety measures in the coalmines and conducts research and development around screening tools and techniques; however, these services do not focus on prevention and /or treatment of CWP.

Evidence: The Federal Register, 42 CFR 55(a) prohibits BLCP grantees from using funds to supplant existing services. Although some states provide funding to support care for black lung patients, BLCP grantees must use these federal dollars to supplement activities, not supplant state funds. The DOL Coal Mine Workers' Compensation program provides monthly compensation and covers the cost of medical care to miners with total disability from black lung disease. However, the DOL only approves 11% of miners who file claims to receive DOL benefits.

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The BLCP design allows for multiple service models that include Health Centers (330 funded programs), standalone Black Lung Clinics, mobile outreach vans and Hospital clinics. These models provide outreach, screening, diagnosis, treatment, patient and family education, health counseling and benefits counseling, as well as patient care coordination and pulmonary rehabilitation. This approach allows each grantee to create and implement the most efficient and effective service model to coordinate benefits and to minimize any possible service gaps in providing treatment to coal miners.

Evidence: There is no evidence (evaluations, GAO/OIG reports) that another service model or funding source would be more effective or efficient in delivering care to this population. The BLCP is the payer of last resort and maximizes third party reimbursements (Federal Register, 42 CFR 55(a)).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: The 2003 needs assessment conducted by Peterson Consulting found weaknesses in outreach and benefits counseling. The BLCP has been operating for more than 20 years; however, few grantee awards have changed over this period even though evidence suggests that this population has spread to new geographic areas and health care providers have expanded to rural areas. The BLCP does not assess the location of current grantee services in proximity to other health care services (hospitals, community health centers, and primary care physicians) to ensure BLCP grantees are targeting areas where there is limited access to health care. In addition, the BLCP lacks a reporting system for BLCP payment leaving it unclear as to what the BLCP is paying for within each clinic/grantee and the relative effectiveness of the services and programs funded. Furthermore, the needs assessment found that a few of the grantees operate clinics less than four hours of operation per week.

Evidence: The 2003 Needs Assessment of the Black Lungs Clinics Program conducted by Peterson Consulting found that the BLCP needed a better understanding of the allocation of grant funds and more consideration given to individual site circumstances. The CDC MMWR weekly (July 23, 2004) found that the state of residence at death is not always the state in which the decedent's causative exposure occurred, especially given the latency and chronic course of the pneumoconiosis. In addition, the BLCP does not have a verification process to ensure they are providing services to coal miners.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: The BLCP has a long-term measure: to increase the percent of miners that show functional improvement following completion of a pulmonary rehabilitation program. Although the BLCP only has one long-term measure, it encompasses the goal of the overall program, which is to improve the quality of life for miners.

Evidence: To increase the percent of miners that show functional improvement following completion of a pulmonary rehabilitation program. The BLCP will collect data on this measure after implementation of the updated collection tool in FY 2007

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The BLCP does not have historical data for their long-term measure. Although the BLCP has estimated baseline information, the program has not estimated annual targets. The BLCP will not collect data from all grantees until the data collection tool is updated in FY 2007.

Evidence: There is no historical data for the BLCP long-term measure. Although the BLCP has estimated baseline information for this measure, they do not have annual targets.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The BLCP program collects data on two annual performance measures. The BLCP can improve the quality of life of the coalminer population by increasing the number of miners that come into the clinic and receive screening, treatment and other medical services.

Evidence: The FY 2007 annual performance measures: 1) Increase the number of miners served each year, and 2) Increase the number of medical encounters for Black Lung each year.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The BLCP goal is to increase the number of total miners by two percent in FY 2008. This is an ambitious target considering that this population has been stable over the past five years. In addition, the BLCP intends to increase the number of medical encounters by two percent by FY 2008.

Evidence: Increase the number of miners served each year. Baseline 2005 10,790 Targets 2006 10,862 2007 10,933 2008 11,005 Increase the number of medical encounters for Black Lung each year. Baseline 2005 20,844 Targets 2006 20,983 2007 21,121 2008 21,269

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The FY 2005 HRSA BLCP guidance requires grantees to include a health care plan that outlines goals and objectives related to identified health needs/issues of the target population as well as improvement activities that are specific to the area or population (e.g., quality of pulmonary rehabilitation services). In addition, grantees report annually on the number of miners served and number of medical encounters. With adoption of the long-term measure, grantees will commit to work towards improving the miners' quality of life as evidenced by improved pulmonary function.

Evidence: The FY 2005 Non-Competing Continuation Guidance includes information on the grantees' health care plans and annual measures.

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: Since implementation of the BLCP in 1985, HRSA has only conducted one independent assessment of the program. While this assessment included information on the structural and operational characteristics of the BLCP and estimated costs for delivering services, the assessment did not provide a rigorous evaluation of the program's effectiveness. Moreover, the needs assessment did not cover all BLCP clinics. For example, only 66 percent of clinics completed the mail survey and only 63 percent of clinics completed at least one cost-related question. Furthermore, of the community health clinics, only 10 percent completed cost related information.

Evidence: Peterson Consulting provided a power point presentation to HRSA on the BLCP needs assessment on May 20, 2003 and a final report on June 25, 2003. This was the only independent assessment conducted.

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: It is not clear from the BLCP how grantees are using provided resources. The BLCP has not tied annual budget requests to accomplishments of annual and long-term performance goals. Moreover, the BLCP has not fully developed indirect and direct cost allocations related to program goals.

Evidence: HRSA 2007 Performance Budget

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: The BLCP is revising the data-reporting system to collect a wider range of elements relating to program performance, including the newly defined performance measures. The BLCP has also worked with grantees to develop strategic planning in their business and health care plans. To expand outreach mechanisms, the BLCP awarded additional outreach grants in FY 2005 to target the mining population through additional outreach coordinators, printed materials, and radio-TV addresses. HRSA also conducted site visits in 2004 to learn more about grantee operations and issues around Black Lung.

Evidence: The FY 2006 Non-Competing Continuation Black Lung Clinics Announcement that includes the health care and business plan and enhanced outreach grants.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: The BLCP does not collect quarterly performance information from grantees or other program partners. In addition, there is no evidence that the program uses performance information to adjust program priorities, allocate resources, or take other appropriate management action. In addition, the BLCP has not established a mechanism for disseminating information on best practices across grantees. The needs assessment found that sites feel they are not kept up-to-date and that information is not provided uniformly to all sites.

Evidence: The current database does not include performance information discussed in the FY 2005 grant application. The grantees only provide annual data through the application review, which includes basic demographic information. In addition, the 2003 needs assessment found that grantees should meet regularly with sub grantees to provide program updates, share program resources, and provide technical assistance. The baseline assessment also indicated that grantees did not have appropriate planning measures to ensure goals were met.

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Project officers monitor re-budgeting requests from grantees through a formal approval process with the Division of Grants Management and review all annual non-competing budget requests for appropriateness within the approved grantee project scope. Failure to comply with required financial and/or programmatic reporting requirements may result in: 1) restrictions on the ability to drawdown grant funds, and/or 2) removal or reduction of funding for the second or third year of continuation funding. The program managers also annually review performance results on the two annual measures and place conditions on the grantees if they do not meet performance targets.

Evidence: The Policy Information Notice 2002-08, HRSA Black Lung Clinics Program Expectations and Principles of Practice: A Resource Guide for Black Lung Clinic Programs provides grantee compliance requirements.

Are funds (Federal and partners') obligated in a timely manner, spent for the intended purpose and accurately reported?

Explanation: The program obligates grant awards and supporting contracts in a timely manner. Grantees undergo annual audits to ensure funds are accurately reported and the program requires grantees to produce a Financial Status Report (FSR) and reconcile annual A-133 audits to ensure funds are spent for the intended purpose. Programs that are out of compliance or deficient in financial management issues have conditions placed on their grant award that may reduce funding.

Evidence: Grantee financial reporting directions and regulations are included in the BLCP Notice of Grant Award. Financial status is reported in the SF 269 Financial Status Report.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: HRSA has taken several steps over the past few years to improve efficiencies and cost effectiveness in the BLCP through electronic management and consolidating and streamlining the application review process. For example, as of 2006, 100% of grantees submitted annual data electronically; this represents an increase of more than 200% from the past three years. The BLCP also measures the cost effectiveness of the program through the following measure: Increase the number of medical encounters per million of federal funding: Baseline: 2005 3570 encounters/million $ Targets: 2006 3590 encounters/million $ 2007 3610 encounters/million $ 2008 3630 encounters/million $

Evidence: Programmatic requirements for measuring efficiency and effectiveness are located in the HRSA direction to Grantees: Program application guidance (2005).

Does the program collaborate and coordinate effectively with related programs?

Explanation: HRSA collaborates with NIOSH through its program requirements that BLCP programs have access to B-readers to review x-rays of their patients. A B-reader is a certified physician that has the ability to accurately and precisely read and interpret chest x-rays. Although only 48 percent of programs have the B-reader capabilities on site, all clinics have access to a B-reader to provide services as needed through coordination with NIOSH. HRSA BLCP grantees collaborate with DOL to assist their patients with DOL benefit applications. The 2003 needs assessment found that grantees are well connected to community-based organizations and health care providers in their service areas through a variety of formal and informal working relationships. HRSA has worked to strengthen its relationship with the National Coalition of Black Lung and Respiratory Disease Clinics, a BLCP member organization, for program input and recommendations. Finally, the 2003 needs assessment found that the BLCP "appears to be well connected to the community-based organizations and health care providers in their service areas." For example, 68 percent of Black Lung clinics have protocols for coordinating care with other providers.

Evidence: The 2002-08 Black Lung Program Expectations and Principles of Practice outline the requirements that programs comply with both the NIOSH B-reader certification program. The needs assessment final report discusses grantee collaborative efforts in Section 7-1. The DOL Coal Mine Workers' Compensation Program requires x-rays read by B-readers as part of the application process.

Does the program use strong financial management practices?

Explanation: In 2005, HHS received a material control weakness for its financial systems and processes. HRSA contributes to the material internal control weakness identified in the 2005 HHS audit. HHS is in the process of resolving these weaknesses by replacing existing accounting systems within HHS with the Unified Financial Management System (UFMS). UFMS is scheduled to be operational for HRSA in October 2006.

Evidence: Since 2003, HRSA has not been included in a consolidated HHS audit. In a 2005 audit of HHS, Ernest and Young found a material weakness in HHS financial systems and processes. In particular, the audit found that documentation regarding significant accounting events, recording of non-routine transactions and post-closing adjustments, as well as correction and other adjustments made in connection with data conversion issues must be strengthened and that processes to prepare financial statements need improvement. In addition, the financial systems are not FFMIA compliant. In addition, the audit found PSC's DFP CORE accounting system, which supports the activities of HRSA, did not facilitate the preparation of timely financial statements and did not have an efficient mechanism in place to compile accounting statements.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Unified Financial Management System (UFMS) is designed to improve funds control and provide real-time data. In addition to streamlining the accounting process through UFMS, the BLCP monitors funds received from grantees through annual Independent Financial Audits. HRSA also implemented a web-based data collection system through the Electronic Handbook on the HRSA GEMS site to improve the data quality and elements collected. To improve program planning, HRSA also worked with consultants to incorporate a health care and business plan in grantee applications to provide HRSA program staff concrete information on grantee goals.

Evidence: HRSA convened a panel of experts from Federal, State, and academic arenas in a Clinical Consensus Conference to create guidelines for the management of lung disease caused by exposure to coal dust. The 2005 Black Lung Clinics Program Non-Competing Continuation Guidance, Section VI describes the health care and business plan. HRSA plans to update the performance database by FY 2007.

Are grants awarded based on a clear competitive process that includes a qualified assessment of merit?

Explanation: The Program issues a new open competitive grant cycle every five years. HRSA distributes all initial (new) awards and competing continuations for BLCP grants through a competitive process, which includes an independent merit review and ranking of applications. The committee reviews the project plan and budget based on criteria (out of 100 points) announced publicly in the application guidance.

Evidence: The procedures for review of grant applications are provided in the HHS Awarding Agency Grants Administration Manual AAGAM Chapter 2.04.104C. Transmittal No. 03.01.

Does the program have oversight practices that provide sufficient knowledge of grantee activities?

Explanation: The HHS Office of Inspector General found that HRSA did not consistently adhere to policies intended to safeguard agency funds by using the Alert List, a mechanism intended to help increase monitoring and oversight of grantees considered at risk

Evidence: A May 2006 OIG Report, "Use of the Departmental Alert List by HRSA", found that HRSA did not consistently adhere to policies governing the use of the Department of Health and Human Services Alert List, a list that alerts agencies of grantees who pose a potential financial risk. The report found that HRSA did not consistently place grantees on the list, nor did it consistently check the list or document monitoring of grantees on the list.

Does the program collect grantee performance data on an annual basis and make it available to the public in a transparent and meaningful manner?

Explanation: Although the BLCP collects data on an annual basis, HRSA does not share this information with the public or the National Coalition of Respiratory and Black Lung Clinics, which plays an active role in promoting activities of the BLCP. Although the BLCP consolidates data at the program level, HRSA does not disseminate best practices at the grantee level. For example, the 2003 needs assessment found that HRSA does not provide sites timely information.

Evidence: Grantees report BLCP data annually in the table format, HRSA 5-H37-06-001 as required by the Non-Competing Continuation Guidance. Peterson 2003 Needs Assessment.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: There is no historical data for the long-term measure.

Evidence: There is no historical data for the long-term measure.

Does the program (including program partners) achieve its annual performance goals?

Explanation: Increase the number of miners served each year. The BLCP has struggled with outreach capacity demonstrated by the fluctuation in the number of miners served. However, the BLCP has taken immediate steps to address this issue by providing additional funds to grantees for targeted outreach grants. Grantees have used these additional funds to hire additional outreach staff, increase pamphlets and/or provide radio/TV addresses. Increase the number of medical encounters for Black Lung each year. Grantees achieved the goal of increasing the number of medical encounters over the last three-year period. Even though the number of miners decreased, the number of medical encounters increased. This demonstrates that miners are receiving treatment that is more appropriate and are returning after the initial screening for treatment.

Evidence: Approved annual goals: The outreach grants have been successful in increasing the number of miners served by 19 percent in FY 2005. In addition, the number of medical encounters per Black Lung increased by 37 percent in FY 2005. 1) Increase the number of miners served each year. 2008: 11,005 2007: 10,933 2006: 10,862 2005: 10,790 2004: 9055 2003: 10,837 2) Increase the number of medical encounters for Black Lung each year. 2008: 21,260 2007: 21,121 2006: 20,983 2005: 20,844 2004: 15,191 2003: 18,963

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BLCP has improved efficiencies and cost effectiveness through electronic management of the application review process and reports on the BLCP efficiency measure. 2008: 3630 encounters/million $ 2007: 3610 encounters/million $ 2006: 3590 encounters/million $ 2005: 3570 encounters/million $ 2004: 2600 encounters/million $

Evidence: Efficiency Measure: Increase the number of medical encounters per $1 million in federal funding. The BLCP increased the number of medical encounters per million from FY 2004 to FY 2005 by 37%.

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: The BLCP is unique in providing targeted treatment and prevention services to the mining population. Although Medicaid and the Department of Labor provide benefits to a portion of this community, they do not provide comprehensive services or education specifically tailored to the needs of the entire mining population.

Evidence: DOL Regulations 20 CFR Part 727 and 30 USC 901.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: The BLCP started in 1985; however, the BLCP lacks an independent evaluation of the program. Peterson Consulting conducted a needs assessment of the BLCP in 2003; however, this assessment only provided a demographic snapshot. The assessment did not cover all clinics and did not evaluate program effectiveness.

Evidence: Needs Assessment of the BLCP conducted by Peterson Consulting in FY 2003.