ExpectMore.gov


Detailed Information on the
Broadcasting to East Asia and Eurasia Assessment

Program Code 10002294
Program Title Broadcasting to East Asia and Eurasia
Department Name Broadcasting Board of Governor
Agency/Bureau Name Broadcasting Board of Governors
Program Type(s) Direct Federal Program
Assessment Year 2004
Assessment Rating Moderately Effective
Assessment Section Scores
Section Score
Program Purpose & Design 100%
Strategic Planning 88%
Program Management 100%
Program Results/Accountability 50%
Program Funding Level
(in millions)
FY2007 $79
FY2008 $89
FY2009 $82

Ongoing Program Improvement Plans

Year Began Improvement Plan Status Comments
2005

Designing and implementing additional performance measures that capture the quality and credibility of the radio and television programs.

Action taken, but not completed BBG added program-wide measures of "credibility" (percent of the audience assessing BBG news and information as trustworthy or very trustworthy) that have been reported by entity since 2004. It is expected that TV quality scores will be defined in 2008 for use in 2009, and the criteria for Internet website quality evaluations has been finalized by the BBG.
2005

Revising the current agency strategic plan to build on a market-based approach to broadcasting.

Action taken, but not completed The BBG continues to finalize the presentation strategy for employees and the general public on the new strategic plan. The 2008-2013 plan carries forward the marrying the mission to the market strategy while acknowledging major new challenges and incorporating new strategies to meet them. The core elements are final and included in the FY 2009 Budget Request.
2007

Designing and implementing additional long-term performance measures that align with the BBG mission and vision.

Action taken, but not completed The BBG Strategic Plan 2008-2013 adds ??to enhance understanding?? to its mission statement. BBG is testing survey questions to capture data that will measure the impact of BBG programming to enhance understanding, through credible news and information to target audiences, in support of U.S. strategic interests. Also, as they are reassessed, programs will add the long-term outcome measure of ??credibility?? for PART reporting.

Completed Program Improvement Plans

Year Began Improvement Plan Status Comments
2005

Ensuring that funding for broadcasting to East Asia and Eurasia is explicitly linked to the accomplishment of relevant performance goals.

Completed BBG's 2006 and 2007 Budget request to OMB presented integrated performance information, and funding for this program was linked to the accomplishment of following performance objectives: "Expand audience reach in strategic locations in the Near East, South, Central Asia and Eurasia;" and "Focus broadcasting to audiences of strategic priority in East Asia."

Program Performance Measures

Term Type  
Long-term Outcome

Measure: Audience Reach across East Asia (millions of listeners and viewers). Note: Since the initial estimate and long-term target, Audience Reach data is not available for most of Asia, primarily due to local restrictions on conducting audience research; therefore, regional estimates are not statistically adequate. New survey or estimation methodologies are being explored, and total Audience Reach for the program will be reported when available. However, where possible, individual language service's Audience Reach measures are reported as annual outcome measures for PART.


Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting to East Asia. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in East Asia.

Year Target Actual
2003 n/a 21.7
2004 n/a n/a
2005 n/a n/a
2006 n/a n/a
2007 n/a n/a
2008 n/a TBD
2009 n/a
2010 n/a
2013 25
Long-term Outcome

Measure: Audience Reach across Eurasia (millions of listeners and viewers). Note: Since the initial estimate and setting the long-term target, Audience Reach data is not available for most of Asia, primarily due to local restrictions on conducting audience research; therefore, regional estimates are not statistically adequate. New survey or estimation methodologies are being explored, and total Audience Reach for the program will be reported when available. However, where possible, individual language service's Audience Reach measures are reported as annual outcome measures for PART.


Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting to Eurasia. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Eurasia.

Year Target Actual
2003 n/a 21
2004 n/a n/a
2005 n/a n/a
2006 n/a n/a
2007 n/a n/a
2008 n/a TBD
2009 n/a
2010 n/a
2013 25
Annual Outcome

Measure: Audience Reach in China, for VOA broadcasting in Mandarin. (%)


Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Mandarin.

Year Target Actual
2002 n/a 0.5%
2003 n/a 0.3%
2004 n/a 0.3%
2005 n/a 0.2%
2006 0.2% 0.2%
2007 0.5% 0.2%
2008 0.3% TBD
2009 0.5%
2010 0.7%
Annual Outcome

Measure: Audience Reach in Russia, for RFE/RL broadcasting (%). Note: in 2005, all international broadcasting was removed from urban FM Russian affiliate stations and popular regional networks and these restrictions continue to be imposed. However, RFE/RL continues to broadcast through limited local FM partners and via shortwave and medium wave frequencies.


Explanation:A key indicator of whether or not BBG broadcasts are reaching their intended audience is the number of people who choose to listen to or view BBG programs. Audience Reach, also called weekly audience, is a measure of the regular listening/viewing audience. Regular audience is defined as all adults in a given population listening/viewing at least once a week, as determined by an audience survey that has an adequately designed sample. The number is a reflection of several annual common measures, such as program quality and the audience's awareness of the program, used for all BBG language services broadcasting. Annual measures of audience size, as well as annual performance goals for each individual service, factor into the overall Audience Reach for broadcasting in Russian by RFE/RL.

Year Target Actual
2002 n/a 1.4%
2003 n/a 5.6%
2004 n/a 3.8%
2005 n/a 2.1%
2006 2.5% 2.6%
2007 3.0% 0.9%
2008 2.0% TBD
2009 2.2%
2010 2.3%
Annual Output

Measure: Audience Awareness in China, for VOA broadcasting in Mandarin. (%) Audience Awareness is the percent of the sampled population, prompted or unprompted, that acknowledges BBG station names in an audience survey that has an adequately designed sample. Note: This measure can fluctuate from year to year based on whether programs are transmitted on preferred media, are placed with popular affiliates, or have sufficient advertising. If annual declines are noted, future targets are adjusted.


Explanation:The statistic is expressed as a percent of the sampled population that knows or recognizes BBG stations, which can then translate to the percent of the entire population of the country or region of which the sample was representative. This measure of the extent to which a target population recognizes BBG is influenced by efforts made by each language service such as ensuring that programs are transmitted on preferred media, are placed with popular affiliates, and have sufficient advertising. Annual measures of the percent of a target audience that is aware of BBG programming are tracked by each individual language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of BBG programming. The data shown here is specific to VOA broadcasting in Mandarin.

Year Target Actual
2003 n/a 2.0%
2004 n/a 6.0%
2005 n/a 6.0%
2006 8.0% 12.0%
2007 8.5% 6.0%
2008 6.5% TBD
2009 7.0%
2010 7.5%
Annual Outcome

Measure: Program Quality Score for VOA broadcasting in Indonesian. (Range 1-4; 4 being highest) Note: Within the range, small variations are common, but maintaining the overall rating, or progressing towards the next rating is significantly ambitious, especially as the rating nears the perfect score of 4.


Explanation:The Program Quality Score is an outcome measure based in part on how the target population perceives the quality of the specific BBG language services. This measure is an industry standard used to determine the target audience's assessment of the quality of media. A monitoring panel of regular listeners from the target population rates the program's accuracy, reliability, authoritativeness, objectivity, comprehensiveness, and other variables reflecting distinct statutory, policy, and mission mandates for the different stations. Another component of the Program Quality Score is the evaluation of IBB delivery systems (engineering and transmission), marketing and program placement efforts, and other broadcasting support functions. Program content and presentation are evaluated by both internal and external reviewers to determine the overall Program Quality Score. In total, the Program Quality Score is a comprehensive, weighted score, which ranges from 1-4, with 1.0-1.3 = poor; 1.4-1.6 = poor to fair; 1.7-2.3 = fair; 2.4-2.6 = fair to good; 2.7-3.3 = good; 3.4-3.6 = good to excellent; 3.7-4.0 = excellent. Annual measures of the perceived quality of BBG programming are tracked by each language service, and directly support the long-term measures of growing the size of the audience and increasing the overall perceived reliability of the programming. The data shown here is specific to VOA Indonesian.

Year Target Actual
2002 n/a 3.0
2003 n/a 3.1
2004 n/a 3.0
2005 n/a 3.3
2006 3.3 3.4
2007 3.4 3.4
2008 3.5 TBD
2009 3.6
2010 3.7
Annual Efficiency

Measure: Cost-per-audience member for Radio Free Asia's broadasting in Khmer to Cambodia. ($)


Explanation:This efficiency measure tracks the cost per audience member of each language service. Cost per audience member is the total cost of programming and delivery divided by the unduplicated audience for all media, and is calculated for each language service. The ratio (measure) must be evaluated in context of the changes in either the cost or audience size. For example, a short-term capital cost increase to improve transmission may not increase audience size until after a lengthy installation. In some cases, audience size may be kept artificially low by restrictions imposed by the host government or increased competition, not necessarily poor programming or inefficient management. Costs per audience member may fluctuate annually and also differ compared to other language services based on a combination of factors in each location, including: preferred media in the region, hours of programming, competition from other broadcasters, jamming, and government restrictions. The data shown here is specific to Radio Free Asia's broadcasting in Khmer to Cambodia.

Year Target Actual
2002 n/a $7.43
2003 n/a $14.40
2004 n/a $5.14
2005 n/a $3.42
2006 $3.25 $1.49
2007 $3.19 $0.81
2008 $0.80 TBD
2009 $0.78
2010 $0.75

Questions/Answers (Detailed Assessment)

Section 1 - Program Purpose & Design
Number Question Answer Score
1.1

Is the program purpose clear?

Explanation: The program purpose is to promote and sustain freedom and democracy by broadcasting accurate and objective news and information about America and the world to audiences overseas (in this case, audiences in East Asia and Eurasia).

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter

Yes 20%
1.2

Does the program address a specific and existing problem, interest or need?

Explanation: The program is meant to address the need for accurate news and information about the United States and events in East Asia and Eurasia in languages spoken in those countries.

Evidence: The dearth of free, unbiased media and the existence of anti-American reporting in these regions creates a need for timely and accurate reporting of U.S. policies and actions as well as events in the regions and the world at large. Freedom House Press Freedom Ratings for East Asia countries is overwhelmingly "Not Free" while only Greece in Eurasia is rated "Free".

Yes 20%
1.3

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The Broadcasting Board of Governors (BBG) is the independent Federal entity responsible for all U.S. government and government-sponsored, non-military international broadcasting. As such, its broadcasts are not duplicative of other Federal contributions to the problem being addressed in East Asia and Eurasia. In those countries that have broadcasting by both Voice of America (VOA) and Radio Free Europe/Radio Liberty (RFE/RL) or Radio Free Asia (RFA), the BBG ensures the missions are distinct, complementary and not duplicative.

Evidence: The U.S. International Broadcasting Act of 1994 and the Foreign Affairs Reform and Restructuring Act of 1998 state that U.S. international broadcasting should not duplicate the activities of private United States broadcasters or the activities of government-supported broadcasting entities of other democratic nations. The FY 2004 Program Plan provides details of reductions and streamlining of dual broadcasting within BBG.

Yes 20%
1.4

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: There is no conclusive evidence that another approach would be more efficient/effective at achieving the program's goals. To further achieve its goals, in 2004 VOA announced a new structure which merges VOA and the former WORLDNET TV to create a focused, multimedia organization that will reach larger audiences with enhanced programming in the most effective media.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter; VOA and WORLDNET TV merger.

Yes 20%
1.5

Is the program effectively targeted, so that resources will reach intended beneficiaries and/or otherwise address the program's purpose directly?

Explanation: The BBG is the sole Federal entity that broadcasts to East Asia and Eurasia for the purpose of delivering accurate and objective news and information about the United States and the world. Where relevant, VOA and the surragate broadcasters (RFA, RFE/RL) "window" their broadcasts to reach the intended audience in the programming they are most interested. "Windowing" is a process by which programming from one broadcaster (i.e., RFA) is included in programming by another broadcaster (i.e., VOA). For example. VOA Albanian has large audience numbers, so to improve efficiency RFE/RL's direct radio broadcasts in Albanian to Kosovo will be cut and RFE/RL's programming will be included in VOA broadcasts. Annual Language Service and Program reviews continually refine the target audience and broadcast in the preferred medium, such as combatting anti-jamming efforts in China and Korea and improved internet capability.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter; East Asia and Eurasia Program Reviews and annual Language Service Reviews

Yes 20%
Section 1 - Program Purpose & Design Score 100%
Section 2 - Strategic Planning
Number Question Answer Score
2.1

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: BBG's East Asia and Eurasia performance plans include a long-term outcome measure that ties annual language service activities to the BBG's overall strategic goals. The primary long-term measure is to increase audience reach. Given that measuring the program's impact on sustaining freedom and democracy in East Asia or Eurasia is virtually impossible, audience reach is the most important and quantifiable measure of the program's success. This measure is a culmination of a number of annual common measures used across all of the VOA, RFA and RFE/RL language services, as well as individual annual action steps for each broadcasting area. The impact of annual activities should result in increased audience awareness across the program and an overall growth in audience size. BBG continues to explore additional or improved long-term measures or goals that address the quality and relevance of broadcasting.

Evidence: BBG Strategic Plan. East Asia and Eurasia individual performance plans.

YES 12%
2.2

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: The target for the primary long-term performance measure of increasing audience reach is sufficiently ambitious given the large populations of the regions BBG is trying to reach and the closed nature of the societies. For East Asia, the FY 2008 target is to increase audience reach by 3% for the region overall. This region includes the world's largest population, with significant jamming of broadcasts coming from China and North Korea. For Eurasia, BBG's FY 2008 target is to increase audience reach by 2% for the region overall. The baseline for these targets is the overall regional FY 2003 audience reach level. The related annual goals and action steps will track the progress toward achieving this long-term goal.

Evidence: East Asia and Eurasia performance plans; Language Service Review; Program Review

YES 12%
2.3

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: The BBG's East Asia and Eurasia broadcasts have annual performance measures that support the long-term goal. There are 5 common measures for the program: 1) percent of weekly audience, 2) percent of audience awarenss, 3) program quality rating, 4) signal strength rating, and 5) cost per listener. The last common measure - cost per listener - is an efficiency measure. BBG is also developing measures to assess the impact of new delivery medium, including the internet. Each language service or area also has, or is developing, individual annual action steps with milestones tailored to the nature of the broadcasting and target population. These contribute to achieving the annual and long-term goals.

Evidence: East Asia and Eurasia performance plans; Language Service Review; Program Review

YES 12%
2.4

Does the program have baselines and ambitious targets for its annual measures?

Explanation: The common measures have separate baselines and annual targets for each broadcasting area, and in many cases for VOA and the grantees separately. BBG established these separate targets so that language service managers could easily track performance for each country or region they broadcast to, helping them identify areas that need improvement. Some of the measures for 2005 have targets that are lower than previous years. The research methodology for several of the measures has been adjusted in the past year and while this has yielded much more specific data on which to base future targets, in some cases it has resulted in the targets for future years being revised downward. However, the targets for BBG's priority language services remain ambitious.

Evidence: Program individual performance plans; Language Service Review; Program Review

YES 12%
2.5

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: As grantees of BBG, RFA and RFE/RL contributed to establishing the new Strategic Plan and committed to the FY 2006 performance plans for East Asia and Eurasia, including the annual action steps for broadcasting that support the overall BBG long-term goals in the program.

Evidence: East Asia and Eurasia performance plans; Annual Language Service Reviews; Program Reviews; BBG Annual Performance Plans and Reports.

YES 12%
2.6

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: BBG's annual Language Service Reviews and Program Reviews conducted by the BBG's Office of Program Review (OPR) serve this purpose. OPR operates independently from all BBG language services. Annual reviews of VOA language services include an initial assessment with regularly scheduled follow-up meetings to check the progress on recommendations generated by the review. The reviews assess radio content, engineering and transmission, marketing efforts, internet development, and production value. BBG contracts with InterMedia, an independent firm that conducts annual surveys for each language service and program,including VOA, RFA and RFE/RL. InterMedia evaluates a variety of factors: market profile, media access and use, and international broadcasting impact and trends. InterMedia also profiles audience listening habits.

Evidence: The annual Language Service Review assesses two basic issues: (1) where should BBG broadcast and (2) how well is BBG broadcasting. The Program Reviews are annaul uality-control mechanisms based on field research and external analyses of program content and presentation. FY 2004 Program Review Schedules.

YES 12%
2.7

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: To date, BBG has not uniformly linked budget requests to its strategic goals or performance goals for its programs. In FY 2006, the BBG plans to include annual performance plans with performance measures for East Asia and Eurasia in the budget request. BBG is also revising its capital planning process. In FY 2005, most budget enhancement requests were accompanied by performance goals. In the FY 2006 budget submission, BBG will ensure that all budget enhancement requests for the Agency will be performance based with measures, goals and annual targets. The BBG intends to build upon this progress with fully integrated performance-based budgets in the future.

Evidence: FY 2005 Budget Request; language Service Review; Program Review; FY 2005 BBG Budget Request

NO 0%
2.8

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: BBG continues to use the individual language service or broadcast area performance plan template, including broadcasting to East Asia and Eurasia. The performance plans are based on the BBG strategic plan, and they attempt to tie individual program goals to the strategic goals of the agency. The annual Language Service Review is used to identify and update strategic broadcasting priorities. BBG also continues to review and revise the current agency strategic plan to build on the innovative new approach to international broadcasting, "Marrying the Mission to the Market." A newly reported common measure of Audience Awareness will further strengthen the goal of reaching the audience and gauging the effectiveness of marketing, branding and affiliations.

Evidence: BBG Strategic Plan, East Asia and Eurasia Performance Plans, FY 2003 Language Service Review

YES 12%
Section 2 - Strategic Planning Score 88%
Section 3 - Program Management
Number Question Answer Score
3.1

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: Using performance data gathered through its annual Language Service Review and Program Reviews, the BBG examines the results to adjust resources and set strategic priorities. Program Reviews also include content and production analysis to assist managers in improving performance. These same data are included in the Performance Plans.

Evidence: Language Service Review, Program Review, East Asia and Eurasia Performance Plans.

YES 17%
3.2

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Senior managers' "Line of Sight" performance appraisal system requires critical elements and ratings that are tied to the objectives of the BBG. The annual Language Service Review holds managers and partners directly accountable for results by the corresponding resource allocations, including the grantees broadcasting to East Asia and Eurasia. Contractors are accountable to the same program standards. The Program Review process also measures the quality of their work. The grantees must report the same common measures. Many contractors work on a "pay-for-product" basis, with payment dependent upon the timely receipt of quality products. For example, many "stringers" receive payment for each quality news story they deliver.

Evidence: "Line of Sight" Performance Standards/Appraisals; Manual of Operations and Administration, Sec. 490 - Performance Appraisal System; Language Service Review

YES 17%
3.3

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All requisitions must include verification that funds are available, and the funds are obligated when a contract or agreement is signed or a purchase order is done. The mid-year and 3rd quarter review provides an opportunity for managers to verify that unliquidated obligations are valid and that once an obligation has been fully expended, remaining balances are deobligated. Financial transactions are coded to both an organization and to a function or project. Given that BBG has a high percentage of fixed costs (salaries, transmission and infrastructure expenses, rent, etc.), funds must be obligated in a timely manner in order to continue operations. The BBG requires an annual financial plan and monthly status reports from its grantees. The financial plans must detail anticipated costs for specific line items by each month of the fiscal year. BBG conducts a mid-year, 3rd quarter, and year-end review of each grantee's financial plans and budgets to monitor funding and expenditures. BBG also requires a monthly status report of expenditures, disbursements and obligations as well as the submission of the SF-169, Financial Status Report and SF-172, Report of Federal Cash Transactions.

Evidence: The reviews and the allotment control procedures enable the Agency to track obligations to ensure that funds are used for the intended purpose by the appropriate Agency office. SF-169, Financial Status Report and SF-172, Report of Federal Cash Transactions. Montly status reports from RFE/RL and RFA on expenditures, disbursements, obligations.

YES 17%
3.4

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: The annual Language Service Review process includes an efficiency measure -- cost per listener. The Program Review process also assesses the efficiency and productivity of each language service. The merger of VOA and WORLDNET TV will improve efficiency through utilization of multimedia resources to effectively reach the audience in the preferred media. BBG recently reorganized its various IT components into one structure to avoid duplication of effort and centralize efficiencies.

Evidence: East Asia and Eurasia Performance Plans; Language Service Reviews; Program Review ; VOA and WORLDNET merger; IT restructuring

YES 17%
3.5

Does the program collaborate and coordinate effectively with related programs?

Explanation: BBG is solely responsible for all non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through broadcasting objective news and information about the U.S. and the world to audiences in Eurasia and East Asia. There are no related programs that share similar goals and objectives with which the BBG can collaborate. Within the BBG, VOA, RFE/RL and RFA collaborate on program content and the BBG has worked to reduce the duplication.

Evidence: U.S. International Broadcasting Act of 1994; Foreign Affairs Reform and Restructuring Act of 1998; Voice of America Charter

NA 0%
3.6

Does the program use strong financial management practices?

Explanation: The BBG is cross-serviced by the State Department to provide financial services using the BFMS system. Internal controls are in place to minimize erroneous payments. The BBG Administrative Officers obligate financial transactions, and the State Department processes the payments, thus ensuring checks and balances on BBG accounts.

Evidence: BFMS System. The State Department has received clean audit opinions.

YES 17%
3.7

Has the program taken meaningful steps to address its management deficiencies?

Explanation: The Office of Program Review conducts an independent analysis of program content and presentation, which includes the management of the program. The Action Plans resulting from the Program Reviews lay out specific and meaningful steps to correct deficiencies when they are identified. The Office of Program Review regularly follows up with managers to track progress. The grantees follow a similar process and report the common measures to the BBG. The senior managers' "Line of Sight" performance appraisal system requires critical elements and ratings based on the mission and is tied to the objectives of the BBG.

Evidence: Program Reviews; BBG Manual of Operations and Administration "Line of Sight" Performance Appraisal System, Section 490

YES 17%
Section 3 - Program Management Score 100%
Section 4 - Program Results/Accountability
Number Question Answer Score
4.1

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The data for East Asian audience reach is unavailable for a number of language services, due in large part because it is difficult to survey listeners and gather accurate data in closed socieities. For this reason, it is difficult to track progress in this region. Those that can be tracked have had small increases in audience reach over the past several years, and indications are that this will continue. Nearly all of the language services in Eurasia show modest increases annually in audience reach, indicating that they are making progress toward reaching the long-term target of increasing by 2% across the region.

Evidence: Language Service Review, Progam Review, East Asia and Eurasia Performanc Plans

SMALL EXTENT 8%
4.2

Does the program (including program partners) achieve its annual performance goals?

Explanation: This is the first year the BBG has provided targets for the common measures for the Eurasia and East Asia language services. For many of the measures, BBG anticipates the level to decrease from FY 2003 to FY 2004, with slight increases the following years. This is due primarily to adjustments to the research methodology that yield more accurate data, resulting in some downward revisions in targets. BBG anticipates maintaining the current levels for many of its lower priority language services, while aiming for improvements in high priority broadcasting areas. Anecdotal evidence also indicates that there has been improvement in broadcasts quality, content, and listener response.

Evidence: East Asia and Eurasia Performance Plans, Program Reviews, Language Service Reviews.

SMALL EXTENT 8%
4.3

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The BBG has a common measure for all language services, cost-per-listener, that tracks the cost effectiveness of broadcasting. Since the weekly audience listening rate is not available for many of the language services in Eurasia and East Asia, BBG cannot track the cost-per-listener of all language services. Those with data indicate that the cost-per-listener, particularly in East Asia, has decreased significantly in the past several years. For most language services the cost-per-listener in future years is anticipated to decrease further. However, the agency anticipates increases in some of its largest services (including Mandarin and Belarusian) in FY 2004, with costs decreasing again in FY 2005 as the number of listeners increases and BBG achieves more efficiencies. RFE/RL's Ukrainian will have an increased cost-per-listener in FY 2005 because the Ukrainian government removed RFE/RL from the only national FM network, decreasing the audience reach. BBG has achieved efficiencies through reducing duplicate broadcasting and integrating content generated from one broadcaster (e.g. RFE/RL) into the programming of another (e.g. VOA).

Evidence: East Asia and Eurasia Performance Plans, Program Reviews, Language Service Reviews.

LARGE EXTENT 17%
4.4

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: BBG is solely responsible for all non-military, U.S. international broadcasting. It is the only entity with the express objective of promoting freedom and democracy through broadcasting objective news and information about the U.S. and the world to audiences in Eurasia and East Asia. There are no related programs that share similar goals and objectives with which to compare BBG's performance.

Evidence:  

NA 0%
4.5

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Annual reviews indicate that BBG broadcasting to East Asia and Eurasia is essential to bring news and information about the U.S. and the world to these repressed regions. The major challenges are reaching the target audience in light of many restrictions placed by governments while also transitioning to the preferred delivery means and format in the diverse media market. This is reflected in the wide range of weekly audience reach and awareness, from less than 1% audience reach, primarily because of jamming, in China to over 50% awareness in all languages in Eurasia. BBG language services in East Asia and Eurasia have achieved results by utilizing the results of research and program review evaluations to improve their effectiveness. The Indonesia Service started with only a 0.5% audience share in 1999 and doubled its audience to 1% by 2000. By building the network of its affiliates in Indonesia to more than 160, and diversifying its programming mix to include television, the service has seen its audience continue to grow ' in 2001 to 2%, in 2002 to 4% and in 2003 to 6.1%. It now attracts more than 8 million regular listeners and viewers. Throughout East Asia and Eurasia, prominent leaders and media experts publicly comment on the high standards, professionalism and improvement in BBG broadcasting. Surveyed listeners point to the modern production, updating of relevant news, and range of subjects as valuable outcomes of ongoing broadcast evaluations.

Evidence: Program Review schedules, results and action plans; Language Service Review analysis and results. InterMedia research, including surveys, monitoring panels and listener panels; expert and government officials public comment or evaluation.

LARGE EXTENT 17%
Section 4 - Program Results/Accountability Score 50%


Last updated: 09062008.2004SPR