Detailed Information on the
Maritime Administration - Merchant Marine Academy Assessment

Develop additional efficiency measures to demonstrate improved cost effectiveness that are easily comparable with other, similar education programs.

Develop policies and procedures to allocate USMMA resources strategically. This effort is intended to both establish an overall cultural environment of systemic and ongoing strategic planning, and provide a strong foundation for those efforts. This initiative is predicated on three key elements: the base-lining of the core academic, extra-curricular and regimental programs; the review of the core mission statement; and the establishment of a strategically-oriented, top-level internal decision making body, responsible for developing the profile and structure of the Academy's core programs, and resource allocation and reinvestment.

Strengthen USMMA's internal control mechanisms. The Merchant Marine Academy underwent an A-123 audit, administered by the Maritime Administration, in mid-FY2008. The Maritime Administration has developed a corrective action plan, which has been approved by the Department of Transportation. That plan is currently being implemented, and is expected to be completed by FY2009. This effort will support and augment existing efforts by USMMA and MARAD to expand the strategic planning program, and improve resource allocation

Measure: Graduation Rate

Explanation:The performance target is to achieve a graduation rate for each incoming plebe (freshman) class that is exceeds the historical five-year moving average. The U.S. Merchant Marine Academy (USMMA) graduation rate targets are ambitious for instituions of higher learning and significantly higher than performance levels at many State and private insititutions. The USMMA graduation rate targets also compare favorably to graduation rates at other Federal service academies. The undergraduate degree program at the USMMA is notable for its rigor, requiring approximate 170 semester hours of education and professional study - three years of study in residence and one year at sea. In comparison, a typical undergraduate program at a State or private institution requires approximately 120 semester hours. In addition to these academic requirements, graduates must complete physical examinations and officer training to recieve a commission as an officer in the U.S. Naval Reserves or U.S. Merchant Marine Reserves, and complete requirements for licensure by the U.S. Coast Guard. It is also noteworthy that unlike other institutions of higher education and other Federal service academies, the numbers and types of degree programs and academic majors at the USMMA are limited to maritime specialties, in accordance with its Congressional charter. These programs are prinicipally engineering in nature and are often associated with high rates of student attrition. The USMMA employs tutoring, mentoring and other enrollment management programs to reduce the likelihood of resignations and academic failures.

Measure: Deferred Graduation Rate

Explanation:U.S. Merchant Marine Academy (USMMA) deferred graduates are midshipman (students) who do not complete their residential or at-sea curriculum with their graduating class. A lower percentage reflects a positive outcome, i.e., fewer deferred graduates is the desired condition/trend. Deferred graduations typically involve minor administrative issues or personal exigencies that delay satisfaction of all graduation requirements. These situations include academic deficiencies, disciplinary infractions, outstanding fee payments, rehabilitation from physical injuries, family crises, incomplete military security clearances, and other factors outside the direct control of the midshipman and/or the USMMA. In such situations, the Superintendent may designate a midshipman a deferred graduate if outstanding graduation requirements can be completed in a matter of months thereafter and it is not appropriate or cost effective to delay the midshipman's graduation for a full year. Similar policies are in place at the other Federal service academies. Since this performance measure was developed in 2004, the USMMA has implemented a number of programs to resolve issues that would impair or prevent a midshipman's ability to complete graduation requirements on schedule. The outcome of these efforts has been a significat reduction in deferred graduations and performance is now at a level that USMMA administration has deemed an acceptable minimum and they have determined that it is no longer cost effective to devote addtional resources to achieve marginal improvements. Therefore, in 2008, targets for fiscal year 2009 and beyond are reset to 5% and will remain at that level for the foreseeable future.

Measure: Cost per graduate in thousands of dollars

Explanation:The U.S. Merchant Marine Academy (USMMA) efficiency performance target is to achieve an annual cost per graduate (CPG) ratio that is less than the prior year CPG ratio adjusted by the annual percentage increase in the Higher Education Price Index (HEPI). As explained below, both ratios use nominal dollars. The HEPI is a measure of the inflation rate applicable to United States higher education; more precisely, the increase in costs in a defined basket of goods and services typically purchased by institutions of higher education. The index is calculated on a fiscal year basis ending each June 30, by the Commonfund Institute, a branch of Commonfund, a non-profit organization devoted to the management of college and university endowments. Each year, the USMMA budget office determines the proportional cost for each class (freshmen, sophomore, junior, senior) of that year's budget. For example, when the Class of 2007 were freshmen (in 2004), they accounted for 29% of the midshipmen-related costs in 2004. By the time they were seniors (in 2007), that cohort represented only 23% of the annual costs in 2007. To get the final cost-per-graduate for each class, the respective annual costs in nominal dollars are added up, and then divided by the number of graduates. The USMMA projects an outyear targets, then calculates final annual targets in conjunction with the annual HEPI calculation. This measure is used to reflect efficiency improvements that create additional purchasing power needed to fund the rising costs of educating the midshipmen.

Is the program purpose clear?

Explanation: "The program is defined as educating U.S. citizens in marine transportaton and marine engineering disciplines who will earn B.S. degrees, USCG unlimited, oceans merchant licenses and commissions in the reserve component of the U.S. Armed Forces while incurring an obligation to serve the U.S. The Merchant Marine Act, 1936 descibes the program and the responsibilities of the USMMA: *Title I ---Declaration of Policy. SEC 101. Fostering Development and Maintenance of Merchant Marine (46 App. U.S.C. 1101 (2003)) *Title XIII??Maritime Education and Training: ---SEC 1301. Congressional declaration of policy (46 App. U.S.C. 1295 (2003) ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (a) ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (e) ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (g) "

Evidence: "All graduates enter careers that include merchant marine service at sea, maritime employment ashore, and reserve commissioned officers of the U.S. Armed Forces. Merchant Marine Act, 1936: *Title I ---Declaration of Policy. SEC 101. Fostering Development and Maintenance of Merchant Marine (46 App. U.S.C. 1101 (2003)) "It is necessary for the national defense and development of its foreign and domestic commerce that the United States shall have a merchant marine . . . " *Title XIII??Maritime Education and Training (MET): ---SEC 1301. Congressional declaration of policy (46 App. U.S.C. 1295 (2003) ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (a) "The Secretary shall maintain the Academy . . . " ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (e) "Commitment agreements." ---SEC 1303. Maintenance of Academy (46 App. U.S.C. 1295b (2003) (g) authorization to award B.S. and Master's degrees "

Does the program address a specific and existing problem, interest, or need?

Explanation: Program educates individuals to serve as well-qualified U.S. maritime labor to support DOD mobilization requirements while sustaining commerce. USMMA provides the merchant and armed forces reserve officers to maintain a viable US flagged merchant fleet for peacetime commerce and wartime mobilization. Graduates represent the largest component of merchant marine officers and a substantial component of reserve officers to support DOD activities.

Evidence: Graduates meet the intent of the declaration of policy by the following ways: (1) The Maritime Security Program which has been increased from 47 to 60 ships requires USMMA graduates to satisfy USCG licensed officer positions; (2) During Iraqi Freedom, the activation of the Ready Reserve Fleet would not have been possible without USMMA graduates; and (3) In general, sufficient licensed mariners would not be available for strategic sealift during times of war or to meet commercial maritime requirements during periods of peace.

Is the program designed so that it is not redundant or duplicative of any other Federal, state, local or private effort?

Explanation: The federal merchant mariner education policy (Merchant Marine Act of 1936) meets MARAD's long-term goals by ensuring that the fed'l-state-private partnership produces a sufficient number of obligated, qualified merchant mariners to meet the military and civilian strategic sealift needs of the nation. USMMA represents the primary federal element of that partnership, and 100% of its graduates have a 5-yr service obligation. SMA's voluntarily sponsor mariner education programs as part of their state-focused instructional program, and only those SMA grads who receive federal tuition assistance have a 3-yr service obligation. USMMA's intentional program design also ensures its graduates meet the needs of the military logistics and maritime intermodal communities, by providing comprehensive high-level transportation, logistics, and military officer training.

Evidence: (1) Federal maritime education policy is set forth in the Merchant Marine Act of 1936 (Title I--Declaration of Policy), et seq. (2) MARAD Report to Congress, "Maritime Education Program Evaluation" (2003), identified that the number of "...officers from MARAD's programs with service obligations [i.e. USMMA and federal tuition assistance program for SMAs] to feed the pool of mariners...." is, however, inadequate to meet the need for qualified officers under a full-mobilization sealift scenario. (3) USMMA program design is established by 46 App. U.S.C. 1295b, identifying three core components (baccalaureate degree authority, USCG license, USN commission).

Is the program design free of major flaws that would limit the program's effectiveness or efficiency?

Explanation: The program is continually revised based upon internal reviews and reviews by external accrediting groups to correct major flaws. Middle States (accredits USMMA as an entity) in its 1996 10-yr review identified the quarterly calendar as a design flaw, since it placed excessive pressures upon students, faculty, time and resources. A trimester program was developed as a result. A 2000 Middle States review identified a need to develop an instructional IT plan; USMMA hired staff and implemented a comprehensive plan. Their next review is in 2006. USMMA has strengthened the HR program, and implemented a Values Survey of all midshipmen in '04, resulting in program actions. The engineering accreditation commission (ABET) in '02 encouraged the use of outcomes assessment measures for program performance; USMMA concurred and will use them for ABET's accreditation visit in '05. In '01, the joint MARAD/USCG Review Committee evaluated and approved USMMA's STCW 95 program. No major flaws were noted; next review is 2007.

Evidence: Internal review processes: (1) institutional policy statements; (2) academic policy statements; (3) academic programs assessment; e.g., Midshipman performance on USCG license examinations; and (4) a standing curriculum committee. External review processes: (1) academic accreditation (Middle States) and (2) specialized engineering accreditation (ABET). Significant challenges: (1) Lack of a federally funded training vessel (Midshipmen sail aboard commercial and military sealift vessels (less faculty control exerted during the sea periods; but the experience and training are more realistic and the number of sea days is 60% higher than their state school counterparts)); (2) Strategic Plan, developed in coordination with DOT and MARAD strategic planning efforts, encourages continuous institutional improvement; and (3) satisfying ever increasing requirements for maritime/port security education (new courses have been added to an already full curricula).

Is the program design effectively targeted so that resources will address the program's purpose directly and will reach intended beneficiaries?

Explanation: Financial resources are applied in a value-added process to educate trained mariners who contribute to national defense and commerce. Service Obligation Contract required by Title 46 App.US Code: 100% of graduates earn a (1) USCG license as deck or engine officers and are obligated to maintain license currency for at least six years following graduation; (2) reserve officer commission in the US Armed Forces and maintain the appointment for a period of eight years following graduation; and (3) serve the foreign and domestic commerce and national defense of the US for a period of at least 5 years following graduation. Of USMMA's FY05 budget of $55M, $42M is allocated to Academy operational funding and an additional $13M is earmarked for capital improvements (buildings, grounds). It is estimated that approx. 80% ($34M) of operational funds will be spent on direct expenses related to education and training (all academic instruction and professional training, regimental officer training (in-residence leadership instruction), shipboard training, health and physical education.)

Evidence: USMMA midshipmen and graduates serve beneficiary organizations in the U.S. Government (Departments of Defense, Homeland Security, and Transportation as well as the National Security Agency.) Graduates contribute to the strategic sealift initiatives in time of national emergencies. Graduates also contribute to the commercial vitality of our nation by virtue of their employment in domestic ports/terminals and transportation and logistics companies. During the past four years 100% of USMMA graduates either began sailing in the Merchant Marine, or entered active duty in one of the military serves or are employed in the maritime industry shoreside as approved by MARAD; e.g., shipyards, ports, terminals. Additionally, alumni records indicate that 20 years after graduation, 70% of USMMA graduates continue to serve in qualifying positions afloat, the military services, or ashore. Cost of Education data from COE Report Dated 12/27/2004.

Does the program have a limited number of specific long-term performance measures that focus on outcomes and meaningfully reflect the purpose of the program?

Explanation: "Both MARAD and USMMA have comprehensive strategic plans in place which address maritime education. These plans include long-term performance measures. "

Evidence: Long-term measures are identified as PART Performance Measures 1, 2, and 3. Additional performance measures are included as part of the following: (1) MARAD 2003-2008 Strategic Plan, Section VI(F), National Security Strategic Objective and accompanying performance measures; and (2) USMMA 2004-2009 Strategic Plan

Does the program have ambitious targets and timeframes for its long-term measures?

Explanation: USMMA's overall program is very rigorous, and annual improvements in performance are inherently ambitious. (For example, USMMA student retention is already well above the national average for 4-year colleges, so increases of 0.5% or more are significant.) USMMA is mandated by Congress to grant 3 credentials--baccalaureate degree, USCG license (including certification through STCW competency endorsements), and USN commission--within a 4-yr window (3 yrs in residence, 1 yr at sea). Achieving these goals means that USMMA must meet the distinct and demanding standards of four independent external agencies: Middle States, ABET, USCG, and USN. USMMA personnel (faculty, medical, regimental, shipboard training and admin. staff) engage in regular, mandatory assessment of midshipmen academic performance and professional development (15 academic performance review points over four years, 300 days of evaluation by professional merchant mariners, continuous residential performance, conduct and medical evaluations) and set goals based upon those findings.

Evidence: Targets are identified as part of accompanying PART performance measures. Additional data: (1) Periodic Review Report, Middle States, April 2000, (2) Decennial Accreditation Review Report, Middle States, June 1996, (3) Accreditation Report, ABET, 2001, (4) USCG/MARAD STCW Review Audit, February 2001, (5) Periodic on-site reviews of the Ensign Training, USN, (6) National Center for Education Statistics (IPEDS) statistics on student retention.

Does the program have a limited number of specific annual performance measures that can demonstrate progress toward achieving the program's long-term goals?

Explanation: Each long-term performance measure established for PART includes annual targets. In addition, the USMMA strategic planning and management assessment programs include comprehensive and evolving annual performance measures.

Evidence: (1) Specific annual targets are identified as sub-measures of PART Performance Measures 1, 2, and 3; (2) USMMA 2004-2009 Strategic Plan; (3) Academic Dean's Assessment Matrix; and (4) USMMA/MARAD Budget annual submission.

Does the program have baselines and ambitious targets for its annual measures?

Explanation: USMMA has identified measurement baselines and established targets for key areas of performance, based upon extensive and ongoing internal study and program review and adjustment. Consequently, ambitious targes for improvement are measured in small increments of 0.5% or more. Targets involve deferred graduates, i.e. those midshipmen unable to meet all graduation requirements by their scheduled graduation date. There are two leading causes of deferred graduates that are outside the direct control of USMMA. 1) A fixed number of commercial ship cadet berths (representing 83% of the available training days required for USMMA midshipmen, and a smaller but growing portion of SMA training days), creates an increasing demand for berths for USMMA and SMAs. This mitigates against timely completion or remediation of sea time requirements. 2) Medical requirements necessary for USCG license and/or USN commission.

Evidence: Baselines and targets are identified as part of accompanying PART performance measures. Additional data: (1) USMMA and State Maritime Academy Cadet Days on Commercial Ships, USMMA, April 2005, and (2) Maine Maritime Seeks Relief from Fuel Costs, Bangor Daily News, April 30, 2005.

Do all partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) commit to and work toward the annual and/or long-term goals of the program?

Explanation: The complementary design of the USMMA and SMA programs meet MARAD's long-term goals. Additionally, direct partners are: 1) US-Flag shipping companies and Military Sealift Command (MSC), who support USMMA shipboard training program (all midshipmen spend 300 days at sea as part of 4-yr education); 2) US maritime companies (shipping, logistics, transportation, shipyards, naval architecture) and military services, who support midshipman internship program (2-6 weeks); 3) DoD agencies, particularly US Navy, who provide personnel for training and education in military specialties (100% of USMMA graduates have a statutory requirement to serve for 8-yrs in the reserves; at graduation, 25%-35% serve on active duty, many in DoD logistics field). USMMA leverages resources with other partners in the training of merchant mariners to support strategic sealift, maintain an industrial shipyard base, and provide maritime transportation security. These include DOT modal agencies, DoD Transportation Command, state and federal maritime agencies, private sector industries.

Evidence: (1) USMMA Midshipman Shipboard Training Program Statistics; (2) USMMA Midshipman Internship Program Statistics; (3) Responsibilities of the Secretary of the Navy under the Merchant Marine Act of 1936 (46 App. USC 1295, Section 2); (4) Participation in MARAD Ship Operator's Cooperative Program (SOCP); (5) Maritime Security Training Program Standards (developed by USMMA for MARAD); (6) Partnership with National Security Agency (NSA) on its Secure Computing initiative; (7) Defense Threat Reduction Agency (DTRA).

Are independent evaluations of sufficient scope and quality conducted on a regular basis or as needed to support program improvements and evaluate effectiveness and relevance to the problem, interest, or need?

Explanation: The USMMA educational program is subject to comprehensive review and audit by several independent federal and professional entities. For example, as part of the 1993 Strategic Plan initiative, USMMA had extensive consultation with government and industry leaders, which resulted in the addition of two new academic majors to address areas of concern to the maritime transportation community (Logistics and Intermodal Transportation, and Shipyard Management). In addition, the most recent decennial Middle States Reaccreditation Report (1996) identified severe deficiencies in the quarterly academic calendar system, which resulted in a major initiative to convert to a trimester calendar (1997-1998). A 2000 Middle States review identified a need to develop an instructional information technology (IT) plan; USMMA hired staff and implemented a comprehensive plan. In 2001, the joint MARAD/USCG Review Committee evaluated and approved USMMA's STCW 95 program. No major flaws were noted; next review is 2007.

Evidence: (1) USMMA subject to oversight by USMMA Advisory Board (46USC1295), reporting to DOT on programs and policy. (2) Add'l oversight by Congressional Board of Visitors (46USC1295), though it has not convened in 10-yrs. (3) Goal 4 of 04-09 Strategic Plan ensures "...USMMA's continuing relevance and viability as a nat'l asset." Associated action items include: industry outreach, goal-setting, and assessment. (4) As institution of higher education, USMMA is subject to mandatory program audits by independent agencies: (a) 10-yr reaccreditation by Middle States Commission on Higher Education (MSCHE); (b) 5-year interim review by MSCHE; (c) reaccreditation of Marine Engineering Systems and Shipyard Management programs by Accrediting Board for Engineering and Technology (ABET); (d) review and audit by USCG of licensing programs (curriculum review, shipboard training time); (e) general program reviews regularly undertaken by MARAD (Report to Congress: Maritime Education Program Evaluation, 4/03); (f) Dept of Education (admissions, financial aid); (g) US Congress (Report 105-636).

Are Budget requests explicitly tied to accomplishment of the annual and long-term performance goals, and are the resource needs presented in a complete and transparent manner in the program's budget?

Explanation: Budget requests directly relate to the accomplishment of annual and long term goals. Resource needs are developed in support of distinct operational programs, which provide for the ongoing Academic, Administrative and Regimental needs of the Institution in support of annual and long term goals. Additionally, a Capital Improvement Program (Master Plan) provides the resources for short and long term infrastructure and modernization requirements. Budget justifications specifically address: (1) past accomplishments; (2) current programs; and (3) future plans. Additionally, the documentation identifies progress on achieving interim goals, e.g. STCW-95 certification, lab modernization, etc. and the ability of the organization to accomplish specified objectives and long term goals.

Evidence: As part of the annual budget cycle the USMMA submits complete budget justifications, as required by the Government Performance and Review Act, to the Maritime Administration and assists with the documentation of the combined MARAD submission to DOT and ultimately OMB (DOT Budget estimates FY 2006-Maritime Administration, Submission to OMB 11/2004. Additionally, the USMMA assists in providing responses to OMB/Congressional inquiries relative to resource needs and applications. The USMMA Facility Master Plan is tied directly to the Capital Improvement Plan budget requests submitted to MARAD. The culmination of these processes ultimately results in the successful achievement of the Academy's goal to produce highly trained, licensed mariners.

Has the program taken meaningful steps to correct its strategic planning deficiencies?

Explanation: USMMA has taken significant and effective steps to manage strategic planning shortfalls, and thereby ensure program currency and added value to the federal government. Examples include: (1) The development of a facility master plan (initiated 1998-1999, funded in FY01 onwards). Capital funding needs have been identified on a 10-year-out basis, which has helped improve the budgeting, maintenance, and capital improvement processes at USMMA, MARAD and DOT. (2) Two new academic majors to meet changing needs of industry and strategic sealift. (3) Establishment of GMATS to provide specialized education to maritime transportation professionals and DOD active and reserve personnel; (4) Implemented a comprehensive strategic planning program to create an stronger culture of planning and assessment. Each department is now responsible for establishing an Assessment Plan, identifying assessment criteria, and then tracking performance. Results are tied to budget or PART targets, and will be used by management in financial and program decisions.

Evidence: USMMA uses the findings and recommendations of independent review and audit processes (see Question 2.6, above), to identify strategic deficiencies and implement program improvement plans. These have included: (1) Facility master plan; (2) Logistics and Intermodal Transportation Major (1998), Shipyard Management Major (1995); (3) GMATS designation as a federal National Maritime Enhancement Institute, NMEI (2002); (4) Development and implementation of a comprehensive strategic planning program (SUPINST 2003-15); and, (5) USMMA 2004-2009 Strategic Plan.

Does the agency regularly collect timely and credible performance information, including information from key program partners, and use it to manage the program and improve performance?

Explanation: USMMA collects performance data on a regular basis, including (1) daily financial review using Academy Resource Management System (ARMS); (2) weekly residential midshipman student count and retention data; (3) annual graduation qualification and Cost Per Graduate (CPG) data. The USMMA CPG has consistently been the lowest CPG for all Federal academies since the statistic was developed by the GAO in the 70's. In FY03, USMMA CPG was only 51% of the average of the other four academies. USMMA assesses other program performance through surveys and questionnaires, in collaboration with key program partners, like the USMMA Alumni Foundation, American shipping companies, accrediting organizations and applicable DoD agencies, e.g. MSC, USN, etc. Independent DOT appointed Advisory Board reported positive impacts of Academy performance on DOT strategic plan. For example, use of weekly midshipman count was used to develop lower cost method for billing for student meals (i.e. use of an actual count vs. estimate), resulting in reduction of food costs.

Evidence: USMMA reporting systems include: (1) Regimental strength/retention data, issued bi-weekly; (2) Annual Cost Per Graduate calculations (Jan 2005) ; (3) Maritime Education Program Evaluations (Congress 2003 -Omnibus App. Act of FY 1999 -P.L. 105-277); (4) Maritime Administration FY 06 OMB budget - 1999-2006 target/actual mariner availability data; (5) Mariner Readiness Survey, August 2001; (6) Academy Performance Survey, March 2004; (7) MARAD Annual Report 2003; (8) MARAD Strategic Plan 2003-2008, September 2003; (9) Middle States Accreditation review (in progress)-identifies areas for improved program performance, and USMMA Strategic Plan (in progress)-reviews every component of each Academy Program; (10) Academy Resource Management Report (In progress); (11) National Center for Education Statistics Report (IPEDS); (12) Advisory Board ltr. to Secretary Mineta (6/12/2003); (13) USMMA M/N Shipboard training stats; (14) USMMA M/N Internship Program; and (15) DOD Recruiting /employment Stats

Are Federal managers and program partners (including grantees, sub-grantees, contractors, cost-sharing partners, and other government partners) held accountable for cost, schedule and performance results?

Explanation: Fulfillment of performance goals are a critical element of USMMA Performance Work Plans for program and department managers, and a critical component in the annual review of program and department manager performance. Such plans are reviewed to assure they conform and support USMMA mission, goals, and objectives. Periodic reviews measure performance for specific goals and objectives to ensure the continuity of academic excellence. Selection of contractors is based on comprehensive evaluation of their past performance, if any. Architectural and Engineering services contractors are required to document their ability to perform. Service contracts are awarded using accepted performance based criteria and measures. In Partnership with the Department of Energy, the Academy requires an annual verification report which assesses the quality and the quantity of the cost savings achieved under a shared energy savings contract; and (6) Periodic accrediting agency reports validate successful completion of Academic performance objectives and goals.

Evidence: (1) The Performance Management System for the USMMA is established by Maritime Administrative Order (MAO) 740-451 - Performance Management System, January 27, 1997, as amended, reflects the requirements of 5 U.S.C. Chapters 43 and 45, and 5 CFR Parts 430, 432 and 451-" . . . to encourage all employees to share actively in improving government operations"; (2) President's annual budget guidance mandates institutional accountability [within the constraints of the Academy Resource Management System] to demonstrate adherence to prescribed budget and performance objectives; (3) Policies prescribed in MAO 150-1, as amended, (delegation of authority and responsibilities to Superintendent and other Academy personnel), (4) Contract requirements conform to Federal Acquisition Regulations (PART 37) and the Transportation Acquisition Manual (TAM); (5) Noresco/Dept. of Energy annual verification report (10/20/2004)

Are funds (Federal and partners') obligated in a timely manner and spent for the intended purpose?

Explanation: All funds for the USMMA program are obligated in a timely manner, and in all cases, are spent for the intended purposes of the programs supported by the President's approved and appropriated budget: Midshipmen Program, Instructional Program, Program Direction and Administration, Maintenance, Repair, and Operating Requirements, Capital Improvements/Master Plan, and Pay and Benefits. In FY 2004 $55.955 Million was available for obligation and ALL funds were executed as planned. Funds are disbursed, as received, through quarterly, or periodic, MARAD Allotment and Advice Reports. Additionally, funds provided by partner organizations are disbursed for the purpose intended, including gifts, bequests, grants, and reimbursable agreements. Audit Findings are promptly addressed upon notification.

Evidence: (1) Financial Management policies and regulations are set forth in Maritime Administrative Orders 400 Series, including 400-3, 5, 8, 10, 410-1, 2, 440-2, 440-5; (2) Procedures documented in the Oracle Federal Financial Program (DELPHI), and the Academy Resource Management System (ARMS) ; (3) Annual Gift Request letter to the USMMA Alumni Foundation; and (4) monthly USMMA Alumni Foundation Gift Fund reconciliation reports.

Does the program have procedures (e.g. competitive sourcing/cost comparisons, IT improvements, appropriate incentives) to measure and achieve efficiencies and cost effectiveness in program execution?

Explanation: USMMA uses an enterprise resource planning system to allow program managers to allocate faculty and support resources to meet a 4-yr graduation schedule for the vast majority of students--despite a national trend of 5-6 years. USMMA conducts exit interviews to identify causes of student resignations. USMMA's procurement team uses GSA prices as a baseline, and regularly negotiates and documents more favorable pricing and/or terms and conditions. USMMA substitutes alternative HVAC systems for conventional fossil fuel systems when studies and evaluations document lower cost and improved performance. USMMA regularly assesses program operations to identify potential outsourcing. USMMA developed a Leadership, Education, and Assessment Program (LEAP), and is evaluating a 2004 LAN audit to provide feedback for IT and other programs. The effectiveness of USMMA programs to achieve maximum efficiencies is evidenced by the annual calculation called Cost Per Graduate. This statistic documents that the USMMA program results in the lowest Cost Per Graduate of the 5 federal academies.

Evidence: (1) Progress System documentation found at the USMMA Department of Information Technology; (2) Graduation data reported by the Integrated Postsecondary Education Data System (IPEDS) at the National Center for Education Statistics; (3) Procurement files of the USMMA Office of Procurement; (4) Contract awards for dining hall and food services, building cleaning and janitorial services, on-going awards for maintenance and repair services, information technology problem reporting and management, and selected email accounts; and (5) USMMA Office of Engineering Resources HVAC analyses and designs.(6) Local Area Network Audit (LAN) 2004; (7) LEAP program (In Progress) (8) Annual USMMA Cost Per Graduate Report.

Does the program collaborate and coordinate effectively with related programs?

Explanation: USMMA collaborates extensively with federal agencies, as well as with SMA's. To meet requirements of the Maritime Security Transportation Act, DOT asked USMMA to develop standards and training req'ts for shipboard officers, terminal operators, etc. which became the US standards and were later adopted internationally by IMO. For DoD Defense Threat Reduction Agency, USMMA developed counter-terrorism analyses and strategies for port security. For Sandia Labs, faculty and students analyzed threat potentials at US ports. The CNO's Strategic Studies Group met with faculty to better understand terrorists threats to US merchant shipping. Faculty gave technical assistance to the City of New York after the Staten Island Ferry accident. USMMA and the SMA's met frequently in the late 1990's and early 2000's to discuss with the USCG and MARAD how to implement the requirements of STCW 95. MARAD has sponsored joint USMMA-SMA conferences on new technologies (e.g. use of bridge simulators to teach navigation).

Evidence: (1) Agreements and relationships with other Federal Agencies, including: USCG, National Oceanic and Atmospheric Administration, Department of the Navy (CNO's Strategic Studies Group), DoD Defense Threat Reduction Agency, DoE Sandia National Labs, US Department of State (USMMA had advised maritime officials in Nigeria and the Cape Verde Islands to the methods and opportunities to improve international trade and transport, and was instrumental in the development and organization of the new Philippines Merchant Marine Academy); and (2) The USMMA is a major contributor in establishing procedures, regulations, and best practices for port security initiatives, i.e., US Maritime Transportation Security Act of 2002, Maritime Security Act of 2002: Section 109 Implementation, A Report To Congress, July 2004, and (3) DTRA program results (In progress).

Does the program use strong financial management practices?

Explanation: All Academy financial resources are centrally budgeted and accounted for thru the Department of Resource Management, in accordance with MAO 150-1. Subsequently, departmental program managers plan, budget, and expend funds according to a series of program / institutional priorities, in accordance with all GAO, OMB, and DOT rules and regulations. Specialized Major Capital Improvement Funding is individually accounted for and executed. Overall annual accounting records are audited by DOT IG and/or GAO, with no known negative findings. Purchase and travel card holders are regularly audited, with appropriate follow-up action taken. USMMA has initiated a comprehensive management and financial audit program for its twelve NAFI organizations.

Evidence: (1) Annual budget and financial management objectives in the President's budget guidance; (2) Policies prescribed in MAO 150-1, as amended; (3) Financial management policies and regulations in Maritime Administrative Orders 400 Series, including 400-3, 5, 8, 10, 410-1, 2, 440-2, 440-5; (4) Policies and procedures documented in the Oracle Federal Financial program (DELPHI); and (5) Contract requirements conform to Federal Acquisition Regulations and the Transportation Acquisition Manual (TAM) Part 37.

Has the program taken meaningful steps to address its management deficiencies?

Explanation: All financial obligations incurred are executed with strict internal procedures, in accordance with strategic goals and the Institutional Assessment Plan. However, the growing scope of the USMMA financial program and the need for corporate governance required substantial changes in the format and process of the financial management program. A Department of Resource Management (DRM) and a Capital Improvement Program (Master Plan), were created. These changes provided an independent, objective, and centralized process to account for and allocate the Academy's budget, based on the successful Resource Management models currently in use at the other Federal Academies, specifically at West Point and the Naval Academy. DRM's purpose is to coordinate all financial and resource activities to more effectively carry out USMMA financial management policy. The Master Plan directs appropriated funds to address the capital needs of the USMMA.

Evidence: (1) USMMA 2004-2009 Strategic Plan (In progress); (2) Program Management Review (PMR) for the Academy Master Plan; (3) MAO 150-1 dated 7/1/2000 (United States Merchant Marine Academy); and (4) Draft Superintendent's Instruction -Institutional Assessment Plan.

Has the program demonstrated adequate progress in achieving its long-term performance goals?

Explanation: The USMMA is demonstrating progress in meeting its long-term performance goals: (1) The number of fully qualified graduates exceeds the target; (2) The actual CPG remain below the HEPI-adjusted CPG (baseline FY99) which is ambitious for an institution whose instructional program is entirely based on technology (the HEPI statistics include data from liberal arts institutions that use less technology), and since USMMA costs include all costs for room & board, medical, shipboard training, travel, and other expenses that are otherwise not borne directly by other colleges or universities; and (3) The graduation rate for each incoming Plebe Class is consistent with the five year moving average.

Evidence: Performance data are to be found in accompanying PART long-term performance measures. GAO guidelines to calculate the average total cost per graduate are referenced in the data column for Question 4.4. In accordance with the GAO formula for calculating the cost per graduate, capital costs are excluded.

Does the program (including program partners) achieve its annual performance goals?

Explanation: USMMA met one of its two annual performance goals : a reduction in the number of deferred graduates. While the Academy did not meet the target to reduce the mean time (in months) which is required to fully credential a deferred graduate (degree, USCG license, USN reserve commission) in 2004, it is important to note that in a class of only 200, one student's deferred status caused the Academy to not meet its goal. This fact does not reflect that the overwhelming majority of deferred graduates resolved their outstanding issues in a shorter period of time than in past years.

Evidence: (1) Dean's Projection of Deferred Graduates, Class 2005 and projections from the Department of Naval Science, USMMA; (2) Status of Deferred Graduates, Class 2004, Dean's Office, USMMA; (3) Status of Deferred Graduates, Class 2003, Dean's Office, USMMA; (4) Cost Per Graduate Report, Director, Resource Management, USMMA, (5) Data available upon request, Dean's Office, USMMA.

Does the program demonstrate improved efficiencies or cost effectiveness in achieving program goals each year?

Explanation: The USMMA program demonstrates improved efficiencies, in terms of cost and output, relative to program goals. Budget execution strategies are designed to maximize midshipman retention and graduation rates and reduce deferred graduate incidences and credentialing times, while reducing the effective cost of producing a qualified graduate. Proven annual financial results include utilization of special pricing for IT procurements - cost avoidance of $0.9M over GSA/Commercial schedules; Energy Savings Performance Contract savings of $1.0M+; in-house vice outsource applications -$0.5M; midshipman book loan vice issue program - $0.1M. Additionally, ongoing academic program evaluations improve retention and graduation rates while reducing deferred graduate incidences, in order to meet program goals and performance measures.

Evidence: "Cost Per Graduate (1) In past years, annual reviews resulted in program changes to reflect the licensure requirements of STCW-95, more recently, the changes for the USCG regulations regarding ECDIS and Person-In-Charge endorsements for dangerous liquid cargo, (2) Most recently the development and implementation of the Master Plan process for managing capital projects, energy conservation projects reviews (Noresco Report), purchase and travel card expenditure analyses, IT procurement decisions (3) Exit interviews with disenrolling midshipmen to identify and mitigate the causes of academic and training issues/problems, (4) Dean's reports, reports from the Department of Career Development and Professional Services, reports from the Departments of Maritime Transportation and Engineering and (5) Annual Cost Per Graduate report "

Does the performance of this program compare favorably to other programs, including government, private, etc., with similar purpose and goals?

Explanation: USMMA produces nearly 80% of the licensed, qualified, obligated merchant mariners appointed each year in the USNR-MMR program. The balance of USNR-MMR accessions come from the six SMA programs. The average total cost per graduate is $187,503, on an average annual basis, $46,876 per student. USMMA cost per graduate is determined as specified in GAO study B-159219 (Financial Operations of the FIve Service Academies). The SMA four year licensed officer cost of $120,196 per graduate The SMA annual cost per graduate is $30,049. The average total cost per graduate at the US Coast Guard Academy is $305,000.

Evidence: 1) DoN Statistics on USNR-MMR Accessions; 2) GAO Study B-159219 establishes the methodology for computing CPG statistics, and is used by all five federal academies. This data is reported annually to MARAD and is included in the document supporting the annual budget request. Capital costs are excluded from the CPG calculation.

Do independent evaluations of sufficient scope and quality indicate that the program is effective and achieving results?

Explanation: Comprehensive, independent evaluations by ABET & Middle States accreditors document USMMA's record of academic excellence. The USCG examines and certifies that almost 100% of mids meet the requirements to serve on vessels of unlimited tonnage, any horsepower, on oceans, as well as meeting all requirements of STCW-95. Some grads receive USCG endorsements as Person-In-Charge (PIC) for vessels carrying dangerous liquid cargo. Evaluations by professional mariners aboard ship report USMMA cadets more than adequately prepared. USMMA grads report that they are very well prepared for service aboard ship. Branches of the Armed Forces establish ann'l recruiting goals at USMMA and direct-commission up to 35% of each USMMA graduating class for active duty assignments. Key DoD communities (surface warfare, logistics and transportation) view USMMA graduates as exceptionally well prepared professionally due to their USCG license, transportation industry experience, and leadership capabilities; USMMA grads are available to serve immediately, while other service acad. grads require add'l training.

Evidence: (1) ABET and Middle States review reports renewing USMMA's accreditations (reaccreditation involves comprehensive review by visiting teams consisting of professionals recognized within their fields of expertise to review USMMA governance, administration, strategic planning, resource allocation and execution, assessment and evaluation activities); (2) Approval of USMMA's licensing program by the USCG/MARAD Review Committee on STCW-95; (3) Report to Congress: Maritime Education Program Evaluation, DoT/MARAD, April 2003; (4) Survey data provided by the Office of Career Services, USMMA; (5) USMMA Alumni Foundation "2004 Alumni Survey"; (6) Service quotas reported by the Dept of Naval Science at the USMMA; (7) annual review of financial practices and internal control (MAO 900-3, implementing OMB Circulars A-123, A-127, and DOT 5100.4A); (8) annual certification of USMMA LAN (NIST 800-53).