Review of the Multicultural Mental Health Australia (MMHA) project

1. Introduction

Page last updated: November 2009

1.1 Background
1.2 The national Multicultural Mental Health Australia project
1.3 This review
1.4 Review methodology

1.1 Background

Cultural diversity is an important aspect of Australia's community with over 40% of the population either born overseas or has a parent born overseas.8 Migrants and their descendants have played an important role in shaping Australia's history and the nation today, in terms of population and workforce growth, and in contributing to Australia's culture and society.9

Mental health issues can affect people of all ages and from different cultural and social groups.10 In 2007, one in five Australians aged 16–85 years had a mental disorder,11 and many studies have highlighted that people from diverse backgrounds will face numerous barriers when accessing timely and appropriate mental health care. These barriers include:
  • Language - reluctance to use services due to language and cultural differences, lack of interpreter use or misuse of interpreters; limited information available in community languages;

  • Cultural differences between client and clinician - culturally insensitive attitudes of organisations and service providers, lack of appropriately trained staff to work with CALD consumers;

  • Knowledge - limited of awareness/knowledge about available services and GP referral patterns,

  • Experience - pre-migration experience and/or trauma experiences; and

  • Stigma - differences in cultural explanations and perceptions surrounding mental health.12 13 14
National mental health policy, through the National Action Plan on Mental Health 2006-2011 and other documents recognises the need for culturally relevant services. For example, the Mental Health Promotion and Prevention National Action Plan identifies people from diverse CALD backgrounds as a priority population group; and the implementation framework for the Plan specifically focuses on the needs of the Australian CALD communities.15 Top of page

1.2 The national Multicultural Mental Health Australia project

The national Multicultural Mental Health Australia (MMHA) project (hereafter referred to as the Project) is targeted towards providing leadership and direction in transcultural mental health and suicide prevention. It is one of a number of national projects targeting the needs of specific population groups, and aims to raise community and professional awareness to support a national focus on the mental health issues faced by Australia's CALD communities.

The work by the MMHA must support those objectives of the National Mental Health Strategy and the National Suicide Prevention Strategy, with consideration given to the Council of Australian Governments' (COAG) National Action Plan on Mental Health (2006-2011) and other relevant national plans/initiatives. To achieve this, the MMHA project has five priority work areas in the current Funding Agreement:
  • Policy / Secretariat;
  • Community Capacity Building and Development;
  • Communication, Education and Information Dissemination;
  • Consumer and Carer Support and Representation; and
  • Workforce.16
A transcultural mental health program has been funded since 1995 by the Commonwealth under the National Mental Health Strategy. However, Multicultural Mental Health Australia has specifically been funded since May 2003. Sydney West Area Health Service (SWAHS) is the fund-holder (i.e. contractor) for the Project. SWAHS auspices the Project and reports to DoHA on achievements against the Funding Agreement. SWAHS and DoHA have recently entered into a new three year Funding Agreement to end 30 June 2011.

MMHA has been reviewed on two previous occasions: in 2001 when MMHA was formally known as the Australian Transcultural Mental Health Network; and in 2006 as MMHA. The findings and recommendations have been considered in the conduct of the current review.

1.3 This review

As part of the current Funding Agreement, the Department of Health and Ageing (DoHA) engaged Health Outcomes International (HOI) to undertake a review of the MMHA project. This review has been commissioned nearly one year into the new Funding Agreement. Stakeholders were informed by DoHA in November 2008 that this review (to be finalised in 2008-09) would identify priorities and future scope of work. The review outcomes and recommendations are intended to be considered in the implementation of the project during the remainder of the current funding period.

1.3.1 Review aim and objectives

The overall aim of the review is to determine the appropriateness, effectiveness and efficiency of the Project. In doing so, the review aims to determine whether MMHA is:
  • Achieving the outcomes that were originally identified;
  • Internally structured in a manner whereby the responsibilities and decision making process is clear, ethical and transparent;
  • Operating appropriately, effectively and is sustainable under the current model; and
  • Identifying appropriate priorities and scope of work for future consideration.
The specific objectives of the review include the provision of analysis and recommendations with regard to:
  • Effectiveness, suitability and sustainability of the current program model including:
    • Implementation and impact of previous evaluation recommendations.
    • Instances where there have been delays in implementation of the project and reasons for those delays.

  • Effectiveness, representation and suitability of the governance and accountability model of MMHA, including decision making processes and identification of any areas requiring improvement including:
    • Impact of previous evaluation recommendations;
    • Identifying whether clear ethical and transparent decision making processes exist.

  • Effectiveness of the financial and service management structures and identification of any areas requiring improvement including:
    • The extent to which the project's inputs have been minimised, or outputs maximised, in achieving the project's intended products and services;
    • Trends over time in the ratio of administrative to program costs;
    • Under or overspends in years to date.

  • Review of program planning and needs analysis design and processes:
    • Extent to which the project is currently consistent with Australian Government priorities;
    • The nature and extent of continuing community need;
    • Alternative strategies available to address the need, and any implications for future delivery of the project.

  • Identification of priorities and scope of work for future consideration; and

  • Current reporting requirements including identification of clear and meaningful performance measures for future consideration and implementation.
Top of page

1.4 Review methodology

The review was undertaken in three stages, each comprising distinct tasks as described below.

Stage 1: Review design and documentation review

Stage 1 involved various planning activities in preparation for fieldwork. A range of relevant documentation pertaining to the Project was provided to the review team by DoHA and MMHA (see appendix A). A list of relevant stakeholders to be consulted in the course of the review was provided by MMHA and finalised in consultation with DoHA (see appendix B). These included representatives of:
  • Department of Health and Ageing (as project funder)

  • Sydney West Area Health Service Management (as contract holder)

  • MMHA staff (as funding recipient)

  • CALD consumers and carers (as project stakeholders)

  • MMHA National Partner agencies/organisations such as Beyondblue, Mental Health Council of Australia, National Ethnic Disabilities Association, SBS and the Migrant Resource Services Council of Australia (as project stakeholders)

  • State and Territory Partners such as state health departments, Migrant Resource Centres, Transcultural Mental Health Centres, former Consortium members, Ethnic Communities Councils and the University of South Australia (as project stakeholders).
The other key task for this stage involved the development of the review framework detailing the review methodology and data collection tools to be used to answer the review objectives. In doing so, the review objectives were categorised according to the following broad domains:
  • Project governance and accountability
  • Project planning
  • Project model
  • Project financial and service management
  • Future considerations.
These domains were then used to inform the development of research questions. Table 1.1 shows the review matrix which sets out the review objectives, domains, specific research questions and data sources.

The research questions included in the Review matrix served to inform the development of more detailed lines of enquiry with each of the stakeholder groups. Appendix C provides the lines of enquiry explored with representatives of DoHA, SWAHS management and MMHA management and staff. All other stakeholders completed either a telephone interview (see appendix D) or a written survey questionnaire (see appendix E). The content of the telephone interview and written survey questionnaire was identical.

Table 1.1 - Review matrix

Table 1.1 is separated into 5 smaller tables in this HTML version for accessibility reasons. It is presented as one table in the PDF version. Top of page
MMHA Governance and Accountability
Research questionDomain (based on review objective)Consultations with MMHA
management & staff
Consultations
with SWAHS
Consultations with
former consortium
Consultations
with DoHA
Consultations with
other stakeholders
ReportsFinancial recordsOther documents
Is the MMHA governance model documented and agreed?Appropriateness
tick
tick
Are the roles and responsibilities of all stakeholders clear and appropriate to the project?Appropriateness
tick
tick
tick
tick
tick
tick
What are the powers of delegation and decisionmaking within MMHA. Are these transparent?Appropriateness
tick
tick
tick
tick
tick
tick
tick
Are there alternative governance models that should be considered?Appropriateness
tick
tick
tick
tick
Are there agreed performance indicators and are they appropriate?Appropriateness
tick
tick
tick
tick
Is the governance model workable for this type of project?Effectiveness
tick
tick
tick
tick
tick
Is there sufficient and timely communication between all stakeholders to ensure effective & efficient management of MMHA?Effectiveness
tick
tick
tick
tick
tick
What opportunities are there for improving the governance and accountability of MMHA?Effectiveness
tick
tick
tick
tick
tick
tick
tick
MMHA planning
Research questionDomain (based on review objective)Consultations with MMHA
management & staff
Consultations
with SWAHS
Consultations with
former consortium
Consultations
with DoHA
Consultations with
other stakeholders
ReportsFinancial recordsOther documents
Is there a clear program logic to MMHA? Is it evidence based? Is it documented?Appropriateness
tick
tick
tick
tick
tick
Who is MMHA's target group?Appropriateness
tick
tick
tick
tick
tick
How are consumer needs identified and planned for (i.e. needs analysis)?Appropriateness
tick
tick
tick
tick
Have recommendations from previous reviews been implemented? What has been the impact of these?Appropriateness
tick
tick
tick
tick
How does MMHA planning align with current Government mental health policies and priorities?Effectiveness
tick
tick
tick
tick
Have future priorities been planned for the program? If so what are they?Effectiveness
tick
tick
tick
tick
tick
tick
What opportunities are there for improving MMHA planning?Effectiveness
tick
tick
tick
tick
tick
Top of page
Suitability and sustainability of MMHA Model
Research questionDomain (based on review objective)Consultations with MMHA
management & staff
Consultations
with SWAHS
Consultations with
former consortium
Consultations
with DoHA
Consultations with
other stakeholders
ReportsFinancial recordsOther documents
Is there agreement that current program logic is addressing needs? Is the current model the best fit?Appropriateness
tick
tick
tick
tick
tick
What are the linkages with the National Mental Health Strategy and other related policies?Appropriateness
tick
tick
tick
tick
tick
Do other Commonwealth or health jurisdiction programs overlap/duplicate MMHA or are the programs complementary? Are the programs well coordinated?Appropriateness
tick
tick
tick
tick
tick
tick
tick
What has been the result against the performance indicators (e.g. are the objectives being met?) Have the results met or exceeded expectations?Effectiveness
tick
tick
tick
tick
tick
tick
What have been the short and medium term impacts and outcomes from MMHA activities? Have they been reported?Effectiveness
tick
tick
tick
tick
tick
Have some MMHA activities worked well more than others?Effectiveness
tick
tick
tick
tick
To what extent has the project facilitated the establishment, expansion or support of complementary services?Effectiveness
tick
tick
tick
tick
tick
Have there been any unintended consequences (positive or negative)?Effectiveness
tick
tick
tick
tick
What opportunities do you see for improving the a) suitability and b) sustainability of MMHA?Effectiveness
tick
tick
tick
tick
tick
tick
Is the funding level adequate to make a realistic contribution towards achieving the objectives of MMHA?Efficiency
tick
tick
tick
tick
tick
MMHA Financial and service management
Research questionDomain (based on review objective)Consultations with MMHA
management & staff
Consultations
with SWAHS
Consultations with
former consortium
Consultations
with DoHA
Consultations with
other stakeholders
ReportsFinancial recordsOther documents
What are the financial and service arrangements? Are they documented?Appropriateness
tick
tick
tick
tick
tick
Are financial reports provided on a timely-basis that allows the program to be managed?Effectiveness
tick
tick
tick
tick
tick
Do service and governance arrangements enhance or impede the program operation?Effectiveness
tick
tick
tick
tick
What opportunities are there for improvement in MMHA's financial and service management?Effectiveness
tick
tick
tick
tick
tick
What are the trends over time in the ratio of administrative to project costs?Efficiency
tick
Has any thought been given to how MMHA may be continued without DoHA funding?Efficiency
tick
tick
tick
What is the reason for MMHA's request to DoHA in January 2009 to carryover funds totalling $644,744?Efficiency
tick
tick
tick
tick
tick
Top of page
Future considerations
Research questionDomain (based on review objective)Consultations with MMHA
management & staff
Consultations
with SWAHS
Consultations with
former consortium
Consultations
with DoHA
Consultations with
other stakeholders
ReportsFinancial recordsOther documents
What is the nature and extent of continuing/future community need? How can MMHA meet these needs?Appropriateness Effectiveness Efficiency
If MMHA was re-established, what (if anything) would be done differently?Appropriateness Effectiveness Efficiency
tick
tick
tick
tick
What lessons can be learnt if other similar projects are implemented in the future?Appropriateness Effectiveness Efficiency
tick
tick
tick
tick
tick
Is there a mechanism for inter jurisdictional learning (i.e. can we learn from what has or is happening on other projects) and is this appropriate?Appropriateness Effectiveness Efficiency
tick
tick
tick

Stage 2: Field work

This stage involved two major tasks – documentation review and stakeholder consultations. A comprehensive and objective review and assessment of all provided documentation against the review objectives was undertaken by the review team. In addition, all nominated stakeholders were contacted and invited to participate in the review. With the exception of one SWAHS representative who was interviewed by telephone, all interviews with representatives of DoHA (n=4), SWAHS (n=1) and MMHA (n=4) were conducted face-to-face on site. All other stakeholders were invited to participate in the review either via a telephone interview or a written survey questionnaire. All respondents were emailed the interview questions prior to the scheduled interview to ensure they were well prepared. Of the total number (70) of stakeholders invited to take part in the review, 36 agreed. This represents a response rate of 51.4%. Of the 36 participants, 23 completed an interview (21 via telephone, 2 via face-to-face) and 10 completed the survey questionnaire.

Stage 3: Analysis and reporting

The final stage of the review involved collation and analysis of the data collected during stage 2 in order to answer the review objectives and inform conclusions against the objectives and formulation of recommendations. This work in turn informed the preparation of the draft report on the review findings. Following review of the Draft report by DoHA, all feedback received was incorporated into the development of the final report.

Footnotes

8 Beyondblue. (2008). Diversity, Multiculturalism and Mental Health. Available on the beyond blue website (www.beyondblue.org.au)
9 Australian Bureau of Statistics. (2008). Themes - Migrant and Ethnicity.
10 Australian Bureau of Statistics. (2008). Media Release: One in five Australians have a mental illness: ABS. Available on the Australian Bureau of Statistics website (www.abs.gov.au)
11 Australian Bureau of Statistics. (2007). National Survey of Mental Health and Wellbeing: Summary of Results, 2007. Available on the Australian Bureau of Statistics website (www.abs.gov.au)
12 Beyondblue. (2008) op.cit.
13 Multicultural Mental Health Australia, National Ethnic Disability Alliance, Australian Mental Health Consumer Network and the Australian Government Department of Health and Ageing.(2004). Reality Check: culturally diverse mental health consumers speak out.
14 Long, H., Pirkis, J., Mihalopoulos, C., Naccarella, L, Summers, M., & Dent, D. (1999). Evaluating Mental Health Services for Non-English Speaking Communities. Australian Transcultural Mental Health Network.
15 Australian Health Ministers (2004). Framework for the Implementation of the National Mental Health Plan 2003-2008 in Multicultural Australia, Department of Health and Ageing, Canberra.
16 Standard Funding Agreement between Department of Health and Ageing and Sydney West Area Health Service. July 2008 to 30 June 2011.