In Chapter 4 and Chapter 5 respectively, we presented the key findings from the documentation review and our consultations with the key stakeholders. This chapter presents our conclusions against the review objectives centred around five broad domains. These include MMHA governance and accountability; planning processes; sustainability and sustainability of the service model, financial and service management and future considerations.
The conclusions are grounded in a comprehensive and objective review and analysis of all documentation provided to the review team. The results of this work have then been reviewed alongside our analysis of the findings of all stakeholder consultations. A final analysis of this collective body of work has informed the formulation of conclusions against the review objectives.
The following sections present our conclusions with respect to the suitability, effectiveness, efficiency and sustainability of the project. For consistency with the reported findings, conclusions are presented in line with the four broad domains explored in the review:
- Project governance and accountability
- Project model
- Project planning
- Project financial and service management
- Project reporting and performance measures
6.1 Project governance and accountabilityThe criteria used to assess this objective included the:
- Governance documentation outlining decision making processes, committee structures, lines of accountability, etc.
- Organisational charts
- Minutes of meetings of Consortium, JOG and other relevant committees or forums
- Best practice in management of public organisations as identified in the literature and by such bodies as The Australian National Audit Office, Australian Public Service Commission
- Progress reports from MMHA to DoHA as specified in Funding Agreements.
6.1.1 ConclusionWe conclude that the governance arrangements for the MMHA are ineffective and undermine the projects ability to achieve its objectives in an efficient manner.
MMHA is funded by the Department of Health and Ageing (DoHA) with the contract to manage the MMHA project being held by the Sydney West Area Health Service. A secretariat to the project has been established and resides within the Sydney West Area Health Service. The role of the Consortium as identified in the MMHA Consortium Governance 2006-2008 document is to "ensure equity and priority setting, to foster collaboration between members and to ensure equitable representation of the needs of all members. It provides the MMHA with the conduit to identify the needs of the CALD community in the area of mental health services and acts in an advisory capacity. This group was disbanded in November 2008. In order to formalise MMHA's linkages with States and Territories it was agreed, in conjunction with DoHA, to create a Joint Officers Group consisting of State and Territory representatives and the Australian Government, to facilitate implementation of the Framework for Implementation of the National Mental Health Plan 2003-2008 in Multicultural Australia nation-wide.
The reporting and accountability lines for the MMHA depict only the relationship between the Sydney West Area Health Service and the secretariat of the MMHA. It does not acknowledge the roles, responsibilities or the relationships between the Consortium, JOG, the secretariat, Sydney West Area Health Service and DoHA. There is a lack of clarity of the respective roles and responsibilities in the minds of nearly all stakeholders and this has caused significant disruption and inertia in the project.
While the auspice arrangements provide certain benefits/cost savings (utilisation of existing infrastructure etc.) in this case it has served (in the eyes of most stakeholders and as evidenced by documentation reviewed to undermine the effective operation of the project.
At least one of the previous reviews (2001) identified these same issues, namely that the project is not driven by strong strategic planning and is not underpinned by sound governance and accountability arrangements. Irrespective of the fact that the management structure and auspice body has changed since this review, it is of concern that these issues remain significant impediments to the projects ability to achieve its objectives eight years later.
6.2 Program modelThe criteria used to assess this objective included the:
- alignment with the National Mental Health Strategy, National Suicide Prevention Strategy and other related Government policies
- extent to which model addresses the needs of CALD communities and service providers
- nature and extent of continuing need and unmet need
- drivers and barriers to project success
- processes to facilitate stakeholder communication
- project has achieved its objectives
- best practice in management of public organisations as identified in the literature and by such bodies as The Australian National Audit Office, Australian Public Service Commission.
6.2.1 ConclusionWe conclude that the MMHA model is based on a highly collaborative and partnership engendering philosophy which is appropriate for the issues of the transcultural mental health and suicide prevention. However the model has not been effectively implemented.
The MMHA model as one of collaboration and partnership aimed at addressing issues of transcultural mental health and suicide prevention and is appropriate to the needs of this sector. The MMHA model has taken into account the appropriate key stakeholders within the transcultural mental health and suicide prevention arena.
It has established various forums with which to engage with these stakeholders however these forums have not worked effectively for a range of reasons. The key issues relate to the size of some of the forums, the lack of transparency and clear role delineations as well as overall management of the project (discussed in Chapter 5 and Section 6.1 above). For example, the Consortium was charged with needs identification analysis and priority setting for the project. As such the Consortium should have an established mechanism of monitoring the project. No reporting lines were identified between the Consortium and the MMHA secretariat in the organisational and reporting structure chart provided to the review team by MMHA.
In its current form the MMHA model is not sustainable. Clearer reporting lines need to be established, greater clarity in the respective roles and responsibilities of the respective stakeholders of the MMHA project need to be defined and stronger governance arrangements need to be implemented.Top of page
6.3 Project planningThe criteria used to assess the effectiveness of planning processes within the MMHA project included:
- the quality of previous and existing strategic plans for MMHA
- the alignment of these plans with the NMHS objectives, community needs as articulated by stakeholder views
- the quality of previous and existing operational plans for MMHA
- stakeholder acceptance of the planning processes
- best practice in management of public organisations as identified in the literature and by such bodies as The Australian National Audit Office, Australian Public Service Commission
- likely consequences of not having in place appropriate strategic and project planning processes and the subsequent capacity for MMHA to address community need.
6.3.1 ConclusionWe conclude that the MMHA planning processes are ineffective.
Evidence of strategic planning specifying the direction of the MMHA project had not been undertaken for the current funding round at the time at which the review was undertaken. Not with standing that the review team were advised that there was an intention to do so once the review was completed we do not believe that the development of a strategic plan should be contingent on the outcomes of a review. It forms part of a sound business management and this deficit, namely the absence of a strategic plan and only working from an operational plan has been commented upon in previous evaluations.
While different planning mechanisms and processes have been established over the course of the project's life, overall their effectiveness has been undermined. This applies in particular to the Consortium. While an appropriate forum at the onset for planning it has become less effective for a number of reasons, including:
- the multicultural mental health landscape changed
- was not inclusive of newer players
- was seen as having somewhat tokenistic representation of carers and consumers
- was increasingly costly to run
- had poor governance
- too complex in composition (diversity in membership, competing agendas, size, etc.) to enable effective decision making to occur. In turn this resulted in the operation became increasingly dysfunctional/mostly related to the nature of the diverse membership. Tensions between progressing local/state needs and driving the national agenda, dominant personalities etc.
There is a role for the inclusion of a broader stakeholder audience who are best informed about multicultural mental health needs at the local level. The current MMHA structure in our opinion does not provide the appropriate expertise from which to develop a comprehensive strategic plan. There are no obvious committee structures present in the MMHA organisational chart and documentation. The establishment of a Strategic Planning Committee with the specific role of developing the project's strategic plan, reviewing and updating that plan is one possible mechanism that could be instigated to progress this work. Caution needs to be extended that in establishing such a committee the MMHA does not end up reconstituting the Consortium or a group similar in nature. The Strategic Planning Committee should be small in numbers (thereby ensuring it is manageable) and its scope should be limited to that outlined above.
There is a current lack of baseline data from which to undertake effective strategic and project planning and this will need to be addressed in the immediate future.Top of page
6.4 Project financial managementThe criteria specified in the tender brief to assess/determine the efficiency of the Initiative includes:
- the extent to which the Project inputs have been minimised, or outputs maximised, in achieving the Project's intended products and services
- the impact of the Project on costs borne by the community, clients and other Governments
- trends over time in the ratio of administrative to program costs
- instances where there have been delays in implementation of the Project and reasons for those delays. Explanation of under or overspends in the years to date
- identification of areas for improvement.
6.4.1 ConclusionWe conclude that the project's ratio of administrative to program costs are within the expected benchmarks established in the evidence.
We currently question whether the level of funding currently provided to the project is warranted given a three year history of carry over of significant funds.
We conclude that the project in its current form is not sustainable.
The evidence collected in the review points to a need for better monitoring and management of the contract. The MMHA project has requested to carry over funds for the last three years. This inability to expend funds on designated activities does not reconcile with the views expressed by stakeholders that the funding levels for the project are inadequate.
The issue of timely distribution of funds from DoHA accounts to SWAHS accounts attributing to this carry over is questioned given that the carry over is not a one off phenomenon. The question arises as to whether the current MMHA is equipped in its current format to utilise the existing funds.
The transparency of the project in terms of consistency in which budgets are established and funds acquitted is questioned given the significant differences in budgeted funds for expenses such as Committee Activities. This transparency is further at risk when the audited financial statements are prepared by a division within the fund holder organisation. The perception of having audited financial statements prepared by an external auditor, one independent from the project, fund holder and funder – a requirement for the majority if not all projects funded by DoHA is difficult to maintain under the current circumstances.
6.5 Project reporting and performance measuresThe criteria used to assess this objective includes the:
- progress reports provided submitted by MMHA to DoHA as part of the Funding Agreements
- identification of existing performance measures contained within the Funding Agreements.
6.5.1 ConclusionWe conclude that the MMHA project has met its reporting requirements from a process perspective but current performance measures do not adequately inform the Government about the quality or value of the project.
The Funding Agreement between DoHA and MMHA provides for regular monitoring and reporting of agreed work priorities. The MMHA has met its reporting requirements as set out in the Funding Agreement.
The current suite of performance indicators are relevant in respect to the objectives of the project however they are highly quantitative and process output focussed. The existing key performance indicators do not adequately inform the Government about the effectiveness and quality of the project. To do so requires the development of more qualitative and outcome focussed indicators.