4.4.1 Impact of the CHCI on staff and services
4.4.2 Impact of the CHCI on the NT PHC system
4.4.3 Health service use
4.4.4 Will the impacts continue?
Considering the impact of the CHCI on health service delivery and the NT PHC system more broadly, this section focuses on four main questions:
- What impact did the roll out of child health checks and follow-up services have on existing health service staff and services?
- What did the CHCI do to the existing system?
- Did the general use of health services increase for Aboriginal children?
- What happened afterwards—was there a continuing impact?
- checks done by visiting Australian Government taskforce teams in a separate building with resident staff having little involvement (community D)
- checks done by visiting Australian Government taskforce teams in the existing clinic, closely assisted by resident staff (communities A, B and C)
- checks done in the existing clinic by staff recruited by the local service
- checks done as part of existing MBS Item No. 708 child health checks, using existing staff and referral systems (community E).
Negative impacts of the child health checks on resident servicesWhere a clinic recruited its own dedicated staff to do the child health checks, other services had to be suspended for a week so the staff could focus on doing the checks. This put staff under pressure to get the checks completed so normal services could resume as soon as possible. We were not able to quantify the impact of suspending normal services for a week, but the opportunity cost is likely to have been significant.Top of page
Where the checks were done by visiting Australian Government taskforce teams the clinics and services continued to run as normal and the impact depended on how much support the resident staff provided. In three of the four communities we studied that involved visiting taskforce teams, the resident teams worked closely with the visiting teams in providing support, advice and direction. This led to high workloads as resident teams continued to provide normal services. One service (in community B) closed its clinic for the first week that the visiting team were there so that all the clinic staff could focus on assisting the visiting staff.
The following contributed to the impact of the CHCI on staff and services:
- Administrative support had to be provided to teams. This included entering paper-based forms onto electronic patient record systems, retrieving forms from files and photocopying. Although this support is critical staff often viewed it as unrewarding.
- Staff had to do a lot of work to reassure the community about the checks and to overcome feelings that ranged from general scepticism about the checks to real fear that people’s children would be taken away. Some staff put a lot of effort and thought into how to get the community on board and selected specific people they were sure would attend to encourage others to follow.
- There were tensions between visiting teams and resident staff. Some staff were angry at first that outside staff were being brought in to do the checks, feeling their previous work was not valued. The substantially higher salaries of the Australian Government taskforce teams that visited in Phase 1 may have contributed to tensions.21
- There were tensions between members of the visiting teams. They were not used to working together and were placed in remote communities with conditions and levels of need that they were not necessarily familiar with.
- The skill set of the visiting teams had an impact on existing staff. Some teams lacked experience in remote Aboriginal health and child health and needed support from local staff.
- The checks affected Aboriginal health professionals at various levels. At a local level, some services reported that they lost AHWs who walked off the job to look after and protect their children. We do not have recent data on the AHW workforce to validate this. At the NT and even national level the NTER affected, and continues to have a residual effect on, the way Aboriginal people were portrayed across the country. The AIDA commented on the effects on its members, ‘NTER was a drain on Aboriginal professionals—they had to be a clinician in the day, strategist at night, and a community member 24/7’. This view was supported by interviews with several staff of ACCHOs who spoke about the huge workload and stress at the time of the CHCI.
- The workload presented by the referrals swamped a number of services, both at the local health service/clinic where staff were left with the responsibility of organising follow-up referrals, and at specialist services which were overwhelmed by the number of referrals.
A number of participants in the evaluation case studies felt that the negative impacts of the CHCI on staff and services was minimised by delivering the child health checks in a similar way to routine services/checks. In a number of communities resident staff assisted visiting teams with delivering the checks and integrated the checks into normal clinic business. This was no different from how the services delivered ongoing checks such as the HSAK program. In these instances, the community, staff and the health service were used to operating like this and the checks were almost indistinguishable from normal clinic services.
Staff in the health centres generally considered the visiting teams to be friendly and professional and this helped reduce the potential negative impact of the CHCI. A number of evaluation participants also noted that when the visiting teams treated patients at the same time as doing the checks (this mainly occurred in Phase 2), this helped avoid overloading the resident clinic PHC team with referrals.Top of page
Benefits of child health checks for staff and servicesCase study participants also noted a number of positive impacts of child health checks on existing staff and services:
- The extra resources were welcome. These included capital infrastructure in clinics, audiology equipment and additional staff time for follow-up services.
- There were noticeable improvements in the delivery of some health services. There were more dentists on the ground, including an increased provision of dental services through ACCHOs and in very remote communities, and improved audiology services.
- Health knowledge could be shared between the staff and visiting teams.
Any improvements in health service delivery seem to only be only noticeable for follow-up services. The child health checks alone cost around $17.935 million and reached approximately 10,605 Aboriginal children ($1,691 per child). Including 50 per cent ($10 million) of the DoHA departmental expenditure associated with policy, planning and implementation functions (around $20 million in total to 30 June 2010), and 50 per cent ($7 million) of the cost of Operation OUTREACH (the ADF spent around $14 million in total) brings the costs of the child health checks to $34.935 million, or $3,294 per child. At $1,691–$3,294 per child, the checks reached 57–65 per cent of the target population, a group that was largely made up of children who were
already accessing health services and screening programs.
Many community members, health centre staff and health administrators saw the child health checks as duplicating existing child health check services such as MBS Item No. 708 checks and the GAA and HSAK programs. This probably provides the best explanation for why we have identified little improvement in health service delivery as a result of the checks. Child health checks were, and continue to be, provided regularly in many remote Aboriginal communities and an additional one-off child health check program delivered largely by people outside the community had limited opportunity to bring real, sustainable improvements.
One possible exception to this is that the extra resources may have enabled checks on children’s health to be done, particularly in larger and/or less well-resourced communities.22 One government official commented that ‘a heck of a lot of kids got their PHC for the first time’ but we have found no evidence to support this view.
Service delivery modelsThe child health check program model explains a number of limitations for the impacts of the CHCI. It provided a one-off surge in capacity but there was not always the organisation or infrastructure within the system to respond to this surge. As a result, there was a ‘bubble’ of Aboriginal children aged 0–15 years who had a child health check but in a system that was not organised enough to be able to provide a clinical pathway through to secondary and tertiary care.
The nature of the ‘fly-in/fly-out’, one-off service model did not always achieve sufficient engagement with existing staff and services to have any real, ongoing influence on either the staff and services or the PHC system more generally. Other than the legacy of the child health check data (the limitations of which are discussed below), the child health checks left little else for services and the system to work with. We did not identify any health check practices, procedures or systems that were taken up as a result of the child health checks in the five case study communities. It is possible that some changes in practices were not attributed to
the child health checks because the staff we spoke to felt they were due to separate initiatives, including the EHSDI and other funding programs. We might assume that any such practices or systems used in child health checks that may have influenced future practices did not have the opportunity to become embedded or properly tested within a ‘normal’ remote NT PHC setting.
The exception to this is a number of service delivery models used as part of the child health check follow-up phase. Through the influence of staff familiar with the existing system and as a result of new funding this is leading to improved delivery models. This is most evident in the areas of hearing and dental services as discussed in Box 3 and 4 below.
While these service delivery models are positive developments, there would be benefit in monitoring and evaluating these models once they are fully operational to assess what impact they are having on health outcomes. The sustainabiity of these models also needs to be carefully considered, particularly the hearing/ENT model with so few staff positions being filled and the large number of children requiring ongoing clinical review.
There is also a need to ensure these service delivery models are not funded as vertical programs in isolation from frontline PHC services. Ensuring the services are part of comprehensive PHC is likely to lead to improved integration with other health care, improved capacity for and effectiveness of health promotion activities that can be directly linked to the relevant programs (such as nutrition advice with dental services) and potentially better coverage in high need populations due to an improved ability to case manage patients and guide them through different services.Top of page
Box 3: ENT and hearing services
The service delivery model developed in response to the need to provide CHCI referral follow-up
- developing health promotion materials
- providing health promotion in remote communities
- employing ENT specialists from interstate to provide non-acute surgery
- employing additional audiologists from interstate
- developing ENT outreach teams
- employing two case managers
- employing 16 Aboriginal Hearing Workers
- developing an Aboriginal Hearing Worker curriculum and training program
- providing a skills upgrading program for health workers
- procuring hearing booths and other equipment to be located in remote communities.
Much of this investment now supports ongoing hearing and ENT health service delivery. This ongoing service delivery includes continuing to provide services to children referred from the CHCI and also children who did not have a CHCI referral. At August 2010 there were approximately 3,000 children requiring ongoing hearing review.
There is no significant evidence that the effects of otitis media in remote Indigenous children in NT has changed in the last 40–50 years. The Burden of Disease report (WHO 2004) indicates that NT Indigenous populations have chronic ear disease at a prevalence rate of 31 per cent. The nature of otitis media in NT Indigenous children is complex and the disease needs to be managed simultaneously at several different service levels including primary health, secondary specialist services (audiology and ENT) and tertiary rehabilitation activities. The wider determinants of health also need to be addressed. Individual case management, rehabilitation or surgical interventions depend on integrated primary, secondary and tertiary hearing health information. Audiological and ENT specialist service provision resides in the secondary and tertiary health care systems and has various interactions with primary and other tertiary healthcare providers for the referral pathway and sharing and exchanging information.
The following diagram represents a ‘child-centred’ case management service delivery system. The success of the model relies on having a centralised and comprehensive data management system that allows all service providers access to real time information about a child’s care.
Conceptual diagram of ‘child-centred’ ENT and hearing case management service delivery model
Source: DHF (2010a).
Until effective preventative measures are identified, children living in remote Aboriginal communities are likely to continue experiencing high rates of otitis media and consequently hearing impairment. The provided an impetus to move to developing a new model of service for ENT and hearing. The systems developed for supporting the new model are immature. Recently some of this impetus has been lost with only three of 16 Aboriginal Hearing Worker positions being filled and the two regional coordinator positions remain vacant. Active recruitment to these positions is underway. These vacancies indicate the difficulties associated with maintaining programs and having them operating at high levels in remote NT communities.
Box 4: Dental services
To overcome the workforce shortage, a new service delivery model was developed based on recruiting staff from interstate for three-week placements to provide mobile dental services (dental vans), outreach services and secondary hospital (surgical) services. Helping Hands staff booked the flights and accommodation, set up human resource processes, facilitated NT professional registration and criminal record checks, and coordinated orientation and supervision for short-term staff. To promote high clinical standards all staff recruited received comprehensive clinical orientation on arrival in Darwin or Alice Springs before being ‘deployed’ to communities to provide services.
The program was well-supported by oral health professionals from across Australia and many of the staff recruited for short-term contracts were very satisfied with their NT experience and indicated a willingness to return. The Helping Hands team provided a high level of support to interstate recruits with the goal that they would enjoy their NT experience and return to complete more placements. In the second year of the service many did return for another short-term placement. Because these staff were already linked in to the DHF personnel system, the work required to set up the second visit was less labour intensive. Remote communities also indicated an increasing level of trust in the service when they knew the same team was coming back for a second visit.
The Helping Hands Child Oral Health Program, now known as the Closing the Gap Child Oral Health Program, continues to use this system. The standalone unit no longer exists and responsibilities for continuing to provide follow-up CHCI services has been integrated into DHF’s oral health program. In July 2009 the contract with RAHC was extended to include dental practitioners. RAHC takes responsibility for recruitment and cultural orientation of dental staff, while DHF continues to provide clinical orientation, support and supervision of teams in the field, as well as debrief and exit interviews at the end of placements.
‘Fly-in/fly-out’ models of service delivery that rely on interstate recruits do not provide the most ideal service delivery model. The constantly changing teams offer little continuity in care. Regular oral health services in the NT are also provided by visiting staff but for much shorter visits (between one and three days). The longer stays of up to three weeks in the new service delivery model were successful and proved useful in providing a wide range of dental services. This model of oral health service delivery, with mechanisms in place to ensure a high quality service is provided by appropriately qualified personnel, offers a feasible service delivery model alternative to ensure children living in remote Aboriginal communities have access to the dental services that they need.
WorkforceAlthough some practitioners from the Australian Government taskforce teams did return to the NT to work within the PHC system, this was the exception, and the child health checks have had limited direct impact on workforce capacity.
Cooperation and linksThe CHCI, and subsequently the EHSDI, have contributed to improved cooperation and understanding among service providers and other core stakeholders in the system of the challenges and needs in delivering PHC to remote communities. This was most evident from the two evaluation workshops where participants noted that the level of trust and good relationships between the three core partners (AMSANT, DHF and DoHA) would not have been foreseen 10 years ago. The AIDA also found improved cooperation and understanding among service providers to be a positive impact of the CHCI (AIDA and Centre for Health Equity Training, research andEvaluation 2010). More generally, the CHCI has had the impact of raising the profile of Aboriginal child health.
Information and dataThe CHCI has added a lot of information and data to the PHC system. During the case studies we heard that larger organisations with more resources have been able to use the data to improve programs and inform reviews on the treatment of some conditions. This includes the DHF which, as the main provider of oral health services in remote communities, has used data from the CHCI dental collection to inform planning for regular oral health services. Smaller organisations and providers with fewer resources, however, do not have the capacity to make use of the CHCI data.
There is a widespread view among NT health professionals, including paediatricians, that the CHCI data has ‘told us nothing we didn’t already know’, leaving no legacy to the system in terms of increased knowledge of disease and treatment in Aboriginal children. One health professional stated:
What was learnt? Zero, nil, nada, not a thing . . . we knew everything the child health checks told us about remote Aboriginal children. The way the health checks were done is simply not a basis for accurate data on prevalence.
There are two points in this quote—first that the data did not increase understanding of the health needs of Aboriginal children, and second that the data was inaccurate anyway.
Taking the latter point first, there would appear to be numerous reasons why the CHCI data would potentially be unreliable. This includes deficiencies described in previous sections such as how the checks and follow-up services were done, the skills and experience of the teams doing the checks and the numerous changes to the forms used to collect data. We do not have evidence to quantify the degree of inaccuracy. We do have the views of a number of health professionals working in the NT who believe this data is unreliable. Due to the consistency in views from several sources, we consider this information is very important; however, the information is from a single method (interviews) and we have been unable to corroborate it with evidence collected through other methods.
In its attempts to validate the CHCI data, the AIHW compared the results of the CHCI data with data from other sources (as discussed in Section 2.3.4) and found that the child health check results were similar to findings from other data sources, although they noted a number of limitations in making such comparisons. This is also a valid finding.
Based on interviews and the comparative analysis, we believe that the overall accuracy of the CHCI data is likely to be somewhere in between. It is neither a precise measure of the prevalence of health conditions in the target population, nor is it a totally unreliable estimate of prevalence.
There are also different perspectives on whether the data then increased understanding of the health needs of Aboriginal children. Once again, the consistency of the view, particularly in the NT, that the data ‘told us nothing we didn’t already know’ suggests that this is probably the case for many professionals working within the health system.
There is emerging evidence on the way the CHCI data has been used that suggests it has supported a greater understanding, particularly within the Australian Government, about the needs of Aboriginal children in the NT and the extent to which the services and systems are able to respond to this need.
Since the program was implemented, DoHA has regularly been asked at Senate Estimates hearings about the progress of the checks and follow-up referrals and the prevalence of the specific conditions found. The CHCI data provided the basis of the response to the committee. In responding to a series of questions on the progress of the CHCI at the Senate Standing Committee on Community Affairs, a senior manager within DoHA made the link between the data, better understanding and the PHC reforms in the NT:
It [the regular analysis and publication of CHCI data] is part of the reason why the significant change in primary health care is taking place in the Northern Territory and remote services—partially in response to a better understanding of the types of conditions that children have and the types of services that primary health care needs to be able to deliver (Commonwealth of Australia 2009).
It is likely that the regular CHCI progress reporting, coupled with the use of CHCI data in Senate hearings and similar fora, provided decision makers with quantifiable, tangible and timely evidence of the needs of Aboriginal children in ways that had not occurred previously. This information motivated the Australian and NT governments to address the issues that the children face.
Some participants in the evaluation remain hopeful that the wider evidence base of the CHCI will help build the basis for future policies and programs for Aboriginal child health, and learnings from the CHCI and the NTER may have helped to inform the Closing the Gap strategy. The CHCI data has considerable potential to be used as evidence of need in this area and may be a valuable resource for informing future policy and resourcing strategies.
Others are less optimistic, believing that limitations of the reliability of the data and the systems used to generate it mean it is unlikely to usefully contribute to ongoing development of health programs and services.Top of page
EngagementThe lack of engagement with people and structures within various parts of the health system at a policy and planning stage, before the child health checks, appears to have been quite destructive and may explain a number of less desirable impacts of the program.
We found that the lack of engagement contributed to resistance in parts of the system which led, for example, to children being viewed by health services as ‘your kids’ or ‘our kids’ depending on whether they were accessing a service through a child health check referral or a referral from normal service delivery. Staff morale and motivation at various levels suffered within the system and within the government and ACCHO sectors. One Australian Government official described the difficulty in keeping staff motivated and onside after the announcement of the NTER. This person had been unable to engage with staff before the announcement and could not prepare to manage any potential fallout associated with it. Similarly, we heard that the NTER had an adverse effect on staff motivation and morale in a number of ACCHOs. In its health impact assessment, the AIDA identified disempowerment of local services as one of eight negative impacts of the CHCI, quoting a senior Aboriginal official:
It disheartened a lot of people who’d been doing a lot of hard work in the Northern Territory for a long time. I think that was negative. People feeling, you know who’ve worked for 10, 15 years, working very hard, suddenly felt that, you know, their contribution wasn’t valued (AIDA and Centre for Health and Equity Training, Research and Evaluation 2010).
Opportunity costsWe discussed opportunity costs briefly in Section 4.4.1. There were opportunity costs associated with the CHCI and these were raised by participants in a number of interviews. Although we are unable to quantify these costs, they appear to fall in the following general areas:
- closing clinics and/or suspending normal PHC services to deliver the checks
- duplicating assessment when the child health checks were not coordinated with existing services including routine health checks of children (AIDA and Centre for Health Equity Training, Research and Evaluation 2010)
- the administrative burden of processing child health check forms and managing referrals (which generally fell on the existing service)
- existing specialist follow-up appointments being ‘bumped’ by child health check-generated referrals
- over-referrals taking up the time of specialists
- the loss of capacity from staff, particularly Aboriginal, who either left the workforce or whose morale was damaged by feeling disempowered or stigmatised by the NTER process.
Figure 5 shows participation in the GAA over the period 2003–2010. This was leading up to the CHCI and immediately following the delivery of the child health checks. Participation in GAA has been fairly consistent over the sevenyear period, showing a slight increase over time. Participation was lowest in 2006 but the child health checks do not appear to have influenced participation from 2007–2009.
From our interviews with health centre staff in the case study communities, it is evident that where they could, the local services coordinated the GAA and HSAK programs with the child health check visits. For example, they would delay the GAA and HSAK so that these checks were not scheduled for nine to twelve months after the child health check visit. It was not always possible to coordinate the services. Some communities had only completed their GAA or HSAK checks two to three months before the teams arrived to do the child health checks. If a child had had a MBS Item No. 708 health check within nine months of the child health check they were not supposed to receive a check, but this did not apply to GAA and HSAK health checks.Top of page
Source: Data supplied by DHF (October 2010).[D]
Figure 6 shows the number of hospital admissions for all NT Aboriginal children aged 0–4 years and 5–14 years from 1992–93 to 2007–08. The hospitalisation rate for Aboriginal children aged 0–4 years increased slightly over the period but there was no noticeable impact from the child health checks in the final year of data (2007–08). The hospitalisation rate for older Aboriginal children also increased over the period and there was a marked increase in hospitalisations in the final year of data. This may be the result of referrals from child health checks.
Source: Li et al (2010).[D]
The impact of the CHCI data is unlikely to have a lasting effect, partly because it was not integrated into the existing NT PHC information systems but mainly because it will date. There is a risk that knowledge of the health needs of Aboriginal children and the performance of services in meeting these needs will diminish, particularly at the federal level. There is the potential for the NT AHKPIs to be used to provide evidence at this level, but the KPIs currently only include three indicators relevant to child health outcomes and none relating to critical areas such as ear and oral health.
The experience of delivering services in different ways to remote communities and the specific models developed through the CHCI for doing this may continue to have a positive impact. The greater cooperation and shared understanding between service providers and the key PHC sector bodies (AMSANT, DHF and DoHA), at least in part facilitated through planning and delivery of the CHCI, has extended into the EHSDI program and we would expect this to continue.4.4.5 Conclusions
It ignored the service and system that were already in there, not to mention the staff (people) ... it wasn’t coordinated ... and it was wasteful ... it brought in, on the whole, over-priced and under-experienced staff—and on top of this the army—in to communities to do what we already do.
This perception was shared by many participants in the evaluation. For these people the CHCI was about making the best of a bad thing. To some extent this was achieved, albeit at some cost in personal stress and opportunity costs in service delivery. The extra resources were welcome—and needed. There were some successes, particularly delivering dental follow-up care. These successes depended on the people working within services and the system that set out to do things to make the CHCI work. Examples included the local health centre staff who invested considerable effort in the program to minimise negative effects on their community and to reduce community resistance, and the commitment of the majority of the visiting staff to ensure they coordinated delivery with the local health service.
The evidence collected in this evaluation suggests that the CHCI has helped to patch some areas of service delivery to children and some areas of the PHC system in the NT. At a cost of around $1,691–$3,294 per child for the child health checks alone, coupled with significant opportunity costs associated with disruption to existing services, the benefits that the CHCI has added to the health service and system are small. This suggests that the CHCI was inefficient. The real benefits that occurred in areas such as follow-up dental and hearing services could have been achieved without the substantial investment in the child health checks. In judging efficiency it is important to also consider the effectiveness of the program on improving the status of Aboriginal children’s health.
To improve any future child health check program in the NT, consideration should be given to developing policy and guidelines that are appropriate for remote Aboriginal communities. The policy and guidelines will need to consider issues such as:
- the objective or outcome of the intervention
- target population
- scope of the intervention
- screening tools and methods
- skills and resources required to do the intervention
- issues relating to the clinical setting for the intervention
- issues concerning the cultural preferences of patients
- data collection and access
- an analysis of the risks and benefits of the intervention
- an appraisal of different options and approaches.
- the process should focus on outcomes
- the guidelines should be based on the best available evidence and include a statement concerning the strength of recommendations
- the method used to synthesise the available evidence should be the strongest applicable
- the development process should be multidisciplinary and include consumers early in the process, and involve a range of generalist and specialist clinicians, allied health professionals and experts in methodology
- guidelines should be flexible and adaptable to varying local conditions
- guidelines should consider resources and should incorporate an economic appraisal, which may assist in choosing between alternative treatments
- guidelines should be developed for dissemination and implementation with regard to their target audiences
- the implementation and impact of the guidelines should be evaluated
- guidelines should be updated regularly.
20 - The actual ‘models’ used were not always as clear-cut as this list suggests. For example, while checks may have been done by a taskforce team in a separate building, the resident team may have provided support and assistance.
21 - In Phase 2 DHF and ACCHOs paid the salaries of visiting teams. These were much more comparable to the salaries of resident staff.
22 - This is an assumption that we have not tested by looking at uptake of existing child health services at a community level.