The mostly online 2018 Census had the lowest response rate in modern history – nearly five per cent lower than in 2013.
In some health catchments it is estimated that fewer than 80 per cent of people were counted – a major concern for public health advocates when the census is used to calculate the population-based funding for district heath boards. Also those with some of the highest health needs – such as the elderly, Māori, the homeless and those with a serious mental illness – are feared to be the hardest hit by undercounting.
The reaction from those working at the coal face of the public health system varies only in the level of its intensity. They range from a diplomatic “where was the risk assessment?” to a tired, almost angry, incredulity.
As one unnamed public health doctor from the Otago School of Medicine put it “it’s a complete balls up and I don’t know if we’ll ever recover”.
The five-yearly census was run by Stats NZ on March 6 but the online option and follow-up visits for hard copy forms continued for weeks after that date.
The census not only counts how many people live in households across the country, but also gathers other information including ethnicity, income, smoking-status and education. In turn district health boards are allocated population-based health funding based largely on the census data.
The online census option was first introduced in 2013 when about a third of census forms were completed that way. In 2014, after a budget cut and staff reductions delivered by the cabinet of the time, Stats NZ set a new target for an 70% online completion of the 2018 Census – with back-up, hard copy census forms. That target was surpassed with 82 per cent completing the 2018 Census online but there was much criticism of this year’s census including people complaining of not enough hard copies for people to fill out – and posted-out access codes never reaching people’s mailboxes.
Stats NZ announced in June that it had a lower response rate than expected – with full or partial data collected for an estimated 90 per cent of people compared to 94.5 per cent in the previous census – which meant achieving a high quality dataset was “more challenging” than anticipated.
In mid-July it confirmed that it was delaying the release of initial Census findings from October until March next year as it needed to revise its methods for ‘compensating’ or ‘backfilling’ missing data because there was more data missing than in previous censuses.
The data gaps are ‘plugged’ using dataset sources such as the Department of Internal Affairs, the Ministry of Business, Innovation and Employment (MBIE), the Ministry of Education and Inland Revenue.
Stats NZ says that that 2018 Census will still deliver ‘accurate and high quality data’ that communities and decision-makers can rely on; and they have downplayed the delay in release of information saying that every census requires gap filling.
This is true, but not to the extent of the 2018 Census, says many in the business of data collection. Likewise Australia also moved to an “online first” census model in 2016 and despite the census website crashing on census night the response rate only dipped slightly (from 96.3% in 2011 to 94.8%) compared to how far it dipped in New Zealand.
What are the health impacts?
The anguish in the voice of Northland DHB Nick Chamberlain is palpable as he describes how the census played out in the north like “a perfect storm”.
“The census being run correctly is absolutely critical for us. About 98-99 per cent of our funding is determined by our population as shown in the census. If the census is saying it had 90 per cent coverage overall, that can actually mean four times less that figure participation of Māori.”
He says access to the census for Northland – with the second highest Māori population and one of the most rural and oldest populations in the country – has been a “huge issue”.
“People have to have access to digital – our high rurality means a lot of people have PO Boxes – you have to have mailboxes for the access code to be left in. Postal delivery people have told me they’ve found access codes rotting in mailboxes.”
The Northland DHB invested in an education campaign that ran ads online and on radio to try and encourage people to fill out their forms, says Chamberlain.
“Lots of people never received their access code, which means the whole campaign could have been a waste of time…I’m all for technology but we needed a dual system.
‘We’ve had emails back and forth with Stats NZ. I don’t know how they will be able to correct the figures for us as there has been huge growth here in recent years.
“For a long time we have been underfunded – we were tracking $20 million behind. We have only just remedied that. You never get that back – you are always playing catch up…” He pauses. “And for that to happen again…is just really disappointing.”
The extent of the census undercount – and what it will mean for health funding – especially for Māori across the country, has appalled researcher Andrew Sporle (Ngati Apa, Rangitane, Te Rarawa) who is a University of Auckland research fellow in statistics and Deputy Director of the Healthier Lives National Science Challenge.
“The regional count determines the allocation of resources to those regions. Good quality information about the age, gender and ethnicity of regional populations is essential as different groups have different health needs.
“Fewer people taking part in the census can have major consequences for funding health and social services. If it looks like we have fewer people than there actually are, then the local services will be under-funded.
“There have been concerns about the regional and ethnic counts of the previous census, but it looks like this census will be worse, which is not good news for regional services and not good news for Māori.”
Any significant changes to such a huge piece of work as a census needed meticulous planning, says Professor Tahu Kukutai (Ngāti Tīipā, Ngāti Kinohaku and Te Aupōuri), a Professor of Demography at Waikato University. Prior to this year’s census New Zealand had been known for running a highly effective census.
“If we look back at 2013 only 34 per cent of census respondents did it online and Stats NZ was trying to double that in just five years. That is very ambitious and there was always the risk that things would not go as planned,” says Professor Kukutai.
“There needed to be a robust risk analysis and then risk mitigation and residual risk strategies – basically they needed a back-up plan and a back-up for the back-up. The situation was entirely avoidable and now Māori will likely bear the costs of poor quality data.”
Māori needed to be more involved in census decision-making
Professor Kukutai is a founding member of Te Mana Raraunga (TMR) – a Māori Data Sovereignty* Network of academics and practitioners who informally advocate for Māori rights and interests in data to be protected as the world moves into an increasingly open data environment.
In 2015 the Māori Census Advisory Committee which she was a part of, was dissolved. “I’m not saying it was a perfect model but it was an advisory group of people of very diverse backgrounds and a wide range of expertise which operated as a very robust sounding board for those planning and creating the census.” She says it is really important to have Māori included not only in the delivery of the census but in all stages of decision-making. Also that the census is essential for planning and for resource allocation and has profound real-world consequences.
This census’s low quality information may also impact on how confident health researchers and officials can be using population-related data, says health researcher Dr Donna Cormack (Kāti Mamoe, Kai Tahu).
“It introduces doubt. We won’t know if we’re looking at an actual trend or just a change because of the data issues …it’s hard to know the extent of the problem because we still don’t know the extent of the undercount. We don’t yet know what the impact of the census not reaching everyone in the population will be.”
Uptake of the census has historically been lower for Māori, says Cormack who holds joint research positions at the University of Otago’s Eru Pomare Māori Health Research Centre and University of Auckland’s Te Kupenga Hauora Māori.
“There is a need to involve Māori in conversations and decision-making around the changes that are made. We knew that if the census went fully online that there’s not equal access for Māori households to the internet. We know this from data from the last Census. The lower Māori participation will have a disparate impact on Māori because the smaller the group in population terms the bigger the impact on the data if they are not all counted.
“Our Treaty relationship wasn’t really factored in and I would expect that there would be a Treaty-based relationship with Stats NZ. That we might be collecting data that is better for some population groups than for Māori means that it has been inequitable, even if that was not the intent.”
Ngaire Button, the Pacific Portfolio Manager for Canterbury DHB was pro-active in encouraging people to do this year’s census. In the lead up to the big event she says she wore her purple census shirt close to every day for a fortnight.
“But that’s not actually my job to encourage people to do the census. It helped raise awareness of my colleagues and my own whanau helped my dad fill out the census but …so many of our people didn’t even know the census was happening or they expected a nice lady to come to their door to help them fill it out.
“I am supportive of the move to go digital but to go from around a third of the census being done online in 2013 to aiming for (nearly) everyone to do it online this year was too much too soon.”
What is the impact of absent-census-completing-Cantabrians on an already stretched Canterbury DHB?
“It’s horrendous for us. Every funding dollar counts. Currently we get $3500 health funding per person. We are already paying an extra capital tax towards the Christchurch hospital rebuild. For every person missed – that missing data means we will have to take an already incredibly stretched dollar and stretch it even further.”
Lance Norman, chief executive of the Māori public health organisation Hāpaite Hauora, says that the failed census is part of a bigger problem for Māori where at best lip service has been paid to the concerns of Māori around the sovereignty of their own data; and at worst the information gathered about Māori hasn’t always been for Māori.
“There is a broader question to be answered here because ‘access’ is determined by more than just one thing. Sure, the digital nature of the census may have had an impact, but it’s also to do with the acceptability of this sort of engagement for Māori as well. What we know from Kaupapa Māori research principles is that the gathering of information about Māori hasn’t always been to our benefit. So there’s a way to go for the Government to convince our communities that this will serve our people in a positive way – and in a way that’s meaningful for us, not just to suit a Pākehā funding or data collection paradigm”.
“Indigenous data sovereignty is an issue that is becoming more and more well understood thanks to groups like Te Mana Raraunga, we suggest these issues need to be more closely interrogated during the next census period”.
“So obviously we think the inaccurate data provided by the census will ultimately hurt Māori because the way things are organised in Aotearoa is that you have to be ‘visible’ in a way that suits the policy settings determined by the Government. But the more important point we would make is around culturally acceptable data collection and we think this should be urgently addressed”.
How to stop it happening again?
The problem with plugging the largest census gap in recent times is that it is expensive and could cancel any savings made by putting the census online. It is also highly technical so the resulting data will be less easily interpreted and more difficult to understand.
So what are the lessons to be learned to ensure the next census gets a better response rate from Kiwis?
Sporle appreciates that Stats NZ has responded to criticism by organising not one but two reviews to see what happened and to prevent it ever happening again. Their response so far, in his view – has been, “quick, open and comprehensive – but this approach needs to be continued”.
“This is an opportunity to work out how to do a census well and rebuild public confidence in the quality and value of the census.”
He points to under-resourcing as a partial contributor to the failure of the census.
“To some extent Stats NZ was given a hospital pass. I believe that they were not given enough budget to do this census. The digital-first focus was part of a cost reduction policy of the previous government to reduce the cost of doing the census by 5-10 per cent over ten years. Each census costs around $120 million, creates one billion dollars of value to the economy and they aimed on saving $24 million over ten years? The result is we are likely to get less accurate information and get it six months later.”
“The value of a census is that it is done well and the results are available to be used as soon as possible. It’s such a foundational part of running our democracy. The reviews need to focus on what is required to achieve that quality. That’s more than budget, it’s also re-engaging with people and communities who provide and use the information.”
He said he’d like to see the census be online “where appropriate”.
“As someone who has worked on the census from door knocking, handing out forms to doing complex analyses on the data, I’m aware how important face-to-face is to a successful census.
“Having someone from your community who looks like you and talks like you visiting you to explain why this process is important – it helps get real buy-in. A census produces information about people and communities. But that information is also for people and communities, so it needs to be high quality. That quality requires a level of engagement on lots of levels that wasn’t apparent in this census.”
Dr Cormack also points to a lack of widespread awareness on the part of New Zealanders of the administrative data that may be used for imputation to fill in the missing data.
“People may not expect the information they’ve shared with other agencies for example, potentially being used to fill in the census.
“If there was consultation with Māori and rather than viewing Māori simply as one of many clients or as a customer… if Māori were engaged as a partner according to Te Tiriti, these issues may not have happened. If there was greater Māori involvement in governance and decision-making, this census may not have failed for Māori as it appears to have.”
Professor Kukutai says it is vital for Te Mana Raraunga to be part of the “what-do-we-do-now” about the census process.
“Stats NZ are not resistant to Māori data governance; the Government statistician is very aware of it and wants to partner. It may be too late for 2018 but they appreciate that there’s a need to do things differently. You can’t make decisions and impose them on the New Zealand public, you’re got to get their buy-in and maintain their trust and confidence. This is critical for Māori as we have a rather fraught history when it comes to state controlled data collection.”
Kakutai says a very clear example of the importance of census data is the counting of the Māori descent population, which is a part of the statutory formula for determining the boundaries and number of Māori electoral seats.
“If Māori descendants are missing in large numbers, this will reduce the size of the Māori electoral population and, potentially, the number of electorates.For the 2018 census much more data will need to be imputed and there needs to be a high degree of transparency about how that will be done.”
*Māori Data Sovereignty recognises that Māori data should be subject to Māori governance and supports tribal sovereignty and the realisation of Māori and iwi aspirations. Its kaupapa is: He whenua hou,Te Ao Raraunga, Te Ao Raraunga, He whenua hou.