When research is not Research
Speech by Theo Muller to the Rotary Club of Eastern Hutt on 2 November 2009
President Judy, Ladies and Gentlemen,
I appreciate the invitation to present to you today. Thank you.
Before I get into the topic of today's presentation, I need to tell you that I will be using the term research fairly frequently. Please, do not think of me as the man in a white coat concerned with physics or scientific laboratory research. Research in the context it is used today means market research or social research. As you have heard from the introduction, I am a market researcher; I study people's behaviour, attitudes and actions on behalf of the many clients we have in the public and private sectors of this country.
Several weeks ago, Councillors of The Hutt City Council approved a budget item of $2.2 million for the upgrade of the dilapidated bus garage at Korohiwa at the southern end of Eastbourne. Two Councillors voted against. How it arrived at that decision is a story worth telling.
Enter ERI or Eastbourne Rights Inc. - a small group of Eastbournites who felt, rightly or wrongly, that Eastbourne got a raw deal from HCC and they threatened to start secession procedures aimed at breaking away from Hutt City and joining Wellington City. A bold move, but it certainly got the attention of the Council and particularly the Mayor. He was not interested in losing millions of dollars in rates and handing it over on a silver platter to Kerry Prendergast. From a person who was accused of not paying enough attention to Eastbourne's needs, overnight, he became an Eastbourne advocate. You and me would probably find that difficult to understand, but let's not forget that it is a politician's prerogative to swing with the mood of his constituents. That's how you stay in power.
ERI have never formally called off the secession process as they are keeping a wary eye on every move of the Council, particularly the Mayor, and there is no doubt that they have created a bit of clout, in fact a substantial counter weight to the whims of Mayor and Councillors. And they have not been afraid to use that power when they were amongst the prime movers of having the Korohiwa bus garage classified Category Two of The Historic Places Trust. And, as I said earlier, the Council has set aside $2.2 million for its refurbishment and resource consent for the work has been granted and tenders are being let.
I am not qualified to argue whether this should have happened or not - my interest in this saga lies in the way in which the decision of setting aside a budget of $2.2 million for the refurbishment of the bus garage appears to have been made.
Max Shierlaw, City Councillor, wrote in the Hutt News of 20 October 2009, and I quote "Council has allocated $2.2 million to this project, largely on the grounds that they have been advised that it had significant support from the Eastbourne community".
Now, I want you to remember the words "significant support". Councillor Shierlaw continues, "... feedback presented to the Council has been solely from the Eastbourne Community Board, who have been content to limit their consultation to a handful of people, who support the project". Again, remember the words "consultation of a handful of people".
Ladies and Gentlemen, I will now tell you what my talk will be about. For the next fifteen minutes or so, I would like to address the question "When is research not research?"
In the Hutt News of 13 October 2009 a lone contributor to the "Your Views" section of this newspaper, Mr Greg Dellabarca, called it economic lunacy to spend that sort of money on, what is essentially, a dilapidated building suitable only for demolition. He may not have realised it, but his letter to the Editor unleashed a mini flood of reactions, including Mr Shierlaw's piece of prose cited earlier.
The project should not proceed; the money should be spent on other more worthwhile causes. These were the views of some of the letters to the Editor of the Hutt News of the 20th of October.
Now, these are not necessarily my view. My view on this matter is entirely irrelevant, particularly for the purpose of today's presentation. I am looking at this issue from a researcher's perspective, so my personal view shall remain concealed.
The decision to refurbish the bus garage was entirely based on assumption; the assumption that the Eastbourne community and the rate payers of Lower Hutt wanted this building preserved and refurbished. A handful of individuals (remember that phrase), and I mainly refer to ERI and the Eastbourne Community Board, who felt that this was a worthwhile cause, pushed ahead under the guise of "significant support from the Eastbourne community" - the other phrase that I wanted you to remember. They bent the Mayor's ear, both of which were very pliable, because he wanted to be seen to do good for Eastbourne and thus maintain his power base.
How often have you heard the phrase "research has shown ..." - politicians use it all the time in order to advance their own cause or pet projects. Or they say "people in my electorate come and talk to me all the time ... or I talk to people every day", suggesting or inferring that what they hear is the collective opinion of (the vast majority) in their constituency, when in fact it is nothing more than the opinion of a handful of people. This is what is happening in Eastbourne with the bus barn. It is a handful of people who infer that the majority of the people in the community share their enthusiasm for having the bus barn refurbished. Whilst not saying in so many words that they have done their research, they certainly put a spin on things to suggest as such.
Ladies and gentlemen, this is not research; this is nothing more than a small group of enthusiasts abusing the basic principles of democracy. Whilst they have every right to get in behind a project or issue; it is devious or at least disingenuous to suggest that they are acting on behalf of the collective will of the wider community. They have not done their research, but they infer that they have.
I can cite another issue that also involves the Eastbourne Community Board, who have asked the Hutt City Council to reduce the speed limit on a stretch of Marine Parade from 70K down to 50K.
Under the guise of "this is what the community wants" where in fact it was just a handful of people who have a bee in their bonnet on this issue. There has not been any proper research and community consultation has been very poor, resulting in this awful mess. It now looks like the status quo (70K) will remain.
I am well aware that it is impractical to research the community on every proposed change that affects that community. The central government agrees with this notion having just vetoed Rodney Hide's proposal for local authorities to conduct non-binding referenda on rate rises. In a democracy when constituents elect their representative, they automatically give this person a mandate to act on their behalf. However, it is plainly mischievous and deceiving, if that representative then pushes ahead on his or her agenda, inferring that they have done their research, when in fact they have not - they have spoken only to a few people who agreed with their agenda, and if there were people who did not agree, they conveniently forget to mention them.
Ladies and gentlemen, be very aware next time when you hear a politician utter the words "research has shown that ... or this is what the people want". Demand to see the research or at least demand an explanation on how the research was carried out, how many people did they in fact speak to, how was the data analysed, how many were for and how many were against and does the research truly support the proposed action to be taken. Be critical and do not allow yourself to be bamboozled by those official sounding phrases such as "research has shown that ... or it is the collective will of my constituents".
Deception is not the prerogative solely of politicians. The media are also very skilled in this area. How often have we been invited by Paul Holmes, Mark Sainsbury or John Campbell to call this number if you agree and that number if you disagree on a particular issue with the promise that the results of "this survey or poll" will be revealed at the end of the show. Calls cost $1.99 and children, ask your parents first.
This is not research, even though they give the impression that it is. Issues subject to this type of treatment are often very polarising and have a strong emotional element - there are those who are strongly in favour and those who are equally strongly opposed. Both groups of people however are strongly driven by their emotions and personal convictions, in fact so strongly, that they pick up the phone and spend the money. These tactics were often used at the time when the anti-smacking legislation went through the House.
True research, on the strength of which investment decisions are to be made, is all about obtaining the opinion of a representative sample of the wider population or, as we call it in research jargon, the target audience. Those strongly opinionated people, driven by powerful personal convictions at either end of the issue, were so motivated that they grabbed the phone and cast their vote. Do those people represent the wider population? The answer is a resounding no.
Yet, the information we have just gleaned from our TV screens make us believe that X% of New Zealanders agree and Y% disagree. Ladies and gentlemen, like in politics, this was nothing more than the vote of a very small minority of driven individuals.
Am I suggesting that politicians should not talk to people, should not be interested in their opinion, should not canvas their constituency? Of course not. They need to keep their ears very close to the ground and listen very carefully to what people say. Smart politicians hold regular community clinics, attend meetings of the local business community and other community groups and conduct focus groups on topics of local, regional and national interest. That's how they stay in touch with the people.
Whilst it could be argued that, in the broadest sense of the word, they are conducting research, this type of obtaining community feedback - we call it qualitative research - should never be used to support important investment decisions. Investment decisions can only be supported by what we in the trade call, quantitative research. The type of research I referred to earlier that involves obtaining the opinion of a representative sample of the wider population.
In conclusion, partly in support of this presentation to you today and partly to satisfy my own curiosity as a researcher, I would like to demonstrate to you with a practical example, of how the people of Eastbourne feel about the upgrade of the bus barn and the Council spending $2.2 million of rate payers' money.
In a small telephone survey carried out between 30 October and 2 November 2009, we have asked 100 residents of Eastbourne and the bays two simple questions.
1. Do you agree with the Council's plan to upgrade the bus garage, including three flats?
2. Do you think that the allocated $2.2 million for the bus garage upgrade is good use of rate payers' money?
The margin of error on this survey is ±9.7%.
Before giving you the results - and you are the privileged few who get to hear the results before anybody else - I would like to tell you how we collected the information.
We obtained a random sample from the most recent Hutt Valley telephone directory, which is simple to do as addresses in the directory are followed with the abbreviated name of the bay or local area, like Muritai, Days Bay or Lowry Bay. Businesses, local politicians and individuals with a known perspective on the issue, were excluded. Individuals who had no knowledge or awareness of the issue were also excluded.
Secondly, we asked one resident per household 18 years or over if they were willing to participate in a confidential survey by introducing the survey with the following sentence: "The Korohiwa bus garage is located at the southern end of Muritai Road in Eastbourne. It has a category 2 classification of The Historic Places Trust. Earlier this year, the Hutt City Council agreed to spend $2.2 million on the upgrade of the bus garage, including three flats, which are currently in a dilapidated condition."
Thirdly, participation in the survey was voluntary and completely anonymous. Only, if the person agreed to participate, we asked them the two questions mentioned earlier.
Lastly, we recorded the response to the questions, the area the person lives in and the gender. The reason for the last two items is so that we obtained balanced result. We were limited by the available listings in the telephone directory*.
And now for the results.
Question 1: Do you agree with the Council's plan to upgrade the bus garage, including three flats?
32% agree, 64% disagree and 4% Don't know/have no opinion.
Question 2: Do you think that the allocated $2.2 million for the bus garage upgrade is good use of rate payers' money?
18% agree, 72% disagree and 10% Don't know/have no opinion.
So, where to from here? Councillor Baird, who also sits on the Eastbourne Community Board, claims that it is too late to make objections and reverse the decision made by the Council to proceed with the upgrade. She argues that residents should have spoken sooner during the consultation process.
Reversing the decision is fraught with obstacles. As I mentioned earlier, the bus garage has a Category Two classification of The Historic Places Trust. If the Eastbourne community and the wider Lower Hutt community don't want this to go ahead and want the building demolished, they will first have to make a visit to the Historic Places Trust and negotiate approval of the Category Two listing to be lifted.
What are the lessons here?
I asked you to remember the phrases "significant support from the Eastbourne community" and that the Eastbourne Community Board seemed "content to limit their consultation to a handful of people". I believe that, whilst this was only a small survey in terms of sample size, it does provide a reasonably strong indication that the Eastbourne Community Board and the Hutt City Council may have misread the collective opinion of the people of Eastbourne on this issue.
Secondly, be critical. Don't allow politicians or pressure groups make representations on your behalf under the guise of research.
Thirdly, as individuals and communities, we need to participate in the democratic process of consultation and decision making. Stand to be counted.
Thank you. I would be happy to answer any questions you may have.