Tell me again how this should be allowed!

So this little inaccuracy caused a bit of damage.
It is rural areas that will suffer - particularly the fishery.

Funny how the PC vote was pretty dead on - but the other little mess probably cost some seats for the Liberals.

More INFO regarding MQO poll coming soon

I have received a further response from Ms. Carolyn O'Keefe - President MQO Research.
I will post this and commentary over the weekend. Stay tuned.

Has the NL Media unwittingly interfered with our Election? Polls should be Investigated?

Just listening to a VOCM report on the latest poll this one done by Environics - reportedly for the Canadian Press.

What was reported was that this was a poll done with a group of people selected from a panel and who were compensated for it.

Further they properly stated that these types of polls do not have a predictable level of accuracy - therefore they do not ascribe a + - percentage to it.

This is exactly what should have been done by MQO and their two polls. If one reads the standards for polling stated by the MRIA an association for pollsters - one should conclude that the MQO polls should not have stated an accuracy +-.

Now let's look at what the cost is to the people, our election, and the outcome.

In the case of many reputable media giants like the New York Times and others - simply do not report these polls because of the potential for significant inaccuracies and thereby possibility of improperly swaying opinions during an election campaign.

It's one thing to report these types of polls when talking about choosing between Crest and Colgate or Pepsi and Coke - it's quite another thing to upset election impartiality.

So why are the media stations in Newfoundland and Labrador even reporting these types of polls and what is the rolling impact of them?

Please remember this statement by Stanford University Researchers on these types of polls:

The inconsistency is a challenge because it means the accuracy of one measure from an opt-in panel survey can’t reliably be taken to mean that other measures are accurate, the researchers noted. And there are other problems: While results of one of the seven opt-in online panels was “strikingly and unusually inaccurate,” they said, “the rest were roughly equivalently inaccurate,”

This means that the polls by MQO and Environics can be equivalently strikingly and unusually inaccurate. So once again why is our media reporting them? Now that we have a CRA poll which may be based on random sampling meaning an accuracy percentage can be applied - have the other polls now interfered with peoples opinions prior to a "random sampling" poll being done?

Have they been lined up this way?

One thing we should all be - is very mad at companies conducting "innovative" polling in the middle of an election and also that our media has broadly reported it.

Although the Environics poll has fully explained that their poll cannot attach itself to an accuracy percentage - why has the Canadian Press released it and why is our media reporting it?

This should all be investigated after the election.

Response from Carolyn O'Keefe President MQO Research on the Poll

Well we now have part of the response from the MQO Research Company.

Now we can start to put in perspective what the polls value is.

As all the questions have not been answered I will ask again for further information.

First - Here is a statement from a report lead by Stanford University Researchers.

While the random-sample surveys were “consistently highly accurate,” the Internet surveys based on self-selected or “opt-in” panels “were always less accurate, on average, than probability sample surveys, and were less consistent in their level of accuracy,” the researchers said. Further, they said, adjusting these samples to known population values had no effect on accuracy (and in one case even worsened it) as often as that process, known as weighting, improved it.

The inconsistency is a challenge because it means the accuracy of one measure from an opt-in panel survey can’t reliably be taken to mean that other measures are accurate, the researchers noted. And there are other problems: While results of one of the seven opt-in online panels was “strikingly and unusually inaccurate,” they said, “the rest were roughly equivalently inaccurate,” suggesting that no one or another of these producers can claim to have perfected the approach.

One ABC reporter said:

The results shouldn’t be a surprise; as I’ve reported previously, non-probability samples lack the theoretical underpinning on which valid and reliable survey research is based; our policy at ABC News, as at several other national news organizations (including The Associated Press, The New York Times and The Washington Post) is not to report them.
The poll conducted by MQO was one of these polls. Although Ms. O`Keefe says the poll is a probability based one. This is not exactly accurate as it is a panel. This poll is not scientific (under the definition under MRIA) and may be very inacurate. You will note that Ms. O`Keefe does not say it is scientific. That is something she should have answered.

Further the MRIA - the Association MQO is a member of - public information states:

Online survey participants are self-selected, meaning that a margin of sampling error cannot be calculated or quoted for online panel research studies. Using margin of sampling error to describe the accuracy of online polling is misleading, and prohibited under MRIA’s Code of Conduct.

Does this mean that adding some phone panel participants in the midst of online polling makes this one different.

Below is what Ms. Carolyn O`Keefe provided me on the poll. As not all the information requested was NOT included - I will ask again for complete information.

I would be pleased to respond to your request. Please find attached a release outlining the methodology employed for the poll. The poll was paid for by MQO Research, it was not commissioned by any outside organization. I designed the questions for the survey.

The survey conducted was quantitative in nature, not qualitative research, so Questions 5-7 do not apply.

The poll was conducted through iView Atlantic, a proprietary research panel developed by MQO Research. This panel consists of over 12,000 Atlantic Canadians who have agreed to provide their opinions on Atlantic, provincial and local topics that matter to them. To ensure the representativeness of all Atlantic Canadians, iView Atlantic also includes a component of phone-based membership to reflect those Atlantic households who still spend limited time in an online environment. iView Atlantic is unique from other opt-in online panels in that panellists are recruited randomly by telephone, and indicate their preferred method of contact (either by phone or online).This means that individuals without Internet access are recruited and can participate in our panel surveys therefore enabling coverage of the entire population. iView Atlantic is a probability-based panel, meaning that every member of the population has an equal chance of being selected for participation.

The poll was based on a sample of 464 Newfoundlanders and Labradorians, with a margin of error of +/- 4.6%.  It was conducted between Wednesday, September 28 and Friday September 30, 2011.   The respondents were weighted regionally to ensure proper representation of the province as a whole.

 

Letter to MQO President Carolyn O'Keefe requesting information on Polls

Attention: Carolyn O’Keefe

President

October 3^rd 2011

Dear Ms. O’Keefe,

I am writing this letter to request the following information on the past two polls you have conducted on the Newfoundland and Labrador election. 

I am a member of the general public – interested in the validity of polls.

This request is being made as per the standards of the MRIA – an organization you are a member of.

Please advise when this material will be provided. 

• Sample size, and population surveyed (who was included)
• Sponsor of study (who commissioned the research)
• Survey method (e.g. telephone, on-line, intercept)
• Timing (when the survey was done)
• Statement of sample error/margin of error (i.e. "+/- 2.5% 19 times out of 20")

• Name of practitioner (company conducting research)
• Sampling method (e.g. random, custom list)
• Weighting procedures (statistical weights, if used)
• Exact wording and order of questions

3) Always differentiate between scientific (most public opinion polls) and non-scientific studies (reader/viewer polls or other "self-selection" methodologies).

4) Where appropriate, use the caveat that research is not necessarily predictive of future outcomes, but rather captures opinion at one point in time.

Additional Standards for Reporting on Qualitative Research
Additional considerations should be adhered to when reporting on qualitative research observations (such as focus groups, or other small sample in-depth research).

5) Clearly defined recruiting specifications (i.e. product usage, party affiliation, specific demographic specifications, etc...)

6) Inclusion of statement of non-projectability. Results of qualitative research are not statistically projectable to the population at large.

7) Qualitative reporting should not include percentages or precise proportions. Expressions such as some, most, or a few, may be used.

I look forward to your earliest reply.

Thank you,

Sue Kelland-Dyer

No answer from the polling company! Public Servants restricted in Private Life?

MQO has not answered my request for specific information on the election poll. They claim to be an upstanding member of MRIA - an agency that sets standards for polling and research.

The information I am seeking - according to MRIA - should be given to a member of the public that asks for it. So where is it. The only answers MQO has given is a self proclamation of their goodness.

Then we move to a Telelink Poll that has 42% undecided. That number could form its own government.

Now word that the PC government policy is that public servants are being controlled in their private lives by their employer - the province.

Rules like - can't make a political contribution, can't display a sign on their own lawn, can't wear a button outside the workplace, and can't stuff envelopes!

Who does this apply to?
- report directly to, speak on behalf of, or provide advice, opinions or recommendations to the Premier, Cabinet, Treasury Board, a Minister, or a Deputy Minister

That could be anybody.


What's next?

This is a very dangerous and not democratic. Also is this applying to the Tories - and the political emails from the Confederation Building - managers and all - promoting Tory things. Okay let's start pumping these out.

Then there's this.

Political Support Staff

For the purpose of this Policy, political support staff members are considered to be politically non-restricted. The appropriateness of the political activities undertaken by this group will be determined by the respective Minister.

So if the PC's win - does that mean that Ross Reid and Len Simms cannot come back - because maybe they would not be impartial and may treat certain members of the public unfairly.

Finally this one takes the cake.

The policy actually says:

Employees
All employees can vote in an election.

Gee Thanks!

Polling Standards - I am awaiting a reply from MQO - based on MRIA standards.

Today in response to Liberal leader Kevin Aylward's assertion that the MQO poll was bought and paid for - MQO released a statement countering the claims.

Part of the MQO rebuttal follows:

MQO Research is an accredited Gold Seal member of the only national research organization, Marketing Research and Intelligence Association (MRIA). Every piece of research carried out by the firm meets and exceeds the Code of Conduct and Good Practice put in place by MRIA. The same is true of the recent election polling done in Newfoundland and Labrador. 

So in the spirit of fairness - I viewed the MRIA standards and followed up with a call to Carolyn O'Keefe - President of MQO. I got her answering machine so I left a message.

I have asked her to supply the following to me:

In order to preserve public confidence in public opinion research-a science of mutual importance to the Media and Market Research industries-please consider our following recommendations before releasing survey data to the public:
1) Please include the following key facts in the report:

• Sample size, and population surveyed (who was included)
• Sponsor of study (who commissioned the research)
• Survey method (e.g. telephone, on-line, intercept)
• Timing (when the survey was done)
• Statement of sample error/margin of error (i.e. "+/- 2.5% 19 times out of 20")

2) Please make the following facts available to the public upon request (if not included in report):

• Name of practitioner (company conducting research)
• Sampling method (e.g. random, custom list)
• Weighting procedures (statistical weights, if used)
• Exact wording and order of questions

3) Always differentiate between scientific (most public opinion polls) and non-scientific studies (reader/viewer polls or other "self-selection" methodologies).
4) Where appropriate, use the caveat that research is not necessarily predictive of future outcomes, but rather captures opinion at one point in time.

Additional Standards for Reporting on Qualitative Research
Additional considerations should be adhered to when reporting on qualitative research observations (such as focus groups, or other small sample in-depth research).
5) Clearly defined recruiting specifications (i.e. product usage, party affiliation, specific demographic specifications, etc...)
6) Inclusion of statement of non-projectability. Results of qualitative research are not statistically projectable to the population at large.
7) Qualitative reporting should not include percentages or precise proportions. Expressions such as some, most, or a few, may be used.

I have not received a return telephone call yet. 

Snake Oil Poll? Do we really know? Oh yes and I hear a poll is being conducted NOW!

DARRELL BRICKER: IPSOS REID
Open letter to Ontario’s journalists

Posted on Wed, Sep 14, 2011, 10:02 pm by Darrell Bricker

We’ve all seen over the last few days a confusing cacophony of polls on the Ontario election.
Depending on what poll you read, McGuinty’s Liberals are on a roll, Hudak and the Tories are comfortably ahead, or the Grits and Tories are neck and neck. How can this be? It is because all polls are NOT created equally. And, in spite of what you may assume, pollsters are never held to account for their indiscretions, incompetence and mistakes (there is no “polling police”). Some marginal pollsters count on your ignorance and hunger to make the news to peddle an inferior product. Others are using your coverage to “prove” that their untried methodology is the way forward for market research in Canada. Instead of being their own biggest skeptics (which is what our training tells us to be), they’ve become hucksters selling methodological snake oil.
Remember, the term “pollster” is derived from the term “huckster.”
Journalists are no mere dupes in this process. We’ve also seen a disturbing trend of late in which questionable polls find their way into an outlet’s coverage because they appear to match an editorial line, or present a counter-intuitive perspective. After all, if a poll is wrong it’s easy to throw the pollster under the bus and walk away with clean hands.
All of this MUST stop. We are distorting our democracy, confusing voters, and destroying what should be a source of truth in election campaigns – the unbiased, truly scientific public opinion poll.
To be clear, this is not about banning media polls during election campaigns. That would just take us back to the old days of backroom boys leaking false polling, and to the practices we see in less stable democracies around the world. What we need is better, more informed reporting of polls. Here are six easy rules to get us started.
1. IVR polls (robo-calling) are NOT telephone polls.
They are tremendously biased in terms of sample coverage. In the last federal election IVR polls were MASSIVELY off on the final vote. Why? Because they systematically under-represented the Tory vote. But, these same pollsters have picked up again in this election without skipping a beat. Here’s the question you should be asking them – show us your UNWEIGHTED results. That’s prior to adjusting for both demographics and political support. You’ll be surprised by what you see.
2. The same question should be asked of pollsters using on-line methodologies to predict vote.
Ask to see their results prior to all weighting. You will find that some heavy thumbs are being applied to adjust for under-represented voting groups. While the weighting can produce very good results, it really amounts to no more than an educated guess. And, if that’s the case, the results should be reported as such.
3. Disclosing margin of error and the questions asked doesn’t represent meaningful disclosure: a rogue poll, a bad poll, and a good poll all look the same on these points.
Be honest when something looks dodgy — either don’t publish it, or publish it with an editorial disclaimer. Huge shifts in public opinion, even in an election, are the exception, not the rule and should be treated as dubious until confirmed by other polls.
4. A moratorium should be placed on all “new” polling techniques until the pollster has tested them in parallel with more traditional polling methods with a record of success.
This should have absolutely been the case with IVR in the federal election. It’s a pretty minimum standard to be right at least once with a new methodology before you get to lead a newscast or get the front page headline in a newspaper.
5. The conventional election scenario that’s tested in a vote question is – if you had to vote tomorrow, how would you vote?
If your pollster asked something different, even if the poll was properly done, it should NOT be reported as “current vote.” It is misleading to do so.
6. Spend some time with pollsters.
Not just the one you know or use, but with the ones you’re hearing about. They are NOT all created equal. Find out about their business. Is this a one person show that subcontracts all of their data collection, and only shows up at election time? What research are they doing on research? What resources do they have available – is it one way for everything, or do they have the resources to adjust depending on what’s needed? What are their motives? Are they just publicity hounds trying to promote their business, or are they serious researchers with both the desire and capabilities to perform at the highest level? Shouldn’t you do at least this minimal background check on whomever you are trusting with your lead story?
These six rules require journalists to do two things – kick the tires before publishing a poll, and make it harder for bad or misleading polls to get published. That’s the way it’s done in jurisdictions that take polling seriously. It’s sad that this isn’t the case in Ontario today.
Darrell Bricker and John Wright lead all of Ipsos Reid’s political polling for the media in Canada. They have done polls in every major election in Canada since 1988. Ipsos Reid is Canada’s largest market research firm, and is owned by Ipsos, the world’s third largest market research firm.

More on the Poll by the M5 subsidiary! Oh there is a real problem here!

Further to my previous post on the latest political poll conducted by a subsidiary of M5 are the following details.

First M5 contributed $13,400 to the PC's since 2007 and we have NO idea what they have contributed in 2011 - until 2012.

Then there is the statement, "a survey with a probability sample of this size has a margin of error of plus or minus 4.9 per cent." Not necessarily this poll - but a like poll - right?

Then there is the kicker - this poll...involved 413 adults, who were contacted over the phone or online.
How does one confirm - that the person on the end of a computer is an adult? How can they be sure? How many were done online?

This is junk! If this is how this election is going to proceed with the media dutifully reporting - the information going to the public is not acceptable.

Let me predict that these polls will be coming fast and furious so that PC's can claim that you need to vote for their candidate - in order to be on the government side. This is a bastardization of democracy and we deserve better. In order to be sure we get better - we must be responsible for our decisions and ask questions that are valid and ensure that information we are getting is not tainted.
 
I will have further info on what happens when online polling is used and what happens to the scientific validity - in a future post. Stay tuned.

The Polls - MQO let me see that again.

MQO poll - MQO a subsidiary of M5 - completed a poll. Who commissioned the poll?
Breakdown of people polled by region?
M5 Client List
MMSB
Rio Tinto
Newfoundland and Labrador Hydro
Ocean Choice International OCI
Pennecon Energy
Government of Canada
and so on.

The story in rural Newfoundland and Labrador does not match the polls.