• Moderators, please send me a PM if you are unable to access mod permissions. Thanks, Habsy.

OT: American Politics

The elemental basis of the validity of survey data collection is probability sampling. Opt-ins do not use this and so are garbage.

But there are many variables involved in conducting proper probability sampling. One of them is coverage -- so, selecting just landlines is very poor because it excludes the vast majority of the population.
 
The biggest bias in virtually all polls is undersampling younger adults. They are much less likely to respond to surveys, regardless of the mode used. Conversely, older adults, and especially seniors, are much more likely than younger age subgroups to respond.

This is why Nate and so many polls under reported Bernie's support in 2016. It looks to me like they've tried to address this bias this time around by including large subsamples of cell phones. I expect the polling in 2020 to be at least somewhat better but there's other issues going on as well.
 

Sampling Methods for Political Polling - AAPOR

www.aapor.org www.aapor.org

For more than five decades probability sampling was the standard method for polls. But in recent years, as fewer people respond to polls and the costs of polls have gone up, researchers have turned to non-probability based sampling methods. For example, they may collect data on-line from volunteers who have joined an Internet panel. In a number of instances, these non-probability samples have produced results that were comparable or, in some cases, more accurate in predicting election outcomes than probability-based surveys.
Click to expand...
 
zeke said:

Sampling Methods for Political Polling - AAPOR

www.aapor.org www.aapor.org
Click to expand...
And do they say that makes them valid? No.

Read on ...

In self-selected or opt-in samples, respondents have selected themselves, and this means their answers may not be representative of the larger population. ... These polls may sometimes be accurate, but it is very hard to evaluate whether they are accurate simply because of good luck or because they were able to capture good information about the population they were trying to represent.
Click to expand...

From the AAPOR task force:

“AAPOR has long maintained reporting margin of sampling error with opt-in or self-identified samples is misleading”.
Click to expand...

“The dramatic rise in the use of opt-in panels has been premised on a willingness to accept overwhelming coverage and selection error.”
Click to expand...
 
Here's Pew on opt-ins:

Even the most effective adjustment procedures were unable to remove most of the bias. The study tested a variety of elaborate weighting adjustments to online opt-in surveys with sample sizes as large as 8,000 interviews. Across all of these scenarios, none of the evaluated procedures reduced the average estimated bias across 24 benchmarks below 6 percentage points – down from 8.4 points unweighted. This means that even the most effective adjustment strategy was only able to remove about 30% of the original bias
Click to expand...

www.pewresearch.org

For Weighting Online Opt-In Samples, What Matters Most?

A growing share of polling is conducted with online opt-in, or nonprobability, samples. This trend has raised some concern within the industry because, while low participation rates pose a challenge for all surveys, the online opt-in variety face additional hurdles.
www.pewresearch.org www.pewresearch.org
 
BeLeafer said:
Never.

Unless you think including landlines in sampling creates bias. I've certainly posted many polls that include both landline and cellphone.
Click to expand...

Yes I do think that - and it's the main driver as to why probability polling has struggled lately - there's just no easy way to correct for landline bias at the moment.
 
zeke said:
Yes I do think that - and it's the main driver as to why probability polling has struggled lately - there's just no easy way to correct for landline bias at the moment.
Click to expand...
What bias is that, exactly?

A significant portion of the population still uses landline. The vast majority use cell phones now. Excluding landline creates significant coverage bias.

The ideal is to give everyone in the population you are targeting an equal probability of landing in your sample. If you do not include landlines, you create serious bias.
 
BeLeafer said:
What bias is that, exactly?

A significant portion of the population still uses landline. The vast majority use cell phones now. Excluding landline creates significant coverage bias.

The ideal is to give everyone in the population you are targeting and equal probability of landing in your sample. If you do not include landlines, you create serious bias.
Click to expand...

I didnt say don't include them.

But we simply do not understand mobile phone user polling response anywhere near as well as landline users. And now even our understanding of landline users has changed considerably now that so many of certain demographics have stopped using them.
 
zeke said:
I didnt say don't include them.
Click to expand...
Good but it sounded like you think it's inherently biased. The mode has very little bias beyond the coverage issue.

But we simply do not understand mobile phone user polling response anywhere near as well as landline users. And now even our understanding of landline users has changed considerably now that so many of certain demographics have stopped using them.
Click to expand...

The latter point is simply a coverage issue. The first point, I don't get what you're trying to say.

You can absolutely calculate the sampling error with both landline and cellphone, provided you use probability sampling.

The biases are largely two-fold: (a) social desirability bias due to live interviewers, and; (b) non-response bias (vastly overstated by purveyors of opt-in panels who can't even calculate response rate).
 
There's other things that create bias, depending on the way the phone sampling is conducted. For instance, persistence of effort. One and done samples skew to people who are more likely to pick up the phone (e.g., not very busy). In traditional media measurement, it was not uncommon to use up to 10 or more call backs to mitigate this bias.
 
I’ve said for years, this won’t get him. Yes I’m jaded but Washington has given me every reason to be.
 
hockeylover said:
Those emails are pretty important but don't seem like they're being talked about a lot. Ditto for the money going from Putin buddy right to Rudy and the GOP. Wonder what else congress wasn't able to get a hold of before the hearings.
Click to expand...

Mueller had access to it all. It’s not being talked about because it’s yet another nothing burger.
 
BeLeafer said:
and; (b) non-response bias (vastly overstated by purveyors of opt-in panels who can't even calculate response rate).
Click to expand...

well, that's fine, and that's where the disagreement lies.

the thing is, though, the results are not supporting your position here much lately.
 
You must log in or register to reply here.
Back
Top