Blog



River Sampling

People who are willing to be members of panels differ in many ways from people who aren’t. This too lessens the representativeness of panel research. 

For instance, in a recent CASRO webinar, NPD Group reported that 70% of their panel members are introverted, compared to approximately 50% of the U.S. population. Panel members most likely differ in other key ways regarding their personalities; Inna Burdein, direct of panel analytics for NPD, reported that diligent panelists are “high on introversion, have a high need for cognition, enjoy thinking, and prefer complex to simple problems, and they like surveys – they find surveys worthwhile.” 

A particular type of individual keeps completing surveys – someone who actually enjoys the introspection of the process. And the experience of staying in the panel changes their answers over time. On the positive side, the practice effect sees them improve the accuracy of their responses and they become more self-aware; on the negative side, panel conditioning exposes them to more studies, making them less like the general population being researched. Given these findings, and given that panel tenure can fluctuate as new panel sources are added, many organizations weight their data by panel tenure to keep findings consistent over time. Weighting in this case not to improve representativeness but to improve replicability. 

To offset practice effects and panel conditioning, river sampling is meant to provide fresh meat – fresh eyes – to surveys. These respondents are intercepted by banner ads and popups as they surf the web and are recruited to take a survey. Such respondents take many fewer surveys than panelists, with a high proportion reporting this is the only survey they’ve taken. Such people are hard to build a panel on (they don’t come back for subsequent surveys). They are often integrated into surveys through routers that try to balance the proportion of new respondents with established panelists.

This is an excerpt from the free Researchscape white paper, "Improving the Representativeness of Online Surveys”. Download your own copy now.

 

Recent Headlines

  1. Postgraduate Degree Survey Jeffrey Henning 10-Jul-2019
  2. Voice Assistant Survey: Let the Young Speak Jeffrey Henning 09-Jul-2019
  3. PDI Releases New C-Store Shopper Report Jeffrey Henning 26-Jun-2019
  4. Issues with Grid Questions Jeffrey Henning 24-Jun-2019


White Papers


Consumer Research

  1. Postgraduate Degree Survey Jeffrey Henning 10-Jul-2019
  2. Voice Assistant Survey: Let the Young Speak Jeffrey Henning 09-Jul-2019
  3. Grocery Shopping Survey Jeffrey Henning 22-Jun-2019
  4. 2019 Summer Vacation Survey Jeffrey Henning 11-Jun-2019


From Jeffrey Henning's Blog

  1. Issues with Grid Questions Jeffrey Henning 24-Jun-2019
  2. Asking Too Much Jeffrey Henning 07-Jun-2019
  3. Helping You Through the Entire Survey Research Process Jeffrey Henning 02-Mar-2019
  4. Market Research Conference Calendar - 2019 Jeffrey Henning 12-Dec-2018