The plus side of disproportionately attracting young, highly educated males from urban areas

​Critics argue that one of the main drawbacks of VAA data is the lack of representativeness. Indeed, VAA sites disproportionately attract young, highly educated males from urban areas who have high levels of interest in politics. Consequently, the data generated through these sites mostly reflects the views and attitudes of specific sectors of the population. This, technically speaking, constitutes a so called self-selection bias. However the self-selection bias can in some settings bring about certain advantages does make the VAA data unique and valuable.
When comparing VAA data with traditional survey data, one would expect the latter to be undeniably superior, but that is too simplistic and not exactly true. For the sake of comparison, we take a sample of UK VAA users that consulted the tool for the 2009 elections for the European Parliament and participated in the follow-up survey, and the UK sample from the 2009 European Election Studies (EES). The VAA sample (N=1104) suffers from a double self-selection bias: besides the “usual” self-selection bias affecting VAAs, only a fraction of the participants filled up the additional follow-up questionnaire  containing some of the relevant political questions. The EES sample (N=1000) has national coverage, targeted the population aged 18 and above, and used RDD for selecting the households and the most recent birthday in the household for the selection of respondents. Up until this point, the EES sample seems the better one, as expected.

​A more careful look at the differences between the two samples challenges that ranking. The EES sample has a response rate of only 18%, but to correct for sampling disparities, the data was weighted on age, sex and region. The chart presented above illustrates the main differences between the VAA and the EES sample (which serves here as an example of traditional survey) in terms of respondents’ overrepresentation.
While the younger age categories are overrepresented in the VAA sample, they are underrepresented in the EES one. While the highly educated make up the most of the VAA sample, the traditional one over represents those with low levels of educational attainment. While males are overrepresented in the VAA sample, in the EES one are the females. Further on, when it comes to party affiliation, one can notice that while mainstream party supporters are overrepresented in the EES sample, the VAA one offers the opposite picture, capturing opinions of the smaller parties’ supporters. Finally, while most of the EES respondents present low and very low levels of political interest, the VAA users are very interested in political matters.
One can argue that the VAA sample lacks important information on specific demographic categories and other variables of interest. But a trained eye can see that the VAA sample brings valuable information on the political preferences of those demographic categories that are usually underrepresented by the traditional surveys, such as age, gender and educational attainment. Furthermore, it offers information on the niche party supporters that are harder to reach through traditional means.
 
Resources:
http://eeshomepage.net/ees-2009-study/voter-study/
http://dvn.eudo.eu/dvn/dv/euprofiler