{"id":118,"date":"2020-04-29T18:48:31","date_gmt":"2020-04-29T18:48:31","guid":{"rendered":"http:\/\/blogs.messiah.edu\/politicsinternationalrelations\/?p=118"},"modified":"2020-06-08T21:31:06","modified_gmt":"2020-06-08T21:31:06","slug":"citizen-hack-deciphering-polling-and-survey-results","status":"publish","type":"post","link":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/2020\/04\/29\/citizen-hack-deciphering-polling-and-survey-results\/","title":{"rendered":"Citizen Hack \u2013 Deciphering Polling – and Survey – Results (Civic Mind)"},"content":{"rendered":"

\"magnifying<\/a><\/p>\n

 <\/p>\n

\u201cDewey beats Truman!\u201d<\/p>\n

This headline, from the 1948 election, has become a historical lesson on the potential dangers of polls.\u00a0 In this case, due to the lack of a representative sample and the ability to determine whose vote was decided and would be cast, the results did not accurately reflect the intentions of the larger voting public<\/a>.\u00a0 Flash forward to the 2016 presidential election, when some polls leading up to Election Day did not quite square with the results of who won in some state<\/a>s.\u00a0 Just as with any other means of gathering data to further knowledge, knowing what to look for when reading polling results can help you determine how much stock to put into the findings.\u00a0 But what is a poll, and in what ways does it differ from a survey?<\/p>\n

Both formats of questions allow us to gather information from individuals, known as respondents, in order to analyze patterns in the public.\u00a0 Surveys provide a means for professionals to ask people directly \u2013 in a variety of formats \u2013 about their opinions, behaviors, experiences and personal characteristics, whereas polls focus on one or a few narrow questions. (Think about a field survey or survey course, which can offer a view of the larger landscape.) Surveys typically take much longer to craft and administer; polls can be run quickly and frequently.\u00a0 Each, then, serves a different purpose even though their methods may be very similar.\u00a0 However, in order to benefit from the availability of any results, we need to be aware of several key factors that shape whether we should put credence in them, as well as the inherent limits of this type of data.<\/p>\n

Although surveying as a means of gathering data other than opinions can be used more extensively to capture all members of a group \u2013\u00a0 like the US Census \u2013\u00a0 our focus here is on the use of smaller subsets of individual, known as samples, in order to draw conclusions about a larger group of people, known as a population. Except for rarer \u2013 and expensive \u2013 types of data collection from the public, such as the census, almost all poll data comes from subsets of the population. The form and size of the sample help us know how reliable the findings may be.<\/p>\n

Both surveys and polls come in many forms, only some of which provide valid information that you can generalize<\/em> to the larger environment.\u00a0 In general, Scientific polls<\/strong> provide the most useful results<\/a><\/em>, because the people responding are picked by the use of a random sample<\/a>, <\/strong>in which each person has an equal likelihood of being chosen.\u00a0 Such an approach is most likely to result in a sample that is representative of the larger whole. Therefore, to have the most reliable information possible, you want to see that the poll was conducted with this method.<\/p>\n

Non-scientific polls<\/em><\/strong>, which rely on convenience or other means to capture responses (i.e. clicking from a website), may have some useful information for limited purposes, but cannot be evaluated as representative of a larger whole.\u00a0 One of the least reliable types of non-scientific polls is a push poll, in which people contact respondents for other purposes and use the opportunity to press (push) ideas upon them using leading language.\u00a0 (Individual questions in a poll or larger survey can also be misleading, but push polls are designed to sway opinion as a whole, rather than measure it. For that reason, research often considers them faux polls.)\u00a0 However, the means of selecting the people to respond to the questions is just one element to consider; the number of individuals responding also matters to the reliability of results.<\/p>\n

Sample size<\/strong> also helps us understand how seriously to take results.\u00a0 In essence, the larger the sample, the more reliable the results.\u00a0 In fact, it generally takes about 1,500 responses in a random sample to get helpful results.\u00a0 More is better, of course, but gathering more takes more time and money.\u00a0 Thus, researchers and other pollsters are always trying to balance accuracy with cost. However, they provide information that allows you to assess how accurate the results may be based on the number of respondents.<\/p>\n

Poll results for specific questions should include a margin of error<\/a><\/em> that indicates the range of confidence in the results.\u00a0 The smaller the margin of error, the more reliable the results.\u00a0 The closer the results \u2013 for example between two candidates \u2013 the greater chance that an outcome, such as an election win, could be unpredictable.\u00a0 If you have a 2-3% point difference between candidates and a 3% margin of error, then the leading candidate could be ahead further or could actually be behind the other candidate; that shifting is why some races are considered \u201ctoo close to call\u201d.\u00a0 The\u00a0same could be said for differences of opinion on issues or events, all of which presume that we have well-formed questions.<\/p>\n

Even with a reliable sample form and size, survey\/ polling results can still be impacted by the wording of the questions<\/strong> asked.\u00a0 Individuals who use polls for research purposes spend a good deal of time refining the wording of a question, both to ensure that everyday respondents can understand it and to ensure that the question itself does not push a respondent towards a particular response (a leading question). \u00a0A brief discussion of the factors that researchers take into consideration when drafting questions may be found here<\/a>. In addition, other question-wording issues should be noted as well.\u00a0 Some research on surveys has even looked at the order of options presented in question and responses, finding that it can impact on the ways that people answer questions.<\/p>\n

Ultimate, as with any data collection for any topic, we are best served by looking at the collective results of multiple polls or studies that use reliable methods to gather and process the data.\u00a0 For this reason, we often see some sources report moving averages of results across multiple polls collecting the same data over the same period of time.\u00a0 In addition, looking at results of larger-scale research projects, not simply those focused on electoral horserace or other sensational items, will yield more helpful information.\u00a0 For example, the American National Election Study<\/a> and the General Social Survey<\/a> are have been collected on a regular basis for decades \u2013 though the researchers gathering these data sets do not focus on prediction measures like news outlets do, their findings do allow users to make deeper meaning of results and their likely causes.<\/p>\n

We can become more critical evaluators of data, using more than our own existing opinions to evaluate it by becoming better informed about these elements.\u00a0 Herbert Asher\u2019s Polling and the Public<\/em>: What Every Citizen Should Know<\/em><\/a> provides a concise overview of the key elements discussed in this post and more.\u00a0 In addition, the American Association for Public Opinion Researchers (AAPOR) provides ethical guidelines, including disclosure expectations<\/a>, and best practices for professionals who conduct polling; published results should always include these basic pieces of information.\u00a0 In addition, this recent episode of WITF\u2019s Smart Talk<\/em> (an NPR affiliate)<\/a> offers some great insights on polling from Berwood Yost, currently the Director of the Center for Opinion Research at Franklin and Marshall College, and a former colleague of mine when I taught at Millersville.<\/p>\n

We also need to keep in mind that there are limits to survey data and the conclusions that we can draw from it.\u00a0 Unless data is captured over time \u2013 usually referred to as a panel \u2013 the results are simply a snapshot, even though we need to consider the context in which these responses are gathered.\u00a0 \u00a0In addition, unless data is gathered on other opinions, behaviors and respondent characteristics, we cannot draw larger conclusions as to the cause and effect of a specific opinion.<\/p>\n

Polling and survey data, like other methods of gathering information, are not perfect.\u00a0 However, they do provide a much more reliable source of evidence than our own impressions based on people around us, who might not represent the larger whole, or on assumptions that our opinions reflect those of a larger portion of the public (known as false consensus<\/a>). Ultimately, we should push ourselves to ask for data beyond electoral horseraces and simplistic issue stances.\u00a0 By harnessing effective data drawn from reputable sources, we can not only better understand the public mood, but we can better craft solutions to societal issues.<\/p>\n

 <\/p>\n

 <\/p>\n","protected":false},"excerpt":{"rendered":"

  \u201cDewey beats Truman!\u201d This headline, from the 1948 election, has become a historical lesson on the potential dangers of polls.\u00a0 In this case, due to the lack of a representative sample and the ability to determine whose vote was decided and would be cast, the results did not accurately reflect the intentions of the […]<\/p>\n","protected":false},"author":58,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[11462],"tags":[],"class_list":["post-118","post","type-post","status-publish","format-standard","hentry","category-civic-mind"],"_links":{"self":[{"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/posts\/118","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/users\/58"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/comments?post=118"}],"version-history":[{"count":8,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/posts\/118\/revisions"}],"predecessor-version":[{"id":154,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/posts\/118\/revisions\/154"}],"wp:attachment":[{"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/media?parent=118"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/categories?post=118"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.messiah.edu\/politicsinternationalrelations\/wp-json\/wp\/v2\/tags?post=118"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}