background image

Tips for Reporting on Poll Data

Carret icon

Overview

A brief guide to poll reporting

The importance of covering poll data with clarity and accuracy


Polls and public opinion research are integral to a thriving democracy and a robust free press. They are the most accurate way to measure public opinion and give a voice to the people. Polls are important because they inform public officials of their constituents’ opinions and also provide useful sources of information for journalists who strive for accurate and unbiased reporting.

With a proliferation of polls flooding the information marketplace, Quinnipiac University’s School of Communications and the Quinnipiac University Poll recognize the value of assisting journalists in their coverage and analysis of poll data. Journalists have a professional mandate to examine a poll’s methodology, origin and results; to expose bias; and to report results with accuracy and clarity for the public good.

Understanding How Poll Data is Gathered

Understanding how poll data is gathered

Before journalists can report the statistical findings of a poll, it is essential they understand the methodology involved in the development of the poll and the data collection. The first thing to determine is whether the poll is transparent about its methods. There are many factors that go into creating a poll, and each one can have a big impact on the poll’s results. Try to answer each of the questions below before writing about a poll.

Who


Consider who are the respondents of the poll and how many there are. A poll of adults is different than a poll of registered voters and should not be compared. The number of respondents mainly determines the margin of error for the poll.

Also, who conducted the poll? Is it an experienced, reputable polling organization? Polling organizations with a reputation for accuracy are more likely to produce reliable results, although it is not always a sure thing.

What


Consider what questions were asked, whether they were worded simply and clearly, and whether they were asked without bias. Even subtle differences in the order and wording of questions can change the results. For example, the Quinnipiac University Poll found that respondents who were asked whether they supported or opposed “stricter gun control laws” opposed them, 51–46 percent. A different group of respondents in the same poll who were asked whether they supported or opposed “stricter gun laws” supported them, 52–45 percent. Not including the word “control” showed a notable difference in the results of the question, which is why specific and thoughtful wording of questions is so important.

When


Consider when the interviews were conducted in relation to the timeliness of a current event or topic. Was the poll conducted before or after a major event that could affect the respondents’ opinions? For example, if a presidential election poll was mainly conducted before a presidential debate but released after it, many voters who watched the debate could have changed their minds.

Why


Consider why the poll was conducted. Does the organization who paid for it and/or conducted it stand to benefit from the results of the poll? The reason why the poll was conducted could bias the methodology or even the results themselves. For example, an organization may withhold data that reflects negatively on its goal and only release data that supports it.

How


Consider how the polling organization obtained the sample and reached the respondents. A random sampling method gives all people in a target population an equal chance of being surveyed. This is how to represent a specific population such as American adults or registered voters. Telephone surveys conducted by random digit-dialing are considered the best practice to obtain a random sample. Telephone surveys conducted with voter registration lists can be a good sampling method for the registered voter population. When samples are not chosen randomly, the results are not as accurately generalized for an entire population. Online surveys using internet-based groups of people are common for non-random sampling.

When it comes to telephone surveys, there is another factor that can have a major influence on the randomness of the sample, and therefore, the accuracy of the results. Telephone polls can be conducted by live interviewers or by robocallers. There are a few ways this can affect the results. The most important one is that live interviewers are allowed to call people on cell phones, while robocallers are not. Since more than half of the U.S. population is only reachable by cell phone, this has a significant effect on who can be surveyed by the polling organization.

It’s also important to determine how the surveys are weighted. Weighting adjusts poll data to correct any relevant ways the sample results might overrepresent or underrepresent certain groups in the population. Polls in the United States are commonly weighted according to Census population and demographic data, including but not limited to gender, age, race, region and education.

Again, be very skeptical if the above information is not made easily accessible.

Tips for Writing About Poll Data

Tips for writing about poll data

No reflection of perfection


Reporting a poll’s survey results must reflect that the figures are estimates, not exact numbers. Do not use decimal points when writing about polling results. Using a decimal place implies a level of precision that isn’t achievable in public opinion research. The margin of error also should be included in a story about a poll’s results.

Watch your language


Good writers avoid repetitive or dry language to tell an engaging story. However, being “writerly” often can lead to problems with the accuracy and clarity of reporting about poll results. When referencing questions asked in a polling survey, stick to the original phrasing. Otherwise, paraphrase the question(s) very carefully. 

Don’t trip over the election poll


Election polls are unique. When writing about them, remember they are not meant to predict the outcome of an election. They simply illustrate public opinion at a certain point in time and give you a snapshot of what voters are thinking at that moment. As a rule of thumb, a candidate typically needs to be ahead by 1.5 to 2 times the margin of error for a lead to be significant.

Sometimes, the ‘don’t knows’ are the story


If there is a high percentage of survey respondents who are undecided or have no opinion, that is important and should be reported. These responses can be just as meaningful or even more meaningful than firmer answers — especially with political surveys — because they can indicate how many people haven’t made up their minds or are unfamiliar with an issue or a candidate.