Comparing Two Proportions

“New survey shows left-wing parties gathering strength”
A typical news story but is it reliable? Is the difference of the two sample proportions statistically significant? Use the calculator to find out.

%

%

Use the Calculator

A survey shows that 35% of 980 respondents would vote for left-wing parties.
Two months later, a new survey shows that 42% of 1020 respondents would vote for left-wing parties.
Is the increase in left-wing voters from the first to the second survey statistically significant? Can we conclude that support for left-wing parties has actually increased? And if so, by how much?

  • Enter 35 into the “Proportion 1” box
  • Enter 980 into the “Sample Size 1” box
  • Leave “Population 1” empty
  • Enter 42 into the “Proportion 2” box
  • Enter 1020 into the “Sample Size 2” box
  • Leave “Population 2” empty
  • Choose a 95% confidence level

 

Interpret the Results

In the example, the confidence interval for the difference of the two proportions is 2.7 to 11.3 percentage points.
This means we can be 95% certain that from the first to the second poll left-wing support among voters has increased by 2.7 to 11.3 percentage points. Hence, the difference of the two proportions is statistically significant.
Note the range from 2.7 to 11.3 percentage points is not a flat range. It’s a bell curve which means that the mid-range values are much more likely than the two extremes (2.7 and 11.3) to be the true population difference.

Communicate the Results

How should we deal with all this statistical information? What should be communicated? It depends on our audience. But we shouldn’t underestimate their demand for fact-based and reliable stories. Statistical data rarely fits in an article, a press release, or similar. We suggest that you use a box to state

  • Sample sizes and response rates
  • Whether the samples are representative of the population or not
  • Statistical uncertainty
  • The wording of the question

You may also add information such as who paid for the surveys, who conducted the surveys, and when/how they were done.

Recommendations

Confidence level: We recommend you choose a 95% confidence level. With this level, you can be 95% certain that the true population difference of proportions lies within the estimated confidence interval. Most researchers use this level.

Population Size: Population size affects the range of the confidence interval. You can enter the population size in the calculator to calculate this. However, we recommend this only in the rare cases where the sample is a substantial part of the population (50% or more). In all other cases, you can leave the population box empty.

The Math Behind This

Formula to calculate the confidence interval for the difference of two proportions:

P1 and p2: Proportions (in our example p1 and p2 are 35 og 42)

z: Confidence level (for a 95% confidence level the Z value is 1.96)

n1 and n2: Sample sizes (in our example n1 and n2 are 980 and 1020)

The correction factor which can be multiplied to the confidence interval if you want to correct the interval with the ratio of the sample and the population.

n: Sample size

N: Population size

In case of two different sample sizes, use the average of the two ratios (n/N) in the calculation.