Same Ending ... but Four Stories

It was "A Good Day for the Pollsters," according to a Pew Research Center commentary issued last Thursday, two days after the New Hampshire primary.

"Despite the jokes about the glut of pre-election tracking polls in New Hampshire, these surveys were remarkably accurate in predicting the outcome of the race … Added together, the surveys predicted the level of support for every candidate within a single percentage point."

The closeness of the polls to the actual results presumably should mean that the polls all accurately measured public opinion. Because four of the five polls provided tracking data from just before the Iowa caucuses until the New Hampshire election, we might expect their portrayal of the campaign dynamics -- the changes over the tracking period -- to be roughly comparable.

The key to the election was the contest between former Vermont Gov. Howard Dean and Massachusetts Sen. John Kerry. Early polls in New Hampshire (prior to the Iowa caucuses) showed Dean comfortably ahead of Kerry. After Kerry's surprising victory in Iowa, the contest in New Hampshire underwent a profound transformation, with Kerry emerging the clear victor. How closely did the four tracking polls agree on how this change occurred? Not very. Instead of one story as to what happened, we get four.

All results are based on three-day tracking polls, except Gallup’s last two results, which were based on two-day tracking.

Four Stories

Although all four polls arrived at the same ending -- that Kerry would win by a low double-digit margin -- they presented quite different stories as to how that result was achieved. 

Gallup: After the Iowa caucuses, Kerry jumped to a low double-digit lead (12 points), and maintained that lead over the next several days. There were minor variations, but no significant changes.

UNH: After the Iowa caucuses, Kerry jumped to a high double-digit lead (18 points), and over the next several days gradually lost a little support.

ARG: After the Iowa caucuses, Kerry jumped to a low double-digit lead (13 points), and continued to gain strength, reaching a peak margin of 22 points. But Dean recovered over the next two days, with an especially strong showing on Monday, cutting Kerry's highest lead by half at the election.

Zogby: After the Iowa caucuses, Kerry jumped into a single-digit lead (eight points), maintaining that lead (within one point) for the next two days. But a Dean surge almost wiped out Kerry's lead (down to three points), until Kerry recovered big time on Monday.

Dueling Polls

Some of the differences in what the polls found were both substantively and statistically significant.

The first results based entirely on post-Iowa interviewing, Jan. 20-22, showed the full impact of the Iowa caucuses. But the polls gave different snapshots of what was happening:

• The UNH Survey Center showed Kerry with an 18-point lead, while Zogby found just an 8-point lead. 
• The American Research Group (ARG) and Gallup fell in between, with Kerry ahead of Dean by 13 points and 12 points, respectively.

Two days later, based on Jan. 22-24 polling, Zogby and ARG showed widely divergent results. Zogby had Kerry with only a 7-point lead, while ARG found Kerry leading by 22 points. 

The Jan. 24-25 polling continued to find Zogby and ARG disagreeing about Kerry’s lead by 15 percentage points. Zogby’s results showed Kerry leading Dean by just three points, while ARG said Kerry still enjoyed a comfortable 18-point lead. Had the two polling organizations stopped polling at that time, as did UNH and Gallup, the Pew accolades about the accuracy of the polls would not have been appropriate.

Only ARG and Zogby polled on Monday and, again, their results were wildly at odds with each other: ARG reported an 8-point erosion in Kerry's lead, while Zogby found a 10-point surge.

But the final results of each polling organization ended up close to the results found by UNH and Gallup the previous day, prompting Pew to issue its congratulations to all the polling groups.

Bottom Line

What are we to make of these divergent stories? There is no way to verify which is correct, and each polling organization can claim its story is valid, because the final polling results were close to the actual election results. But I know which one I believe....