Posted on 01/14/2003 4:47:09 AM PST by visagoth
In November 2000, a "perfect storm" of vote-counting miscues and polling problems led the major TV networks repeatedly to change their minds as to whether Al Gore or George Bush was the next president. In November 2002, a second storm whipped through the networks' election broadcasts.
Unfinished and mismanaged efforts to update the computer systems used by Voter News Service forced executives at the consortium's owners—ABC, CBS, CNN, Fox News, NBC and the Associated Press—to abandon the use of exit polling data before it even got all collected. Indeed, by mid-January the failures led to the disbanding of VNS itself. On Jan. 13, the six organizations said only that they were "collectively reviewing a number of strong options'' to avoid another fiasco in the future.
Back up to Election Day, Nov. 5. The balance of power in Congress was up for grabs. Yet by 10 a.m., the TV networks confirmed what they had feared for months: They couldn't derive any meaningful exit-polling data from a system they had just spent between $10 million and $15 million to overhaul.
Disasters were almost comical. Many of the more than 30,000 temporary workers collecting exit-poll information were disconnected from VNS' new voice-recognition system before they could finish inputting data over the phone. Some poll workers were unable to access the system at all. Live operators weren't always a help, as the phone system periodically crashed under the crush of callers dialing in.
Using computers was not much of an alternative. News organizations and other VNS subscribers were repeatedly instructed to log off their machines, so the new servers running BEA Systems' WebLogic application server could be rebooted.
When users finally were able to access the system, they quickly discovered they were being presented with incomplete and inaccurate information. For instance, early exit-polling data indicated that Erskine Bowles was leading Elizabeth Dole in the North Carolina senatorial race. As the day progressed and more exit-poll data was added, that margin grew.
However, when the actual votes were tallied, Dole won the election by almost 200,000 votes, a convincing victory.
"Everyone could smell this coming months in advance," says Joseph Lenski, co-founder of Edison Media Research, a Somerville, N.J., firm that provided supplemental polling data for CNN. "VNS had been trying to rewrite and retool the system for years. This was just the most recent attempt and it failed miserably."
Among the causes of the second "perfect storm," culled from participants, were:
Delays in delivering and testing the new voice recognition software used to capture polling data from around the country
Complications in consolidating IBM DB2 and Oracle databases, which housed election and demographic data from every state and precinct for 30 years, and transferring the information to a new Oracle 8i database.
Insufficient testing of the new Java-based WebLogic application server that replaced mainframe computers running IBM's Operating System 390.
"It was a joke," one political analyst at a major television network told Baseline. "It became obvious to everyone that this wasn't going to work. There wasn't enough testing. There was not enough collaboration between the networks and the IT people. And, worse, there was nothing we could do about it. You can't postpone an election."
Network executives quickly concluded they would not use the bulk of the data they were able to collect, particularly the exit-polling information. Projecting winners and losers in various races would take several hours longer than in the past.
Also, the networks would be unable to give the type of detailed explanations as to why voters voted the way they did this time around. For example, according to TV network analysts working the election, the networks wouldn't be able to tell viewers why particular demographic groups voted for specific candidates nor the issues that they considered most or least important when voting.
Game Over for VNS
This second debacle meant the end of VNS, as the news organizations said they would look at new ways of tabulating national and state results. Insiders close to VNS say the media organizations will likely rely more on their own individual exit polling and that of the Associated Press exit polling data in future elections. Battelle Memorial Institute, the Columbus, Ohio-based technology firm charged with overhauling the VNS system, was terminated.
"There's no way the networks are going to do anything that's connected to Battelle going forward," says one network analyst who spoke on condition of anonymity. "That ship has sailed."
Battelle Memorial Institute representatives declined to comment. Even before it got the boot, the organization appeared to distance itself from VNS. Its corporate Web site made no mention of its work or association with VNS.
The networks now face the prospect of returning to the days of conducting nationwide polls separately—at cost of roughly $10 to $12 million per four-year election cycle—rather than shelling out between $5 million $7 million per cycle by sharing operations. .
The idea behind forming VNS was to pool funds. That way, news organizations could pare costs and still get a much larger and, theoretically, more accurate sample from voters on Election Day. All members, including 19 newspapers, shared in the management of the company and oversaw its $33 million operating budget for the current four-year election cycle.
Warren Mitofsky, a media consultant who developed an exit-polling analysis system for CBS in 1967 and worked for VNS in the early 90s, says the project was doomed from the beginning.
"You simply can't have six different competing news agencies jointly managing a technology project of this scope," he says. "That's why I left VNS. Everyone is trying to decide what should be done and how. If you don't have a final decision maker who takes the responsibility for a project like this, you end up with what we saw in November."
And it is that structure that proved stormy, not just the technology.
"The truth is that had there been better collaboration between the people who collected and passed along the data to those who were analyzing the data and publicly projecting the results, this all could have been avoided," says Michael Traugott, a University of Michigan professor who worked at VNS for more than 10 years. After the 2000 fiasco, though, "VNS decided it was time to make a real effort to fix the system.''
The first step was to change the VNS board of directors. Before the 2000 meltdown, the board was composed of representatives from the election units of each network. After the 2000 fiasco, a vice president from each network was on the board.
That new board took bids from computing companies to completely rewrite the VNS system. One stipulation: That the new system use more flexible and current programming languages—Java and the Extensible Markup Language— rather than OS 390 to gather, compute and deliver data to the media outlets.
The idea: Data could then easily be provided instantaneously to subscribers over the Internet.
Database Debacles
As part of the effort, voting databases developed over a span of more than 30 years were to be consolidated into one. That's where the rubber never met the road.
"There were a lot of IT issues here that I'm not sure Battelle was qualified to handle," Lenski says. "They routinely missed deadlines for delivering the different components for testing."
The databases which housed the election results and local demographics for more than 4,600 precincts were running on both IBM's DB2 and a version of Oracle 7. They were to be consolidated into Oracle 8i database software.
"This caused all kinds of problems," one source close to VNS says. "You're not only talking about a clash in culture and expertise but you're also talking about trying to create places for data to fit that just aren't there."
For example, participants say the new system wasn't able to compare previous election results. If a network analyst wanted to know how independent voters in a particularly county were voting compared to the 1996 or 2000 election, the system couldn't deliver the data quickly, if at all.
"The fields just didn't match up," one network analyst says.
Also, traditionally Democratic precincts scattered throughout the country were, according to the flawed exit polling figures, showing unprecedented strength for Republican candidates. And vice versa.
An estimation model, which was developed by longtime VNS staffers, was implemented by Battelle in time for the election. But it wasn't sufficiently tested.
"We had absolutely no confidence in the numbers we were getting from the estimation model," says one CNN analyst.
CNN, sensing doom, partnered with Edison Media Research as an alternative to the VNS data. Its "RealVote" system gathered exit-polling data from 10 states to beef up CNN's analysis as the raw vote totals trickled in. ABC, NBC and Fox News all either sent reporters out to some of the precincts or conducted telephone polls throughout the day to bolster their broadcasts.
The dissolution of VNS leaves the networks with less than 51 weeks to develop an alternative before the Iowa caucuses—the start of the 2004 presidential campaign.
Launching on a Deadline
Test early: Stress-test your system at least six months before launch
Test heavy: Put it through at least 10 times as much activity as you really expect
Trick yourself: Establish a deadline at least two months in advance of the "real" deadline and make all project managers and vendors comply
Name one chief: Regardless how many partners, consultants and vendors are involved, give one person ultimate decision-making power
Don't reinvent the wheel: Make good use of existing personnel and technology, where possible
Disclaimer: Opinions posted on Free Republic are those of the individual posters and do not necessarily represent the opinion of Free Republic or its management. All materials posted herein are protected by copyright law and the exemption for fair use of copyrighted works.