The Pardu

The Pardu
Watchful eyes and ears feed the brain, thus nourishing the brain cells.

Monday, November 26, 2012

Election 2012: Battle Of Polling Databases, ORCA vs. NARHWAL

The Horror of  a Loss!


It is almost as if reflecting back on past wars.  No war has been won (for that matter lost) without strategy and strategic execution that involved intelligence (AKA INTEL). And, to the winner of the INTEL wars goes the spoils.  Political campaigns are much like a form of war.  The presidential campaigns leading to November 6th elections, certainly involved war-like strategy.  Both success in INTEL execution and failure to execute was evident throughout the 2012 Campaign. The Obama Team was successful; the Romney team failed miserably. While success and failures span many campaign fronts, campaign use of technology was an area that the Obama campaign won handily. No war is won without effective INTEL. Details of behind the scenes technology wars are starting to come forth.

Election 2012 was perpetrated with assistance from technology born just ahead of the 2008 Elections.  The Obama Administration and software professionals continued development of what has come to be known as Project Narwhal (and DreamCatcher). Romney's Polling INTEL came to be known as ORCA. In the end Romney's  system failed. As stated in previous TPI pieces and restated below,  Anonymous claims to have interfered with ORCA. Ultimately, and regardless of failure the system failed miserably. 

A team of high tech software engineers trumped the Romney ORCA Team and they went about their mission with the precision of well planned and executed military strategy. Yet, I am certain not one of the team credits their success to military like acumen. 

Meet the Obama Narwhal Team.

techteam.jpg
Three of the 40 engineers. 
They'd been working 14-hour days, six or seven days a week, trying to reelect the president, and now everything had been broken at just the wrong time.
Narwhal: high-tech 
voter-targeting system, 
after the Arctic sea mammal.
Josh Thayer, the lead engineer of Narwhal, had just been informed that they'd lost another one of the services powering their software. That was bad: Narwhal was the code name for the data platform that underpinned the campaign and let it track voters and volunteers. If it broke, so would everything else. (The Atlantic)
Josh Taylor is a member of a team of elite, talented and committed software professionals (Specialist and technicians) who worked behind the scenes to re-elect a president. 

The Atlantic's  
Alexis C. Madrigal  on November 16th, published an intriguing look at the innards of President Obama's technology team. It is impossible to fathom how President Obama could have garnered as many electoral votes as he factually won without the team of "30 somethings". 

Narwhal...


(The Atlantic

When the Nerds Go Marching In
Excerpt 
The election was still 17 days away, and this was a live action role playing (LARPing!) exercise that the campaign's chief technology officer, Harper Reed, was inflicting on his team. "We worked through every possible disaster situation," Reed said. "We did three actual all-day sessions of destroying everything we had built." 
Hatch was playing the role of dungeon master, calling out devilishly complex scenarios that were designed to test each and every piece of their system as they entered the exponential traffic-growth phase of the election. Mark Trammell, an engineer who Reed hired after he left Twitter, saw a couple game days. He said they reminded him of his time in the Navy. "You ran firefighting drills over and over and over, to make sure that you not just know what you're doing," he said, "but you're calm because you know you can handle your shit." 
The team had elite and, for tech, senior talent — by which I mean that most of them were in their 30s — from Twitter, GoogleFacebookCraigslistQuora, and some of Chicago's own software companies such as Orbitz and Threadless, where Reed had been CTO. But even these people, maybe *especially* these people, knew enough about technology not to trust it. "I think the Republicans fucked up in the hubris department," Reed told me. "I know we had the best technology team I've ever worked with, but we didn't know if it would work. I was incredibly confident it would work. I was betting a lot on it. We had time. We had resources. We had done what we thought would work, and it still could have broken. Something could have happened."
(Immizen View)
Reported as early as Feb, 2012 in a Slate article titled “Obama’s White Whale”, project Narwhal, Obama’s top-secret campaign project until then, featuring high tech data integration and data mining techniques,promised to change the ways in which campaigns are fought and won.

(Slate Dot Com) Feb 15, 2012
.......as part of a project code-named Narwhal, Obama’s team is working to link once completely separate repositories of information so that every fact gathered about a voter is available to every arm of the campaign. Such information-sharing would allow the person who crafts a provocative email about contraception to send it only to women with whom canvassers have personally discussed reproductive views or whom data-mining targeters have pinpointed as likely to be friendly to Obama’s views on the issue.

 There are growing numbers of articles about the NARWHAL Vs. ORCA Systems, their missions and outcomes. I have posted a few links at article end.   I find the most informative a  piece about the two systems was written on November 12, 2012 by Tomi T. Ahoen, Orca meets Narwhal - How the Obama Ground Game Crushed Romney - A look behind the math.  
                                           

The article is very long and probably only suited for those who are enthralled with the inner workings of Presidential Elections Circa 2000 and forward.

Tomi T. Ahonen..... [Tread no farther if you abhor long reads]


Excerpt


The Romney machine called Orca vs the Obama machine called Narwhal. This is truly a classic match-up, like Godzilla vs King Kong. (be prepared, this is a long article, nearly 9,000 words, and there is a lot of math in it..)

PROJECT ORCA



Romney had built a brand new database and voting monitoring and voter activation system called Orca. It is credited with catching the voting behavior of about 14 Million Romney supporters. It also is famous for crashing many times during the day and that many Romney supporters had trouble logging into the system, some never made it all day. Project Orca had inputs coming in from voting places from 34,000 Romney supporters, who would report how each voting precinct was performing. Based on that info, Romney headquarters in Boston could monitor in real time what was the voting in every state, by precinct, and then make their estimates of what the situation was on the ground, and then direct GOTV efforts, primarily robocalls, directed by a phone banking staff of 800.



They would make 6 million calls during the day - a number never attempted before by the Republicans. The Orca system would guide which states were trending 'safely' for Romney, and where their voter numbers were down from projections. This meant, Romney could direct their resources efficiently, to make calls in those states where needed. What is more, the Orca system had identified the needed phone number area codes, so they would call those regions that were strongly Romney and Republican-leaning, so if a given county went for McCain over Obama in 2008 at a ratio of say 70 to 30 or for example 60 to 40, when the whole country went against McCain 47 to 53, such counties were good places to target, with a lop-side Republican supporter base, and make sure, all of their voters would come to the polls, because those would be strongly leaning to the Republicans.. We do also know, that Romney's team explicitly did not attempt to build its own lists of supporters (that would take months, years even) but rather, it bought commercially available lists, email lists, phone number lists, which are of less than perfect accuracy and relevance.

Understand, this all was new. McCain had nothing like this. McCain's campaign was still paper-based with 'strike lists' ie paper slips with names and numbers, you called them and then 'striked' a name off the list, when that person said they had voted. Romney's ground game was a whole generation ahead of McCain's in 2008 and by mid October, Romney's campaign boasted that it had made 50 million actual contacts with voters. Those contacts ranged from visits at home and phone calls, to emails and Facebook contacts, to robocalls and traditional mailings via the post, and depositing doorknob hangers. Out of a nation of 130 million actual voters and some 145 million registered voters, and a target level of about 60 to 65 million Republican-leaning supporters, to make 50 million contacts is very impressive (it would include duplicates, so the aggregate total contacts would be lower). The ABC/WaPo poll of registered voters in mid October found that 18% of the voter base had been contacted by Romney, which would be about 26 million individual voters. So Romney's team had had hit their targets typically twice.

Unfortunately it was indeed, all new. The team was not trained to use this complex system. It required users in the field to access the system from smartphones and needed user IDs and passwords. The volunteers had to read a 60 page manual, which was only sent to the volunteers the night before the election. In the morning, many of the users of the system weren't able to access it, and there was no help desk. The actual decision-making desk at Project Orca in Boston was unable to rely on data from the field as expected, and often got the best info from TV news like CNN and Fox, rather than their own field machine. This part of Orca was under performing hideously, compared to what it could have been, if it had been a well-tested, well-run and well trained system.

There is an interesting parallel to 2008. In 2008, Obama's team had intended to collect voting data per voter, in key districts, and send it in via mobile phones to their central database, through an automated telephone switchboard just by keying in numbers and using IVR (Interactive Voice Response) just like we all do when we call a calling center and they ask us to punch in numbers, press 1 to find out about your phone bill, press 2 to hear our new offer, etc.. The system was called Houdini, and it too crashed on election day due to completely overwhelming telecoms traffic that they had not anticipated. But at least Axelrod and gang had been smart enough to pre-test it and to build a work-around in case it failed on election day. They went back to paper and pencil, and runners, and did get useful data while not anywhere near as efficiently as hoped, in 2008. The difference in Orca in 2012, is that Romney's team had no alternative and when Orca crashed, it was unable to help in any way, and there was no workaround.


Also, just as a note. The Obama system was known for months, that it is internally called Project Narwhal. Narwhal or Narwhale, is an arctic whale. And the only known predator in nature (other than man) that hunts the Narwhal is - the killer whale, Orca (ORCA. So when Romney's team wanted to best Narwhal, and beat team Obama, they named their data-mining project Orca. Yeah, that's intimidation right there. Both sides were making occasional revelations about their systems to psyche out the other side.


PROJECT NARWHAL


Where the Romney campaign was running fast to try to get into the 21st century with their GOTV plans, Obama's team, led by Axelrod, Plouffe and Messina had already done all that. Their 2008 campaign had been brought to fully digital communications, and was very cutting edge from the SMS announcement of Obama's choice of Vice President, to using Facebook. Its not just that the Obama team was fully digital and using social media back in 2008, they had now had four years of time to plan and build and revise and expand their system. Boy did they ever.


We do not have direct apples-to-apples comparison numbers on every metric, but consider the few where we do. Romney's team was proud of achieving 50 million contacts by mid October with their base (which included non-personal contacts such as mailings, doorknob hangers and robocalls). Obama's team announced they had hit 125 million personal contacts. Personal contacts! Visits at home or phone calls. Wow. Based on that same ABC/WaPo survey of total contacts made, Obama's team had hit about 28 million registered voters. 125 million total contacts meant, that Obama's team had talked to - talked to - their voters/supporters on average 5 times in the election cycle so far. You really get to know your voters this way! And Obama got just over 60 million votes in the end. By mid October, the Obama team in Chicago had personally spoken to almost half of those who would eventually vote for them - and on average spoken with each voter 5 times. Not 'sent a letter' but actually spoken, face-to-face, or via phone, on average 5 times!


Romney's team consistently every month spent most of its massive millions in campaign funds on TV ads. Obama's team not one month had its biggest spend in TV advertising. Its biggest spend every month was the mysterious Project Narwhal. They spent millions on this project months before voting day? Why? Because it was a grand plan and it was truly nefarious. First, Team Chicago had decided that the info coming from regular pollsters was not accurate enough. We saw how many daily polls every day? And yet the Obama team felt that was not accurate enough. So they constructed the biggest polling panel ever made, of 29,000 voters in the state of Ohio alone (nearly 1% of the total voting population in Ohio). I am pretty sure they didn't get this level in all states, but probably did, in the targeted 9 battleground states. If we assume the same ratio held for all 9 battleground states, then Narwhal would have a voter panel of approximately 300,000 total voters - just in those 9 states. Double or triple that for the rest of the country to cover all 50 states (probably with less people on the panels in the other states, as those were not goin g to be as critical to victory). This is by two whole orders of magnitude more accurate than anything any professional pollster ever runs. A 'huge' consumer survey might run 3,000 surveys. Obama's team ran 100 times that size (or likely even bigger yet).


This is the essence of Project Narwhal (and DreamCatcher) which was not primarily a voting day machine, but something to give voter insights all throughout the campaign season. The accuracy level of the Obama internal poll totally blasts away any other professional pollsters - usually major national polls have sample sizes from 1,000 to 2,600 voters. State-wide polls often have sample sizes as small as 600 voters. Obama's machine accuracy was so precise, they matched actual voter results on election day, to their panel projection, within a fraction of a percent in almost every battleground state. In Colorado, where Obama's winning victory was far bigger than any public polling had suggested, Narwhal still predicted the landslide victory for Obama, but undercounted it by only 1%. Even when it was 'massively' off, it was off by one percent and did not miss who would be the winner. Team Obama knew they had won. And this level of accuracy is the management insight which drove the Obama campaign.


And this system was built, tested, improved, and its users trained. Users on Narhwal had received training on it a year before election day. The system was truly tested and used daily, not unveiled on election day.


Project Narwhal was also a massive election simulation tool - I really like this part (eleven years ago, back when I was still employed over at Nokia, as part of the Nokia consulting assets and tools, I was heading Nokia's set of simulation tools, so this is also a personal interest of mine). So Obama's team could simulate anything, what if one of the candidates had a heart attack, what if there was a blizzard on voting day, what if there was a military scandal and the military vote support was suddenly lost, etc. The Obama team ran... get this.. 2 MILLION simulations of election outcomes through Narwhal !!! They knew every conceivable scenario, well before it played. They KNEW if there was an Atlantic hurricane that would appear in the last week of the election, what to do with it. Why? Because they had modelleled it through no doubt over 1,000 separate simulations, what if it only hits southern Atlantic, what if it causes Manhattan to flood, what if it rushes up to Boston, what if it blocks out electricity to fifteen Northeastern states - and what if Obama stays campaigning during the Hurricane, what if he stops campaigning but Romney continues, etc etc etc. 2 million total simulations, powered with the most complete consumer survey data of any country any election electorate, ever. Its not just that Narwhal knew what was happening on election day, minute by minute. Narwhal had pre-anticipated any conceivable variation. This is like playing chess against a computer - and the computer is powerful enough to calculate essentially every variation from that move onto check mate. What if there is a terrorist attack in Houston, what would that do to Texas vote and the national vote etc etc etc..


The power of simulation is incredible, if your model is reasonably accurate - which is why simulation is used to train supercritical experts such as astronauts, submarine captains, fighter pilots, Formula 1 drivers, tank commanders etc. So what could Chicago do with these simulations? They soon were able to identify the exact parameters of what type of person would be likely to support the campaign by donating via online - and what messages would trigger that donation; compared to who would rather donate via mail (ie send a cheque) or via SMS etc. They were able to find what types of people were willing to volunteer to help the team etc. How powerful? They raised almost a Billion dollars in donations this way, far and away more than in 2008 - when the Obama campaign set the records for most unique individual donators, and the most money ever raised in an election. The benefit of such simulation is to test out which method works the best, and that, in turn helps guide management into using scarse resources more efficiently. The Obama campaign was always many steps ahead of the Romney campaign.


About those comparisons. We do know Project Orca for Romney was able to capture voting info for '91%' of the intended target voters, and thus it has at a miminum a capacity of collecting data on about 16 million voters. It did not contain this info on election day - so Orca did not know which voters were going to vote for Romney, etc, but as the data came in, Orca added to the info, so that data on about 14 million Republican supporters can be used next time in the 2016 Presidential election. Remember, Orca was only taken into use on election day, and Romney staff had not meticulously been collecting voter preferences in the weeks and months leading into the election (like how Obama's team had done).


Compare to Narwhal. Obama's team had preloaded its massive database with 175 million voting age Americans already before election day! They knew who were registered voters, and they had a very good understanding who were likely Romney or Republican supporters, who were Obama or Democratic supporters and who were swing voters. The Obama campaign had contacted nearly half of its own voter base, and knew which were the 28% who had voted early - those did not need reminder calls on election day - and knew which were the four million who had made a campaign contribution - very high level VIP supporters of the Obama campaign, etc. This is like comparing a navy with a saiboat to a navy with an aircraft carrier. Yes, Romney's Orca was a quantum leap compared to the paper-based system McCain's team had used. But it was peanuts compared to Obama's massive multi-multi-million dollar system. The 2008 version was the biggest data mining effort in any election. The 2012 Obama edition had 5 times the staff compared to 2008 and cost a massive 100 Million dollars to build. Whatever Orca was or could have been, was utterly dwarfed by Narwhal and its sisters. So what did the two campaigns do with their systems?

As you know from reading the TPI, Anonymous claims to have placed a block on the Romney campaign ORCA.  We will never know if the Anonymous claim is accurate. We do admit to an inclination to believe the Hacktavist Group as the coincidence Anonymous in Ohio were too much like the Karl Rove manipulated election in 2004. 


Whether true of not, other writers are writing about the differences in the Obama and Romney technology foundation and operational effectiveness throughout the 2012 election campaign. As we approached the election, I often wondered why the Obama campaign stated late in the campaign, and as November 6th approached, they felt comfortable with, and confident of, their "Ground Game". It is now glaring evident the Obama campaign 'outfoxed and out geeked' the Romney campaign.



The significance of the last point is beyond critical. I learned many years and campaigns ago the presidential campaign is a long and draining process. The process also includes testimonial to how well a candidate and his/her team can perform even after an election win an ascension to the Oval Office. If the last campaign was any indication of the fortunes of America and the simultaneous shallowness of the 47% who voted for Mitt Romney, the following piece from The Daily Beast drives the point to the core. 


The Daily Beast:

The Romney Campaign’s Ground Game Fiasco


Excerpt 
The story starts in 2008. The Romney campaign sought to counter Team Obama’s highly touted, high-tech voter-targeting system, nicknamed Narwhal after the Arctic sea mammal. Narwhal provided the Obama campaign with reams of specific data on voters—finding single women in conservative counties, for instance, or families with children who have disabilities. 

In 2012, the Romney campaign unveiled its own killer app and called it Project Orca—the fierce great whale that is the natural predator of the Narwhal. The only problem: Boston’s Orca turned out to be toothless. 

The system was different from Narwhal. It was designed to allow Romney poll watchers, in real time, to identify likely Romney supporters who still had not shown up at polling stations on Election Day. By uploading the names of people who had voted, the computers back in Boston could figure out who still needed to be targeted and turned out. 

At least that’s the way it was supposed to work. But on Tuesday, it became clear that the deployment of Orca was doing more harm than good. “I think it’s fair to say that pretty much everything about the system that was supposed to work actually failed,” said one campaign official who witnessed the breakdown from the Romney war room on the floor of Boston’s TD Garden. 

The Romney high command had cloaked the system in secrecy to maintain what it hoped would be a true competitive turnout advantage. But by limiting the number of people with access to Orca, the campaign was not able to train its field operatives to use it or do the necessary beta-testing to work out the kinks that typically plague new software.

Reasons for any quest to understand the details of the November 6th win and the loss on  goes well beyond simply politics and social views.  The issues are as serious as any letter from the IRS.  The political campaign really does reveal the effectiveness of leadership, planning, strategy,  and execution.  Of course, the first lesson of any campaign, "...how well does the leader of the campaign and her/his close advisers staff the remainder of the campaign team?"  To explore the continuum a bit more, how effectively can a group of people (w/resources) manage.  In this case, manage affected each of us and our families  our jobs and our literal lives.  Both the McCain and Romney Campaigns stumbled miserable.  And, each stumbled from the top down.

We posit the American voter must be extra careful in exercising their legal right to cast a ballot on who leads our governments.   The Romney campaign showed from day one it was not a campaign of truthfulness, it was a campaign of dirty tricks, and the party behind the campaign executed un-American voter suppression.  The voter must also pay close attention to the quality of management team necessary in today's world for maintaining and advancing the quality of life in our country. 

NARWHAL Vs. Romeny's ORCA is another example the effectiveness of leadership and management that leads to successful outcomes.


Sources:
  
Communities Dominate Blog (ORCA Meets NARWHAL....)

The Business Insider


The Daily Beast (Romney Campaign Ground Game Fiasco)











No comments :

Post a Comment