Election Audits

THE NEW MEXICO 2006 POST ELECTION AUDIT REPORT

gbain — Wed, 03 Dec 2008 11:36:23 -0800

During the 2007 legislative session, the New Mexico Legislature passed a bill and Governor
Richardson signed it into law, which provides for random voting system audits after every
statewide general election (see §1-14-13.1, NMSA). Specifically, the law provides that county
clerks are to compare the total votes tallied in the general election for the office of president or
governor from a random selection of 2% of the voting systems used during the election
throughout the state to a hand count of the ballots cast on that system. A voting system is
defined as a vote-tabulating machine (§1-9-1(B), NMSA). In the event the hand count varies
from the total system count by more than 1.5%, the Secretary of State is required to conduct a
recount of the specified office in the precincts of the legislative district in which the discrepancy
occurred. The random system audit is to be completed within five days of the county canvass
board certification of the county election results to the Secretary of State. (See Part II of this
report for a copy of the New Mexico law).

The purpose of the law is to verify the accuracy and efficacy of the voting systems or vote
tabulating machines in tabulating votes. Thus, the audit is meant as a performance audit of the
voting machines. The New Mexico law is similar to laws that have been passed or are being
considered in legislative committees across the country that require manual counts of paper
ballots and voter-verifiable paper records in randomly selected units (e.g. precincts or voting
systems) and comparing them to the corresponding electronic or manual tallies, for the purpose
of verifying the election result with a high level of confidence.1 The broader purpose of these
measures is to strengthen voter confidence in the administration process and its outcomes.
In the spring of 2008, we had the opportunity to develop and test audit procedures in Bernalillo
County, New Mexico for the purpose of recommending specific recommendations and
guidelines to the New Mexico Secretary of State in preparation for the 2008 post election audit
and more broadly to consider and test methods that would be effective for other states and
localities as they grapple with this issue. At the same time, our study is also meant to inform the
public debate on the accuracy and integrity of the new optical scan voting systems deployed for
the first time in the 2006 election.

Therefore, we simulated the 2006 election using a random sample of 25% of the ballots cast
(47,481 ballots) and recounted those ballots by 2-person and 3-person hand counts and by a
second machine count. Bernalillo County, along with all other counties in New Mexico, uses an
optical scan paper ballot system to administer their elections. Bernalillo tabulates its ballots
using the ES&S M100 optical scan precinct ballot counter for Election Day and early ballot
processing and the ES&S M650 for absentee ballot processing. Early voting machines and the
M650 are programmed with 431 ballot styles and 78 unique ballot combinations, while Election
Day voting machines usually are programmed with only 1 ballot style. Our election audit
focused on the race for governor and land commissioner. 1 H.R. 811 (op cite) also has provisions in it for post-election audits in federal elections.

This study allowed us to evaluate, assess and experiment with procedures to provide
recommendations on post election performance audits. During our study we experimented with
counting procedures, developed sampling procedures, chain of custody procedures and carefully
examined all of the parameters necessary for a successful and complete post election audit. This
included: ballot reconciliation, pre-election preparation for the post election audit, the
importance of transparency to the voting process, sampling methods, example forms, audit team
selection standards, reporting of audit outcomes, voter intent standards and various hand
counting procedures. These recommendations, along with explanations and justifications, are
presented in Part I.

Part II of our report provides a detailed description of our study, research design, and process.
We provide background information on the New Mexico election administration context, the
election audit law and the location and set-up for our study. We also describe the documents that
we relied on to account for our work (e.g. tally sheets, audit logs, forms) chain of custody rules,
sampling methods, the data we collected and the machine and hand-counting procedures.
Part III of our report examines the machine and hand count data that were generated over the
course of our study and allows us to answer a number of questions about the reliability and
precision of machine and hand counts across different counting modes. We compare the machine
count to the actual number of ballots processed, machine to hand counts, hand counts to hand
counts and machine to machine counts to assess the reliability and validity of the various
counting methods. We also examine how long machines and humans took to count ballots,
providing information on how long audits can take.

Each part of our study is prefaced with an executive summary that briefly discusses the major
findings. We also provide an extensive appendix that documents all aspects of our research
design, provides examples for suggested forms, and provides detailed information on the data we
collected.

We hope that our research is a useful tool as election administrators across New Mexico and the
country prepare for the upcoming elections in 2008 and beyond. Our detailed examination of the
audit process is meant to provide practitioners and stakeholders with valuable information in
preparation for their own election audits.

We wish to make clear that this work was a partnership between the non-profit sector,
government, and academia and the strength of these relationships was key to a successful study
design. As scholars we learned much about the election administration process over the course
of our work and have a deeper understanding of the complexities of election administration. We
hope our experiences, detailed throughout this report, provide useful information to practitioners.
This study’s primary financial sponsors were the Pew Charitable Trusts, Center for the States,
and JEHT Foundation’s “Make Voting Work” Initiative. Without their support this project would
not have been completed. In addition, it is important to recognize that our partnership with local
and state government both financially and administratively was also critical to our success.

Bernalillo County Clerk, Maggie Toulouse Oliver and her Deputy Clerk Robert Adams, provided
valuable financial resources to support the election audit at the Voter Warehouse, provided us
with the training and background to operate machines and gave us the freedom, in a potentially
politically sensitive situation, to perform our task independently and experiment with a variety of
methods. Her staff was incredibly supportive before, during and after the audit and made our
jobs both productive and fun. The Secretary of State Mary Herrera also provided key financial
and administrative support and observed our process throughout. Her staff provided key input
on the New Mexico law and on the process of elections in the State of New Mexico and was a
needed asset and ally throughout our work. The partnership has provided important insights for
all parties in creating a stronger, more efficient and confident election system.
In addition to these individuals, the University of New Mexico, the University of Utah and the
California Institute of Technology also provided key financial and administrative support. A
detailed listing of all of the individuals involved in making this project a success are listed in the
acknowledgement section at the end of this document.

The New Mexico Election Administration Report: The 2006 November General Election

admin — Thu, 26 Jun 2008 14:36:08 -0700

The New Mexico Election Administration Report on the 2006 November General Election is the product of three independent research projects focused on New Mexico’s election administration efforts in the 2006 election. New Mexico has recently implemented a number of significant election reforms intended to create fair, accurate and voter-verifiable election administration systems. New Mexico, for example, is the first state that moved from a predominantly electronic voting system to one that mandated optical scan paper ballots statewide, with the intent of providing a paper trail so that elections could be audited for accuracy and to provide an environment that would promote greater voter confidence. The statewide implementation of this program and the use of ES&S M-100 and M-650 occurred in the fall 2006 general election. Prior to that there were at least six different voting technologies used throughout the state. Each of the three research studies summarized here examine a different aspect of the election process, each providing a very important look at the election administration question and an overall look at the 2006 election experience and environment in New Mexico.

Assessing the impact of voting technologies on multi-party electoral outcomes

admin — Fri, 20 Jun 2008 17:20:55 -0700

This paper presents the first study on the impact of different voting technologies on election outcomes in multi-party elections, analyzing data from a large-scale voting experiment conducted in the 2005 congressional election in Buenos Aires, Argentina. Combining different regression models and matching methods, we estimate the effect of alternative voting technologies on the probability of support for the competing parties in the elections for congress and state legislature. The results of the different statistical techniques indicate that voters are extremely receptive to the information cues provided by the different voting technologies and associated ballot designs, and that particular voting devices have a significant impact on voter choice, systematically favoring some parties to the detriment of others. We conclude that the choice of alternative electronic voting devices might have considerable effect on electoral outcomes in multi-party electoral systems.

Russian Elections: An Oxymoron of Democracy

admin — Fri, 20 Jun 2008 17:18:58 -0700

Considerable controversy swirls around the extent to which Russia’s elections have been falsified. We argue here on the basis of an assessment of aberrant distributions of turnout in official election returns for each or Russia’s national elections beginning in 1995, that falsifications in the form of stuffed ballot boxes and artificially augmented election counts, whose significance was first apparent in its ethnic republics, has now spread to and metastasized within both rural and urban oblast districts. That spread, moreover, unashamedly accelerated during the Putin administration – notably the 2004 election – and has sustained itself thru the 2007 Duma parliamentary vote.

Ukraine’s 2007 Parliamentary Elections: Free and Fair or Fraud Once Again and the Argument for Election Observers

admin — Fri, 20 Jun 2008 16:42:05 -0700

No abstract available.

On Auditing Elections When Precincts Have Different Sizes

admin — Fri, 20 Jun 2008 16:12:49 -0700

We address the problem of auditing an election when precincts may have different sizes, and suggest methods for picking a sample of precincts to audit that takes precinct size into account. one method yields optimal auditing strategies together with an exact measure of its effectiveness (probability of detecting corruption of a given size).

We restrict attention to basic auditing strategies, in which each precinct P is audited independently with some probability p determined by the auditor. The auditing probability for a precinct will depend on the size of the precinct, with larger precincts audited more frequently; when all precincts have the same size they will have the same probability of being audited.

On Estimating the Size and Confidence of a Statistical Audit

admin — Fri, 20 Jun 2008 16:09:18 -0700

We consider the problem of statistical sampling for auditing elections, and we develop a remarkably simple and easily-calculated upper bound for the sample size necessary for determining with probability at least c whether a given set of n objects contains b or more “bad” objects. While the size of the optimal sample drawn without replacement can be determined with a computer program, our goal is to derive a highly accurate and simple formula that can be used by election officials equipped with only a simple calculator.

Qualitative Evaluation: "VotoElectronico: Prueba Piloto 2005, Ciudad De Buenos Aires"

admin — Fri, 20 Jun 2008 15:42:23 -0700

The Buenos Aires “VotoElectronico” pilot project took place on October 23, 2005, in the City of Buenos Aires. It involved a pilot test of four different electronic voting systems, in at least 43 voting stations located throughout the City. The purpose of this report is to provide initial qualitative assessments of the pilot project. Quantitative analysis will hopefully be presented at a later date, once detailed data from the pilot project are available for examination.

The key recommendations for the current pilot project evaluation effort and for the possible future continuation of this pilot project, made in this report are:

Much care is needed in analysis and study of the evaluation data.
Determine the extent of usability problems.
Take a careful look at the optical scanning system.
Develop and implement procedures for physical security.
Develop and implement procedures for pilot testing security.
Continue pilot testing, but narrow down the number of options being tested.

Below, I discuss the pilot project in more detail, provide a detailed discussion of my observations, and provide more discussion of these six key recommendations.

Auditing Technology for Electronic Voting Machines

admin — Thu, 19 Jun 2008 16:37:05 -0700

Direct Recording Electronic (DRE) voting machine security has been a significant topic of contention ever since Diebold voting machine code turned up on a public internet site in 2003 and computer scientists at Johns Hopkins University declared the machine “unsuitable for use in a general election.” Since then, many people from computer scientists to politicians have begun to insist that DREs be equipped with a paper trail. A paper trail provides a paper printout for the voter to approve at the end of each voting session. Although there have been strong political efforts to place paper trails on DRE machines, there have not been any scientific studies to indicate that paper trails are effective audits. This work describes a user study done to compare paper trails to audio audits, a new proposal for DRE auditing. Participants in the study completed four elections on a voting machine with a paper trail and four elections on a machine with an audio trail. There were purposeful mistakes inserted into the audits on some of the machines. Results from the study indicated that participants were able to find almost 10 times as many errors in the audio audit then they were able to find in the paper trail. Voters’ attitudes towards the paper audit were extremely apathetic, and voters did not spend much time reviewing their paper record. When asked which type of audit voters would prefer for their own county elections, almost all voters preferred the VVPAT. These results indicate that newer alternative audit technology holds great promise in delivering a safe and accurate audit and further that paper trails have some significant design obstacles that need to be overcome before they will be effective audits.

Measuring the Improvement (Or Lack of Improvement) In Voting Since 2000 in the U.S.

admin — Thu, 19 Jun 2008 15:58:45 -0700

This paper summarizes what systematic evidence exists about the performance of the American voting process in 2004 and proposes a comprehensive system of performance measures that would allow citizens and officials to assess the quality of the voting system in the U.S.

Despite the great deal of attention paid to voting reform from 2000 to 2004, and billions of dollars spent, there is surprisingly little systematic evidence of improvement in how elections are conducted in the United States. The best evidence of improvement comes in assessing the overall quality of voting machines that were used, and here the news is good. Nonetheless the measures used to assess voting machines could be greatly improved. There is little systematic, nationwide evidence of whether registration problems declined, polling places were administered better, or whether voter tabulations were more accurate.

In thinking about how to improve data gathering about the election system, we first need to specify four principles guiding data gathering (uniformity, transparency, expedition, and multiple sources) and three major obstacles (federalism, state and local officials, and disputes over the purpose of elections). With these principles and obstacles in mind, I sketch out a basic data gathering agenda intended to allow the public to assess the quality of voting in the United States.