1 Besides having a paper trail, we need to always audit it

There is widespread support for Voter-Verified Paper Audit Trails (VVPAT). But the paper trail needs to be looked at after the voter verifies it. Therefore an automatic Manual Count Audit of at least some part of the paper trail is also a necessary part of verifiable elections. This is also known as an automatic manual tally audit, random hand count, manual recount, independent hand audit, etc.

Unfortunately, election audits are hardly ever done properly.

Manual count audits can catch a wide variety of problems, ranging from faulty software or hardware, or human error, to manipulation of the machines or results by outsiders or fraud by manufacturers, election officials or volunteers.

For example, exactly the sort of audit described here detected problems with a vote in California's Napa Valley. An audit also detected an incorrect result due to operator error and confusing user interfaces in a system used in Boulder Colorado.

The prestigious Brennan Center Report carefully examines actual risks to elections, and concludes that good audits are among the most effective tools in ensuring election accuracy and integrity.

Manual counts, when properly done and cross-checked, are also very accurate, and typcally find more unambiguously cast ballots than any equipment. The CalTech/MIT study says this about hand-counted ballots:

The central finding of this investigation is that manually counted paper ballots have the lowest average incidence of spoiled, uncounted, and unmarked ballots.

Audits are needed not only for the VVPATs associated with electronic voting devices, but also for the counts produced by optical scans of paper ballots, and the final tallies which combine early voting, absentee voting, and precinct voting. A good audit of a properly auditable election vote count report is the only way to detect many forms of bugs and tampering with the final tally systems.

This discussion focusses on auditing the votes that show up in the final tally, but of course other aspects of the process also need to be audited, as part of the overall canvass, including ballots that are disqualified, counts of ballots from poll books and other election records, etc.

There is widespread citizen support for an independent audit in which a hand count of of voter-verified paper ballots is compared to the results of the current voting system. The hand count must be performed in public by representatives of the various candidates/parties/issue campaigns.

Manual count audits are supported in HR 550 (still pending) by Rep Holt in NJ, and by Colorado's SB-079 (now law) (see http://www.leg.state.co.us/)

2 State audit requirements

Several states currently require manual counts in between 1% and 5% of precincts. As of early 2005:

California: During the official canvass of every election in which a voting system is used, the official conducting the election shall conduct a public manual tally of the ballots tabulated by those devices cast in one percent of the precincts chosen at random by the elections official.

Maine: Paper recount of up to 2% of voting places (Law LD-1759)

Illinois: Audit 1% of precincts after each election (Public Law 093-573)

Kentucky: As part of the official canvass, a manual recount of randomly selected precincts representing three to five percent of the total ballots cast in each election shall be completed.

West Virginia: During the official canvass and any requested recount, at least five percent of the precincts are to be chosen at random and the ballot card cast therein counted manually.

Colorado: Following each primary, general, coordinated, or congressional district vacancy election, the secretary of state shall publicly initiate a manual random audit to be conducted by each county and shall randomly select not less than five (5) percent of the voting devices used in each county; except that, where a central count voting device is in use in the county, the rules promulgated by the secretary pursuant to subsection (5) of this section shall require an audit of a specified percentage of ballots counted within the county. (From Colorado Revised Statutes CRS 1-7-514. See also Secretary of State Election Rules 8 CCR 1505-1, rule 11.5.4)

Colorado also has past experience with audits via manual counting. In Boulder County an error in the original count was discovered and confirmed during a manual count of the St Vrain School District on Dec 7 2004 (see ManualCountAuditStVrain for details). Boulder used a Manual Count Audit for the March 8 2005 municipal election also. But in November of 2005, a test with live ballots was done instead of an audit, because of improper rules from the Colorado Secretary of State, which called for a partial test, rather than an audit as required by state law. The same thing happened in the 2006 Primary election.

3 Need for auditable vote count reports

1: Use the election system to tally up the votes.

2: Produce an auditable vote count report, in which results for each race are calculated for the votes cast on each individual voting device (for eSlate) and in each batch of ballots (for BallotNow). The totals across all devices and batches must match the totals for the election.

3: In a statistically random manner, select devices and batches and races to be audited, such that each device has an equal probabability of being selected. No one should know which devices and batches will be audited until after the auditable vote count report is produced. See below for more discussion on the selection methods.

4: Hand-count the paper records (VVPATs and ballots) to get results for the selected races on the selected devices and batches. The counters must not know what the results calculated by the system were.

5: Compare the numbers from the hand counts and the system counts for each device and each batch. They should each match exactly. If not, any discrepancies must be eliminated or explained and resolved.

4 Sample auditable vote count report

Below is a sample report for auditing. Of course in a real election it would be larger, including all devices and batches, with totals matching the overall election results. Note inclusion of undervotes (UV) and overvotes (OV). In this example, the presidential race, Device 1002, and Batch 101 are selected for audit (marked in bold).

Device/Batch	President				Proposition X				Total
	Jefferson	Adams	UV	OV	Yes	No	UV	OV
Device 1001	53	46	1	0	30	66	3	1	100
Device 1002	55	44	0	0	33	63	1	2	99
Batch 101	125	73	1	1	79	120	0	1	200
Batch 102	123	76	2	1	80	122	0	0	202
Total	356	239	4	2	222	371	4	4	601

5 How to select samples

As noted by elections expert Professor Douglas Jones, the power of a hand audit can be improved by picking some of the precincts (perhaps half of them) based on "measures of suspicion":

Permit the parties, for example, to name the precincts they're most suspicious of, or look at the vote distributions district wide and recount the precincts that represent extremes or that depart the farthest from what exit polling or other predictions suggested for those precincts. Allowing the parties to name their targets simply lets the parties compare the exit polls or statistical predictions. You must, of course, keep some percentage of the targets for the audit random.

If early and absentee ballots from all the precincts are mixed together, as they usually are, sorting thru them for the ballots for a particular precinct can be time-consuming. This is one reason for allowing batches which don't represent precincts, e.g. they could represent individual machines or stacks of paper ballots fed into an optical scanner.

But note that an election vote count report that provides results by precinct is also very important, for comparison with historical trends. Precinct results are also important to candidates and political party organizations, who judge campaign effectiveness on these results.

Jonathan Wand at Stanford has a paper [Auditing an Election using sampling: the impact of bin size on the probability of detecting manipulation http://wand.stanford.edu/elections/probability.pdf] about the degree of confidence that manual counts can provide, which depends in large part on the number of batches examined. If the goal is e.g. detecting a bug or hack that causes _all_ machines to give incorrect results, not many batches need to be counted.

More savvy hackers might manipulate just 10% of machines. They would have to swing ten times as many votes in each hacked machine in order to have the same overall effect on the election. That helps evade detection, but still with a random sample of 20 batches, there is only a 12% chance that they will get away with it. Auditing 40 batches would reduce that to 1.4%, and for 60 batches there is a 0.17% chance of getting away with it. The size of the batches again doesn't matter much. So in circumstances like this, counting more batches/precincts, even if they are smaller, is better than counting fewer big ones.

Even more efficient and robust audits are possible when each paper record has a serial number that allows it to be matched with an electronic representation of the same vote. (Of course this has to be done in a way that preserves the anonymity of the voter.) If they can be truly randomly selected, even an audit of only 230 individual ballots would be likely to detect a problem (it would have a 90% chance of finding a mismatched ballot if there are 1% mismatched ballots.)

One context in which this is useful is auditing a DRE via a paper trail that it produces. The document Preliminary Report: NIST Approach to VVPAT Requirements for the VSS 2002 Addendum by John Wack discusses this possibility.

Paul Walmsley's submission of Overview of Live Auditing Procedures for Incorporation in the Voluntary Voting System Guidelines to the Election Assistance Commission provides more detail on this sort of audit.

It may be surprising that it is easy to make random selections in publicly verifiable ways. This is in fact regularly used in an elections context, when organizations like the Internet Engineering Task Force choose a nominating committee. This technique should be used by elections officials for all random selections. See Publicly Verifiable Nominations Committee (NomCom) Random Selection - RFC 3797 for details and free source code.

6 See also

• Utah proposal for Mandatory Vote Count Audit 2006 by Kathy Dopp and Joycelynn Straight.
• Electronic Voting, by Rebecca Mercuri, Ph.D. whose thesis laid the groundwork for verifiable voting via the "Mercuri Method".
• Story on hand recount in King County Washington
• DiFranco et al, "Small Vote Manipulations Can Swing Elections," Communications of the ACM, Vol. 47, Issue 10, October 2004, pages 43-45, http://www.wheresthepaper.org/p43_di_franco.pdf