Auditing and Election Data Formats

Post-election Vote Tabulation Audits

• Post-election vote tabulation audits require voter-verifiable paper audit trail (VVPAT)
• Publish preliminary vote-count data by batch before the audit
• Batches should reflect how the ballots are stored, e.g. by precinct by method of voting
• Select batches randomly, using verifiably random numbers
• Hand count batches, without the hand counters knowing what the preliminary results were, and then compare vote counts
• Select and count more batches if there are material discrepancies

Growing momentum behind good audits

• 2008 Principles and Best Practices for Post-Election Audits: http://electionaudits.org/principles
• 2009 League of Women Voters Report of the Election Audits Task Force
• 2009 Law in Colorado: risk limiting audits by 2014; EAC grant funding
• 2010 law in CA to pilot risk-limiting audits; EAC grant funding
• NM: state-wide coordinated audit starting 2010
• Many other EAC audit grants funding risk-limiting and small batch audits
• Open source ElectionAudits software can produce EML 510

Risk-limiting audits

• Goal: limit the risk of declaring the wrong winner

• Much more efficient at meeting that goal than traditional percentage-based audits (e.g. California 1% manual tally)

• An audit is risk-limiting if it has a pre-specified minimum chance of requiring a full hand count whenever the apparent outcome of the contest is wrong.

• "Wrong" means that a hand count of the audit trail would show a different outcome. The "risk" is the chance that an outcome that is wrong will be certified anyway.

• Risk-limiting audits can determine with precision how much hand counting is necessary to confirm election results with a high confidence

• If the evidence from the hand counts does not limit the risk to the given risk level, a risk-limiting audit will lead to a full hand count

• E.g. the California pilot program audits provide 90% confidence that the audit will require a full hand count if the voting system tally was so wrong that the incorrect winner was called.

• The size of the initial sample of ballots depends on the margin of victory in the contest: the narrower the margin, the larger the initial sample.

Evidence-Based Elections

• Evidence-Based Elections combine compliance audits (chain of custody, ballot accounting, security etc.) and Risk-Limiting Audits.
• See P.B. Stark and D.A. Wagner: http://statistics.berkeley.edu/~stark/Preprints/evidenceVote12.pdf
• Most efficient given ballot-level audit data - even for a close race in a huge county, only need to select and inspect a few hundred ballots
• Pilot audits done in California, 2011 in 8 counties, most at ballot-level, one ballot-polling audit, for when data collection is difficult and margin isn't too tight
• Colorado Best Practices and Vision Commission endorsed ballot-level risk-limiting audits in December 2011

Data requirements

• For our reporting work, e.g. EML 510 and 520, even for election-night reporting, we should emphasize the need to report undervote and overvote counts, both overall and for each reporting unit. This is important not only for auditing, but as the most basic measure of election quality, the "residual vote".

• The most efficient audits require cast vote records, or small batches with vote count data with subtotals for each

• Number of ballots audited is proportional to the size of the batches

• Most important: still just getting auditable data out of systems

  • desire for interoperability, i.e. out of the box
  • different from integrability - requires consultants
  • ideal: ask election officials to press button labeled "export audit report"

• If batches are very small, need to take care to preserve anonymity of the ballots, e.g. by combining batches, or not tying the data directly to a set of voters

• Secrecy-preserving Observable Ballot-level Audit (SOBA), and related techniques, can be used to publish anonymous data that is still auditable

• See EML 510 example at http://bcn.boulder.co.us/~neal/electionaudits/eml510-example-nostyle.xml

  • What is StatusDetails element for? Why required in EMLv6?

Ballot-level audit data

• Cast Vote Records which can be linked back to the original paper ballot

• Some systems that currently produce auditable Cast Vote Records:

  • Clear Ballot Group
  • Open source software from Kai Wang (UCSD) used in California audits, 2011, being further developed now with help from David Wagner's group at Berkeley

Note volume of data e.g. Los Angeles County:

• turnout of perhaps 3.5 million
• dozens of votes on a typical ballot
• hundreds of characters per Cast Vote (EML 440)

We need some real examples of EML 440 and 460, but I'd guess many gigabytes per Votes message (EML 460), even compressed

Consider various "binary XML" formats like EXI (Efficient XML Interchange), Fast Infoset.

Or a customized binary format which encodes ballot id, ballot style, and one bit per choice, along with a way to transform it to and from EML 460.

State-level audits

• Coordinating audits across multiple jurisdictions
• New Mexico doing this since 2010: "Voting system check"
• See P1622 audit use case for data flows back and forth to coordinate multi-county audit

Background

• P1622 Audit use case: http://www.nist.gov/itl/vote/upload/audit_use_case-02.pdf

• Principles and Best Practices for Post-Election Audits: http://electionaudits.org/principles

• California's EAC-funded Post Election Risk-Limiting Audit Pilot Program: http://www.sos.ca.gov/voting-systems/oversight/risk-limiting-pilot.htm

  • Includes extensive information on recent audits there, as well as Draft Audit Instructions