Boulder County 2010 General Election Risk-Limiting Audit - County Coroner race
In the 2010 general election, Boulder County piloted a new audit method which allows us to deal precisely and flexibly with discrepancies and auditing statistics. It meets the standard for a "risk-limiting" audit established in the scientific literature (Hall, 2009), in Colorado law 1-7-515, and in the more precise definition in California bill AB 2023. It incorporates current best practices based on experiences in Colorado, California and elsewhere. This improved method will be used on only one race, the one for County Coroner, so we can gain experience with it.
Besides this risk-limiting audit, the Boulder County Elections Division continues to do some of the best audits in the country. This year 8 contests were audited using the method we pioneered in 2008, as described in detail at 2010 General Election Audit. For background on election auditing in general, see "Principles and Best Practices for Post-Election Audits", the https://www.amstat.org/policy/pdfs/StarkEtAlLetterOfSupport.pdf American Statistical Association statement endorsing risk-limiting audits, and the 2012 California white paper, "Risk-Limiting Post-Election Audits: Why and How".
County Coroner auditThe audit of the County Coroner's race was set up with precise error bounds and Kaplan-Markov discrepancy analysis, and an alpha value of 0.5. After auditing 9 audit units, we met our target for risk reduction, with only one discrepancy.
- Unofficial Vote Tabulation Report
- Audit Selections and Results - using the random seed 286630844411402 from the dice rolls by the canvass board
Note that this audit accounts for the 5347 ballots that were tallied after the original audit data dump, after they underwent manual pre-processing as provisional ballots, ballots that were damaged and couldn't go through the scanners, mail-in ballots without signatures on the enclosing envelopes, etc. (Originally we thought there wouldn't be more than 3947, but that was due to a misunderstanding.) These ballots won't be audited, so the statistical calculations assume very conservatively that the ballots could all turn out to be cast for the person currently shown to be losing.
When Boulder pioneered improved audits for CO in 2008, we used a recently-published margin-based method by Aslam, Popa and Rivest. It reduced the effort required to audit contests in which the preliminary results report a wide margin, and more closely audited the close contests. But if even a few discrepancies between the hand counts and the machine counts are found, e.g. based on close hand inspection of voter intent that machines can misinterpret, the Aslam method isn't able to calculate a corresponding confidence level. It also makes questionable assumptions about the maximum "Within Precinct Miscount" that an error or attack may involve.
Since then more flexible and precise methods that can precisely determine the level of risk-reduction have been introduced by Stark et al. They simplify the math, and also deal precisely with discrepances and establish a true upper error bound for miscounts. We will be using the "Kaplan-Markov" method - see Stark 2009 and Lindeman 2010.
We'll use the open source ElectionAudits software to run the audit in a transparent way that allows observers to verify that everything from the prior posting of the data to be audited, to the publicly verifiable random selection, to the calculation of confidence levels and the achievement of the pre-determined level of risk limitation was done properly.
- Hall 2009: Hall, J.L., L.W. Miratrix, P.B. Stark, M. Briones, E. Ginnold, F. Oakley, M. Peaden, G. Pellerin, T. Stanionis and T. Webber, 2009. Implementing Risk-Limiting Audits in California, USENIX EVT/WOTE. (preprint: http://arxiv.org/abs/0905.4691)
- Amendment to California AB 2023 requiring pilot of risk-limiting audits, published 26 March 2010 http://www.leginfo.ca.gov/pub/09-10/bill/asm/ab_2001-2050/ab_2023_bill_20100325_amended_asm_v98.html
- Stark 2009: Philip B. Stark, "Risk-limiting post-election audits: P-values from common probability inequalities," IEEE Transactions on Information Forensics and Security. http://statistics.berkeley.edu/~stark/Preprints/pvalues09.pdf
- Lindeman 2010: Mark Lindeman, A Kaplan-Markov auditing example using 2008 California data simple, but realistic, hypothetical example of risk-limiting audit using existing California data.
- A "Sum of Square Roots" (SSR) Pseudorandom Sampling Method For Election Audits by Ronald L. Rivest.
Credit for a multifaceted effort like this goes to many people: Boulder Clerk Hillary Hall for the leadership and resources, Crystal Christman and Don Hayden for going beyond the call of duty, the members of the Canvass Board for their assistance and ideas, and the hand counters and many others in the elections division. Thanks to Philip Stark for developing the underlying statistical procedures, and Mark Lindeman for describing them with such clarity.
Responsibility for any problems with the audit design or implementation lies with Neal McBurnett
Some other audits:
2009 Coordinated Election Audit
and the Unofficial, Independent Audit Data for 2009 Boulder Election
- Boulder County 2008 General Election Audit
- Audit report, Boulder County Primary on 2008-08-12
- Longmont council race audit, Jan 2008
ElectionAudits Data Site
The data on this site was generated by the ElectionAudits software, which provides software and documentation in support of the Principles and Best Practices for Post-Election Audits and the other work of ElectionAudits.org, the nation's clearinghouse for election audit information.
The software is open source and can be followed at its Launchpad.net site:
That is the place to get the source code, file bugs, make contributions, learn about election auditing, etc.