Boulder County 2010 General Election Risk-Limiting Audit - County Coroner race

In the 2010 general election, Boulder County piloted a new audit method which allows us to deal precisely and flexibly with discrepancies and auditing statistics. It meets the standard for a "risk-limiting" audit established in the scientific literature (Hall, 2009), in Colorado law 1-7-515, and in the more precise definition in California bill AB 2023. It incorporates current best practices based on experiences in Colorado, California and elsewhere. This improved method will be used on only one race, the one for County Coroner, so we can gain experience with it.

Besides this risk-limiting audit, the Boulder County Elections Division continues to do some of the best audits in the country. This year 8 contests were audited using the method we pioneered in 2008, as described in detail at 2010 General Election Audit. For background on election auditing in general, see "Principles and Best Practices for Post-Election Audits", the American Statistical Association statement endorsing risk-limiting audits, and the 2012 California white paper, "Risk-Limiting Post-Election Audits: Why and How".

County Coroner audit

The audit of the County Coroner's race was set up with precise error bounds and Kaplan-Markov discrepancy analysis, and an alpha value of 0.5. After auditing 9 audit units, we met our target for risk reduction, with only one discrepancy.

Note that this audit accounts for the 5347 ballots that were tallied after the original audit data dump, after they underwent manual pre-processing as provisional ballots, ballots that were damaged and couldn't go through the scanners, mail-in ballots without signatures on the enclosing envelopes, etc. (Originally we thought there wouldn't be more than 3947, but that was due to a misunderstanding.) These ballots won't be audited, so the statistical calculations assume very conservatively that the ballots could all turn out to be cast for the person currently shown to be losing.

More details

When Boulder pioneered improved audits for CO in 2008, we used a recently-published margin-based method by Aslam, Popa and Rivest. It reduced the effort required to audit contests in which the preliminary results report a wide margin, and more closely audited the close contests. But if even a few discrepancies between the hand counts and the machine counts are found, e.g. based on close hand inspection of voter intent that machines can misinterpret, the Aslam method isn't able to calculate a corresponding confidence level. It also makes questionable assumptions about the maximum "Within Precinct Miscount" that an error or attack may involve.

Since then more flexible and precise methods that can precisely determine the level of risk-reduction have been introduced by Stark et al. They simplify the math, and also deal precisely with discrepances and establish a true upper error bound for miscounts. We will be using the "Kaplan-Markov" method - see Stark 2009 and Lindeman 2010.

We'll use the open source ElectionAudits software to run the audit in a transparent way that allows observers to verify that everything from the prior posting of the data to be audited, to the publicly verifiable random selection, to the calculation of confidence levels and the achievement of the pre-determined level of risk limitation was done properly.


Credit for a multifaceted effort like this goes to many people: Boulder Clerk Hillary Hall for the leadership and resources, Crystal Christman and Don Hayden for going beyond the call of duty, the members of the Canvass Board for their assistance and ideas, and the hand counters and many others in the elections division. Thanks to Philip Stark for developing the underlying statistical procedures, and Mark Lindeman for describing them with such clarity.
Responsibility for any problems with the audit design or implementation lies with Neal McBurnett

Some other audits:

See also the Boulder Clerk Elections Home Page and the 2010 Unofficial General Election Results

ElectionAudits Data Site

The data on this site was generated by the ElectionAudits software, which provides software and documentation in support of the Principles and Best Practices for Post-Election Audits and the other work of, the nation's clearinghouse for election audit information.

The software is open source and can be followed at its site:

That is the place to get the source code, file bugs, make contributions, learn about election auditing, etc.