The Colorado Risk-Limiting Audit Project (CORLA)

Colorado is rolling out Risk-Limiting Audits (RLA) so that we can transparently provide citizens with scientific evidence that the outcomes of our elections correctly reflect the votes that were cast.

After over a decade of gathering experience with election audits in Colorado, in 2017 the Secretary of State's office convened a working group including auditing experts from the Election Verification Network (EVN), experienced election administrators from five counties, and representatives from voting system vendors, establishing the Risk-Limiting Audit Representative Group. The group will build on our experience to help the SoS establish official rules for how RLAs will be conducted, as required starting with the 2017 elections.

The kickoff presentation sets the scene for the next several months of work.


Paper ballots are critical, but we also need to look at a sample of them by hand to check up on the equipment and procedures: good audits are critical also. This has been recognized by the Presidential Commission on Election Administration. They say:

Recommendation: Audits of voting equipment must be conducted after each election, as part of a comprehensive audit program, and data concerning machine performance must be publicly disclosed in a common data format. ....
The Commission endorses both risk-limiting audits that ensure the correct winner has been determined according to a sample of votes cast, and performance audits that evaluate whether the voting technology performs as promised and expected.

Colorado had first-hand experience with the benefits of audits, and the failures of certification. We've done innovative audits starting in 2004 in Boulder County and did the first risk-limiting audit outside of California. The Conroy v. Dennis lawsuit in 2006 documented problems with relying on equipment certification for security. So we adopted a flexible framework for voting system approval, and a state-wide requirement for risk-limiting audits by 2017 (Colorado Revised Statute 1-7-515).

An excellent, comprehensive introduction to risk-limiting audits is Risk-Limiting Post-Election Audits: Why and How., by Bretschneider, J., S. Flaherty, S. Goodman, M. Halvorson, R. Johnston, M. Lindeman, R.L. Rivest, P. Smith, and P.B. Stark, 2012.

Grant from Election Assistance Commission (EAC)

In 2011, Colorado was the recipient of a grant from the Election Assistance Commission (EAC) to implement Risk-Limiting Audits.

Arapahoe County Risk-Limiting Audit, 2013

The CORLA project in Colorado has now given us experience with Evidence-Based Elections as described by Stark and Wagner.

On November 14, 2013, we did a risk-limiting ballot-level audit of three contests for the general election in Arapahoe County. We used the website Tools for Comparison Risk-Limiting Election Audits to audit cast vote records generated by hardware and software from Clear Ballot Group.

The full report for the EAC is available at Risk-Limiting Audit – Final Report (pdf)

Arapahoe County Audit with OpenCount software, August 2014

In August of 2014, after the CORLA project was over, Arapahoe County experimented with a follow-on audit, this time of the 2014 primary election, using the open source OpenCount software. It got good news coverage:

There are some misconceptions in the articles though. The ballot comparison audit is the most efficient way to determine if the outcome was correct, but it requires more detailed reports (cast vote records) than most tabulation systems now provide. The ballot polling audit, demonstrated in Colorado for the first time in this 2014 audit, is much less efficient for close contests, but can be used with any kind of tally equipment. In other words, both achieve the same goal (verifying the outcome), but differ in their efficiency and applicability.

2015 ballot-level Mock Audits

Given the legislative mandate, and based on the positive experience with CORLA, as part of Colorado's Uniform Voting System project, a series of "Mock Risk-Limiting Audits" were performed in several counties using equipment from several different manufacturers. They included innovative demonstrations during the 2015 general election in:

Uniform Voting System

Four major vendors, including ES&S, Clear Ballot, Dominion, and Hart, now provide Cast Vote Records that can be used (despite some unfortunate formatting problems) to do ballot-level Risk-Limiting Audits, so Colorado has really helped move the world of elections forward.

In 2015, Colorado selected a voting system based on Dominion's proposal.

For more information, see:

Related information

Site run by Neal McBurnett