Colorado is rolling out Risk-Limiting Audits (RLA) so that we can transparently provide citizens with scientific evidence that the outcomes of our elections correctly reflect the votes that were cast.
After over a decade of gathering experience with election audits in Colorado, in 2017 the Secretary of State's office convened a working group including auditing experts from the Election Verification Network (EVN), experienced election administrators from five counties, and representatives from voting system vendors, establishing the Risk-Limiting Audit Representative Group. The group is helping the SoS establish official rules for how RLAs will be conducted, as required starting with the 2017 elections.
The kickoff presentation provides an official introduction. A good summary of procedures recommended by auditing experts is at Concise recommendations for Colorado Risk-Limiting Audit rule in 2017, but there is a wealth of other information at the Risk-Limiting Audit Representative Group site.
In a historic move, on May 5, 2017, the Colorado Department of State issued a solicitaion (using their fast-track "Documented Quote" procedure) for audit support software for the upcoming 2017 elections and beyond: DQ_RLA_0500517_FE_KRT_VAAA_2017-1420.pdf. An official copy of it, along with two attachments, can be found by logging in (e.g. for "Public Access") and searching for "Election" at the ColoradoVSS site.
Bids are on a fast track, due by May 24 2017.
They want bids which develop open source software, but regardless, they intend to disclose the source code in the interest of transparency.
The next major step is a draft of the official audit rules, expected within weeks. We hope the state continues to audit all the contests, as they have since 2005, rather than settling for the minimum of one statewide and one in each county which is listed in the DQ.
Paper ballots are critical, but we also need to look at a sample of them by hand to check up on the equipment and procedures: good audits are critical also. This has been recognized by the Presidential Commission on Election Administration. They say:
Recommendation: Audits of voting equipment must be conducted after each election, as part of a comprehensive audit program, and data concerning machine performance must be publicly disclosed in a common data format. ....
The Commission endorses both risk-limiting audits that ensure the correct winner has been determined according to a sample of votes cast, and performance audits that evaluate whether the voting technology performs as promised and expected.
Colorado had first-hand experience with the benefits of audits, and the failures of certification. We've done innovative audits starting in 2004 in Boulder County and did the first risk-limiting audit outside of California. The Conroy v. Dennis lawsuit in 2006 documented problems with relying on equipment certification for security. So we adopted a flexible framework for voting system approval, and a state-wide requirement for risk-limiting audits by 2017 (Colorado Revised Statute 1-7-515).
An excellent, comprehensive introduction to risk-limiting audits is Risk-Limiting Post-Election Audits: Why and How., by Bretschneider, J., S. Flaherty, S. Goodman, M. Halvorson, R. Johnston, M. Lindeman, R.L. Rivest, P. Smith, and P.B. Stark, 2012.
In 2011, Colorado was the recipient of a grant from the Election Assistance Commission (EAC) to implement Risk-Limiting Audits.
The CORLA project in Colorado has now given us experience with Evidence-Based Elections as described by Stark and Wagner.
On November 14, 2013, we did a risk-limiting ballot-level audit of three contests for the general election in Arapahoe County. We used the website Tools for Comparison Risk-Limiting Election Audits to audit cast vote records generated by hardware and software from Clear Ballot Group.
The full report for the EAC is available at Risk-Limiting Audit – Final Report (pdf)
In August of 2014, after the CORLA project was over, Arapahoe County experimented with a follow-on audit, this time of the 2014 primary election, using the open source OpenCount software. It got good news coverage:
There are some misconceptions in the articles though. The ballot comparison audit is the most efficient way to determine if the outcome was correct, but it requires more detailed reports (cast vote records) than most tabulation systems now provide. The ballot polling audit, demonstrated in Colorado for the first time in this 2014 audit, is much less efficient for close contests, but can be used with any kind of tally equipment. In other words, both achieve the same goal (verifying the outcome), but differ in their efficiency and applicability.
Given the legislative mandate, and based on the positive experience with CORLA, as part of Colorado's Uniform Voting System project, a series of "Mock Risk-Limiting Audits" were performed in several counties using equipment from several different manufacturers. They included innovative demonstrations during the 2015 general election in:
Four major vendors, including ES&S, Clear Ballot, Dominion, and Hart, now provide Cast Vote Records that can be used (despite some unfortunate formatting problems) to do ballot-level Risk-Limiting Audits, so Colorado has really helped move the world of elections forward.
In 2015, Colorado selected a voting system based on Dominion's proposal.
For more information, see:
Site run by Neal McBurnett