The Colorado Risk-Limiting Audit Project (CORLA)

Colorado was the recipient of a grant from the Election Assistance Commission (EAC) to implement Risk-Limiting Audits. These notes from Neal McBurnett describe the goals and status of the project.

Program Name: Colorado Risk-Limiting Audit Project (CORLA)

Funding Level: $230,000

Duration: 2011 to 2014

Project Description

The Colorado Department of State plans to pilot risk-limiting audits in five counties in a mix of areas – urban, suburban, and rural. Colorado will develop, test, and implement a risk-limiting audit system that helps meet legislation requiring the state’s jurisdictions to implement a risk-limiting audit system by the 2014 elections. The research team will review current election processes in target counties and solicit input from other stakeholders, including county election officials, voters and concerned citizen groups. The team will also review existing best practices in the state and the nation, selecting the most promising practices for testing the appropriate target counties. Due to the wide array of election methods, vendors, and types of voting equipment used in the state’s 64 counties, the recommendations developed by the research team will be applicable in many of the jurisdictions across the nation.


Paper ballots are critical, but we also need to look at a sample of them by hand to check up on the equipment and procedures - good audits are critical also. This has been recognized by the Presidential Commission on Election Administration. They say:

Recommendation: Audits of voting equipment must be conducted after each election, as part of a comprehensive audit program, and data concerning machine performance must be publicly disclosed in a common data format. ....
The Commission endorses both risk-limiting audits that ensure the correct winner has been determined according to a sample of votes cast, and performance audits that evaluate whether the voting technology performs as promised and expected.

We had first-hand experience with the benefits of audits, and the failures of certification. We've done innovative audits starting in 2004 in Boulder County and did the first risk-limiting audit outside of California. In contrast, the state lost a lawsuit due to inadequate certification and flawed security. So we adopted a flexible framework for voting system approval, and a state-wide requirement for risk-limiting audits by 2017.

Arapahoe County Risk-Limiting Audit, 2013

The CORLA project in Colorado has now given us experience with Evidence-Based Elections as described by Stark and Wagner.

On November 14, 2013, we did a risk-limiting ballot-level audit of three contests for the general election in Arapahoe County. We used the website Tools for Comparison Risk-Limiting Election Audits to audit cast vote records generated by hardware and software from Clear Ballot Group.

The full report for the EAC is available at Risk-Limiting Audit – Final Report (pdf)

See also the presentation Philip Stark made on ballot-level auditing supported by ballot scans done with open-source software. He delivered it at the Colorado Best Practices and Vision Commission on Dec 14, 2011. You can get his slides from

and listen to the audio there also. The commission likes the approach, as do many other lawmakers and election administrators, including the secretary of state. There are several options for open source software to do state-of-the-art, easily auditable scanning and tabulation, including the OpenCount system for Operator-Assisted Tabulation of Optical Scan Ballots. OpenCount has been used to audit several California elections.

An excellent, comprehensive introduction to risk-limiting audits is Risk-Limiting Post-Election Audits: Why and How., by Bretschneider, J., S. Flaherty, S. Goodman, M. Halvorson, R. Johnston, M. Lindeman, R.L. Rivest, P. Smith, and P.B. Stark, 2012.

Related information